Background
I've maintained Pageviews Analysis for a decade now. It is littered with bugs and is difficult to maintain. I've received a lot of interest from potential contributors, but in the end I think most people just find it too unorthodox of a codebase to contribute to. I even find myself reluctant to touch it because I know it will be a headache. It is essentially a glorified jQuery app and its age is showing.
The current implementation also chiefly relies on the clientside for processing, prohibiting us from any large-scale processing that requires more RAM. Not to mention the Massviews has an unnecessarily large memory footprint as it is.
Needless to say, I think it's time to rewrite this thing.
Goals
- The clientside should be 100% Codex and Vue
- Use any and all applicable MediaWiki conventions
- Provide a backend API (preferably RESTful) for most or all functionality
- Allow bulk processing, using a job queue (supersedes T157830)
- Have full support from a WMF team
Where the code should live
MediaWiki
PageViewInfo exists and would seemingly be the right home for this.
Pros:
- More likely to be co-maintained
- Easier backend processing of categories and other MW-specific functionality
- Robust job queue
- Existing action API which can be extended
- Allows us to embed the experience on the wiki (Special pages)
- Possible integration into wikitext and Lua (parser functions, magic words, etc.)
- Charts exist now in production and largely offer what we need.
Cons:
- No WMF team ownership so it may be hard to get the new features reviewed and deployed
- Possible difficulties conforming functionality to the agnostic PageViewService
Wikimedia Statistics
https://stats.wikimedia.org (Data-Engineering-Wikistats) is the official Wikimedia Stats tool.
Pros:
- Consolidated with other official stats tooling
- Already in production
- Data Engineering team may be available to help
Cons:
- No Codex or Vue (though supposedly this is slated to change?)
- Unfamiliar codebase for most
- May not be possible to have chart feature parity with what we're currently doing with Chart.js (?)
- Unclear if there is a backend we could leverage for bulk processing and job queuing
External tool (again)
This is most likely what will happen, with the repo on GitLab.
Pros:
- We can still write the Codex and Vue code with MediaWiki in mind so it's mostly copy/paste to the extension (if we later decide to do that)
- Code moves faster
- Cloud Services has superb uptime
- We could explore server-side rendering
- GitLab is significantly easier for newcomers to contribute to than Gerrit (as would be required if we went the extension route)
Cons:
- Does not give this vital tool the WMF ownership it deserves.
- Though maybe what we build could be deployed as a standalone app, akin but separate to stats.wikimedia.org ?
- We need a job queue (RabbitMQ or some kind of message broker)
- Potentially more maintenance with it being outside production
Decision
I've gone with the "external tool" option.
Work-in-progress at:
