Page MenuHomePhabricator

[EPIC] Modernize portals code for browser support and maintainability
Open, LowPublic

Description

When the project to bring the wikimedia portals into Git was started in 2015, the browser landscape and web platform looked much different than they do now. Much of the code in the Portals repository was written still targeting basic support for IE6 and the build pipeline was built using a version of Node that didn't even support native promises.

Since it's original creation, many of the dependancies (especially Gulp plugins) have become abandonware (or have even been formally deprecated, like Phantom.js). These dependancies are prone to security vulnerabilities, and should be replaced.

Since there are so many things that can be upgraded in the portals repo, they can be categorized into the following areas of concern.

Client-side code

Much of the client-side code in the portals repository was written with ancient browsers like IE6 in mind (both JS and CSS). Browsers lower than IE9 cannot even access Wikimedia sites any more due to their limited TLS 1.2 support. For this reason, much of the code in the portals repo can be upgraded to use “modern” ES5 methods, like for e.g. forEach and XMLHTTPRequest requests instead of JSONP (or maybe even fetch() with a polyfill) and CSS hacks for ancient versions of IE can be removed.

Build process

The Gulp.js build process can be split into separate files and eventually refactored to remove the unmaintained Gulp plugins. Eventually, after this build pipeline is cleaned up, a bundler like Rollup.js can be introduced to better handle the client-side JS.

Data pipeline

The data pipeline that currently powers the stats for the wikipedia.org portal is very opaque and outdated and built targeting a version of Node that didn’t yet support native promises. The source of the data depends on a script hosted on: https://pagecounts.toolforge.org/pagecounts.json which is unmaintained, as well as the Wikimedia data dumps https://dumps.wikimedia.org. The entire pipeline should be refactored and built to depend on a maintained data source such as https://stats.wikimedia.org instead.

Event Timeline