Motivation
Our "dependency management" (more accurately, requiring js files/modules within other ones) of javascript files/modules are being currently handled through the Resource Loader (RL).
Resource Loader dependency management feature was not designed to also host this kind of use-case.
Using javascripts's require/import solution is more appropriate for that use-case.
Using a packing solution on top of that is also appropriate to reduce the amount of http traffic overhead if these files were loaded asynchronously (which is still not supported by all browsers to this date https://caniuse.com/#feat=es6-module-dynamic-import) as well as
Problem
- Testing modules and developing them is rather hard due to the deep integration with the Resource Loader.
- Currently we can't use ES6 features in the front end code.
- RL's minification method is also a little bit outdated and inefficient (See T49437: Consider a pipeline for enhanced minification (e.g. support UglifyJS)). Tools like Webpack solves or improves all of these issues.
Other problems, partly related
- Frontend code in Wikibase Repo is pretty modular which is good but we are using RL for dependency management and class loading resulting to have 300 RL modules with a very complex dependency relationship that causes TTFB (time to first byte) to be high and pretty big network overhead (e.g. T203696: Drastically reduce the number of ResourceLoader modules that WikibaseClient registers: "It looks like Wikibase is trying to use ResourceLoader as a class loader(!) which may be difficult to quickly re-architect, but any reduction would be worthwhile.").
Suggested Solution
Try usings Webpack to bundle our javascript modules in single files.
Webpack is chosen for the following reasons:
- it is a mature and widely-used build and packaging tool in javascript ecosystem
- it brings modern tooling and language features from the Javascript ecosystem into our stack through easy integration with Babel
- it allows for easier use of third party libraries through npm and bundling them with the rest of the code
- it has been already done and used in our production for MobileFrontend extension T195475: [EPIC] Automate asset bundling in MobileFrontend, and also related T207787: [EPIC] Reduce the amount of modules in MobileFrontend T194098: Reduce number of different MobileFrontend bundles, hence not completely unheard of in the Mediawiki context
- it has better minification than Resource Loader
- we can combine Resource Loader modules and reduce their number in Wikidata's production to decrease the traffic sent to users and TTFB
Expected Impact
- On dev productivity: can be big, allowing us to use modern javascript through Babel
- On performance (secondary goal here): Webpack is not the only mean to improve performance.
- Every module removed from lib or client will have ~30GB/day reduction on network. We planning to drop probably around 10-20 of them
- Every module removed from view or repo will have ~55 MB/day (wikidata) + 90 MB/day (commons) reduction on network. We are planning to drop probably around 100-200 of them.
- TTFB is hard to measure. Can be measured afterwards.
- The network reduction due to using webpack instead of RL manification is hard to measure. Can be measured afterwards.
Estimated Trailblazing length
couple of weeks (2-3 weeks for 100-150% FTE)