[[ This document is still a work in progress; expect changes ]]
What is the problem or opportunity?
Enable MediaWiki developers to automatically compile front-end assets (scripts, styles, templates, etc) as part of the deployment or CI process for skins and and extensions.
- Developers can run build scripts automatically in the CI process, without having to commit derivative assets to version control. Some extensions area already using a build step, but they must commit the output to source in order to deploy it. This clutters up code review and can produce conflicts in VCS. Running a build script automatically in CI would avoid the need to commit compiled assets this way along with the associated problems.
- Teams have some flexibility to define their own build scripts on a per-project basis. Not every team is going to use the same tools. Some teams may wish to use tools like Typescript; others won't. While there may need to be limits on what kinds of scripts are allowed (see next requirement), the infrastructure should allow a good amount of flexibility. Letting teams define a build script in a location like package.json seems like a good idea.
- The process must not introduce security vulnerabilities: Flexibility is important, but so is security. The current trend in the wider front-end development community is to rely on an elaborate tool-chain, using software like Webpack along with numerous plugins to generate a large bundle of compiled, obfuscated code that is then shipped to browsers. If un-audited packages with large numbers of dependencies participate in the build process, it becomes hard to guarantee the security or stability of the final output. Teams should have some flexibility in terms of tooling choices, but there should also be limits on what packages can be used. Developers could maintain a list of approved packages or even a private NPM registry to help mitigate security risks.
- The build process should be deterministic. The same input should produce the same output at all times.
- Write frontend code with TypeScript.
- Codex, which is the WMF's UI framework of choice, is written in TypeScript. One cannot directly use code snippet examples (e.g. https://doc.wikimedia.org/codex/main/components/demos/card.html) in MediaWiki core/extension/skin code, because the examples are written in TypeScript. This slows the adoption of Codex.
- Rapidly develop frontend components in a standalone environment, and easily re-use these components in MediaWiki code (example: T328125: Spike: Experiment with creating GrowthExperiments components in external library)
What does the future look like if this is achieved?
Achieving this means that developers can rely on an automated build step in production.
The benefits include flexibility and improved developer experience (allowing developers to write code in Typescript or more modern versions of JS) as well as performance optimizations (tree-shaking of external dependencies, Vue template pre-compilation) to send more efficient payloads to the user’s browser. A given extension or skin could specify a build script in a place like its package.json file, without the need for developers to commit compiled code into version control.
What happens if we do nothing?
Doing nothing means that we give up some opportunities to improve the site performance for users and continue shipping unnecessary code in many places. We also will continue to limit developers in that the code they write must be the exact code that is run in all browsers. This means no ability to rely on tools like Typescript (very useful for reducing bugs in software) or to automate the process of supporting legacy browsers. Developers in the wider web community are increasingly used to having these tools, so this will be one way that we continue to separate ourselves from wider trends and best practices.
Any additional background or context to provide?
This work should be seen as the next stage in the larger Vue.js migration / front-end modernization project. The original Vue.js RFC mentioned this task as one potential follow-up. The original Build Step RFC from 2018 is also still valid and provides good background on why this is important to have.
Why are you bringing this decision to the technical forum?
This problem cuts across the concerns of many different teams (product teams, security, release engineering, etc); similarly, any solutions we adopt will impact many teams downstream – enabling or limiting some of their options in terms of development tooling. It is essential to have input from all impacted groups to come up with an adequate solution here.
One example that is worth noting in particular: the Release Engineering team is preparing to roll out some new container-based infrastructure to support the deployment process. It is possible that a front-end build step could "piggyback" on this new infrastructure, providing improved developer experience across all teams without requiring a huge amount of additional work.
- [Draft] Decision Statement Overview document (WMF staff only; includes the above problem statement)
- Original frontend build step RFC (open, stalled): T199004: RFC: Add a frontend build step to skins/extensions to our deploy process
- Discussion of the advantages of Rollup over Webpack in WVUI (may be relevant generally): T272879: Migrate WVUI library to Rollup or Vite
- Hot-module reloading as a new feature for ResourceLoader: T277747: Explore Hot Module Reloading integration of Vite with Resource Loader
- Other changes being considered for ResourceLoader's debug mode: T85805: Introduce ResourceLoader debug mode v2