A quick update here.

It seems like are essentially two ways we could introduce such a process:

1. a centralized build step that is integrated into Mediawiki itself (a build script, basically) and
2. a per-extension build step, where individual codebases (skins and extensions) define their own build script (probably in a package.json file).

I was originally under the assumption that the first approach was going to be more feasible based on the way our infrastructure is set up. Ruby on Rails does something similar with its asset pipeline; you can run a script to precompile assets, something that is often done at the end of the deployment process. One could imagine a MediaWiki build script that looks at all the ResourceModules that have been defined, checks which ones need to be pre-processed, and then runs them all through an identical script and stores the output in a central location (either in /vendor or /resources/ or somewhere else). Flexibility would be limited; everyone would have to use (mostly) the same tools.

However, it sounds like the second approach (extension-defined build step) might actually be more compatible with the upcoming Kubernetes-based deployment system that Release Engineering is working on. Images will be built per-extension anyway during the deployment process. Running an NPM build script as an additional step in the process is not much of a hurdle in terms of infrastructure (as long as the result of the build script is predictable).

Some standardization would still be needed in terms of what script to run and where to look for output. Security considerations will also remain, so allowable packages may need to be confined to a (security-reviewed) white-list. Tools with few or no dependencies (Rollup, Typescript, etc) would likely be preferable over ones like Webpack or Babel that have big dependency graphs. But within certain constraints teams would be free to write code however they want. And tools which are useful but don't participate in the build process at all (Storybook, for example) could be listed as optionalDependencies in package.json so that they don't get downloaded or run in CI at all, meaning they wouldn't need security review.

I think we should still consider both approaches (centralized vs per-extension), but having the extra flexibility of the second option seems helpful. The downside is that 3rd-party users of MediaWiki would not get such capabilities automatically since it would not be part of MW core.

Thanks for the write-up, Eric!

In T272877#6813880, @egardner wrote:

And tools which are useful but don't participate in the build process at all (Storybook, for example) could be listed as optionalDependencies in package.json so that they don't get downloaded or run in CI at all, meaning they wouldn't need security review.

In the case of a future UI library, that would possibly translate to a limitation, I was hoping to address with the introduction of Storybook. Having up-to-date, branch-specific demo of components available in the CI pipeline for, for example, automated accessibility testing or visual regression testing.

In T272877#6814652, @Volker_E wrote:

In the case of a future UI library, that would possibly translate to a limitation, I was hoping to address with the introduction of Storybook. Having up-to-date, branch-specific demo of components available in the CI pipeline for, for example, automated accessibility testing or visual regression testing.

There may be ways to work around this and still get what you are describing: a live, public storybook page for each project on the web somewhere. It might be possible to define a job that installs and runs Storybook in a dedicated container, and then use a different container (without Storybook) for the final build. As long as the optional dependencies were not present during the final build, they would likely not need to be audited.

I've opened an official Technical Decision Making Process task to address this question over at https://phabricator.wikimedia.org/T279108; that ticket can serve as a public document for whatever comes out of the TDMP to ensure that the wider community has some visiblity into our thinking.

I expect many of the questions discussed above will be addressed in the course of this process: centralized, mw-coupled build step vs. extension/skin specific, security review of packages, how much flexibility teams will have to define their own tooling, etc.