Page MenuHomePhabricator

Explore improving build process to avoid duplicate builds that overwrite each other
Closed, ResolvedPublic

Description

Right now, the Codex build process involves several stages that build the same file multiple times, overwriting the result of the previous build. Specifically:

  • The main library is built twice, once unminified in ESM format, and once minified in the CJS and UMD formats. These two builds produce non-overlapping JS files, but they produce the same CSS files. The first build produces unminified versions of the CSS files, the second build then overwrites them with minified versions.
  • For all builds except the sandbox (the main library, the split library, and the search build) we run the build four times, with each build producing different CSS files (LTR, RTL, LTR legacy and RTL legacy). These four builds all produce the same JS files, which keep being overwritten with identical copies of each other. We had to disable the use of hashes in chunk file names, otherwise these four builds would not have been identical, which would have caused problems.
    • Note that the split build also produces four copies of the manifest.json file, each with a different name and slightly different contents, because the names of the CSS files differ

We should explore using Vite plugins to improve this situation in some way. For the additional style builds, maybe we could suppress the generation of the JS files, or get one build to generate all 4 versions of the CSS files. For the different minification settings for different JS formats, maybe we could suppress the generation of the CSS files for the minified build, or build them but with different names, or build them all as one build but use some other way to prevent minification of the ESM build.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 972734 had a related patch set uploaded (by Tyhopp; author: Tyhopp):

[design/codex@main] feat: Deduplicate build emissions

https://gerrit.wikimedia.org/r/972734

@Catrope I took a first pass at this.

Having entirely separate builds for JS and CSS for each build type (main mjs, main cjs/umd, split, search) might have some maintainability upside in that it would probably make the build script code easier to follow, but it would add more build passes than there is already.

That being said, the approach in the patch keeps the build passes the same and adds a Vite plugin to enable declaration of which files get emitted (one of the solutions you mentioned in the description).

Change 972734 merged by jenkins-bot:

[design/codex@main] build: Deduplicate build emissions

https://gerrit.wikimedia.org/r/972734

Change 980902 had a related patch set uploaded (by Anne Tomasevich; author: Anne Tomasevich):

[mediawiki/core@master] Update Codex from v1.0.1 to v1.1.1

https://gerrit.wikimedia.org/r/980902

Test wiki created on Patch demo by ATomasevich (WMF) using patch(es) linked to this task:
https://patchdemo.wmflabs.org/wikis/d419984345/w

Change 980902 merged by jenkins-bot:

[mediawiki/core@master] Update Codex from v1.0.1 to v1.1.1

https://gerrit.wikimedia.org/r/980902

Test wiki on Patch demo by ATomasevich (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmflabs.org/wikis/d419984345/w/