We discussed and gathered feedback on a code splitting approach in T344386, and then laid out a high-level plan in T344386#9168661. This task is for implementing the CodexModule approach (but not the shared module for common components).

Proposed API

(Note: this proposed API may have to change based on implementation considerations; if that becomes necessary, please note it here).

Usage of this Codex code-splitting feature should be as straightforward as standard usage of Codex in a MediaWiki feature. We imagine it would work in the following way:

• Users would access this functionality through the CodexModule class
• Users can specify a list of desired Codex components (and composables) as part of the module definition of the consuming ResourceLoader module. This list should be provided as an array of string component names; the name of this new RL module definition property should be codexComponents.
• Users can provide an additional line in their module definition to receive only the CSS-only version of the specified components: this property should be called codexStyleOnly and it should accept a boolean value.
• Loading the entire Codex library continues to be supported through the existing @wikimedia/codex and codex-styles modules
• Users can access all specified Codex components in their own JS code via require( './codex.js' ); this differs from how Codex components are traditionally accessed; users loading the entire library would use require( '@wikimedia/codex' )

Potential future enhancements:

• Allow code that loaded a subset to also use require( '@wikimedia/codex' ); ideally there should be no distinction here between users who are loading all Codex components and users who are only loading a subset, but this may be more difficult to implement
• Modules which embed different subsets of Codex can depend on one-another. In particular, we want to support a pattern where a CSS-only module is defined first (to style a server-rendered UI), and then a second module which depends on this first one would add the full Vue versions of the given components.

Example 1: A CSS-only use of CodexModule:

"ResourceModules": {
    "ext.foo.cssonlyfeature": {
        "class": "MediaWiki\\ResourceLoader\\CodexModule",
        "styles": [
            "cssonlyfeature.less"
        ],
        "codexComponents": [
            "CdxCard"
        ],
        "codexStyleOnly": true
    }
}

Example 2. A JS-enabled use of Codex Module which depends on the first (CSS-only) module.

"ResourceModules": {
    "ext.foo.hydratedfeature": {
        "class": "MediaWiki\\ResourceLoader\\CodexModule",
        "codexComponents": [
            "CdxCard",
            "CdxMessage",
            "CdxButton"
        ],
        "dependencies": [
            "ext.foo.cssonlyfeature"
        ]
    }
}

Implementation

Already done:

• Codex's build process has already been changed to produce a modular build, with separate .js and .css files for each component, plus a manifest.json file that lists the dependency relationships between these files (T345688)

MVP:

1. Implement basic support for codexComponents in the CodexModule class: load the specified components and make them available in the codex.js virtual file; but without support for CSS-only or dependencies (there's a proof of concept sketch of this in this patch) (T350054)
   • Preserve the existing CodexModule code (which is specific to the codex-styles and codex-search-styles modules) for now; but plan to remove it later
   • Add code that:
     • Determines which manifest file to use (LTR, RTL, legacy LTR or legacy RTL) based on the context
     • Determines which files to load from Codex, based on the codexComponents key in the module definition and the contents of the manifest file
     • Adds those files to the module object's packageFiles and styles properties
     • Adds a virtual codex.js file that gets the requested components from the Codex files and exports them
   • Run that code when getPackageFiles, getStyleFiles or getDefinitionSummary are called, before calling the parent's implementation of those methods
2. Implement support for codexStyleOnly (T350055)
   • When this flag is set, don't add files to packageFiles, only add them to styles
3. Reimplement the @wikimedia/codex-search and codex-search-styles module using these new features (T350058)
   • Replace codex-search-styles with something like { "codexComponents": [ "CdxTypeaheadSearch" ], "codexStyleOnly": true }
   • Replace @wikimedia/codex-search with something like { "codexComponents": [ "CdxTypeaheadSearch" ], "codexScriptOnly": true, "dependencies": [ "codex-search-styles" ] }
4. Remove themeStyles support from CodexModule class (T355842)
5. Remove the codex-search build (T357596)
6. Deprecate @wikimedia/codex-search and codex-search-styles (T356675)

Potential future improvements:

• T358235: [EPIC] Support dependency de-duplication of Codex components in ResourceLoader
• T350052: Code splitting: @wikimedia/codex collector module (simple version)

Open Questions

• How can we provide backwards compatibility for code that depends on the @wikimedia/codex module and expects that to load the full library? Or is this not feasible, and do we have to accept this as a breaking change (making that code use e.g. codex-all instead)?
  • One way we might be able to do this: add some sort of special treatment for the @wikimedia/codex module in RL so that other modules can call require( '@wikimedia/codex' ) even if that module isn't loaded, and can add things to the object that that require call returns. Then modules using CodexModule would not depend on the @wikimedia/codex module and it wouldn't be loaded. The @wikimedia/codex module would contain the full library and would only be loaded if it was explicitly depended on.
    • If we want to provide a custom error message for code that attempts to access missing components (see implementation plan step 1), then we would need a way to wrap the object returned by require( '@wikimedia/codex' ) in a Proxy. Perhaps we could accomplish this through a feature similar to skipFunction that allows a module to run a short snippet of code to provide the value require() should return when the module has not yet been loaded?
• Is there (or could there be) a cleaner way to override the packageFiles property in ResourceLoader\FileModule? Overriding getPackageFiles doesn't work, because getDefinitionSummary calls the private expandPackageFiles method that then pollutes a cache that getPackageFiles uses. We can work around this by overriding getDefinitionSummary, but if any other methods were to call expandPackageFiles in the future we'd have to override them too, so that doesn't seem like a great approach.
• How does a module using CodexModule add components to the collector module when it executes? Does it wrap the main script with a new file that adds the components before calling the old main file?
• How do we ensure cache invalidation behaves correctly for a module using CodexModule? What do we need to do in getDefinitionSummary and/or elsewhere to make this work?
  • We need to invalidate the cache when one of the Codex files changes, or when the set of files that is loaded changes. The latter can happen when the manifest files change, or when the module's list of dependencies changes, or when one of the module's dependencies changes which Codex files it loads.
  • Adding the Codex files to the packageFiles and styles properties should take care of invalidating the cache when those files change. Should we just add the list of files that will be loaded to getDefinitionSummary?
• Should we cache reading the manifest files and deriving the information we need from them? Would it make sense to build a data structure derived from the manifest files that is in a more convenient format for building module contents, and caching that in memcached, invalidated by a hash of the contents of the manifest file or something like that?