We discussed and gathered feedback on a code splitting approach in T344386, and then laid out a high-level plan in T344386#9168661. This task is for implementing the CodexModule approach (but not the shared module for common components).
#### Proposed API
//(Note: this proposed API may have to change based on implementation considerations; if that becomes necessary, please note it here).//
Usage of this Codex code-splitting feature should be as straightforward as standard usage of Codex in a MediaWiki feature. We imagine it would work in the following way:
* **Users would access this functionality through the `CodexModule` class**
* **Users can specify a list of desired Codex components (and composables) as part of the module definition** of the consuming ResourceLoader module. This list should be provided as an array of string component names; the name of this new RL module definition property should be `codexComponents`.
* **Users can provide an additional line in their module definition to receive only the CSS-only version of the specified components**: this property should be called `codexStyleOnly` and it should accept a boolean value.
* **Loading the entire Codex library continues to be supported** through the existing `@wikimedia/codex` and `codex-styles` modules
* **Users can access all specified Codex components in their own JS code via `require( './codex.js' )`**; this differs from how Codex components are traditionally accessed; users loading the entire library would use `require( '@wikimedia/codex' )`
Potential future enhancements:
* **Allow code that loaded a subset to also use `require( '@wikimedia/codex' )`**; ideally there should be no distinction here between users who are loading all Codex components and users who are only loading a subset, but this may be more difficult to implement
* **Modules which embed different subsets of Codex can depend on one-another**. In particular, we want to support a pattern where a CSS-only module is defined first (to style a server-rendered UI), and then a second module which depends on this first one would add the full Vue versions of the given components.
Example 1: A CSS-only use of CodexModule:
```lang=json
"ResourceModules": {
"ext.foo.cssonlyfeature": {
"class": "MediaWiki\\ResourceLoader\\CodexModule",
"styles": [
"cssonlyfeature.less"
],
"codexComponents": [
"CdxCard"
],
"codexStyleOnly": true
}
}
```
Example 2. A JS-enabled use of Codex Module which depends on the first (CSS-only) module.
```lang=json
"ResourceModules": {
"ext.foo.hydratedfeature": {
"class": "MediaWiki\\ResourceLoader\\CodexModule",
"codexComponents": [
"CdxCard",
"CdxMessage",
"CdxButton"
],
"dependencies": [
"ext.foo.cssonlyfeature"
]
}
}
```
#### Implementation
Already done:
- Codex's build process has already been changed to produce a modular build, with separate `.js` and `.css` files for each component, plus a `manifest.json` file that lists the dependency relationships between these files (T345688)
MVP:
1. Implement basic support for `codexComponents` in the `CodexModule` class: load the specified components and make them available in the `codex.js` virtual file; but without support for CSS-only or dependencies (there's a proof of concept sketch of this in [[https://gerrit.wikimedia.org/r/c/mediawiki/core/+/944351/3/includes/ResourceLoader/CodexModule.php|this patch]]) (T350054)
- Preserve the existing CodexModule code (which is specific to the `codex-styles` and `codex-search-styles` modules) for now; but plan to remove it later
- Add code that:
- Determines which manifest file to use (LTR, RTL, legacy LTR or legacy RTL) based on the context
- Determines which files to load from Codex, based on the `codexComponents` key in the module definition and the contents of the manifest file
- Adds those files to the module object's `packageFiles` and `styles` properties
- Adds a virtual `codex.js` file that gets the requested components from the Codex files and exports them
- Run that code when `getPackageFiles`, `getStyleFiles` or `getDefinitionSummary` are called, before calling the parent's implementation of those methods
2. Implement support for `codexStyleOnly` (T350055)
- When this flag is set, don't add files to `packageFiles`, only add them to `styles`
3. Reimplement the `@wikimedia/codex-search` and `codex-search-styles` module using these new features (T350058) then deprecate them (T356675)
- Replace `codex-search-styles` with something like `{ "codexComponents": [ "CdxTypeaheadSearch" ], "codexStyleOnly": true }`
- Replace `@wikimedia/codex-search` with something like `{ "codexComponents": [ "CdxTypeaheadSearch" ], "codexScriptOnly": true, "dependencies": [ "codex-search-styles" ] }`
4. Remove `themeStyles` support from CodexModule class. T355842
Potential future improvements:
- Implement support for dependency deduplication (T350056)
- In CodexModule, expose a new method that returns a list of Codex chunk files that will be loaded by this module based on its codexComponents setting. (This method probably needs to take a Context object as a parameter, both for getting the module's dependencies and to get the right files based on the directionality and skin.)
- After determining which files should be loaded, determine which of the module's dependencies are also CodexModules, and ask them which chunks they're loading. Only load those chunks that won't already be loaded by a dependency.
- It's possible (and encouraged) for a module A to request component X, and for module A to also depend on module B, which includes only the CSS of component X. When that happens, module A should load only the JS for component X, but not its CSS (since that will be provided by module B). Deduplicating on a per-file level should support this use case, since the JS and CSS for component X are in separate files.
- ~~Turn the `@wikimedia/codex` RL module into a collector module (T350052)~~
- This module should expose the Codex components that have already been loaded. In other words, if the CdxButton component was loaded as part of a module that uses CodexModule, then any module should be able to do `const { CdxButton } = require( '@wikimedia/codex' );` to obtain the CdxButton component
- We could explore adding a feature to ResourceLoader that would allow `require( '@wikimedia/codex' )` to be called (and the returned object to be extended) before the `@wikimedia/codex` module is loaded. If we did that, then we would preserve backwards compatibility for code that expects to load the full library by loading this module.
- If we do this, it would be nice if we could wrap the collector object in a Proxy that throws a custom error when a nonexistent property is accessed on it. That way we can provide a helpful error message for code that attempts to access a component without listing that component in its `codexComponents` array.
#### Open Questions
* How can we provide backwards compatibility for code that depends on the `@wikimedia/codex` module and expects that to load the full library? Or is this not feasible, and do we have to accept this as a breaking change (making that code use e.g. `codex-all` instead)?
- One way we might be able to do this: add some sort of special treatment for the `@wikimedia/codex` module in RL so that other modules can call `require( '@wikimedia/codex' )` even if that module isn't loaded, and can add things to the object that that require call returns. Then modules using CodexModule would not depend on the `@wikimedia/codex` module and it wouldn't be loaded. The `@wikimedia/codex` module would contain the full library and would only be loaded if it was explicitly depended on.
- If we want to provide a custom error message for code that attempts to access missing components (see implementation plan step 1), then we would need a way to wrap the object returned by `require( '@wikimedia/codex' )` in a Proxy. Perhaps we could accomplish this through a feature similar to skipFunction that allows a module to run a short snippet of code to provide the value `require()` should return when the module has not yet been loaded?
* Is there (or could there be) a cleaner way to override the `packageFiles` property in `ResourceLoader\FileModule`? Overriding `getPackageFiles` doesn't work, because `getDefinitionSummary` calls the private `expandPackageFiles` method that then pollutes a cache that `getPackageFiles` uses. We can work around this by overriding `getDefinitionSummary`, but if any other methods were to call `expandPackageFiles` in the future we'd have to override them too, so that doesn't seem like a great approach.
* How does a module using CodexModule add components to the collector module when it executes? Does it wrap the main script with a new file that adds the components before calling the old main file?
* How do we ensure cache invalidation behaves correctly for a module using CodexModule? What do we need to do in `getDefinitionSummary` and/or elsewhere to make this work?
- We need to invalidate the cache when one of the Codex files changes, or when the set of files that is loaded changes. The latter can happen when the manifest files change, or when the module's list of dependencies changes, or when one of the module's dependencies changes which Codex files it loads.
- Adding the Codex files to the `packageFiles` and `styles` properties should take care of invalidating the cache when those files change. Should we just add the list of files that will be loaded to `getDefinitionSummary`?
* Should we cache reading the manifest files and deriving the information we need from them? Would it make sense to build a data structure derived from the manifest files that is in a more convenient format for building module contents, and caching that in memcached, invalidated by a hash of the contents of the manifest file or something like that?