Arising from James's previous musing, and discussions at the 2019 Hackathon.
What
- InitialiseSettings.php (and much of CommonSettings.php) is replaced with per-wiki inheritable YAML files (to allow comments).
- Actually-variant config goes into a much slimmer CommonSettings.php (or re-worked to not vary).
- On merge, the YAML files are converted into one JSON file per wiki, for the currently-deployed version(s) of MW, which are stored in git.
- This replaces the opportunistic cache in /tmp that we current have.
Inheritance tree:
allwikis.yaml
| Default values for all wikis (e.g. wgNamespacesWithSubpages which is over-ridden, or wgEnableCanonicalServerLink which isn't)
|
+- wikipedias.yaml
| Standard values for Wikipedias, where they differ from defaults (e.g. wgSitename or the fallback logo) and special inheritances
|
+- dewiki.yaml
Bespoke values for the German Wikipedia (e.g. the logo, or FlaggedRevisions configuration) and other special inheritancesComparison:
| Task | Current situation | Future state |
|---|---|---|
| Config authored in | InitialiseSettings.php | wikipedias.yaml etc. |
| Config build step | Runtime cache, in /tmp/ | Build time static file, in /srv/mediawiki/ |
| mw-config merge | Trivial rebase | Full production build of on JSON static file per wiki |
| Config read step | From cache or computed live | Always read from built static file |
Pros
- Variant configuration will be static, making it more plausible to inject into docker images.
- It will be much clearer exactly which wikis' config is changing, so deployers have more confidence.
- YAML configuration files explicitly set the inheritance pattern.
- Easier to compare one wiki's config with another's (e.g. "how different is dewiki from frwiki?").
- Clear when the rump of CommonSettings refers to undefined variables; variant config forced to be merged first.
Cons
- Merging is harder (and slower?).
- Harder to audit all wikis' config for settings that "shouldn't" be over-ridden, or see how values vary.
- Production branch pruning, currently just a disc operation and a sync, now needs a commit to mw-config as well as a deploy to delete.
First time we're reading YAML files in PHP prod.We're not reading them in prod, only in CI.
Former questions
- Deterministic sort of output files to avoid noise.
- Assuming that alphasort of the array by keys (ksort) sufficient.
- How do we do splicing in private settings at run time?
- Private settings are already spliced in in CommonSettings; no change.
- Syntax for specifying config, and that a document inherits from another.
- Roughly worked out; to be documented.
- Syntax for specifying that descendent config can't over-ride (e.g. wgMiserMode)
- For now, this is just a simple all.yaml file that is re-applied at the end and so can't be over-ridden.
- Do we need to vary on the PHP run-time still? (once HHVM is un-deployed can this go away, or are there reasons beyond PHP serialisation format that we think this might vary?)
- No. Nothing has been variant between HHVM and Zend for a while. No reason to continue.
Open questions
- How does the CI work for this?
- Do we need to check on build time that a vanilla MediaWiki install (i.e., DefaultSettings) doesn't set any config that isn't represented in all.yaml?
- What do we do about variant non-static config?
Planned steps
- Write cached config to JSON as well as serialised PHP https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/533592/
- Read cached config from JSON not serialised PHP https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/533593/
- Stop writing cached config in serialised PHP https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/533594/
- Write YAML files reader and hook into CI
- Migrate dblist definition into the YAML structure and make the dblist files auto-generated
- Write "InitialiseSettings.json" JSON file writer and hook into CI
- Migrate initial settings from InitialiseSettings.php to YAML
- Completely move all static variant config into YAML; scrap dblists and InitialiseSettings.php
- Pre-calculate config for each wiki and store it in config.git https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/507729/
- Make scap steps to change target for each wiki update cached config in config.git, not just wikiversions.json
- Load config from config.git on disc, not from /tmp