Page MenuHomePhabricator

Convert static mediawiki configuration to form more suitable for containers
Open, Needs TriagePublic

Description

Having static configuration built into the mediawiki container means that it would have to be rebuilt every time configuration changed. The container image shouldn't be tied to a specific configuration or envrionment. We need to move the configuration out of the build step so that we have dynamic configuration and a configuration change only requires a re-deploy.

There has been some discussion here: https://www.mediawiki.org/wiki/Talk:Wikimedia_Release_Engineering_Team/MW-in-Containers_thoughts
and Joe has already written up a good starting point here: https://docs.google.com/document/d/1nGv5N1_PYM5Wl5TTPlkFI8TZTSokuJcmdIZD-dKZdEA

This will likely involve some combination of moving configuration to a data store, inserting some as environment variables, and (maybe) attaching a config volume to the container.

Joe has proposed the following in his document:

  • State and wiki-specific configuration will live in a configuration datastore, and will be fetched periodically by MediaWiki. Right now we’re using etcd for state, but this decision might be revisited if we need to store more stuff into this system.
  • Production code and global, “immutable” configuration should be part of the code release
  • Miscellanea should be managed separately, possibly as another repository again. This needs to be better defined.
  • Every MediaWiki branch will be bundled with production code and Miscellanea and built into a container. It probably makes sense to have a layered approach to such containers so that we can exploit copy on write as much as possible (given production code will be the same across all containers).

Event Timeline

jeena created this task.Sep 17 2020, 6:34 PM
jeena added a comment.Oct 27 2020, 7:34 PM

I thought of some more guidelines we could use to help migrate our configuration. Open to feedback.

  • config that is unlikely to change often could go in a datasore (e.g. the wiki-specific configuration mentioned in the task)
  • config that needs to be edited by the application should be stored in a datastore (e.g. the state configuration mentioned in the task)
  • secret config should be stored in a separate database or maybe something like hashicorp Vault is an option?
  • ‘tuning’ config, such as timeouts, etc could be inserted as an environment variable or configmap
  • frequently changed (by humans) config should be inserted as an environment variable or configmap

As usual, one of the challenges is that whatever we do should also downscale/work seamlessly for local development and for third party installations without all the fanciness of our production environment.