Page MenuHomePhabricator

Allow additional LocalSettings to be overriden by environment variables
Closed, DeclinedPublic

Description

It would be helpful if LocalSettings.php could effectively be set through environment variables. This would allow users to set the defaults for all wiki's on a system, but would also let easy configuration within a docker container or vagrant, etc. And would make MediaWiki closer to becoming a 12-factor app.

You would still be able to override the defaults with a real LocalSettings.php file as well. However, a LocalSettings.php would technically not be required at all.

The order would be something like this:
DefaultSettings.php -> Environment Variables (if set) -> LocalSettings.php

The environment variables should take on the same pattern as the variable names for example:
$wgStyleSheetPath would become MEDIAWIKI_STYLE_SHEET_PATH or if we want to get the wg then it would be WG_STYLE_SHEET_PATH, but I think MEDIAWIKI_ is a better prefix.

Event Timeline

In the past we used to get some settings via environment variables like HTTP_PROXY but that turned out to be a security issue: https://httpoxy.org/

I'd rather see us implement something like T115432: Get MW_CONFIG_FILE from env to facilitate virtual hosting - would that address your use-case?

I'd rather see us implement something like T115432: Get MW_CONFIG_FILE from env to facilitate virtual hosting - would that address your use-case?

Unfortunately it doesn't because in the case of Docker, the LocalSettings.php would still need to be in the container and there's no way to put it there unless the user mounts the file from the host or goes into the container and copy it in. But that's the same as they are doing now, so it doesn't make it any easier.

I would like to be able to alter the configuration from outside of the container, which can only be done with environment variables (afaik).

In the past we used to get some settings via environment variables like HTTP_PROXY but that turned out to be a security issue: https://httpoxy.org/

That seems specific to that environment variable, though, unless I'm missing something. @Legoktm do you have broader concerns with using environment variables? If not, could we talk about what the specific implementation could look like (proposal in task description looks good to me) so that work could start on patches?

brennen moved this task from Backlog to Radar on the User-brennen board.
brennen subscribed.

Supporting this in core would take on additional work. Enabling it by default seems problematic for performance. However this seems trivial to do from a Config subclass or extension hook. That might be a smoother way to kickstart this and if/when it becomes popular we could consider the trade-offs with WMF maintaining it and/or with it residing in core.

I think today it's had to make this appealing since (obviously) nobody uses it as it doesn't exist, and there are trivial ways for any given use case to do this in a differnet way which also means it will likely never reach the urgency required to have someone proactively work on this on the idea that it might become useful.

Alternatives:

  • Plant LocalSettings.php with a forwarder to wherever your file is. E.g. for MW_CONFIG_FILE=/etc/elsewhere/my.php to work you only need to place require getenv('ACMECORP_MW_CONFIGFILE')' in ones LocalSettings.php file. Or if it doesn't need to be runtime dynamic, a symlink could work as well (this is what WMF currently does).
  • Populate LocalSettings.php with specific assignments from wherever, e.g. Etcd or indeed Env variables. E.g. for ACMECORP_MW_DBSERVER=mydb to work, one only needs to set $wgDBserver = getenv('ACMECORP_MW_DBSERVER'); in LocalSettings.php. This is how WMF loads some of its DB configuration from Etcd today.
  • Override runtime settings from an extension hook. E.g. load row from mysql, or (automatically) map any number of settings to optionaly existant Env variables. This is how Fandom currently load some of their settings from a MySQL table on a per-wiki basis.

The logic of Etcd was turned into a Config subclass for easier use. I suppose something similar could be done in an extension for Env to be used/maintained by interested parties. All the neccecary bits for this are in place, I think?

Krinkle renamed this task from Allow DefaultSettings to be overriden by environment variables to Allow additional LocalSettings to be overriden by environment variables.Nov 17 2020, 3:39 AM
Krinkle updated the task description. (Show Details)

Supporting this in core would take on additional work. Enabling it by default seems problematic for performance. However this seems trivial to do from a Config subclass or extension hook. That might be a smoother way to kickstart this and if/when it becomes popular we could consider the trade-offs with WMF maintaining it and/or with it residing in core.

I think today it's had to make this appealing since (obviously) nobody uses it as it doesn't exist, and there are trivial ways for any given use case to do this in a differnet way which also means it will likely never reach the urgency required to have someone proactively work on this on the idea that it might become useful.

Alternatives:

  • Plant LocalSettings.php with a forwarder to wherever your file is. E.g. for MW_CONFIG_FILE=/etc/elsewhere/my.php to work you only need to place require getenv('ACMECORP_MW_CONFIGFILE')' in ones LocalSettings.php file. Or if it doesn't need to be runtime dynamic, a symlink could work as well (this is what WMF currently does).
  • Populate LocalSettings.php with specific assignments from wherever, e.g. Etcd or indeed Env variables. E.g. for ACMECORP_MW_DBSERVER=mydb to work, one only needs to set $wgDBserver = getenv('ACMECORP_MW_DBSERVER'); in LocalSettings.php. This is how WMF loads some of its DB configuration from Etcd today.
  • Override runtime settings from an extension hook. E.g. load row from mysql, or (automatically) map any number of settings to optionaly existant Env variables. This is how Fandom currently load some of their settings from a MySQL table on a per-wiki basis.

The logic of Etcd was turned into a Config subclass for easier use. I suppose something similar could be done in an extension for Env to be used/maintained by interested parties. All the neccecary bits for this are in place, I think?

My use case is to simplify the process for allowing developers to update their MediaWiki-Docker environments, whether it's quickly switching the DB backend between SQLite/MySQL/Postgres, or switching configuration needed for one of the recipes for extensions / services. I'd rather be able to provide a snippet for docker-compose.override.yml that provides a set of environment variables and be able to forego instructing users to add content to LocalSettings.php.

To avoid performance issues, could there be a feature flag that defaults to false, and which can be set by an environment variable, that then allows for setting any Config value via an environment variable?

Environment variables are strings while configuration settings are often arrays (sometimes deep arrays) so as a general mechanism env vars seem problematic.

One approach could be having a LocalSettings.d for config file fragments (Vagrant does something like that and it works quite well), feature-flagged or env-flagged for performance reasons. Would that help with the Docker use case or is providing files complicated even outside the repo directory?

Environment variables are strings while configuration settings are often arrays (sometimes deep arrays) so as a general mechanism env vars seem problematic.

We could JSON encode arrays when needed, perhaps?

One approach could be having a LocalSettings.d for config file fragments (Vagrant does something like that and it works quite well), feature-flagged or env-flagged for performance reasons. Would that help with the Docker use case or is providing files complicated even outside the repo directory?

A LocalSettings.d mechanism in core would indeed by nice (Vagrant and Quibble both implement their own, AIUI), but this request is a bit different.

We could JSON encode arrays when needed, perhaps?

That could work, it's not pretty though.

A LocalSettings.d mechanism in core would indeed by nice (Vagrant and Quibble both implement their own, AIUI), but this request is a bit different.

The proposed solution is different. Is the problem different, though?

As long as creating files in some predetermined location is not too complicated in Docker, that could be an alternative solution for the problem of settings overrides. There are trade-offs (it's going to be a lot more readable and easier to use for people with less OS skills; it's probably less secure if used for a live website; it's more static and having different settings for different tests or different wikis of a farm would be harder; multiple sources can provide configuration fragments without having to know of each other, which is hard to achieve with environment variables) but might be worth it.

As long as creating files in some predetermined location is not too complicated in Docker, that could be an alternative solution for the problem of settings overrides.

I guess I'm not sure exactly what this would look like in practice, could you please sketch it out for me when you have time?

My proposal is that developers could do something like this in their docker-compose.override.yml file:

version: '3.7'
services:
  mediawiki:
    environment:
      G_E_NEWCOMER_TASKS_GUIDANCE_ENABLED: true

Which would be the equivalent to the current instructions to put wgGENewcomerTasksGuidanceEnabled in LocalSettings.php.

I want to avoid asking non-technical users who want to run an extension locally from having to mess around with LocalSettings.php; the file is intimidating for one thing in that you have to navigate past a long block of code before you add your overrides, and then it's easy to lose track of what you've added / removed.

This seems like it would be easy to add as an extension (as long as it is just config and not loading other extensions. Datatypes other than strings seem mildly annoying, but nothing impossible). I'm not sure that i think it makes sense in core, or at least not until it has a proven track record of people using it.

Note that MW_CONFIG_FILE shipped in MW 1.38. Per the above, individual setups are free to assign variables in LocalSettings.php from any source, including environment variables.