Page MenuHomePhabricator

Create a list of all known wikis in MediaWikiFarm
Open, MediumPublic

Description

The complete list of managed wikis is not available for now in MediaWikiFarm. It should be created such a list by iterating over defined wikis in the 'variables' files and in the 'versions' file, farm by farm.

Although the basic thing is just iterating, it would be better to add, for each wiki, its complete ID, its suffix, and its URL. To achieve this, the URL regex should be “reversed”, I guess it is not too much complicated but some care must be taken.

A direct use (a specific task can be created for this) will be to update all wikis in one command (update or possibly another maintenance script, e.g. runJobs would be useful also). A second use (also a specific task to be created) will be the basis to create proper statistics about the farm (e.g. how much wikis, how much different MW versions, how much a config parameter is used, how much different values, etc.).

Event Timeline

Seb35 created this task.Apr 11 2017, 7:26 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 11 2017, 7:26 PM
Seb35 triaged this task as Medium priority.Apr 11 2017, 7:26 PM
Seb35 added a comment.Jul 13 2017, 9:27 AM

Other uses of such a list would be to define the virtual hosts in the webserver (see this method for nginx for instance) and in the DNS, in order to avoid either manual editing of these config files either catch-all regexes. This use case could also be delegated to an external configuration management tool (Ansible, etc.), to be decided if it is taken into account specifically by MediaWikiFarm.

Seb35 claimed this task.Aug 21 2017, 9:07 PM

I developed the code for this feature, but I need to tidy it a bit more before releasing. In preparation of this, I just pushed an improvement of the rules to define the wikis, because there was a possibility some wikis exist but were unregistered in MediaWikiFarm (“ghost wikis”), this new commit forbids this behaviour. Without it, the creation of the list of wikis could have missed some wikis.

Change 381236 had a related patch set uploaded (by Seb35; owner: Seb35):
[mediawiki/extensions/MediaWikiFarm@master] Enumerate all known wikis

https://gerrit.wikimedia.org/r/381236

Change 381236 merged by jenkins-bot:
[mediawiki/extensions/MediaWikiFarm@master] Enumerate all known wikis

https://gerrit.wikimedia.org/r/381236

Seb35 added a comment.Sep 28 2017, 5:27 PM

I let open this task because some other work should be done in this scope, mainly execute a bunch of wikis for MediaWiki scripts, I think about a command line such that:

mwscript --wiki=* runJobs
mwscript --wiki=*.example.org runJobs

For each individual wiki, a new PHP instance should be launched (at least for multiversion farms) because MediaWiki versions could be different between wikis and there would be a risk of conflicting versions (e.g. the class Job was already autoloaded but with a different version).

Change 411590 had a related patch set uploaded (by Seb35; owner: Seb35):
[mediawiki/extensions/MediaWikiFarm@master] Execute a script accross multiple wikis

https://gerrit.wikimedia.org/r/411590

Change 411590 merged by jenkins-bot:
[mediawiki/extensions/MediaWikiFarm@master] Execute a script accross multiple wikis

https://gerrit.wikimedia.org/r/411590

Seb35 added a comment.Feb 17 2018, 6:24 PM

It remains a part to fully solve this task: when the 'versions' file is used, these wikis are not in the list of wikis. This specific mechanism should be reversed-computed similarly to the way it was done for the 'variables' mechanism.