Page MenuHomePhabricator

Eliminate the toil in WMF wiki creation
Closed, ResolvedPublic

Description

One part of WMF creation is the huge toil it has:

  • Keeping track all the parts, Is DNS deployed? Is CX config deployed? Are Wikimedia Messages in place?
  • Creating and keeping track of sometimes up to ten patches needed to be merged across the whole fleet (mediawiki core, pywikibot, DNS, RESTbase, etc.)
  • Commands needed to be executed to make sure the wiki is created without explosions (first run this and run that, do it in wrong order and all of Wikipedia will go down)

I wrote a code to actually handle this part and basically produces the checklist and the commands given the wiki creation ticket: https://github.com/Ladsgroup/Phabricator-maintenance-bot/blob/master/new_wikis_handler.py. You can see its output in description of T253029: Create Shan Wiktionary

This ticket is done:

  • If the checklist and the steps needed to be done is added to the ticket automatically.
  • And patches (at least most of them) are created automatically.
  • Subtickets creations and closing them as resolved also handled automatically.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 23 2020, 8:51 AM

For the checklist, perhaps a custom form for task creation?

Ladsgroup updated the task description. (Show Details)May 23 2020, 8:56 AM

For the checklist, perhaps a custom form for task creation?

The wiki creation has a pre-filled form generated from a template: https://meta.wikimedia.org/wiki/Requests_for_new_languages/Wiktionary_Konkani has a section called "Settings", Expand it and click on "submit phabricator task". It means it's already not free-form but we will get to the place to make it better. For now I think this is good enough.

jhsoby added a subscriber: jhsoby.May 25 2020, 4:23 PM

Now the bot automatically adds and updates the description, here's an example: https://phabricator.wikimedia.org/transactions/detail/PHID-XACT-TASK-7yfvmpcvr7dcd5l/ it checks the open tickets of wiki creation project once a day. Do not touch the automatic text as it will get overridden or worse, if you remove the ending or beginning, the bot appends the whole thing again.

Ladsgroup updated the task description. (Show Details)
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptMay 30 2020, 3:09 PM
awight added a subscriber: awight.Jun 23 2020, 8:19 AM

Change 623141 had a related patch set uploaded (by Ladsgroup; owner: Gerrit maintenance bot):
[analytics/refinery@master] Order entries by alphabetical order

https://gerrit.wikimedia.org/r/623141

Change 623143 had a related patch set uploaded (by Ladsgroup; owner: Gerrit maintenance bot):
[operations/dns@master] Order entries by alphabetical order

https://gerrit.wikimedia.org/r/623143

Change 623145 had a related patch set uploaded (by Gerrit maintenance bot; owner: Gerrit maintenance bot):
[mediawiki/services/cxserver@master] Order entries by alphabetical order

https://gerrit.wikimedia.org/r/623145

Change 623145 merged by jenkins-bot:
[mediawiki/services/cxserver@master] Order entries by alphabetical order

https://gerrit.wikimedia.org/r/623145

Change 623475 had a related patch set uploaded (by KartikMistry; owner: KartikMistry):
[operations/deployment-charts@master] Update cxserver to 2020-08-30-011854-production

https://gerrit.wikimedia.org/r/623475

Change 623143 merged by Dzahn:
[operations/dns@master] order langlist.tmpl entries alphabetically

https://gerrit.wikimedia.org/r/623143

Change 623141 merged by Mforns:
[analytics/refinery@master] Order entries by alphabetical order

https://gerrit.wikimedia.org/r/623141

Ladsgroup updated the task description. (Show Details)Sep 8 2020, 6:43 PM

Change 623475 merged by jenkins-bot:
[operations/deployment-charts@master] Update cxserver to 2020-08-30-011854-production

https://gerrit.wikimedia.org/r/623475

Mentioned in SAL (#wikimedia-operations) [2020-09-16T06:20:44Z] <kart_> Updated cxserver to 2020-08-30-011854-production (T253439, T260557)

Ladsgroup updated the task description. (Show Details)Oct 8 2020, 2:43 AM
Ladsgroup closed this task as Resolved.Oct 8 2020, 2:50 AM

So with some recent changes the bot also creates sub tickets for data storage, and parent tickets for RESTbase, pywikibot and wikidata (it doesn't automatically close them though). Also the bot creates patches automatically for anlaytics refinery, DNS, wikimedia messages and CX server. You can see the artwork of the bot in T264859: Create Inari Sámi Wikipedia. The only thing that's not done yet is automating initial config patch which I want to avoid as it's extremely complicated, we can pick it up once wiki configs are moved to yaml files (T223602: Define variant Wikimedia production config in compiled, static files). I create a follow up for that one.

There will be a long tail of things that can be automated but so far we reached an acceptable milestone to call this done.