Page MenuHomePhabricator

Devise means for experimental software working with live data
Open, Needs TriagePublic

Description

This is a very far-fetched feature request, but Željko said that I should create it as a task anyway :)

In a very simplified description, the Wikimedia sites currently work in this way: Changes in core and extensions are merged in Gerrit. Whatever is merged is immediately deployed on the beta sites, and anything that gets in by Tuesday is scheduled for deployment on the live Wikipedia on Thursday. Changes in server configuration and services are deployed separately.

This is a fairly easy-to-handle schedule, but its disadvantage is that it doesn't let people who'd like to test the cutting-edge features do this with real data. Non-Wikipedia sites have data of different nature, and different configurations, too; that is, not the same extensions. The beta sites only have messy dumps of data from a point in the distant past, which are small, which are not maintained as the real data, and which don't have an actual community around them.

We have "Beta Features", but in practice they only enable particular features and are not built to handle the testing of the whole codebase combination of MediaWiki + extensions + configuration + services.

Therefore, it would be cool if the data was shared between the testing sites and the live sites. In such a setup, I, as an adventurous user who wants to help developers test the newest software, would use something like en.alpha.wikipedia.org instead of en.wikipedia.org, or I would enable some kind of an "alpha" option in my preferences on en.wikipedia.org, and then I'd get the latest MediaWiki + extensions + configuration + services code that was merged to the source repo, but I would otherwise operate with the same data - same articles, same talk pages, same preferences, etc.

This is comparable to Mozilla's "Stable" and "Nightly" releases: They use the same user profile, preferences, history and bookmarks, but run different software. Thanks to the shared history, preferences and bookmarks, an adventurous user can test the software as if it was almost the same, and if something is badly broken in the Nightly build, go back to the stable build. It's a great way to engage a diverse range of community members in testing that is both easy and productive.

Yes, it's very far-fetched. For starters, as of 2015, MediaWiki is not quite built to have two instances read the same database, and bugs in the cutting-edge code can corrupt live data. But nevertheless, it's an idea that would be great to give some thought to.

Event Timeline

Amire80 raised the priority of this task from to Needs Triage.
Amire80 updated the task description. (Show Details)
Amire80 subscribed.

Removing the CI project for now as this is really an architectural issue first and foremost (as Amir said).

This is also very similar (at least in spirit) to: T104398: Deploy MW+Extensions by percentage of users (instead of by domain/wiki)

I added the Services tag, and not entirely tongue-in-cheek.

A better frontend / backend separation (ex: T111588) should enable testing of many user-visible changes using live data. There will be instances where a new feature requires backend changes, but over time I would expect this to become less common.

Previewing stateful backends on live production data is a lot harder. This is more likely to happen as canaries & rolling deploys, after fairly thorough testing in a staging environment explicitly not sharing mutable production data.

Krinkle renamed this task from Set up Wikimedia sites in a way that will allow testing experimental software with live data to Devise means for experimental software working with live data.Mar 14 2018, 8:50 PM

It is definitely not possible to feed live production data to beta, for various reasons, including legal ones.

Having said that, I do see a value in having an environment where changes can be staged programmatically and tested with some replica of live data, and plans for building a staging cluster are being discussed for the next FY annual plan as far as I know.

Have we considered importing a dump of several wikis? That may not cover all the cases and is not the same as live data but it should cover legal issues and should be good enough for most testing.