Page MenuHomePhabricator

Create ability to trivially spin up MediaWiki instance of a particular patch/diff
Open, LowestPublic

Description

This may be a very silly idea (or a duplicate task), but here's my use-case: occasionally I want to be able to see the effects of a particular Gerrit changeset on a public MediaWiki installation. This is particularly true of user interface changes, but can also be true of other changes.

It'd be nice if there were a one-click button somewhere that I could press to spin up a MediaWiki instance based on a particular Gerrit changeset. For the purposes of this task, let's assume only MediaWiki core, no extensions or other complexities.

Is this doable?

Event Timeline

MZMcBride raised the priority of this task from to Needs Triage.
MZMcBride updated the task description. (Show Details)
MZMcBride changed Security from none to None.
MZMcBride added subscribers: MZMcBride, Krenair, matmarex.

This came up most recently in the context of looking at https://gerrit.wikimedia.org/r/175394, as an example.

fbstj awarded a token.Nov 28 2014, 8:43 PM
hashar closed this task as Declined.EditedNov 28 2014, 9:42 PM
hashar claimed this task.

https://www.mediawiki.org/wiki/MediaWiki-Vagrant is the most trivial way to achieve this. It is a VM which has all the needed repositories / roles already setup and let you easily fetch a patch to test it out.

In theory we could have a service that spin up a MediaWiki-Vagrant image on labs and apply whatever patchset, but I don't think it is worth the engineering effort. Much cheaper to have tester load the patch on their own Vagrant :-)

Edit: Beta Cluster is out of scope, it is not mean to test in progress patches but as a final staging area before code lands to production.

MZMcBride reopened this task as Open.Nov 28 2014, 9:47 PM

You're free to say "I don't want to do this," but you're not free to say "this shouldn't be done." Not so hastily and dismissively, anyway. Re-opening.

In theory we could have a service that spin up a MediaWiki-Vagrant image on labs and apply whatever patchset, but I don't think it is worth the engineering effort. Much cheaper to have tester load the patch on their own Vagrant :-)

It's cheaper to have individuals spin up their own instances rather than having a shared instance? I don't follow.

We have quite a few test wikis and a beta cluster, so I don't think we need to reëstablish the virtue of having public test instances of MediaWiki in this task. That seems to be undisputed.

The question here is whether we can make it even easier to spin up a test MediaWiki instance so that we can quickly and easily demonstrate functionality of a particular proposed code change. I think it's at least an idea worth considering.

hashar added a comment.EditedNov 28 2014, 9:56 PM

You're free to say "I don't want to do this," but you're not free to say "this shouldn't be done." Not so hastily and dismissively, anyway. Re-opening.

Sorry :-(

<snip>
It's cheaper to have individuals spin up their own instances rather than having a shared instance? I don't follow.

Yeah I meant on a local machine, a user can spin up MediaWiki Vagrant and quite easily apply a Gerrit change to it for local testing :-)

For a shared environment, we could end up with some kind of wikifarm that would install a dummy MediaWiki, apply the patch, install it and make the result available somewhere (like: alpha.wmflabs.org/12345,42/index.php ). That would need to write a tool / job to spin up such install and garbage collect the install once the patch is merged in. It is probably easy to do with just core + vendor + some skin.

Meanwhile, it seems to me that MediaWiki Vagrant fills the gap quite nicely and is already available. Hence cheaper in term of engineering/developer time.

EDIT: you might want to bring up the subject on wikitech-l and gather interested folks. Then write a small RFC we can implement :]

scfc added a subscriber: scfc.Nov 28 2014, 10:24 PM

You can create MediaWiki instances reasonably easy with Labs-Vagrant. This isn't one-click nor particularly resource-efficient, but it works. Also, you get control of the complete environment, so in your testing you are never blocked by waiting for an admin to do this or that.

You're free to say "I don't want to do this," but you're not free to say "this shouldn't be done." Not so hastily and dismissively, anyway. Re-opening.

Sorry :-(

No worries. I've been trying to push back against very quick Phabricator task closures. We want to be welcoming to ideas and discussion!

For a shared environment, we could end up with some kind of wikifarm that would install a dummy MediaWiki, apply the patch, install it and make the result available somewhere (like: alpha.wmflabs.org/12345,42/index.php ). That would need to write a tool / job to spin up such install and garbage collect the install once the patch is merged in. It is probably easy to do with just core + vendor + some skin.

I think there's a lot of value in sometimes having a shared environment where everyone can see exactly what we're talking about (and do manual browser testing!).

Meanwhile, it seems to me that MediaWiki Vagrant fills the gap quite nicely and is already available. Hence cheaper in term of engineering/developer time.

I'm trying to glue the pieces together. It sounds like maybe we could:

  • have a dedicated Labs provisioned instance(?) such as alpha.wmflabs.org
  • create some kind of wrapper Python or shell script that can:
    • clone MediaWiki-Vagrant and/or Labs-Vagrant
    • clone and apply the relevant Gerrit changeset
    • do some very simple Web server configuration to expose the MediaWiki instance to the public Internet

Would that work? Is that crazy?

EDIT: you might want to bring up the subject on wikitech-l and gather interested folks. Then write a small RFC we can implement :]

Yeah, probably. But I'm already severely in RFC debt. :-(

hashar triaged this task as Lowest priority.Dec 1 2014, 9:30 AM

Is that crazy?

Yeah that is crazy, so it is probably something we should do :-}

greg added a subscriber: greg.

I know some places that have a set number of "qa instances" that people can "check out" (like a book from a library) and have a specific tag/revision be deployed there. Something like that could be worthwhile here, maybe?

But that is probably better for the Labs-Vagrant project instead of Beta Cluster (as BC is solely about master).

bd808 added a project: MediaWiki-Vagrant.EditedDec 1 2014, 9:46 PM
bd808 added a subscriber: bd808.

This kind of testing is exactly what I expect developers to be doing (and I do myself) in a local MediaWiki-Vagrant instance using git-review to pull the appropriate changes. When you need to share what you are seeing with others, you can use vagrant share or test in a Labs-Vagrant managed instance (like http://sulfinalization.wmflabs.org/).

The complexity of having a pool of shared servers that can be reserved for doing this kind of thing seems like a lot of machinery to keep working. I personally think that keeping beta running is hard enough without adding a "sort of beta but not really" environment that has shared ownership.

greg added a comment.Dec 1 2014, 9:54 PM

The complexity of having a pool of shared servers that can be reserved for doing this kind of thing seems like a lot of machinery to keep working. I personally think that keeping beta running is hard enough without adding a "sort of beta but not really" environment that has shared ownership.

+1000 *If* this is ever on the Release-Engineering-Team list of things to do, it's below maintenance of Beta Cluster.

So, I've thought about this a lot - essentially a 'Tool Labs' type setup for Mediawiki test setups.

Most promising way to do this efficiently would be to have a cluster of machines with a shared:

  1. Database server
  2. File storage for uploads
  3. Redis/Memcached for caching (with prefixing)

And then run the code itself in a docker/LXC environment. This would allow us to setup a simple web interface that lets people pick vagrant roles they want, and provision a docker environment that runs that set of things. Since all 'state' is kept outside the docker environment, it can be restarted / re-created whenever we want. Providing shell access to the image would also not be too hard, and applying a gerrit change to it would also be fairly easy.

Now, the question is a simple matter of 'is this worth it?'. This would essentially replace the many labs-vagrant instances we have. Considering the lack of manpower available for betacluster issues themselves, the ease-of-use of vagrant share / labs-vagrant, I do not think we'll get to this anytime soon. If there's some volunteer interested in getting this done, all is well! But I do not believe WMF engineering/ops will prioritize this anytime :)

hashar removed hashar as the assignee of this task.Apr 15 2015, 12:27 PM

Unassigning myself since that is merely a crazy idea, feature request for the backlog.

Florian added a subscriber: Florian.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 24 2015, 6:29 AM
greg renamed this task from Create ability to trivially spin up MediaWiki instance of a particular Gerrit changeset to Create ability to trivially spin up MediaWiki instance of a particular patch/diff.Nov 4 2015, 10:42 PM

FWIW I think this is an excellent idea and I would actually like to see us do something like this, especially along the lines of what @yuvipanda described here.

If we were to have differential run some tests on a virtual machine, then we could simply let that machine linger for a while, eventually recycled if it goes unused. Then along with unit test results, the differential build could link to the live instance with that code applied and running. If a test fails you could log in and check out the problem. When someone comes along to review your revision they will have instant access to see the effects of your change without any extra effort. I think it would significantly streamline the code review process, at least for changes that are front-end-visible (that is, when the change does not require manual back-end testing)

scfc added a comment.Nov 5 2015, 4:14 PM

There are about a hundred open changes in Gerrit for mediawiki/core that have been updated in the last week. Could anybody (with an LDAP account) log into any such machine and fiddle with it? How do people simultaneously logged in not interfere with each other's tests?

@scfc: presumably the author of the patch and any reviewers would be able to log in. I'm not sure if it would be appropriate to allow anyone at all to log in without having participated in the review somehow. Anyone could access the instance via http(s) but not necessarily ssh.

The situation currently, on the beta cluster we have many things happening that could interfere with eachother, so a bit of isolation would be an improvement.

hashar removed a subscriber: hashar.Nov 9 2015, 11:41 AM
Restricted Application added a subscriber: Luke081515. · View Herald TranscriptMay 13 2016, 9:17 PM
dbarratt added a subscriber: dbarratt.EditedMar 26 2018, 10:17 PM

Drupal has a tool kind of like this.. https://simplytest.me
you can specify "Drupal core" and under advanced options you can apply a patch or add an extension.

The instance only remains active for 30 minutes (and shows a timer).