Page MenuHomePhabricator

Allow to test a mediawiki-config change to the beta cluster
Open, LowPublic

Description

One of the early beta cluster goal was to allow to test MediaWiki changes prior to production.

Currently, if one writes a patch for operations/mediawiki-config, it must be merged to master to test it on the beta cluster.

We need a process to allow people to test their config changes at writing time on a part of this cluster. If the process could be automated, that would be awesome, but we at least need to manually allow that.

That would have a real impact to reduce prod breakage due to changes to CommonSettings.php.

Event Timeline

For example someone writes a change at gerrit, and we give some people access to git, that thex can fetch it, and checkout master again, later?

But this has a security impact: For example you can reactivate the checkuser-extension, and view private data, by assigning rights to *.

You need access to the deployment-prep labs project and then you can deploy config changes to beta to test them there. That's easy to give out.

Any config change or only stuff already merged on wmf/master branches?

As Krenair said, quick cherry-picks/test changes on Beta Cluster are easy/straight-forward and it's simple to make sure people have the right access.

Any config change or only stuff already merged on wmf/master branches?

Just for context: There's a slight difference between MW+Ext and rOMWC (mw-config) in how code developed and deployed to production is managed.

For MW+Ext it's usually:

  1. Develop against master, which is deployed to Beta within 10 minutes of merge
  2. RelEng cuts a branch from master each week to deploy to production (when we migrate to long-lived branches (T89945) it's still a similar workflow, just semantics change)

That can be put into words as: Develop against what the current production environment (services/mw-config) looks like.

For rOMWC it's:

For Beta Cluster only changes:

  • Develop (on master) using the -labs.php versions of relevant rOMWC files that take effect in Beta Cluster (and let Beta Cluster auto-update)

For Production changes:

  • Write new patch (sometimes, but not always, based on the changes you tested in Beta Cluster) and deploy from master immediately after merge (we actually have an icinga check for when what's in master and what's on tin/mira is out of sync)

So, options:

  1. make rOMWC act more like MW+Ext (develop on master, master on Beta Cluster, create new deploy branches, then deploy new branches to production)
  2. Don't change anything technically but have deployers wait ~10 after each merge into rOMWC (ie: let Beta catch up) before deploying to production
    1. I don't like this one, but it's the right direction (having Beta or something test post merge automatically before push to production)
  3. something else...
hashar claimed this task.

Per @greg explanation. Either cherry pick on deployment-tin and sync (note the patch will get overwritten automatically by Jenkins) or add a hack in one of the -labs.php files.

Dereckson reopened this task as Open.EditedSep 12 2016, 6:48 PM
Dereckson added a subscriber: Urbanecm.

I'm reopening the bug following a discussion with @Urbanecm.

Martin would like to be able to test patches before merge time, to detect pure rOMWC issues like pixellisation logos, missing part of the config.

The best scenario for that would be:

  1. Upload to Gerrit a patch
  2. On Beta Cluster, have a server with a staging area where we can cherry pick a commit not yet merged but waiting on Gerrit, that would apparently be deployment-tin, and ask a server to pull it from there, which is already possible.

I'm not sure that be a comprehensive solution, as we don't have every project mirrored and so specific wiki configuration would be tricky to test.

hashar triaged this task as Low priority.

Bringing this back, I'd like to see the possibility of cherry-pick on one target host, instead of the entire cluster getting the impact. Example: a couple days ago i really wanted to be able to cherry-pick a change to MW on one instance to check something on my snapshot01 instance, without polluting all of the other mediawiki instances. Can we figure out a slick way to do this?