Page MenuHomePhabricator

Beta: submodule update reverts new portals commits
Closed, ResolvedPublic

Description

11:36 <@JGirault> Hi there
11:38 <@JGirault> Yesterday I wanted to update beta portals http://www.wikipedia.beta.wmflabs.org/, so did I (git pull in portals folder, and then sync-dir portals), and the page was up. Today the old page is back though.
11:39 <@JGirault> not sure what caused it, but portals is a submodule, and I’m thinking maybe a git submodule update reverted my stuff
11:40 <@JGirault> how should I do, to make sure what I deploy to portals beta stays in beta?

11:53 <@thcipriani> jgirault: lemme look at how beta updates portals, it's likely it's an automated deploy
11:55 <@JGirault> thcipriani: so far it is manual, but we want to automate it eventually https://phabricator.wikimedia.org/T124848
11:57 <@thcipriani> jgirault: ah, I see. I'll bet you're right that beta-mediawiki-config-update overwrote the change...looking at that job now
11:59 <@thcipriani> yeah, I think that is what did it: https://github.com/wikimedia/operations-puppet/blob/production/modules/beta/templates/wmf-beta-mwconfig-update.erb
12:00 <@JGirault> thcipriani: yeah, looks like it. what do you recommend?

12:05 <@thcipriani> jgirault: that's tough. It seems like this is a little at odds with automated deployment of mediawiki config submodules
12:05 <@thcipriani> I think this patch is what caused the current behavior: https://github.com/wikimedia/operations-puppet/commit/f4baf22e8622c40fed66c49ab38372ddb9af2a13
12:06 <@thcipriani> what we could do is just change it to a submodule update --rebase

12:10 <@JGirault> thcipriani: ok I see.. do you see any issue? I don’t want portals to mess everything else
12:14 <@thcipriani> jgirault: submodule update --rebase shouldn't cause too much of an issue (pretty sure it's the default behavior for core extensions). Does that solve your issue though? you have changes on top of the current commit pointed to by the submodule that you'd like to persist?
12:18 <@JGirault> thcipriani: yes. the submodule points to master in production. and we’d like to run git pull to test new commits on beta before we update the references and deploy to production
12:20 <@thcipriani> jgirault: kk, I think that should work fine for others as well. Mind creating a ticket? I can add a pull request and get a few more folks to check my thinking.

Details

Related Changes in Gerrit:

Event Timeline

JGirault assigned this task to thcipriani.
JGirault raised the priority of this task from to Needs Triage.
JGirault updated the task description. (Show Details)
JGirault added subscribers: JGirault, thcipriani.

Change 268737 had a related patch set uploaded (by Thcipriani):
Beta: Rebase mw-config submodules

https://gerrit.wikimedia.org/r/268737

@thcipriani: This is causing a lot of frustration for our team. Do you have an idea of when the patch will take effect? Thanks!

Change 268737 merged by Dzahn:
Beta: Rebase mw-config submodules

https://gerrit.wikimedia.org/r/268737

I am not sure why beta cluster is being used for live hack. The portal development should really happen at a different place with beta cluster being the staging phase before that lands to production.

For all the rest of repositories we do not ever allow live hacks. Changes get merged first, auto deployed then can be looked at.

Now that I have ranted, indeed using submodule --rebase would keep the local commits around. Note they might disappear at any time though so make sure your hack are always committed and proposed in Gerrit.

@hashar: I'm a bit out of the technical loop, but my understanding is that the portal folks want to use beta exactly as you have outlined. I suspect the problem is that the portal itself is deployed from a completely separate repo from the rest of the site. (Until recently, it was deployed from a wiki page, not a version controlled repo).

The goal is not to do live hacks on the beta portal. The goal is to be able to deploy portal code to beta (from gerrit), and then to test it for however long is necessary without it being overwritten by unrelated deployments. Once the tests are complete, exactly that portal code would get pushed to production.

Of course I could be wrong about some or all of that, but that's my understanding.

Hi @hashar - @ksmith is correct. We want to be able to deploy the code to beta and test it before we launch it into production. This ticket was written so that we can have the ability to test without the beta portal being overwritten by some other deployment at unexpected times.

Thanks @ksmith and @debt. Looks that is a pragmatic compromise. In the first place I would really have preferred the wikiportals to NOT be deployed via operations/mediawiki-config, but I am not going to reopen that heated can of worms :-}

So cherry pick , magic rebase and it is all fine \O/