Page MenuHomePhabricator

1.35.0-wmf.18 deployment blockers
Closed, ResolvedPublicRelease

Details

Release Version
1.35.0-wmf.18
Release Date
1580688000
Backup Conductor
dduvall

2020 week 06 1.35-wmf.18 Changes wmf/1.35.0-wmf.18

This MediaWiki Train Deployment is scheduled for the week of Monday, February 3rd:

Monday February 3rdTuesday, February 4thWednesday, February 5thThursday, February 6thFriday
SWAT deployments only.Branch wmf.18 and deploy to Group 0 Wikis.Deploy wmf.18 to Group 1 Wikis.Deploy wmf.18 to all Wikis.No deployments on fridays

How this works

  • Any serious bugs affecting wmf.18 should be added as subtasks beneath this one.
  • Any open subtask(s) block the train from moving forward.This means no further deployments until the blockers are resolved.
  • If something is serious enough to warrant a rollback then you should bring it to the attention of deployers on the #wikimedia-operations IRC channel.
  • For more info about deployment blockers, see Holding the train.

Related Links

Other Deployments

Previous: 1.35.0-wmf.17
Next: 1.35.0-wmf.19

Event Timeline

mmodell created this task.Sep 25 2019, 10:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 25 2019, 10:14 PM
thcipriani triaged this task as Medium priority.
thcipriani updated Backup Conductor, added: dduvall.

Mentioned in SAL (#wikimedia-operations) [2020-02-04T20:06:00Z] <marxarelli> temporarily holding 1.35.0-wmf.18 [T233866] branch cut and train due to concurrent maps prod issues

Mentioned in SAL (#wikimedia-operations) [2020-02-04T20:20:02Z] <twentyafterfour> branching mediawiki to wmf/1.35.0-wmf.18 from commit 054dd94e97d6 - train blockers should be added as subtasks under T233866

Change 570144 had a related patch set uploaded (by 20after4; owner: 20after4):
[operations/mediawiki-config@master] testwikis wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/570144

Change 570144 merged by jenkins-bot:
[operations/mediawiki-config@master] testwikis wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/570144

Mentioned in SAL (#wikimedia-operations) [2020-02-04T21:41:48Z] <twentyafterfour@deploy1001> Started scap: testwikis wikis to 1.35.0-wmf.18 refs T233866

Mentioned in SAL (#wikimedia-operations) [2020-02-04T22:13:51Z] <twentyafterfour@deploy1001> Finished scap: testwikis wikis to 1.35.0-wmf.18 refs T233866 (duration: 32m 03s)

Change 570157 had a related patch set uploaded (by 20after4; owner: 20after4):
[operations/mediawiki-config@master] group0 wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/570157

Change 570157 merged by jenkins-bot:
[operations/mediawiki-config@master] group0 wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/570157

Mentioned in SAL (#wikimedia-operations) [2020-02-04T22:35:16Z] <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.35.0-wmf.18 refs T233866

Train status: Unblocked. The train will move forward, at least until there is more evidence for production impact or other serious fallout from T244300.

  1. Tentatively removed as a blocker:
  2. Resolved (thank you!):

Mentioned in SAL (#wikimedia-operations) [2020-02-05T20:09:49Z] <twentyafterfour> Preparing to deploy wmf/1.35.0-wmf.18 to group1 wikis refs T233866

Change 570429 had a related patch set uploaded (by 20after4; owner: 20after4):
[operations/mediawiki-config@master] group1 wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/570429

Change 570429 merged by jenkins-bot:
[operations/mediawiki-config@master] group1 wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/570429

Mentioned in SAL (#wikimedia-operations) [2020-02-05T20:44:12Z] <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.18 refs T233866

Mentioned in SAL (#wikimedia-operations) [2020-02-05T20:45:19Z] <twentyafterfour@deploy1001> Synchronized php: group1 wikis to 1.35.0-wmf.18 refs T233866 (duration: 01m 07s)

Change 570710 had a related patch set uploaded (by 20after4; owner: 20after4):
[operations/mediawiki-config@master] all wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/570710

Change 570710 merged by jenkins-bot:
[operations/mediawiki-config@master] all wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/570710

Mentioned in SAL (#wikimedia-operations) [2020-02-06T20:25:24Z] <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.18 refs T233866

Change 570713 had a related patch set uploaded (by 20after4; owner: 20after4):
[operations/mediawiki-config@master] group2 wikis to 1.35.0-wmf.16 refs T233866

https://gerrit.wikimedia.org/r/570713

Change 570713 merged by 20after4:
[operations/mediawiki-config@master] group2 wikis to 1.35.0-wmf.16 refs T233866

https://gerrit.wikimedia.org/r/570713

Mentioned in SAL (#wikimedia-operations) [2020-02-06T21:40:33Z] <twentyafterfour> train blocked due to serious incident related to deploying the latest branch. Incident documentation: https://wikitech.wikimedia.org/wiki/Incident_documentation/20200206-mediawiki refs T233866

Window over so assuming it gets unblocked the train will resume on Monday.

mmodell changed the task status from Open to Stalled.Feb 6 2020, 10:45 PM
mmodell raised the priority of this task from Medium to Unbreak Now!.
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptFeb 6 2020, 10:45 PM

We need help with T240858. It appears like the train deleted a message on all group 1 wikis it should not have been deleted.

Xover added a subscriber: Xover.Feb 7 2020, 11:23 AM
Addshore added a subscriber: Addshore.

Mentioned in SAL (#wikimedia-operations) [2020-02-06T21:40:33Z] <twentyafterfour> train blocked due to serious incident related to deploying the latest branch. Incident documentation: https://wikitech.wikimedia.org/wiki/Incident_documentation/20200206-mediawiki refs T233866

We just tried deploying a config change for T244529 to fix an UBN, and that went wrong, and revealed that infact this config setting being wrong and the defaults changing in Wikibase for .18 is what is blocking the train.

Summary of the blocker at T244529#5859927

Mentioned in SAL (#wikimedia-operations) [2020-02-07T17:22:38Z] <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: roll back all wikis to 1.35.0-wmf.16 refs T233866

Mentioned in SAL (#wikimedia-operations) [2020-02-07T17:32:18Z] <twentyafterfour@deploy1001> Synchronized wmf-config/InitialiseSettings.php: sync https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/570929 refs T233866 (duration: 01m 02s)

Mentioned in SAL (#wikimedia-operations) [2020-02-07T17:36:13Z] <twentyafterfour@deploy1001> Synchronized wmf-config/InitialiseSettings.php: sync InitializeSettings again for lols refs T233866 (duration: 01m 03s)

If .18 restarts next week then the backport for .18 in T244529 is needed for sure and is a blocker.
If next week we roll .19 then T244529 is no longer a blocker, as the patch is already merged in master.

If .18 restarts next week then the backport for .18 in T244529 is needed for sure and is a blocker.
If next week we roll .19 then T244529 is no longer a blocker, as the patch is already merged in master.

I want to deploy the backport as Adam is in vacation. Let me know if you want to roll out wmf.18

Mentioned in SAL (#wikimedia-operations) [2020-02-10T14:40:10Z] <twentyafterfour> MediaWiki Train: Running a full scap to prepare for moving forward to 1.35.0-wmf.18 ( T233866 )

mmodell changed the task status from Stalled to Open.Feb 10 2020, 6:21 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-10T18:21:23Z] <twentyafterfour> MediaWiki train: finally moving forward with group0 wikis to 1.35.0-wmf.18 refs T233866

Change 571331 had a related patch set uploaded (by 20after4; owner: 20after4):
[operations/mediawiki-config@master] group1 wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/571331

Change 571331 merged by jenkins-bot:
[operations/mediawiki-config@master] group1 wikis to 1.35.0-wmf.18 refs T233866

https://gerrit.wikimedia.org/r/571331

Mentioned in SAL (#wikimedia-operations) [2020-02-10T18:38:04Z] <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.18 refs T233866

Mentioned in SAL (#wikimedia-operations) [2020-02-10T18:39:10Z] <twentyafterfour@deploy1001> Synchronized php: group1 wikis to 1.35.0-wmf.18 refs T233866 (duration: 01m 05s)

Mentioned in SAL (#wikimedia-operations) [2020-02-11T18:43:44Z] <twentyafterfour> getting ready to deploy wmf.18 refs T233866

Jdforrester-WMF closed this task as Resolved.Feb 11 2020, 7:25 PM

This is (finally) fully deployed. Declaring it done.

DannyS712 added a subscriber: DannyS712.

Assuming any remaining patch is no longer relevant at this point