1.40.0-wmf.27 deployment blockers
Closed, ResolvedPublic5 Estimated Story PointsRelease
Actions

Details

Backup Train Conductor: jeena
Release Version: 1.40.0-wmf.27
Release Date: Mar 13 2023, 12:00 AM

2023 week 11 1.40-wmf.27 Changes wmf/1.40.0-wmf.27

This MediaWiki Train Deployment is scheduled for the week of Monday, March 13th:

Monday March 13th	Tuesday, March 14th	Wednesday, March 15th	Thursday, March 16th	Friday
Backports only.	Branch `wmf.27` and deploy to Group 0 Wikis.	Deploy `wmf.27` to Group 1 Wikis.	Deploy `wmf.27` to all Wikis.	No deployments on fridays

See https://wikitech.wikimedia.org/wiki/Deployments for full schedule.

How this works

Any serious bugs affecting wmf.27 should be added as subtasks beneath this one.
- Use this form to create one.
Any open subtask(s) block the train from moving forward. This means no further deployments until the blockers are resolved.
If something is serious enough to warrant a rollback then you should bring it to the attention of deployers on the #wikimedia-operations IRC channel.
If you have a risky change in this week's train add a comment to this task using the Risky patch template
For more info about deployment blockers, see Holding the train.

Other Deployments

Previous: 1.40.0-wmf.26

Next: 1.41.0-wmf.1

Subject	Repo	Branch	Lines +/-
all wikis to 1.40.0-wmf.27	operations/mediawiki-config	master	+327 -327
rsyslog: move spam filter into ruleset	operations/puppet	production	+3 -3
rsyslog: drop rdbms log spam pre-kafka	operations/puppet	production	+3 -0
logstash: add rdbms log spam filter	operations/puppet	production	+4 -0
group1 wikis to 1.40.0-wmf.27	operations/mediawiki-config	master	+542 -542
action: Restrict action.delete.js to action=delete pages	mediawiki/core	wmf/1.40.0-wmf.27	+5 -1
group0 wikis to 1.40.0-wmf.27	operations/mediawiki-config	master	+128 -128
testwikis wikis to 1.40.0-wmf.27	operations/mediawiki-config	master	+4 -4
Branch commit for wmf/1.40.0-wmf.27	mediawiki/core	wmf/1.40.0-wmf.27	+986 -3

Customize query in gerrit

Related Objects

Mentioned In: T332228: High log volume from rdbms lib triggering logstash throttler
T321160: Lcobucci\JWT\Signer\InvalidKeyProvided: Key cannot be empty
T332217: Lcobucci\JWT\Signer\InvalidKeyProvided: Key cannot be empty
Mentioned Here: T321160: Lcobucci\JWT\Signer\InvalidKeyProvided: Key cannot be empty

Event Timeline

thcipriani created this task.Feb 21 2023, 5:50 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 21 2023, 5:50 PM

thcipriani assigned this task to brennen.Feb 21 2023, 5:56 PM

thcipriani triaged this task as Medium priority.

thcipriani updated Other Assignee, added: jeena.

thcipriani set the point value for this task to 5.

thcipriani added a project: Release-Engineering-Team (Priority Backlog 📥).

Change 897929 had a related patch set uploaded (by TrainBranchBot; author: trainbranchbot):

[mediawiki/core@wmf/1.40.0-wmf.27] Branch commit for wmf/1.40.0-wmf.27

https://gerrit.wikimedia.org/r/897929

gerritbot added a project: Patch-For-Review.Mar 14 2023, 2:07 AM

Change 897929 merged by jenkins-bot:

[mediawiki/core@wmf/1.40.0-wmf.27] Branch commit for wmf/1.40.0-wmf.27

https://gerrit.wikimedia.org/r/897929

Change 898245 had a related patch set uploaded (by TrainBranchBot; author: MediaWiki PreSync):

[operations/mediawiki-config@master] testwikis wikis to 1.40.0-wmf.27

https://gerrit.wikimedia.org/r/898245

Change 898245 merged by jenkins-bot:

[operations/mediawiki-config@master] testwikis wikis to 1.40.0-wmf.27

https://gerrit.wikimedia.org/r/898245

Mentioned in SAL (#wikimedia-operations) [2023-03-14T03:02:27Z] <mwpresync@deploy2002> Started scap: testwikis wikis to 1.40.0-wmf.27 refs T330205

Mentioned in SAL (#wikimedia-operations) [2023-03-14T03:53:30Z] <mwpresync@deploy2002> Finished scap: testwikis wikis to 1.40.0-wmf.27 refs T330205 (duration: 51m 02s)

brennen changed the task status from Open to In Progress.Mar 14 2023, 6:02 PM

brennen added a project: User-brennen.

Mentioned in SAL (#wikimedia-operations) [2023-03-14T18:03:44Z] <brennen> 1.40.0-wmf.27 train (T330205): no current blockers, rolling to group0.

Change 898818 had a related patch set uploaded (by TrainBranchBot; author: Brennen Bearnes):

[operations/mediawiki-config@master] group0 wikis to 1.40.0-wmf.27

https://gerrit.wikimedia.org/r/898818

Change 898818 merged by jenkins-bot:

[operations/mediawiki-config@master] group0 wikis to 1.40.0-wmf.27

https://gerrit.wikimedia.org/r/898818

Mentioned in SAL (#wikimedia-operations) [2023-03-14T18:13:48Z] <brennen@deploy2002> rebuilt and synchronized wikiversions files: group0 wikis to 1.40.0-wmf.27 refs T330205

Mentioned in SAL (#wikimedia-operations) [2023-03-14T19:30:40Z] <brennen> 1.40.0-wmf.27 train (T330205): uneventful at group0. i'm afk for about an hour.

Change 898867 had a related patch set uploaded (by Umherirrender; author: Umherirrender):

[mediawiki/core@wmf/1.40.0-wmf.27] action: Restrict action.delete.js to action=delete pages

https://gerrit.wikimedia.org/r/898867

A patch set included only in wmf.27 can result in javascript error on Special:MovePage, please deploy https://gerrit.wikimedia.org/r/c/mediawiki/core/+/898867 as a fix for that. Thanks.

Change 898867 merged by jenkins-bot:

[mediawiki/core@wmf/1.40.0-wmf.27] action: Restrict action.delete.js to action=delete pages

https://gerrit.wikimedia.org/r/898867

Mentioned in SAL (#wikimedia-operations) [2023-03-14T23:19:22Z] <brennen@deploy2002> Started scap: Backport for [[gerrit:898867|action: Restrict action.delete.js to action=delete pages (T330205)]]

Mentioned in SAL (#wikimedia-operations) [2023-03-14T23:20:55Z] <brennen@deploy2002> brennen and umherirrender: Backport for [[gerrit:898867|action: Restrict action.delete.js to action=delete pages (T330205)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-03-14T23:29:55Z] <brennen@deploy2002> Finished scap: Backport for [[gerrit:898867|action: Restrict action.delete.js to action=delete pages (T330205)]] (duration: 10m 32s)

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.27; 2023-03-13).Mar 15 2023, 12:01 AM

Mentioned in SAL (#wikimedia-operations) [2023-03-15T18:06:01Z] <brennen> 1.40.0-wmf.27 train (T330205): no current blockers, rolling to group1.

Change 899709 had a related patch set uploaded (by TrainBranchBot; author: Brennen Bearnes):

[operations/mediawiki-config@master] group1 wikis to 1.40.0-wmf.27

https://gerrit.wikimedia.org/r/899709

Change 899709 merged by jenkins-bot:

[operations/mediawiki-config@master] group1 wikis to 1.40.0-wmf.27

https://gerrit.wikimedia.org/r/899709

Mentioned in SAL (#wikimedia-operations) [2023-03-15T18:18:46Z] <brennen@deploy2002> rebuilt and synchronized wikiversions files: group1 wikis to 1.40.0-wmf.27 refs T330205

Mentioned in SAL (#wikimedia-operations) [2023-03-15T18:24:55Z] <brennen@deploy2002> Synchronized php: group1 wikis to 1.40.0-wmf.27 refs T330205 (duration: 06m 08s)

Logstash is experiencing a high rate of Wikimedia\Rdbms\Platform\SQLPlatform::isWriteQuery fallback to regex.

I will put in a spam filter to mitigate lag.

Change 898915 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/puppet@production] logstash: add rdbms log spam filter

https://gerrit.wikimedia.org/r/898915

Change 898915 merged by Cwhite:

[operations/puppet@production] logstash: add rdbms log spam filter

https://gerrit.wikimedia.org/r/898915

brennen mentioned this in T332217: Lcobucci\JWT\Signer\InvalidKeyProvided: Key cannot be empty.Mar 15 2023, 7:45 PM

Change 898916 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/puppet@production] rsyslog: drop rdbms log spam pre-kafka

https://gerrit.wikimedia.org/r/898916

brennen mentioned this in T321160: Lcobucci\JWT\Signer\InvalidKeyProvided: Key cannot be empty.Mar 15 2023, 7:56 PM

Change 898916 merged by Cwhite:

[operations/puppet@production] rsyslog: drop rdbms log spam pre-kafka

https://gerrit.wikimedia.org/r/898916

Change 898917 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/puppet@production] rsyslog: move spam filter into ruleset

https://gerrit.wikimedia.org/r/898917

Change 898917 merged by Cwhite:

[operations/puppet@production] rsyslog: move spam filter into ruleset

https://gerrit.wikimedia.org/r/898917

I will be out of office Thu, 16 Mar–Tue, 21 Mar

For 🚨 Emergency deploys, @brennen will be the point of contact (for the remainder of this week)

colewhite mentioned this in T332228: High log volume from rdbms lib triggering logstash throttler.Mar 15 2023, 9:03 PM

jeena added a subtask: T321160: Lcobucci\JWT\Signer\InvalidKeyProvided: Key cannot be empty.Mar 15 2023, 9:33 PM

End-of-workday status: Not rolling back for T321160, but holding at group1.

Mentioned in SAL (#wikimedia-operations) [2023-03-16T20:36:49Z] <brennen> 1.40.0-wmf.27 train (T330205): blockers hopefully resolved, rolling to all wikis

Change 900444 had a related patch set uploaded (by TrainBranchBot; author: Brennen Bearnes):

[operations/mediawiki-config@master] all wikis to 1.40.0-wmf.27

https://gerrit.wikimedia.org/r/900444

Change 900444 merged by jenkins-bot:

[operations/mediawiki-config@master] all wikis to 1.40.0-wmf.27

https://gerrit.wikimedia.org/r/900444

Mentioned in SAL (#wikimedia-operations) [2023-03-16T20:47:49Z] <brennen@deploy2002> rebuilt and synchronized wikiversions files: all wikis to 1.40.0-wmf.27 refs T330205

brennen removed a subtask: T321160: Lcobucci\JWT\Signer\InvalidKeyProvided: Key cannot be empty.Mar 16 2023, 8:54 PM

brennen closed this task as Resolved.Mar 16 2023, 10:32 PM

brennen moved this task from Backlog to Done or Declined on the User-brennen board.May 8 2023, 11:35 PM

1.40.0-wmf.27 deployment blockersClosed, ResolvedPublic5 Estimated Story PointsReleaseActions