1.35.0-wmf.31 deployment blockers
Closed, ResolvedPublicRelease
Actions

Details

Backup Train Conductor: hashar
Release Version: 1.35.0-wmf.31
Release Date: May 4 2020, 12:00 AM

2020 week 19 1.35-wmf.31 Changes wmf/1.35.0-wmf.31

This MediaWiki Train Deployment is scheduled for the week of Monday, May 4th:

Monday May 4th	Tuesday, May 5th	Wednesday, May 6th	Thursday, May 7th	Friday
Backports only.	Branch `wmf.31` and deploy to Group 0 Wikis.	Deploy `wmf.31` to Group 1 Wikis.	Deploy `wmf.31` to all Wikis.	No deployments on fridays

See https://wikitech.wikimedia.org/wiki/Deployments for full schedule.

How this works

Any serious bugs affecting wmf.31 should be added as subtasks beneath this one.
- Use this form to create one.
Any open subtask(s) block the train from moving forward. This means no further deployments until the blockers are resolved.
If something is serious enough to warrant a rollback then you should bring it to the attention of deployers on the #wikimedia-operations IRC channel.
If you have a risky change in this week's train add a comment to this task using the Risky patch template
For more info about deployment blockers, see Holding the train.

Other Deployments

Previous: 1.35.0-wmf.30

Next: 1.35.0-wmf.32

Subject	Repo	Branch	Lines +/-
All wikis except test to 1.35.0-wmf.30	operations/mediawiki-config	master	+941 -941
Revert "all wikis to 1.35.0-wmf.31"	operations/mediawiki-config	master	+304 -304
group1 wikis to 1.35.0-wmf.31 (2)	operations/mediawiki-config	master	+510 -510

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved	Release	brennen	T249963 1.35.0-wmf.31 deployment blockers
Resolved		DannyS712	T251386 Special:RecentChanges broken: MutableRevisionRecord: Bad value for parameter $id: must be a integer
Resolved	PRODUCTION ERROR	DannyS712	T251952 CoreParserFunctions::revisionuser: Call to a member function getUser() on boolean
Resolved	PRODUCTION ERROR	DannyS712	T251950 MutableRevisionRecord: Bad value for parameter $visibility: must be a integer
Resolved	PRODUCTION ERROR	DannyS712	T252043 Special:DeletedContributions: $row does not contain fields needed for comment rev_comment
Resolved		• mmodell	T252052 Fix Undefined property: stdClass::$ar_id from Revision/RevisionArchiveRecord.php line 74
Resolved	PRODUCTION ERROR	DannyS712	T252072 Special:RevisionReview: PHP Unknown error: Object of class MediaWiki\Revision\RevisionStoreRecord could not be converted to string
Resolved		DannyS712	T252179 Edits saved via PageUpdater need autopatrol status set

Event Timeline

thcipriani created this task.Apr 10 2020, 10:29 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 10 2020, 10:29 PM

thcipriani assigned this task to brennen.Apr 10 2020, 10:30 PM

thcipriani triaged this task as Medium priority.

thcipriani updated Other Assignee, added: hashar.

thcipriani added a project: Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)).

thcipriani moved this task from INBOX to Deployment on the Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)) board.

brennen added a project: User-brennen.Apr 13 2020, 4:21 PM

Nikerabbit added a subtask: T251386: Special:RecentChanges broken: MutableRevisionRecord: Bad value for parameter $id: must be a integer.Apr 29 2020, 10:36 AM

DannyS712 closed subtask T251386: Special:RecentChanges broken: MutableRevisionRecord: Bad value for parameter $id: must be a integer as Resolved.Apr 29 2020, 2:03 PM

brennen moved this task from Backlog to Next on the User-brennen board.May 4 2020, 7:17 PM

Mentioned in SAL (#wikimedia-operations) [2020-05-05T16:18:53Z] <brennen> notice: planning branch cut for 1.35.0-wmf.31 (T249963) at 16:30 UTC

Mentioned in SAL (#wikimedia-operations) [2020-05-05T16:34:37Z] <brennen> triggering branch cut for 1.35.0-wmf.31 (T249963) via https://releases-jenkins.wikimedia.org/job/MediaWiki%20Train%20Branch%20Cut/build?delay=0sec

Mentioned in SAL (#wikimedia-operations) [2020-05-05T16:48:09Z] <brennen> 1.35.0-wmf.31 was branched at 4d3fed31a435e7bd24925a154f89a9407670986d for T249963

Mentioned in SAL (#wikimedia-operations) [2020-05-05T19:02:27Z] <brennen> train status: 1.35.0-wmf.31: presently pressing enter through scap-cdb-rebuild; at 8% (T249963, T223287)

Stashbot mentioned this in T223287: Investigate scap cluster_ssh idling until pressing ENTER repeatedly.May 5 2020, 7:02 PM

brennen mentioned this in T251952: CoreParserFunctions::revisionuser: Call to a member function getUser() on boolean.May 5 2020, 8:59 PM

Are T251952: CoreParserFunctions::revisionuser: Call to a member function getUser() on boolean and T251950: MutableRevisionRecord: Bad value for parameter $visibility: must be a integer blockers / should they be?

I'm not rolling back at present, but yeah, marking those as blockers for going forward.

brennen added a subtask: T251952: CoreParserFunctions::revisionuser: Call to a member function getUser() on boolean.May 5 2020, 9:21 PM

brennen added a subtask: T251950: MutableRevisionRecord: Bad value for parameter $visibility: must be a integer.

In T249963#6110751, @brennen wrote:

I'm not rolling back at present, but yeah, marking those as blockers for going forward.

...so I caused three blockers for the train? Wow. Anyway, patches pending for both tasks, unable to test locally unfortunately

...so I caused three blockers for the train? Wow. Anyway, patches pending for both tasks, unable to test locally unfortunately

This seems to be pretty finicky work. Thanks for the rapid response.

In T249963#6110777, @brennen wrote:

...so I caused three blockers for the train? Wow. Anyway, patches pending for both tasks, unable to test locally unfortunately

This seems to be pretty finicky work. Thanks for the rapid response.

Both had +2 and are merging. Backports available:

https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/594572/ - backport for T251952
https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/594574/ - backport for T251950

I should be available to fix any more issues discovered with my revision work :)

<humor>jouncebot announces SWAT saying Note: If you break AND fix the wikis, you will be rewarded with a sticker. - what if you break AND fix core? three times?</humor>

DannyS712 closed subtask T251950: MutableRevisionRecord: Bad value for parameter $visibility: must be a integer as Resolved.May 5 2020, 9:56 PM

DannyS712 closed subtask T251952: CoreParserFunctions::revisionuser: Call to a member function getUser() on boolean as Resolved.May 5 2020, 10:03 PM

brennen added a subtask: T252043: Special:DeletedContributions: $row does not contain fields needed for comment rev_comment.May 6 2020, 4:24 PM

DannyS712 closed subtask T252043: Special:DeletedContributions: $row does not contain fields needed for comment rev_comment as Resolved.May 6 2020, 6:29 PM

• mmodell added a subtask: T252052: Fix Undefined property: stdClass::$ar_id from Revision/RevisionArchiveRecord.php line 74.May 6 2020, 6:41 PM

• mmodell closed subtask T252052: Fix Undefined property: stdClass::$ar_id from Revision/RevisionArchiveRecord.php line 74 as Resolved.May 6 2020, 7:00 PM

Mentioned in SAL (#wikimedia-operations) [2020-05-06T19:03:05Z] <brennen> 1.35.0-wmf.31 train unblocked (T249963), rolling forward to group0

Mentioned in SAL (#wikimedia-operations) [2020-05-06T19:03:44Z] <brennen> CORRECTION: 1.35.0-wmf.31 train unblocked (T249963), rolling forward to group1

brennen added a subtask: T252072: Special:RevisionReview: PHP Unknown error: Object of class MediaWiki\Revision\RevisionStoreRecord could not be converted to string.May 6 2020, 8:57 PM

brennen added a subtask: T252077: WikimediaPrometheusQueryServiceLagProvider: PHP Warning: A non-numeric value encountered.May 6 2020, 10:04 PM

DannyS712 closed subtask T252072: Special:RevisionReview: PHP Unknown error: Object of class MediaWiki\Revision\RevisionStoreRecord could not be converted to string as Resolved.May 6 2020, 10:27 PM

brennen removed a subtask: T252077: WikimediaPrometheusQueryServiceLagProvider: PHP Warning: A non-numeric value encountered.May 6 2020, 10:34 PM

End-of-workday notes before I sign off: Stable on group1 as of this comment. It's confirmed that T252077 is not a blocker.

LucasWerkmeister added a subtask: T252079: mw.wikibase.getLabelByLang('Q1','en') returning nil today.May 7 2020, 12:27 AM

Rolled back to group0 at 02:56 UTC for T252079; writing a "blocked" status update mail.

Logging off; reassigning to Antoine for now as backup conductor in case the train can roll before I'm awake.

Addshore removed a subtask: T252079: mw.wikibase.getLabelByLang('Q1','en') returning nil today.May 7 2020, 12:13 PM

Change 594947 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/mediawiki-config@master] group1 wikis to 1.35.0-wmf.31 (2)

https://gerrit.wikimedia.org/r/594947

gerritbot added a project: Patch-For-Review.May 7 2020, 1:08 PM

Change 594947 merged by jenkins-bot:
[operations/mediawiki-config@master] group1 wikis to 1.35.0-wmf.31 (2)

https://gerrit.wikimedia.org/r/594947

brennen claimed this task.May 7 2020, 3:42 PM

brennen added a subscriber: hashar.

Change 595030 had a related patch set uploaded (by Brennen Bearnes; owner: Brennen Bearnes):
[operations/mediawiki-config@master] Revert "all wikis to 1.35.0-wmf.31"

https://gerrit.wikimedia.org/r/595030

Change 595030 merged by jenkins-bot:
[operations/mediawiki-config@master] Revert "all wikis to 1.35.0-wmf.31"

https://gerrit.wikimedia.org/r/595030

brennen added a subtask: T252156: Increase in "Main slot of revision [number] not found in database!" after deploy of 1.35.0-wmf.31 to all wikis.May 7 2020, 7:25 PM

Addshore mentioned this in T252079: mw.wikibase.getLabelByLang('Q1','en') returning nil today.May 7 2020, 9:21 PM

brennen removed a subtask: T252156: Increase in "Main slot of revision [number] not found in database!" after deploy of 1.35.0-wmf.31 to all wikis.May 7 2020, 11:09 PM

DannyS712 added a subtask: T252179: Edits saved via PageUpdater need autopatrol status set.May 7 2020, 11:53 PM

Change 595075 had a related patch set uploaded (by Brennen Bearnes; owner: Brennen Bearnes):
[operations/mediawiki-config@master] All wikis except test to 1.35.0-wmf.30

https://gerrit.wikimedia.org/r/595075

Change 595075 merged by jenkins-bot:
[operations/mediawiki-config@master] All wikis except test to 1.35.0-wmf.30

https://gerrit.wikimedia.org/r/595075

Current status of T252179: Edits saved via PageUpdater need autopatrol status set:

All (5/5) reverts merged to respective master branches
No (0/4, one not needed) reverts merged to wmf.31
No (0/4) reverts deployed to wmf.31

For the train to proceed:

the 4 backports need to be merged
the 4 backports need to be deployed

Once unblocked, if mediawiki.org is updated to wmf.31, it should be easy to confirm that the issue was fixed

DannyS712 mentioned this in T250446: CheckUser internal API error.May 11 2020, 7:15 AM

Mentioned in SAL (#wikimedia-operations) [2020-05-11T15:42:28Z] <brennen> syncing backports to 1.35.0-wmf.31 (T249963) for T252179

Stashbot mentioned this in T252179: Edits saved via PageUpdater need autopatrol status set.May 11 2020, 3:42 PM

Mentioned in SAL (#wikimedia-operations) [2020-05-11T16:13:46Z] <brennen@deploy1001> rebuilt and synchronized wikiversions files: mediawikiwiki to 1.35.0-wmf.31 (T249963) for testing T252179

DannyS712 closed subtask T252179: Edits saved via PageUpdater need autopatrol status set as Resolved.May 11 2020, 4:57 PM

Appears stable on all wikis, but will continue monitoring. Thanks all for assistance with this one.

We've had a huge spikes on INSERTS on our primary masters - even though it has happened before and it is being tracked at T247028, this happened again right after yesterday's release see T247028#6128426
Not sure if we should reopen this task.

In T249963#6128435, @Marostegui wrote:

We've had a huge spikes on INSERTS on our primary masters - even though it has happened before and it is being tracked at T247028, this happened again right after yesterday's release see T247028#6128426
Not sure if we should reopen this task.

Let's do, so we can block the next train while we investigate.

thcipriani added a parent task: T249964: 1.35.0-wmf.32 deployment blockers.May 12 2020, 3:22 PM

thcipriani mentioned this in T249964: 1.35.0-wmf.32 deployment blockers.