1.36.0-wmf.38 deployment blockers
Closed, ResolvedPublicRelease
Actions

Assigned To

Authored By

	thcipriani
	Mar 24 2021, 3:46 PM

Details

Backup Train Conductor: • mmodell
Release Version: 1.36.0-wmf.38
Release Date: Apr 5 2021, 12:00 AM

2021 week 14 1.36-wmf.38 Changes wmf/1.36.0-wmf.38

This MediaWiki Train Deployment is scheduled for the week of Monday, April 5th:

Monday April 5th	Tuesday, April 6th	Wednesday, April 7th	Thursday, April 8th	Friday
Backports only.	Branch `wmf.38` and deploy to Group 0 Wikis.	Deploy `wmf.38` to Group 1 Wikis.	Deploy `wmf.38` to all Wikis.	No deployments on fridays

See https://wikitech.wikimedia.org/wiki/Deployments for full schedule.

How this works

Any serious bugs affecting wmf.38 should be added as subtasks beneath this one.
- Use this form to create one.
Any open subtask(s) block the train from moving forward. This means no further deployments until the blockers are resolved.
If something is serious enough to warrant a rollback then you should bring it to the attention of deployers on the #wikimedia-operations IRC channel.
If you have a risky change in this week's train add a comment to this task using the Risky patch template
For more info about deployment blockers, see Holding the train.

Other Deployments

Previous: 1.36.0-wmf.37

Next: 1.37.0-wmf.1

Related Objects

Mentioned Here: T279585: Content serialization failed: Failed to decode as application/json (when parsing edit summary via API)

Event Timeline

thcipriani created this task.Mar 24 2021, 3:46 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 24 2021, 3:46 PM

thcipriani assigned this task to dduvall.Mar 26 2021, 5:58 PM

thcipriani triaged this task as Medium priority.

thcipriani updated Other Assignee, added: • mmodell.

thcipriani added a project: Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4))).Mar 26 2021, 6:01 PM

thcipriani edited projects, added Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4)); removed Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4))).Mar 26 2021, 6:04 PM

thcipriani moved this task from INBOX to Deployment on the Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4)) board.

Jdlrobson added a subtask: T279008: mw-parser-output now clears after block, which is a noticeable change for Category pages where infoboxes used to render floated alongside the automatic content.Mar 31 2021, 8:23 PM

Jdlrobson added a subtask: T279113: Unable to parse title at new Title in Special:MediaSearch.Apr 1 2021, 11:08 PM

Risky Patch! 🚂🔥

Change: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/FlaggedRevs/+/674423
What it does/What it's risky:
- The patch is large and touches core logic of the extension.
- FlaggedRevs is not well covered by tests and has a ton of edge cases
- FlaggedRevs has like twenty different modes based on configuration and uncountable number of combinations making it really hard to test for every case.
Test Plan:
- FlaggedRevs are enabled on test2wiki. Mostly manual testing unfortunately and log watching.
Places to Watch for Breakage
- Logstash MediaWiki Errors Dashboard: https://logstash.wikimedia.org/app/kibana#/dashboard/mediawiki-errors
- Grafana Production Logging Dashboard: https://grafana.wikimedia.org/d/000000102/production-logging
Revert Plan: rollback and yell at @Ladsgroup
Wikis Affected:

alswiki
arwiki
bewiki
bnwiki
bswiki
cawikinews
cewiki
ckbwiki
dewiki
dewikiquote
dewiktionary
elwikinews
enwiki
enwikibooks
enwikinews
eowiki
eswikinews
fawiki
fawikinews
fiwiki
frwikinews
hewikisource
hiwiki
huwiki
iawiki
idwiki
iswiktionary
kawiki
lawikisource
mkwiki
plwiki
plwikisource
plwiktionary
ptwikibooks
ptwikinews
ptwikisource
ruwiki
ruwikinews
ruwikiquote
ruwikisource
ruwiktionary
sqwiki
tawikinews
test2wiki
trwiki
trwikiquote
ukwiki
ukwiktionary
vecwiki
zh_classicalwiki

IRC Contact: @Ladsgroup (Amir1)
UBN Task Projects/tags: FlaggedRevs

Jdlrobson removed a subtask: T279008: mw-parser-output now clears after block, which is a noticeable change for Category pages where infoboxes used to render floated alongside the automatic content.Apr 5 2021, 5:39 PM

@Ladsgroup: Thank you for the risky patch notice. That is always very helpful and appreciated.

Parsoid missed the branch cut again (sorry!) so https://gerrit.wikimedia.org/r/c/mediawiki/vendor/+/676957 was cherry-picked to the wmf.38 branch mediawiki-vendor this morning before the first train. This shouldn't cause any issues, but making sure you're aware of it.

@cscott thanks for the heads-up

Mentioned in SAL (#wikimedia-operations) [2021-04-06T19:40:16Z] <marxarelli> 1.36.0-wmf.38 rolled to group0. error rates steady and no new errors spotted (T278344)

T279585: Content serialization failed: Failed to decode as application/json (when parsing edit summary via API) seems to be a new error on wmf.38. Don’t know if it should block the train, just mentioning it.

Just FYI, earlier today I rolled enwiki and enwikibooks (on mwdebug1002 only obviously) to wmf.38 to make sure flagged revs doesn't cause any massive outage or anything like that and logs were clean as well. So fingers crossed.

Thanks for that, @Ladsgroup!

@LucasWerkmeister that bug seems worrying to me and unfortunately I can't get a read on how often it's occurring since it doesn't show up in logstash. I'll reach out to folks for an opinion on severity.

dduvall added a subtask: T279585: Content serialization failed: Failed to decode as application/json (when parsing edit summary via API).Apr 7 2021, 8:45 PM

dduvall removed a subtask: T279585: Content serialization failed: Failed to decode as application/json (when parsing edit summary via API).Apr 7 2021, 11:04 PM

dduvall removed a subtask: T279113: Unable to parse title at new Title in Special:MediaSearch.Apr 7 2021, 11:06 PM

dduvall closed this task as Resolved.Apr 8 2021, 11:27 PM

1.36.0-wmf.38 deployment blockersClosed, ResolvedPublicReleaseActions