Risky Patch! 🚂🔥

• Change: https://gerrit.wikimedia.org/r/900691
• Summary:
  • The change affects MediaWiki:GrowthMentors.json (example), a source of the list of mentors, by adding a username field for each mentor. However, GrowthExperiments interprets unrecognized keys as a validation error (validations runs on page saves and on using the list of mentors).
  • This means that if this patch gets reverted, we will likely receive many validation errors to GrowthMentors.json. Adding the field on its own is not risky; getting the change undeployed because of train rollback is the rsky part here.
• Test plan:
  • Manual verification (trigger an edit to GrowthMentors.json via Special:ManageMentors and see it works), monitoring of validation errors
• Places to monitor:
  • Logstash: mediawiki-errors
  • Logstash: mediawiki-client-errors
  • Logstash: GrowthExperiments channel
    • An entry to look for would be "failed to load mentor list"
  • Grafana: mediawiki-errors
• Revert plan: When the patch needs to be undeployed (train rollback/patch revert doesn't make a difference), the field username needs to be manually removed from MediaWiki:GrowthMentors.json (needs advanced on-wiki permissions; I can do that should that be needed).
• Affected wikis: growthexperiments.dblist

abwiki
acewiki
adywiki
afwiki
akwiki
alswiki
altwiki
amiwiki
amwiki
angwiki
anwiki
arcwiki
arwiki
arywiki
arzwiki
astwiki
aswiki
atjwiki
avkwiki
avwiki
awawiki
aywiki
azbwiki
azwiki
banwiki
barwiki
bat_smgwiki
bawiki
bclwiki
be_x_oldwiki
bewiki
bgwiki
bhwiki
biwiki
bjnwiki
bmwiki
bnwiki
bowiki
bpywiki
brwiki
bswiki
bugwiki
bxrwiki
cawiki
cbk_zamwiki
cdowiki
cebwiki
cewiki
chrwiki
chwiki
chywiki
ckbwiki
cowiki
crhwiki
crwiki
csbwiki
cswiki
cuwiki
cvwiki
cywiki
dagwiki
dawiki
dewiki
dinwiki
diqwiki
dsbwiki
dtywiki
dvwiki
dzwiki
eewiki
elwiki
emlwiki
enwiki
eowiki
eswiki
etwiki
euwiki
extwiki
fawiki
ffwiki
fiu_vrowiki
fiwiki
fjwiki
fowiki
frpwiki
frrwiki
frwiki
frwiktionary
furwiki
fywiki
gagwiki
ganwiki
gawiki
gcrwiki
gdwiki
glkwiki
glwiki
gnwiki
gomwiki
gorwiki
gotwiki
guwiki
gvwiki
hakwiki
hawiki
hawwiki
hewiki
hifwiki
hiwiki
hrwiki
hsbwiki
htwiki
huwiki
hywiki
hywwiki
iawiki
idwiki
iewiki
igwiki
ikwiki
ilowiki
inhwiki
iowiki
iswiki
itwiki
iuwiki
jamwiki
jawiki
jbowiki
jvwiki
kaawiki
kabwiki
kawiki
kbdwiki
kbpwiki
kgwiki
kiwiki
kkwiki
klwiki
kmwiki
knwiki
koiwiki
kowiki
krcwiki
kshwiki
kswiki
kuwiki
kvwiki
kwwiki
kywiki
ladwiki
lawiki
lbewiki
lbwiki
lezwiki
lfnwiki
lgwiki
lijwiki
liwiki
lldwiki
lmowiki
lnwiki
lowiki
ltgwiki
ltwiki
lvwiki
madwiki
maiwiki
map_bmswiki
mdfwiki
mgwiki
mhrwiki
minwiki
miwiki
mkwiki
mlwiki
mniwiki
mnwiki
mnwwiki
mrjwiki
mrwiki
mswiki
mtwiki
mwlwiki
myvwiki
mywiki
mznwiki
nahwiki
napwiki
nawiki
nds_nlwiki
ndswiki
newiki
newwiki
niawiki
nlwiki
nnwiki
novwiki
nowiki
nqowiki
nrmwiki
nsowiki
nvwiki
nywiki
ocwiki
olowiki
omwiki
orwiki
oswiki
pagwiki
pamwiki
papwiki
pawiki
pcdwiki
pdcwiki
pflwiki
pihwiki
piwiki
plwiki
pmswiki
pnbwiki
pntwiki
pswiki
ptwiki
pwnwiki
quwiki
rmwiki
rmywiki
rnwiki
roa_rupwiki
roa_tarawiki
rowiki
ruewiki
ruwiki
rwwiki
sahwiki
satwiki
sawiki
scnwiki
scowiki
scwiki
sdwiki
sewiki
sgwiki
shiwiki
shnwiki
shwiki
simplewiki
siwiki
skrwiki
skwiki
slwiki
smnwiki
smwiki
snwiki
sowiki
sqwiki
srnwiki
srwiki
sswiki
stqwiki
stwiki
suwiki
svwiki
swwiki
szlwiki
szywiki
tawiki
taywiki
tcywiki
test2wiki
testwiki
tetwiki
tewiki
tgwiki
thwiki
tiwiki
tkwiki
tlwiki
tnwiki
towiki
tpiwiki
trvwiki
trwiki
tswiki
ttwiki
tumwiki
twwiki
tyvwiki
tywiki
udmwiki
ugwiki
ukwiki
urwiki
uzwiki
vecwiki
vepwiki
vewiki
viwiki
vlswiki
vowiki
warwiki
wawiki
wowiki
wuuwiki
xalwiki
xhwiki
xmfwiki
yiwiki
yowiki
zawiki
zeawiki
zh_classicalwiki
zh_min_nanwiki
zh_yuewiki
zhwiki
zuwiki

• IRC contact: Urbanecm, tgr, kostajh
• UBN Task Projects/tags: Growth-Team (Sprint 0 (Growth Team)), GrowthExperiments-Mentorship
• Would you like to backport this change rather than ride the train?: Yes: backporting seems like a safer option here.

FTR: I've backported the patch I mentioned as T330208#8717262.

Risky Patch! 🚂🔥

• Change: https://gerrit.wikimedia.org/r/445185
• Summary:
  • Why is it risky? Large patch impacting several files, including code paths interacting with the database.
• Test plan:
  • How will it be verified? See T188180#8726858.
• Places to monitor:
  • Logstash: mediawiki-errors
  • Logstash: mediawiki-client-errors
  • Grafana: mediawiki-errors
  • Grafana: mediawiki-abusefilter-profiling (need to select one wiki)
• Revert plan: Revert patch
• Affected wikis: AbuseFilter-deployed wikis (i.e., most production wikis)
• IRC contact: matej_suchanek (timezone UTC+2/CEST)
• UBN Task Projects/tags: AbuseFilter
• Would you like to backport this change rather than ride the train?: No

Change 902622 had a related patch set uploaded (by TrainBranchBot; author: trainbranchbot):

[mediawiki/core@wmf/1.41.0-wmf.2] Branch commit for wmf/1.41.0-wmf.2

https://gerrit.wikimedia.org/r/902622

Change 903656 had a related patch set uploaded (by Jforrester; author: trainbranchbot):

[mediawiki/core@master] Branch commit for wmf/1.41.0-wmf.2

https://gerrit.wikimedia.org/r/903656

Change 903656 abandoned by Jforrester:

[mediawiki/core@master] Branch commit for wmf/1.41.0-wmf.2

Reason:

Gerrit, you irritate me.

https://gerrit.wikimedia.org/r/903656

Change 903657 had a related patch set uploaded (by Jforrester; author: trainbranchbot):

[mediawiki/core@wmf/1.41.0-wmf.2] Branch commit for wmf/1.41.0-wmf.2

https://gerrit.wikimedia.org/r/903657

Change 902622 abandoned by Jforrester:

[mediawiki/core@wmf/1.41.0-wmf.2] Branch commit for wmf/1.41.0-wmf.2

Reason:

Re-pushed with the updated EventLogging patch as Ie2b61256fc6538f2da9e8a546b8eb45ee00ad28a because gerrit refused to let me.

https://gerrit.wikimedia.org/r/902622

Change 903657 merged by jenkins-bot:

[mediawiki/core@wmf/1.41.0-wmf.2] Branch commit for wmf/1.41.0-wmf.2

https://gerrit.wikimedia.org/r/903657

OK, branch creation is merged, but the pre-deploy will have to be triggered manually. Over to @dduvall.

Change 903660 had a related patch set uploaded (by TrainBranchBot; author: Jaime Nuche):

[operations/mediawiki-config@master] testwikis wikis to 1.41.0-wmf.2

https://gerrit.wikimedia.org/r/903660

Change 903660 merged by jenkins-bot:

[operations/mediawiki-config@master] testwikis wikis to 1.41.0-wmf.2

https://gerrit.wikimedia.org/r/903660

Mentioned in SAL (#wikimedia-operations) [2023-03-28T15:05:34Z] <jnuche@deploy2002> Started scap: testwikis wikis to 1.41.0-wmf.2 refs T330208

Mentioned in SAL (#wikimedia-operations) [2023-03-28T15:20:20Z] <jnuche@deploy2002> Started scap: testwikis wikis to 1.41.0-wmf.2 refs T330208

Mentioned in SAL (#wikimedia-operations) [2023-03-28T16:10:12Z] <jnuche@deploy2002> Finished scap: testwikis wikis to 1.41.0-wmf.2 refs T330208 (duration: 49m 52s)

Mentioned in SAL (#wikimedia-operations) [2023-03-28T19:19:27Z] <dduvall@deploy2002> Started scap: testwikis wikis to 1.41.0-wmf.2 refs T330208

Mentioned in SAL (#wikimedia-operations) [2023-03-28T19:26:51Z] <dduvall@deploy2002> Finished scap: testwikis wikis to 1.41.0-wmf.2 refs T330208 (duration: 07m 24s)

Risky Patch! 🚂🔥

• Change: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/883501, https://gerrit.wikimedia.org/r/c/mediawiki/core/+/800769, https://gerrit.wikimedia.org/r/c/mediawiki/core/+/901705
• Summary:
  • Why is it risky? Anything that touches ParserCache is risky by definition. But the particular issue we're looking at here is increased ParserCache miss rates due to the useParsoid ParserOption being added to the used options set. The scenario where this is relevant is fairly unusual -- on a page with user language in the 'used options' list, a user visiting an enwiki page for the first time in (for example) Spanish, will also end up invalidating the cached English version of the page when they write the cache entry for the Spanish version. This only happens on the first visit, only happens if they are visiting a forked cache entry with previously-cached content, etc. The increase in cache miss rate shouldn't be significant, but it will likely persist for the full 21 day parser cache TTL, after which all entries will have "useParsoid" set in their used options list.
• Test plan:
  • How will it be verified? Monitor ParserCache miss rates and confirm that the increase actually is minimal, as we expect.
• Places to monitor:
  • Grafana: parser cache miss rates in particular the miss_absent rate on cache pcache
• Revert plan: Rollback train is probably the most reasonable initial step. The three patches listed above would eventually need to be reverted, or alternatively a one-line patch to remove $parserOptions->getUseParsoid() from line 365 of WikitextContentHandler.php would also work. Either would temporarily block the parsoid read views work, but that is experimental and not yet exposed to production.
• Affected wikis: all
• IRC contact: cscott @cscott (UTC-4) , dues/duesen @daniel (timezone UTC+-??)
• UBN Task Projects/tags: Content-Transform-Team
• Would you like to backport this change rather than ride the train?: No

*On further review of the code* I don't think this will actually impact cache miss rate, since although the usedOptions change in the ParserMetadataCache, since for the legacy parser new value of useParsoid matches the default value, it shouldn't be included in the parser cache key even after the update to ParserMetadataCache. Still worth watching the ParserCache out of an abundance of caution.

Change 904243 had a related patch set uploaded (by TrainBranchBot; author: Dan Duvall):

[operations/mediawiki-config@master] group0 wikis to 1.41.0-wmf.2

https://gerrit.wikimedia.org/r/904243

Change 904243 merged by jenkins-bot:

[operations/mediawiki-config@master] group0 wikis to 1.41.0-wmf.2

https://gerrit.wikimedia.org/r/904243

Mentioned in SAL (#wikimedia-operations) [2023-03-29T18:16:44Z] <dduvall@deploy2002> rebuilt and synchronized wikiversions files: group0 wikis to 1.41.0-wmf.2 refs T330208

Change 904267 had a related patch set uploaded (by TrainBranchBot; author: Dan Duvall):

[operations/mediawiki-config@master] group1 wikis to 1.41.0-wmf.2

https://gerrit.wikimedia.org/r/904267

Change 904267 merged by jenkins-bot:

[operations/mediawiki-config@master] group1 wikis to 1.41.0-wmf.2

https://gerrit.wikimedia.org/r/904267

Mentioned in SAL (#wikimedia-operations) [2023-03-29T18:39:20Z] <dduvall@deploy2002> rebuilt and synchronized wikiversions files: group1 wikis to 1.41.0-wmf.2 refs T330208

Mentioned in SAL (#wikimedia-operations) [2023-03-29T18:45:07Z] <dduvall@deploy2002> Synchronized php: group1 wikis to 1.41.0-wmf.2 refs T330208 (duration: 05m 48s)

Change 904611 had a related patch set uploaded (by TrainBranchBot; author: Dan Duvall):

[operations/mediawiki-config@master] all wikis to 1.41.0-wmf.2

https://gerrit.wikimedia.org/r/904611

Change 904611 merged by jenkins-bot:

[operations/mediawiki-config@master] all wikis to 1.41.0-wmf.2

https://gerrit.wikimedia.org/r/904611

Mentioned in SAL (#wikimedia-operations) [2023-03-30T18:33:12Z] <dduvall@deploy2002> rebuilt and synchronized wikiversions files: all wikis to 1.41.0-wmf.2 refs T330208