Page MenuHomePhabricator
Create Task
Maniphest T408272

1.46.0-wmf.2 deployment blockers
Closed, ResolvedPublic5 Estimated Story PointsRelease
Actions

Details

Backup Train Conductor
jeena
Release Version
1.46.0-wmf.2
Release Date
Nov 10 2025, 12:00 AM

2025 week 46 1.46-wmf.2 Changes wmf/1.46.0-wmf.2

This MediaWiki Train Deployment is scheduled for the week of Monday, November 10th:

Monday November 10thTuesday, November 11thWednesday, November 12thThursday, November 13thFriday
Backports only.Branch wmf.2 and deploy to Group 0 Wikis.Deploy wmf.2 to Group 1 Wikis.Deploy wmf.2 to all Wikis.No deployments on fridays

How this works

  • Any serious bugs affecting wmf.2 should be added as subtasks beneath this one.
  • Any open subtask(s) block the train from moving forward. This means no further deployments until the blockers are resolved.
  • If something is serious enough to warrant a rollback then you should bring it to the attention of deployers on the #wikimedia-operations IRC channel.
  • If you have a risky change in this week's train add a comment to this task using the Risky patch template
  • For more info about deployment blockers, see Holding the train.

Related Links

Other Deployments

Previous: 1.46.0-wmf.1
Next: 1.46.0-wmf.3
Related Changes in Gerrit:
SubjectRepoBranchLines +/-
group2 to 1.46.0-wmf.2operations/mediawiki-configmaster+341 -341
group1 to 1.46.0-wmf.2operations/mediawiki-configmaster+578 -578
group0 to 1.46.0-wmf.2operations/mediawiki-configmaster+578 -578
group1 to 1.46.0-wmf.2operations/mediawiki-configmaster+578 -578
group0 to 1.46.0-wmf.2operations/mediawiki-configmaster+140 -140
testwikis to 1.46.0-wmf.2operations/mediawiki-configmaster+3 -3
Branch commit for wmf/1.46.0-wmf.2mediawiki/corewmf/1.46.0-wmf.2+1 K -3
Customize query in gerrit

Related Objects
Search...

Event Timeline

thcipriani created this task.Oct 25 2025, 12:05 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 25 2025, 12:05 AM
thcipriani assigned this task to Aklapper.Oct 25 2025, 12:16 AM
thcipriani triaged this task as Medium priority.
thcipriani updated Other Assignee, added: jeena.
thcipriani set the point value for this task to 5.
thcipriani added projects: Release-Engineering-Team (Priority Backlog 📥), Essential-Work.
ABreault-WMF mentioned this in T409607: CTT tasks week of 2025-11-07.Nov 8 2025, 12:55 AM
Krinkle added a subtask: T409619: [Regression pre-1.46wmf.2] Clicking "Edit" button makes its label go missing in VE.Nov 8 2025, 10:18 PM
Edtadros mentioned this in T409650: Verify 1.46.0-wmf.2 deployment.Nov 9 2025, 7:08 AM
Zabe added a subtask: T409740: Beta cluster is down due to 'No such service: WikibaseRepo.Settings'.Nov 10 2025, 3:11 PM
Zabe closed subtask T409740: Beta cluster is down due to 'No such service: WikibaseRepo.Settings' as Resolved.Nov 10 2025, 4:53 PM
Aklapper removed a subtask: T409619: [Regression pre-1.46wmf.2] Clicking "Edit" button makes its label go missing in VE.Nov 10 2025, 9:30 PM
gerritbot added a comment.Nov 11 2025, 2:07 AM

Change #1203580 had a related patch set uploaded (by TrainBranchBot; author: trainbranchbot):

[mediawiki/core@wmf/1.46.0-wmf.2] Branch commit for wmf/1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1203580

gerritbot added a project: Patch-For-Review.Nov 11 2025, 2:07 AM
gerritbot added a comment.Nov 11 2025, 2:18 AM

Change #1203580 merged by jenkins-bot:

[mediawiki/core@wmf/1.46.0-wmf.2] Branch commit for wmf/1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1203580

Maintenance_bot removed a project: Patch-For-Review.Nov 11 2025, 2:30 AM
gerritbot added a comment.Nov 11 2025, 4:02 AM

Change #1203592 had a related patch set uploaded (by TrainBranchBot; author: MediaWiki PreSync):

[operations/mediawiki-config@master] testwikis to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1203592

gerritbot added a project: Patch-For-Review.Nov 11 2025, 4:02 AM

Change #1203592 merged by jenkins-bot:

[operations/mediawiki-config@master] testwikis to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1203592

Stashbot added a comment.Nov 11 2025, 4:03 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-11T04:03:26Z] <mwpresync@deploy2002> Started scap sync-world: testwikis to 1.46.0-wmf.2 refs T408272

Maintenance_bot removed a project: Patch-For-Review.Nov 11 2025, 4:30 AM
Stashbot added a comment.Nov 11 2025, 4:49 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-11T04:49:53Z] <mwpresync@deploy2002> Finished scap sync-world: testwikis to 1.46.0-wmf.2 refs T408272 (duration: 46m 27s)

gerritbot added a comment.Nov 11 2025, 9:23 AM

Change #1203761 had a related patch set uploaded (by TrainBranchBot; author: Andre Klapper):

[operations/mediawiki-config@master] group0 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1203761

gerritbot added a project: Patch-For-Review.Nov 11 2025, 9:24 AM

Change #1203761 merged by jenkins-bot:

[operations/mediawiki-config@master] group0 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1203761

Maintenance_bot removed a project: Patch-For-Review.Nov 11 2025, 9:30 AM
Stashbot added a comment.Nov 11 2025, 9:36 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-11T09:36:54Z] <aklapper@deploy2002> rebuilt and synchronized wikiversions files: group0 to 1.46.0-wmf.2 refs T408272

taavi added a subtask: T409831: Previous link in page history is broken.Nov 11 2025, 2:12 PM
Aklapper closed subtask T409831: Previous link in page history is broken as Resolved.Nov 12 2025, 10:40 AM
gerritbot added a comment.Nov 12 2025, 10:42 AM

Change #1204358 had a related patch set uploaded (by TrainBranchBot; author: Andre Klapper):

[operations/mediawiki-config@master] group1 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1204358

gerritbot added a project: Patch-For-Review.Nov 12 2025, 10:42 AM

Change #1204358 merged by jenkins-bot:

[operations/mediawiki-config@master] group1 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1204358

Stashbot added a comment.Nov 12 2025, 10:49 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-12T10:49:28Z] <aklapper@deploy2002> rebuilt and synchronized wikiversions files: group1 to 1.46.0-wmf.2 refs T408272

Aklapper mentioned this in T409876: WikiLambda's StringForLanguageBuilder relies on LanguageFallback's mode being an int, not the new LanguageFallbackMode enum, breaking views (and tests).Nov 12 2025, 10:59 AM
gerritbot added a comment.Nov 12 2025, 11:03 AM

Change #1204362 had a related patch set uploaded (by TrainBranchBot; author: Andre Klapper):

[operations/mediawiki-config@master] group0 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1204362

Aklapper added a subtask: T409876: WikiLambda's StringForLanguageBuilder relies on LanguageFallback's mode being an int, not the new LanguageFallbackMode enum, breaking views (and tests).Nov 12 2025, 11:04 AM
gerritbot added a comment.Nov 12 2025, 11:05 AM

Change #1204362 merged by jenkins-bot:

[operations/mediawiki-config@master] group0 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1204362

Stashbot added a comment.Nov 12 2025, 11:17 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-12T11:17:16Z] <aklapper@deploy2002> rebuilt and synchronized wikiversions files: group0 to 1.46.0-wmf.2 refs T408272

Maintenance_bot removed a project: Patch-For-Review.Nov 12 2025, 11:31 AM
Aklapper added a comment.Nov 12 2025, 11:41 AM

Rolled back group1 from 1.46.0-wmf.2 to 1.46.0-wmf.1 due to T409876: WikiLambda's StringForLanguageBuilder relies on LanguageFallback's mode being an int, not the new LanguageFallbackMode enum, breaking views (and tests) making wikifunctions.org render an exception on nearly every page, only in wmf.2.

Also, canary check output while rolling back was a bit confusing (while informative, it likely makes more sense as a warning when moving forward):

11:09:36 Logstash checker Counted 81 error(s) in the last 20 seconds. The threshold is 10.
11:09:36 Top 2 errors:
[41 hits] PHP Warning: foreach() argument must be of type array|object, null given
[40 hits] PHP Warning: Undefined array key "extensionData"

These two errors listed by the canary check were a spike happening on old wmf.1, which I reported as T409910: PHP Warning: foreach() argument must be of type array|object, null given / PHP Warning: Undefined array key "extensionData" but technically is not a train blocker.

Jdforrester-WMF subscribed.Nov 12 2025, 1:57 PM
In T408272#11365992, @Aklapper wrote:

Rolled back group1 from 1.46.0-wmf.2 to 1.46.0-wmf.1 due to T409876: WikiLambda's StringForLanguageBuilder relies on LanguageFallback's mode being an int, not the new LanguageFallbackMode enum, breaking views (and tests) making wikifunctions.org render an exception on nearly every page, only in wmf.2.

Thanks; fix landing in master now, then we'll back-port.

Jdforrester-WMF closed subtask T409876: WikiLambda's StringForLanguageBuilder relies on LanguageFallback's mode being an int, not the new LanguageFallbackMode enum, breaking views (and tests) as Resolved.Nov 12 2025, 3:24 PM
gerritbot added a comment.Nov 12 2025, 7:01 PM

Change #1204650 had a related patch set uploaded (by TrainBranchBot; author: Andre Klapper):

[operations/mediawiki-config@master] group1 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1204650

gerritbot added a project: Patch-For-Review.Nov 12 2025, 7:01 PM

Change #1204650 merged by jenkins-bot:

[operations/mediawiki-config@master] group1 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1204650

Aklapper edited projects, added Release-Engineering-Team (Doing 😎); removed Release-Engineering-Team (Priority Backlog 📥).Nov 12 2025, 7:06 PM
Stashbot added a comment.Nov 12 2025, 7:10 PM

Mentioned in SAL (#wikimedia-operations) [2025-11-12T19:10:34Z] <aklapper@deploy2002> rebuilt and synchronized wikiversions files: group1 to 1.46.0-wmf.2 refs T408272

Maintenance_bot removed a project: Patch-For-Review.Nov 12 2025, 7:30 PM
gerritbot added a comment.Nov 13 2025, 9:08 AM

Change #1204803 had a related patch set uploaded (by TrainBranchBot; author: Andre Klapper):

[operations/mediawiki-config@master] group2 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1204803

gerritbot added a project: Patch-For-Review.Nov 13 2025, 9:08 AM

Change #1204803 merged by jenkins-bot:

[operations/mediawiki-config@master] group2 to 1.46.0-wmf.2

https://gerrit.wikimedia.org/r/1204803

Maintenance_bot removed a project: Patch-For-Review.Nov 13 2025, 9:30 AM
Aklapper added a comment.EditedNov 13 2025, 9:33 AM

scap train failed:

09:10:47 [mediawiki-publish-83] received unexpected HTTP status: 500 Internal Server Error
[...]
subprocess.CalledProcessError: Command '['sudo', '/usr/local/bin/docker-pusher', '-q', 'docker-registry.discovery.wmnet/restricted/mediawiki-multiversion-debug:2025-11-13-090954-publish-83']' returned non-zero exit status 1.

Full log: P85305

I asked for input on #wikimedia-sre and #wikimedia-operations as recommended by hashar.

Note to myself: I copied the logfile to /home/aklapper/scap-image-build-and-push-log-20251113-fail on deploy2002.

Interestingly, if I tried to run scap train again, first dialog says that group2 is already on wmf.2. While https://versions.toolforge.org/ correctly shows wmf.1. Filed as T410018.

Stashbot added a comment.Nov 13 2025, 10:03 AM

Mentioned in SAL (#wikimedia-operations) [2025-11-13T10:03:56Z] <aklapper@deploy2002> rebuilt and synchronized wikiversions files: group2 to 1.46.0-wmf.2 refs T408272

Aklapper added a comment.Nov 13 2025, 10:05 AM

Deployment worked on second attempt trying to run scap train.

In T408272#11369913, @Aklapper wrote:

Interestingly, if I tried to run scap train again, first dialog says that group2 is already on wmf.2. While https://versions.toolforge.org/ correctly shows wmf.1.

<hashar> my guess is scap think that because it reads the local /srv/mediawiki-staging/wikiversions.json

Aklapper mentioned this in T410018: Initial group versions displayed by "scap train" can be wrong if previous scap attempt failed.Nov 13 2025, 10:33 AM
Aklapper closed this task as Resolved.Nov 13 2025, 11:00 AM

Declaring victory. See you again next week, little locomotive.

train-fire-u1f525_u1f682-emojikitchen-dev.png (534×534 px, 17 KB)

Aklapper mentioned this in T412265: Pushing to the docker registry fails with 500 Internal Server Error.Dec 11 2025, 9:34 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits