Page MenuHomePhabricator
Create Task
Maniphest T408277

1.46.0-wmf.7 deployment blockers
Closed, ResolvedPublic5 Estimated Story PointsRelease
Actions

Details

Backup Train Conductor
hashar
Release Version
1.46.0-wmf.7
Release Date
Dec 15 2025, 12:00 AM

2025 week 51 1.46-wmf.7 Changes wmf/1.46.0-wmf.7

This MediaWiki Train Deployment is scheduled for the week of Monday, December 15th:

Monday December 15thTuesday, December 16thWednesday, December 17thThursday, December 18thFriday
Backports only.Branch wmf.7 and deploy to Group 0 Wikis.Deploy wmf.7 to Group 1 Wikis.Deploy wmf.7 to all Wikis.No deployments on fridays

How this works

  • Any serious bugs affecting wmf.7 should be added as subtasks beneath this one.
  • Any open subtask(s) block the train from moving forward. This means no further deployments until the blockers are resolved.
  • If something is serious enough to warrant a rollback then you should bring it to the attention of deployers on the #wikimedia-operations IRC channel.
  • If you have a risky change in this week's train add a comment to this task using the Risky patch template
  • For more info about deployment blockers, see Holding the train.

Related Links

Other Deployments

Previous: 1.46.0-wmf.6
Next: 1.46.0-wmf.8
Related Changes in Gerrit:
SubjectRepoBranchLines +/-
group2 to 1.46.0-wmf.7operations/mediawiki-configmaster+340 -340
group1 to 1.46.0-wmf.7operations/mediawiki-configmaster+578 -578
group0 to 1.46.0-wmf.7operations/mediawiki-configmaster+142 -142
testwikis to 1.46.0-wmf.7operations/mediawiki-configmaster+3 -3
Branch commit for wmf/1.46.0-wmf.7mediawiki/corewmf/1.46.0-wmf.7+1 K -3
Customize query in gerrit

Related Objects
Search...

Event Timeline

thcipriani created this task.Oct 25 2025, 12:05 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 25 2025, 12:05 AM
thcipriani assigned this task to dancy.Oct 25 2025, 12:19 AM
thcipriani triaged this task as Medium priority.
thcipriani updated Other Assignee, added: hashar.
thcipriani set the point value for this task to 5.
thcipriani added projects: Release-Engineering-Team (Priority Backlog 📥), Essential-Work.
ihurbain mentioned this in T411951: CTT tasks week of 2025-12-12.Dec 7 2025, 9:58 AM
Michael added a subtask: T412265: Pushing to the docker registry fails with 500 Internal Server Error.Dec 12 2025, 12:23 PM
Etonkovidova mentioned this in T412737: Verify 1.45.0-wmf.7 deployments.Dec 15 2025, 4:51 PM
gerritbot added a comment.Dec 16 2025, 2:09 AM

Change #1218393 had a related patch set uploaded (by TrainBranchBot; author: trainbranchbot):

[mediawiki/core@wmf/1.46.0-wmf.7] Branch commit for wmf/1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1218393

gerritbot added a project: Patch-For-Review.Dec 16 2025, 2:09 AM
hashar subscribed.Dec 16 2025, 9:54 AM
In T408277#11462645, @gerritbot wrote:

Change #1218393 had a related patch set uploaded (by TrainBranchBot; author: trainbranchbot):

[mediawiki/core@wmf/1.46.0-wmf.7] Branch commit for wmf/1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1218393

The commit failed to merge due to the apitests job failing with npm error code ECONNRESET. This in turn caused the train-presync job to fail:

04:00:05 Clone https://gerrit.wikimedia.org/r/mediawiki/core (wmf/1.46.0-wmf.7 branch) in /srv/mediawiki-staging/php-1.46.0-wmf.7
04:00:58 https://gerrit.wikimedia.org/r/mediawiki/core checked out at commit aa121a4b4e989b3917823b381ad1b72283050f16
04:00:59 Finished scap prep 1.46.0-wmf.7 (duration: 00m 53s)
/srv/mediawiki-staging/php-1.46.0-wmf.7/.gitmodules does not exist. Did the train branch commit get merged?
gerritbot added a comment.Dec 16 2025, 9:59 AM

Change #1218393 merged by jenkins-bot:

[mediawiki/core@wmf/1.46.0-wmf.7] Branch commit for wmf/1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1218393

Stashbot added a comment.Dec 16 2025, 10:03 AM

Mentioned in SAL (#wikimedia-operations) [2025-12-16T10:03:08Z] <hashar> Started MediaWiki train task train-presync. It did not run overnight due to a CI failure | T408277

gerritbot added a comment.Dec 16 2025, 10:03 AM

Change #1218720 had a related patch set uploaded (by TrainBranchBot; author: MediaWiki PreSync):

[operations/mediawiki-config@master] testwikis to 1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1218720

gerritbot added a comment.Dec 16 2025, 10:04 AM

Change #1218720 merged by jenkins-bot:

[operations/mediawiki-config@master] testwikis to 1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1218720

Stashbot added a comment.Dec 16 2025, 10:05 AM

Mentioned in SAL (#wikimedia-operations) [2025-12-16T10:05:08Z] <mwpresync@deploy2002> Started scap sync-world: testwikis to 1.46.0-wmf.7 refs T408277

Maintenance_bot removed a project: Patch-For-Review.Dec 16 2025, 10:31 AM
Dreamy_Jazz subscribed.Dec 16 2025, 10:48 AM

The train still has not moved to 1.46.0-wmf.7 and is blocking deployments? Is this a known issue?

hashar added a comment.Dec 16 2025, 11:04 AM

Yes it failed overnight and I have restarted the images building this morning. It was stuck.

The image was created at 10:06:58 with a size of 9.23GB. scap was waiting for docker-pusher to push the image. From the deploy2002 network view:

docker_push_deploy2002.png (470×860 px, 42 KB)

And surely eventually it failed to be pushed because of some error happening on the Docker registry. From /srv/mwpresync/scap-image-build-and-push-log:

10:09:23 [mediawiki-publish-83] Pushing docker-registry.discovery.wmnet/restricted/mediawiki-multiversion:<...>
10:09:23 [mediawiki-publish-83] Running sudo /usr/local/bin/docker-pusher -q docker-registry.discovery.wmnet/restricted/mediawiki-multiversion:<...>
10:58:05 [mediawiki-publish-83] received unexpected HTTP status: 500 Internal Server Error
10:58:05 [mediawiki-publish-83] Traceback (most recent call last):
  File "/srv/mwbuilder/release/make-container-image/app.py", line 145, in join
    future.result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 433, in result
    return self.__get_result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/mwbuilder/release/make-container-image/build-images.py", line 548, in build
    mw_mv_image, mw_mv_debug_image, mw_mv_cli_image = app_instance.build_mediawiki_images(
  File "/srv/mwbuilder/release/make-container-image/build-images.py", line 260, in build_mediawiki_images
    image = build_image_incr.App(
  File "/srv/mwbuilder/release/make-container-image/build_image_incr.py", line 171, in run
    self.push_image(report["image"])
  File "/srv/mwbuilder/release/make-container-image/app.py", line 91, in push_image
    self.check_call(["sudo", "/usr/local/bin/docker-pusher", "-q", image_ref])
  File "/srv/mwbuilder/release/make-container-image/app.py", line 72, in check_call
    return subprocess.check_call(
  File "/usr/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '/usr/local/bin/docker-pusher', '-q', 'docker-registry.discovery.wmnet/restricted/mediawiki-multiversion:<...>']' returned non-zero exit status 1.

Traceback (most recent call last):
  File "/srv/mwbuilder/release/make-container-image/build-images.py", line 619, in <module>
    main()
  File "/srv/mwbuilder/release/make-container-image/build-images.py", line 603, in main
    threaded_app.join()
  File "/srv/mwbuilder/release/make-container-image/app.py", line 151, in join
    raise ThreadedAppJoinException("One or more tasks raised an exception")
app.ThreadedAppJoinException: One or more tasks raised an exception

I'll retry.

Dreamy_Jazz added a comment.Dec 16 2025, 11:08 AM

I wonder if it makes sense for scap to be locked during this? scap proceeded to merge the patches but then was halted by a different lock

hashar added a comment.Dec 16 2025, 3:34 PM

@dancy 1.46.0-wmf.7 has been rolled to testwikis.

The pretrain Jenkins job failed due to some npm connection issue. I reran it, the image took some efforts to get published and finally made it there. So I think you are all set to continue! 🚆

dancy added a comment.Dec 16 2025, 3:37 PM
In T408277#11464912, @hashar wrote:

@dancy 1.46.0-wmf.7 has been rolled to testwikis.

The pretrain Jenkins job failed due to some npm connection issue. I reran it, the image took some efforts to get published and finally made it there. So I think you are all set to continue! 🚆

Many thanks!

gerritbot added a comment.Dec 16 2025, 7:11 PM

Change #1218814 had a related patch set uploaded (by TrainBranchBot; author: SpiderPig jobrunner/apiserver):

[operations/mediawiki-config@master] group0 to 1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1218814

gerritbot added a project: Patch-For-Review.Dec 16 2025, 7:11 PM

Change #1218814 merged by jenkins-bot:

[operations/mediawiki-config@master] group0 to 1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1218814

Stashbot added a comment.Dec 16 2025, 7:18 PM

Mentioned in SAL (#wikimedia-operations) [2025-12-16T19:18:43Z] <dancy@deploy2002> rebuilt and synchronized wikiversions files: group0 to 1.46.0-wmf.7 refs T408277

Maintenance_bot removed a project: Patch-For-Review.Dec 16 2025, 7:30 PM
gerritbot added a comment.Wed, Dec 17, 7:00 PM

Change #1219202 had a related patch set uploaded (by TrainBranchBot; author: SpiderPig jobrunner/apiserver):

[operations/mediawiki-config@master] group1 to 1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1219202

gerritbot added a project: Patch-For-Review.Wed, Dec 17, 7:00 PM

Change #1219202 merged by jenkins-bot:

[operations/mediawiki-config@master] group1 to 1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1219202

Stashbot added a comment.Wed, Dec 17, 7:11 PM

Mentioned in SAL (#wikimedia-operations) [2025-12-17T19:11:40Z] <dancy@deploy2002> rebuilt and synchronized wikiversions files: group1 to 1.46.0-wmf.7 refs T408277

Maintenance_bot removed a project: Patch-For-Review.Wed, Dec 17, 7:30 PM
gerritbot added a comment.Thu, Dec 18, 7:02 PM

Change #1219621 had a related patch set uploaded (by TrainBranchBot; author: SpiderPig jobrunner/apiserver):

[operations/mediawiki-config@master] group2 to 1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1219621

gerritbot added a project: Patch-For-Review.Thu, Dec 18, 7:02 PM

Change #1219621 merged by jenkins-bot:

[operations/mediawiki-config@master] group2 to 1.46.0-wmf.7

https://gerrit.wikimedia.org/r/1219621

Stashbot added a comment.Thu, Dec 18, 7:10 PM

Mentioned in SAL (#wikimedia-operations) [2025-12-18T19:10:00Z] <dancy@deploy2002> rebuilt and synchronized wikiversions files: group2 to 1.46.0-wmf.7 refs T408277

Maintenance_bot removed a project: Patch-For-Review.Thu, Dec 18, 7:30 PM
dancy closed this task as Resolved.Fri, Dec 19, 3:38 PM
cscott added a subtask: T413227: Section edit links appear in wrong user interface language on Parsoid Read View wikis.Fri, Dec 19, 5:14 PM
cscott closed subtask T413227: Section edit links appear in wrong user interface language on Parsoid Read View wikis as Resolved.Fri, Dec 19, 9:56 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits