Page MenuHomePhabricator

⬆️ Update Addshore's status wiki
Closed, ResolvedPublic

Description

Reasons:

  • Get an idea of how updating a large wiki affects production
  • Get an idea of how longer updating a large wiki takes on production
  • Flush out any potential errors we have missed
  • @Addshore won't bite and is currently in the WMDE offices for fast feedback

https://addshore-wikibase-cloud-status.wikibase.cloud/wiki/Main_Page

Event Timeline

kicked off at Wed Dec 3 04:21:45 PM CET 2025

WBS_DOMAIN=addshore-wikibase-cloud-status.wikibase.cloud MW_VERSION=mw1.43-wbs1 MEDIAWIKI_BACKEND_INSTANCE_LABEL=mediawiki-143 ./mediawikiUpdate.sh
job.batch/mediawiki-update-82jgx created

Done in 1 h 23 min 29 s.

logs: https://cloudlogging.app.goo.gl/RnsF4JhzTRxJybRWA

image.png (467×1 px, 57 KB)

I wonder if we could run out of memory here.

Screenshot from the status page side of things showing the period of time it couldn't edit for

Screenshot_20251203-225828.png (960×960 px, 106 KB)

Done in 1 h 23 min 29 s.

logs: https://cloudlogging.app.goo.gl/RnsF4JhzTRxJybRWA

image.png (467×1 px, 57 KB)

I wonder if we could run out of memory here.

I wonder what part of the upgrade ends up consuming that much ram

Done in 1 h 23 min 29 s.

logs: https://cloudlogging.app.goo.gl/RnsF4JhzTRxJybRWA

image.png (467×1 px, 57 KB)

I wonder if we could run out of memory here.

I wonder what part of the upgrade ends up consuming that much ram

fwiw from the logs the part that took the longest looks like this (no idea what its rebuilding there actually):

2025-12-03 16:31:36.485 Rebuilding Q1 till Q250
...
2025-12-03 17:44:19.407 Rebuilding Q367501 till Q367656

https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/29a148c69312339721e9e700178e91d43238380e/repo/includes/Store/ItemTermsRebuilder.php#L71

So rebuilding the terms storage in SQL

It looks like there are no meaningful / complete release notes in the Wikibase extension that mention terms at all between these versions

https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/REL1_40/RELEASE-NOTES-1.40
https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/REL1_41/RELEASE-NOTES-1.41
https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/REL1_42/RELEASE-NOTES-1.42
https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/REL1_43/RELEASE-NOTES-1.43

Looking at the code a little, this might be something that you can skip, as I believe this generation has already been done, and it happening again is a side effect of something else?
The thing to do would be look at a DB you have not yet migrated, look in the update log table https://www.mediawiki.org/wiki/Manual:Updatelog_table for any string like rebuildItemTerms
I believe the update has been there since 1.35.... but will be easiest to check the database before worying about anything else

It looks like the update row log line rebuildItemTerms is only added via update.php, and not when the maintenance script itself that does this job is run.

https://github.com/search?q=repo%3Awikimedia%2Fmediawiki-extensions-Wikibase%20%22%3A%3ArebuildItemTerms%22&type=code vs https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/master/repo/maintenance/rebuildItemTerms.php

And back in the day this will have been done without update.php

The thing to do would be look at a DB you have not yet migrated, look in the update log table https://www.mediawiki.org/wiki/Manual:Updatelog_table for any string like rebuildItemTerms

I see an entry with ul_key | Wikibase\Repo\Store\Sql\DatabaseSchemaUpdater::rebuildPropertyTerms on one of my (non-updated) wikis on production updatelog table

So I think rebuildPropertyTerms is there from an earlier version

Looking at the hex in https://github.com/wbstack/api/blob/main/database/mw/new/mw1.43-wbs1.sql i believe the ItemTerms one is missing, and I believe it should be there for all newly created sites using this schema (and I guess it was missing from some previous too?)

It could be an issue when the SQL was generated, but perhaps not, especially if it is missing from multiple versions.
It could be that something leads to wikibase not adding the row at install time?
Or it also could be that indeed this site, and others of your sites dont have item terms populated in these tables? 1.39 -> 1.43 is a big jump, and it'll be easier to check in your DBs I expect than speculate what might have happened :)

Change #1215194 had a related patch set uploaded (by Addshore; author: Addshore):

[mediawiki/extensions/Wikibase@master] Add rebuildItemTerms log on a fresh install

https://gerrit.wikimedia.org/r/1215194

Change #1215194 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Add rebuildItemTerms log on a fresh install

https://gerrit.wikimedia.org/r/1215194

Change #1216829 had a related patch set uploaded (by Tarrow; author: Addshore):

[mediawiki/extensions/Wikibase@REL1_43] Add rebuildItemTerms log on a fresh install

https://gerrit.wikimedia.org/r/1216829

Change #1216829 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@REL1_43] Add rebuildItemTerms log on a fresh install

https://gerrit.wikimedia.org/r/1216829

Change #1218786 had a related patch set uploaded (by Tarrow; author: Addshore):

[mediawiki/extensions/Wikibase@REL1_44] Add rebuildItemTerms log on a fresh install

https://gerrit.wikimedia.org/r/1218786

Change #1218786 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@REL1_44] Add rebuildItemTerms log on a fresh install

https://gerrit.wikimedia.org/r/1218786

Change #1224954 had a related patch set uploaded (by Ollie Shotton; author: Addshore):

[mediawiki/extensions/Wikibase@REL1_45] Add rebuildItemTerms log on a fresh install

https://gerrit.wikimedia.org/r/1224954

Change #1224954 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@REL1_45] Add rebuildItemTerms log on a fresh install

https://gerrit.wikimedia.org/r/1224954