Page MenuHomePhabricator

Upgrade the MediaWiki servers to ICU 72 (no table swap)
Closed, ResolvedPublic

Description

Goal

In order to upgrade all our MediaWiki clusters to Debian Bookworm, we need to upgrade the current Debian Bullseye installation to use ICU 72.

We will perform the upgrade on Monday, 20th April.

We will update the categorylinks table sortkeys in place (not using the new process, per T419049#11796113). This will take a few days to run on larger wikis, and up to a week on enwiki. Until the script is finished, there will be some weirdness in category lists sorting.

Affected wikis

s1:
  enwiki
s2:
  cswiki fiwiki itwiki nlwiki nowiki plwiki ptwiki svwiki thwiki trwiki
s3:
  mediawikiwiki arwiktionary azwiki azwikibooks azwikiquote azwikisource azwiktionary be_x_oldwiki bewiki bewikisource bswiki bswiktionary bswikiquote cswiktionary cywiki cywikibooks cywikiquote cywikisource cywiktionary dewikisource eswikiversity etwiki etwikibooks etwikimedia etwikiquote etwikisource etwiktionary fawikisource fawiktionary fawikibooks fawikinews fawikiquote fiwikibooks fiwikimedia fiwikinews fiwikiquote fiwikisource fiwikiversity fiwikivoyage frwikibooks frwikinews frwikiversity gawiki gdwiki gewikimedia glwiki grwikimedia hrwiki hsbwiki idwikimedia id_internalwikimedia ilowiki iswiki kkwiki kuwiki kuwiktionary kuwikibooks kuwikiquote lbwiki ltwiki lvwiki mkwiki napwikisource nnwiki nowikimedia olowiki plwikiquote plwikisource plwikivoyage plwiktionary ptwikibooks romdwikimedia rowikibooks rowikinews rowikiquote rowikisource rowikivoyage rowiktionary rswikimedia ruwikibooks ruwikinews ruwikiquote ruwikisource ruwikiversity ruwikivoyage ruwiktionary sewiki simplewiki slwiki slwikibooks slwikiquote slwikisource slwikiversity slwiktionary sqwiki sqwikibooks sqwikinews sqwikiquote srwikibooks srwikinews srwikiquote srwikisource srwiktionary skwiki svwikisource tawiki tawikibooks tawikinews tawikiquote tawikisource tawiktionary testwiki thwikibooks thwikinews thwikiquote thwikisource thwiktionary trwikiquote trwiktionary uawikimedia ukwikibooks ukwikinews ukwikiquote ukwikisource ukwikivoyage ukwiktionary viwikibooks viwikiquote viwikisource viwikivoyage viwiktionary
s5:
  azwikimedia cswikivoyage shwiki srwiki trwikivoyage
s6:
  frwiki ruwiki
s7:
  eswiki fawiki frwiktionary huwiki rowiki ukwiki viwiki

Runbook

  • Prepare packages + base images: T419058
  • Notify communities with MoveComms help
  • prepare dblists for use with updateCollation.php with find_collations.py
  • Migration Day
    • merge MW image change & wait for build
    • during MW infra window so nobody else deploys: scap sync-world --k8s-only --pause-after-testserver-sync -D full_image_build:true, then hope that nothing explodes on the canaries, then deploy everywhere
    • start updateCollation.php scripts:
      • mwscript-k8s --comment 'migrate collation data to ICU72 -- T422544' --local_dblist ./icu-s1.dblist --follow -- updateCollation.php --force
      • mwscript-k8s --comment 'migrate collation data to ICU72 -- T422544' --local_dblist ./icu-s2.dblist --follow -- updateCollation.php --force
      • mwscript-k8s --comment 'migrate collation data to ICU72 -- T422544' --local_dblist ./icu-s3.dblist --follow -- updateCollation.php --force
      • mwscript-k8s --comment 'migrate collation data to ICU72 -- T422544' --local_dblist ./icu-s5.dblist --follow -- updateCollation.php --force
      • mwscript-k8s --comment 'migrate collation data to ICU72 -- T422544' --local_dblist ./icu-s6.dblist --follow -- updateCollation.php --force
      • mwscript-k8s --comment 'migrate collation data to ICU72 -- T422544' --local_dblist ./icu-s7.dblist --follow -- updateCollation.php --force
  • Check that migration scripts completed successfully:
    • s1
    • s2
    • s3
    • s5
    • s6
    • s7
  • Include the icu72 upgrade into the general php images => T424214

Out of scope

  • Upgrading ICU on deployment hosts: tracked separately in T422964.
  • Upgrading ICU on Beta.

Details

Other Assignee
• Scott_French

Event Timeline

Raine changed the task status from Open to In Progress.Apr 7 2026, 7:37 PM
Raine claimed this task.
Raine triaged this task as Medium priority.
Raine updated Other Assignee, added: • Scott_French.

@Trizek-WMF as discussed, we can use the same announcements referenced in T420748#11779097. Thank you for your help!

@Raine Benoit is out of office, I would appreciate it if you help with the wording of the entry.

You can look at this and see if it works, I took it from the task description:

As part of infrastructure upgrades to support Debian Bookworm, all MediaWiki clusters will be updated from Debian Bullseye to use ICU 72 on Monday, 20 April. This includes an in-place update of the categorylinks table sortkeys. The update will take several days on larger wikis and up to a week on English Wikipedia, during which users may notice temporary inconsistencies in category list sorting; normal behavior will resume once the process is complete.

@Trizek-WMF as discussed, we can use the same announcements referenced in T420748#11779097. Thank you for your help!

How about:

  • for Tech News:

Due to a library upgrade, listings on category pages may be displayed out of order starting on Monday, 20th April. A migration script will be run to correct this, and will take hours to days depending on the size of the wiki (up to a week for enwiki). You can read more at $phab.

Or do you think it's too brief?

  • for VPT:

Hi everyone,

Starting on Monday, 20th April 2026, the Wikimedia Foundation Site Reliability Engineering team will migrate MediaWiki application servers to a new release of the ICU Unicode library (to version 72). This unblocks some future work on upgrading the servers to a new Operating System release and will also allow the use of improved internationalisation in the future (as wikis will then be able to use features introduced by the new ICU release such as new collation definitions; this will also allow us to use a more recent version of Unicode in MediaWiki).

This migration will cause some unavoidable temporary user-visible impact: The sorting of some category pages will be distorted – all pages which have been updated with the new software version will use the new sorting while untouched pages still use the old sorting. As such, SRE need to run a maintenance script to update the sorting for old entries.

The distortions may last from a few hours (on medium-sized wikis), up to a few days (on the largest wikis), and a up to a week on English Wikipedia. The start-time will depend upon when the migration script reaches each wiki.

The detailed list and the task for the technical implementation is at T422544. If this message seems familiar, it's because we've done similar updates in the past, the latest one in 2023.

Potential updates will be posted following this message.

This operation will be announced in Tech News. Other impacted wikis will also have a similar message. Please share this message where it needs to be posted!

(You can link to this ticket, I will keep it up to date.)

What do you think?

I'll post at en:WP:VPT on the 17th.

"Starting on November 2, 2023" appears to be a typo, unless you own a time machine ;)

"Starting on November 2, 2023" appears to be a typo, unless you own a time machine ;)

Oops, you're right! I only built my time machine this year, so it doesn't go back to 2023 :P

Thanks <3

I'll post at en:WP:VPT on the 17th.

Posted; thank you for the updated message!

So far so good, updateCollations script is finished on everything but s1 and s2, so those sections take about a day (21 - 28h).

Update scripts all finished succesfully. Marking as resolved, as the remaining follow-up task is not user-visible.