Change Details

I've temporarily upgraded mwdebug2001 to a HHVM build with a ICU 57 backport and ran the updateCollation script in dry run mode to check which wikis with any form of UCA collation are affected and it turns out almost all of them need the migration. In fact, of the ten biggest Wikipedias, only cebwiki and dewiki //do not// need migration. Here's the rundown sorted by shards: s1: [] enwiki s2: [x] cswiki [x] fiwiki [x] itwiki [x] nlwiki [x] nowiki [x] plwiki [] ptwiki [] svwiki [] thwiki s3: [x] be_x_oldwiki [x] bewiki [x] bewikisource [x] bswiki [x] ckbwiki [x] cswiktionary [x] cywiki [x] cywikibooks [x] cywikiquote [x] cywikisource [x] cywiktionary [x] dewikisource [x] eswikiversity [x] etwiki [x] etwikibooks [x] etwikimedia [x] etwikiquote [x] etwikisource [x] etwiktionary [x] fawikibooks [x] fawikinews [x] fawikiquote [x] fawikisource [x] fawiktionary [x] fiwikibooks [x] fiwikimedia [x] fiwikinews [x] fiwikiquote [x] fiwikisource [x] fiwikiversity [x] fiwikivoyage [x] frwikibooks [x] frwikinews [x] frwikiversity [x] gdwiki [x] glwiki [x] hrwiki [x] hsbwiki [x] ilowiki [x] iswiki [x] ltwiki [x] lvwiki [x] mediawikiwiki [x] mkwiki [x] nowikimedia [x] olowiki [x] plwikisource [x] plwikivoyage [x] plwiktionary [x] ptwikibooks [x] rowikibooks [x] rowikinews [x] rowikiquote [x] rowikisource [x] rowikivoyage [x] rowiktionary [x] rswikimedia [x] ruwikibooks [x] ruwikinews [x] ruwikiquote [x] ruwikisource [x] ruwikiversity [x] ruwikivoyage [x] ruwiktionary [x] shwiki [x] skwiki [x] srwiki [x] srwikibooks [x] srwikinews [x] srwikiquote [x] srwikisource [x] srwiktionary [x] svwikisource [x] tawiki [x] tawikibooks [x] tawikinews [x] tawikiquote [x] tawikisource [x] tawiktionary [x] testwiki [x] thwikibooks [x] thwikinews [x] thwikiquote [x] thwikisource [x] thwiktionary [x] uawikimedia [x] ukwikibooks [x] ukwikinews [x] ukwikiquote [x] ukwikisource [x] ukwikivoyage [x] ukwiktionary [x] viwikibooks [x] viwikiquote [x] viwikisource [x] viwikivoyage [x] viwiktionary s6: [x] frwiki [x] ruwiki s7: [x] eswiki [x] fawiki [x] frwiktionary [x] huwiki [x] rowiki [x] ukwiki [] viwiki The migration causes some unavoidable user-visible impact: The sorting of some category pages will be distorted; all pages which have been updated with the new HHVM build using ICU 57 will use the new sorting while untouched pages use the old sorting. As such, this change needs to be coordinated with Community Liaisons. These sorting problems will only be fixed once the updateCollation maintenance script [1] has completed to run for an affected wiki. For past collation changes that took e.g. four hours in 2016 for Swedish Wikipedia and six days for English Wikipedia [2]. Current open questions: - Discuss the possible concurrency of updateCollation runs per shard with DBAs, in particular for wikis in s2 to minimise user visible impact - Check the head start Community Liaisons needs to prepare user notifications The plan for the actual migration would look like this: - Merge a patch to enable component/icu57 for our jessie-based mediawiki app servers - Migrate the canaries to the new HHVM build and keep an eye on logs/metrics for an hour (new packages are running in beta already for about a month, but nothing beats production traffic) - If all is well, upgrade HHVM on the remaining app servers in eqiad and terbium - Initiate updateCollation runs on terbium - Upgrade app servers in codfw Once we've migrated to ICU 57 this unblocks our migration of the app servers to Debian stretch (ICU 57 is backported from stretch) (T174431) and allows us to use a more recent version of Unicode in Mediawiki (T188480) Footnotes: [1] https://www.mediawiki.org/wiki/Manual:UpdateCollation.php [2] https://phabricator.wikimedia.org/T146675#2668367

I've temporarily upgraded mwdebug2001 to a HHVM build with a ICU 57 backport and ran the updateCollation script in dry run mode to check which wikis with any form of UCA collation are affected and it turns out almost all of them need the migration. In fact, of the ten biggest Wikipedias, only cebwiki and dewiki //do not// need migration. Here's the rundown sorted by shards: s1: [] enwiki s2: [x] cswiki [x] fiwiki [x] itwiki [x] nlwiki [x] nowiki [x] plwiki [] ptwiki [] svwiki [] thwiki s3: [x] be_x_oldwiki [x] bewiki [x] bewikisource [x] bswiki [x] ckbwiki [x] cswiktionary [x] cywiki [x] cywikibooks [x] cywikiquote [x] cywikisource [x] cywiktionary [x] dewikisource [x] eswikiversity [x] etwiki [x] etwikibooks [x] etwikimedia [x] etwikiquote [x] etwikisource [x] etwiktionary [x] fawikibooks [x] fawikinews [x] fawikiquote [x] fawikisource [x] fawiktionary [x] fiwikibooks [x] fiwikimedia [x] fiwikinews [x] fiwikiquote [x] fiwikisource [x] fiwikiversity [x] fiwikivoyage [x] frwikibooks [x] frwikinews [x] frwikiversity [x] gdwiki [x] glwiki [x] hrwiki [x] hsbwiki [x] ilowiki [x] iswiki [x] ltwiki [x] lvwiki [x] mediawikiwiki [x] mkwiki [x] nowikimedia [x] olowiki [x] plwikisource [x] plwikivoyage [x] plwiktionary [x] ptwikibooks [x] rowikibooks [x] rowikinews [x] rowikiquote [x] rowikisource [x] rowikivoyage [x] rowiktionary [x] rswikimedia [x] ruwikibooks [x] ruwikinews [x] ruwikiquote [x] ruwikisource [x] ruwikiversity [x] ruwikivoyage [x] ruwiktionary [x] shwiki [x] skwiki [x] srwiki [x] srwikibooks [x] srwikinews [x] srwikiquote [x] srwikisource [x] srwiktionary [x] svwikisource [x] tawiki [x] tawikibooks [x] tawikinews [x] tawikiquote [x] tawikisource [x] tawiktionary [x] testwiki [x] thwikibooks [x] thwikinews [x] thwikiquote [x] thwikisource [x] thwiktionary [x] uawikimedia [x] ukwikibooks [x] ukwikinews [x] ukwikiquote [x] ukwikisource [x] ukwikivoyage [x] ukwiktionary [x] viwikibooks [x] viwikiquote [x] viwikisource [x] viwikivoyage [x] viwiktionary s6: [x] frwiki [x] ruwiki s7: [x] eswiki [x] fawiki [x] frwiktionary [x] huwiki [x] rowiki [x] ukwiki [x] viwiki The migration causes some unavoidable user-visible impact: The sorting of some category pages will be distorted; all pages which have been updated with the new HHVM build using ICU 57 will use the new sorting while untouched pages use the old sorting. As such, this change needs to be coordinated with Community Liaisons. These sorting problems will only be fixed once the updateCollation maintenance script [1] has completed to run for an affected wiki. For past collation changes that took e.g. four hours in 2016 for Swedish Wikipedia and six days for English Wikipedia [2]. Current open questions: - Discuss the possible concurrency of updateCollation runs per shard with DBAs, in particular for wikis in s2 to minimise user visible impact - Check the head start Community Liaisons needs to prepare user notifications The plan for the actual migration would look like this: - Merge a patch to enable component/icu57 for our jessie-based mediawiki app servers - Migrate the canaries to the new HHVM build and keep an eye on logs/metrics for an hour (new packages are running in beta already for about a month, but nothing beats production traffic) - If all is well, upgrade HHVM on the remaining app servers in eqiad and terbium - Initiate updateCollation runs on terbium - Upgrade app servers in codfw Once we've migrated to ICU 57 this unblocks our migration of the app servers to Debian stretch (ICU 57 is backported from stretch) (T174431) and allows us to use a more recent version of Unicode in Mediawiki (T188480) Footnotes: [1] https://www.mediawiki.org/wiki/Manual:UpdateCollation.php [2] https://phabricator.wikimedia.org/T146675#2668367

I've temporarily upgraded mwdebug2001 to a HHVM build with a ICU 57 backport and ran the updateCollation script in dry run mode to check which wikis with any form of UCA collation are affected and it turns out almost all of them need the migration. In fact, of the ten biggest Wikipedias, only cebwiki and dewiki //do not// need migration. Here's the rundown sorted by shards: s1: [] enwiki s2: [x] cswiki [x] fiwiki [x] itwiki [x] nlwiki [x] nowiki [x] plwiki [] ptwiki [] svwiki [] thwiki s3: [x] be_x_oldwiki [x] bewiki [x] bewikisource [x] bswiki [x] ckbwiki [x] cswiktionary [x] cywiki [x] cywikibooks [x] cywikiquote [x] cywikisource [x] cywiktionary [x] dewikisource [x] eswikiversity [x] etwiki [x] etwikibooks [x] etwikimedia [x] etwikiquote [x] etwikisource [x] etwiktionary [x] fawikibooks [x] fawikinews [x] fawikiquote [x] fawikisource [x] fawiktionary [x] fiwikibooks [x] fiwikimedia [x] fiwikinews [x] fiwikiquote [x] fiwikisource [x] fiwikiversity [x] fiwikivoyage [x] frwikibooks [x] frwikinews [x] frwikiversity [x] gdwiki [x] glwiki [x] hrwiki [x] hsbwiki [x] ilowiki [x] iswiki [x] ltwiki [x] lvwiki [x] mediawikiwiki [x] mkwiki [x] nowikimedia [x] olowiki [x] plwikisource [x] plwikivoyage [x] plwiktionary [x] ptwikibooks [x] rowikibooks [x] rowikinews [x] rowikiquote [x] rowikisource [x] rowikivoyage [x] rowiktionary [x] rswikimedia [x] ruwikibooks [x] ruwikinews [x] ruwikiquote [x] ruwikisource [x] ruwikiversity [x] ruwikivoyage [x] ruwiktionary [x] shwiki [x] skwiki [x] srwiki [x] srwikibooks [x] srwikinews [x] srwikiquote [x] srwikisource [x] srwiktionary [x] svwikisource [x] tawiki [x] tawikibooks [x] tawikinews [x] tawikiquote [x] tawikisource [x] tawiktionary [x] testwiki [x] thwikibooks [x] thwikinews [x] thwikiquote [x] thwikisource [x] thwiktionary [x] uawikimedia [x] ukwikibooks [x] ukwikinews [x] ukwikiquote [x] ukwikisource [x] ukwikivoyage [x] ukwiktionary [x] viwikibooks [x] viwikiquote [x] viwikisource [x] viwikivoyage [x] viwiktionary s6: [x] frwiki [x] ruwiki s7: [x] eswiki [x] fawiki [x] frwiktionary [x] huwiki [x] rowiki [x] ukwiki [x] viwiki The migration causes some unavoidable user-visible impact: The sorting of some category pages will be distorted; all pages which have been updated with the new HHVM build using ICU 57 will use the new sorting while untouched pages use the old sorting. As such, this change needs to be coordinated with Community Liaisons. These sorting problems will only be fixed once the updateCollation maintenance script [1] has completed to run for an affected wiki. For past collation changes that took e.g. four hours in 2016 for Swedish Wikipedia and six days for English Wikipedia [2]. Current open questions: - Discuss the possible concurrency of updateCollation runs per shard with DBAs, in particular for wikis in s2 to minimise user visible impact - Check the head start Community Liaisons needs to prepare user notifications The plan for the actual migration would look like this: - Merge a patch to enable component/icu57 for our jessie-based mediawiki app servers - Migrate the canaries to the new HHVM build and keep an eye on logs/metrics for an hour (new packages are running in beta already for about a month, but nothing beats production traffic) - If all is well, upgrade HHVM on the remaining app servers in eqiad and terbium - Initiate updateCollation runs on terbium - Upgrade app servers in codfw Once we've migrated to ICU 57 this unblocks our migration of the app servers to Debian stretch (ICU 57 is backported from stretch) (T174431) and allows us to use a more recent version of Unicode in Mediawiki (T188480) Footnotes: [1] https://www.mediawiki.org/wiki/Manual:UpdateCollation.php [2] https://phabricator.wikimedia.org/T146675#2668367