Page MenuHomePhabricator

Migrate all old DB rows from windows-1252 to UTF-8 on enwiki
Closed, ResolvedPublic

Event Timeline

Change 383012 had a related patch set uploaded (by Zoranzoki21; owner: Zoranzoki21):
[operations/mediawiki-config@master] Migrate all old DB rows from windows-1252 to UTF-8 on several wikis:

https://gerrit.wikimedia.org/r/383012

Change 383012 abandoned by Zoranzoki21:
Migrate all old DB rows from windows-1252 to UTF-8 on several wikis:

Reason:
I will abandon this change. Told me please to restore this patch, if it will be need. Sorry for much emails about this.

https://gerrit.wikimedia.org/r/383012

Ladsgroup added a project: DBA.
Ladsgroup subscribed.

I'll be running this next week Monday.

root@dbstore1003:/srv/sqldata.s1/enwiki# ls -Ssh | grep -i text
 46G text.ibd
4.0K text.frm

Total (of enwiki) 1.1T

mysql:research@s1-analytics-replica.eqiad.wmnet [enwiki]> select max(old_id) from text where old_id < 200000000 and not old_flags like '%external%' limit 50;
+-------------+
| max(old_id) |
+-------------+
|   109744978 |
+-------------+
1 row in set (2 min 37.524 sec)

mysql:research@s1-analytics-replica.eqiad.wmnet [enwiki]> select max(old_id) from text where old_id < 200000000 and not old_flags like '%utf-8%' and not old_flags like '%utf8%' limit 50;
+-------------+
| max(old_id) |
+-------------+
|    30867767 |
+-------------+
1 row in set (2 min 1.201 sec)

mysql:research@s1-analytics-replica.eqiad.wmnet [enwiki]> select old_flags, count(*) from text where old_id < 200000000 group by old_flags limit 50;
+---------------------+-----------+
| old_flags           | count(*)  |
+---------------------+-----------+
|                     |      1053 |
| 0                   |        22 |
| external,utf-8      | 115810372 |
| gzip                |    506567 |
| object              |   1831663 |
| object,utf-8        |      1800 |
| utf-8,gzip          |   2521186 |
| utf-8,gzip,external |  78284822 |
+---------------------+-----------+
8 rows in set (9 min 40.183 sec)

Mentioned in SAL (#wikimedia-operations) [2023-06-12T02:51:27Z] <Amir1> ladsgroup@mwmaint1002:~$ mwscript maintenance/storage/moveToExternal.php --wiki=enwiki --end 32000000 --undo /home/ladsgroup/T128151.undo.sql --iconv DB cluster27 (T128151)

Ran up to 32M, Moved 4.5M edits out of text table:

oldid=31900001, moved=4505322

And most stubs were also cleaned up:

1858 of 2368 stubs resolved

Mentioned in SAL (#wikimedia-operations) [2023-06-12T10:40:54Z] <Amir1> mwscript maintenance/storage/moveToExternal.php --wiki=enwiki --start 31000000 --end 110000000 --undo /home/ladsgroup/T128151.undo.sql --iconv DB cluster27 (T128151)

The second batch:

oldid=110000000, moved=339379
Resolving 234 stubs
100 stubs processed
200 stubs processed
234 of 234 stubs resolved

Now I need to deal with edge cases

Only 510 rows left:

Resolving 510 stubs
Error at old_id 9452844: target 9300930 is a CGZ pointer
Error at old_id 9456361: target 8944420 is a CGZ pointer
Error at old_id 9459452: target 9300930 is a CGZ pointer
Error at old_id 9459555: target 9300930 is a CGZ pointer
Error at old_id 9471959: target 8505221 is a CGZ pointer
Error at old_id 9509611: target 9300930 is a CGZ pointer
Error at old_id 9555610: target 9300930 is a CGZ pointer
Error at old_id 9562362: target 9300930 is a CGZ pointer
Error at old_id 9597530: can't find main text row old_id 741958
Error at old_id 9597531: can't find main text row old_id 741958
Error at old_id 9597532: can't find main text row old_id 741958
Error at old_id 9597533: can't find main text row old_id 741958
Error at old_id 9597534: can't find main text row old_id 741958
Error at old_id 9597535: can't find main text row old_id 741958
Error at old_id 9597536: can't find main text row old_id 741958
Error at old_id 9597537: can't find main text row old_id 741958
Error at old_id 9597538: can't find main text row old_id 741958
Error at old_id 9597539: can't find main text row old_id 741958
Error at old_id 9597540: can't find main text row old_id 741958
Error at old_id 9597541: can't find main text row old_id 741958
Error at old_id 9618362: target 9292452 is a CGZ pointer
Error at old_id 9733208: target 9292452 is a CGZ pointer
Error at old_id 9751729: target 11634650 is a CGZ pointer
Error at old_id 9752546: target 9336211 is a CGZ pointer
Error at old_id 9767540: target 9767230 is a CGZ pointer
Error at old_id 9773867: target 9767230 is a CGZ pointer
Error at old_id 9824188: target 11634650 is a CGZ pointer
Error at old_id 9824232: target 11634650 is a CGZ pointer
Error at old_id 9824539: target 11634650 is a CGZ pointer
Error at old_id 9826008: target 9669466 is a CGZ pointer
Error at old_id 9826346: target 9669466 is a CGZ pointer
Error at old_id 9826429: target 9669466 is a CGZ pointer
Error at old_id 9841318: target 9669466 is a CGZ pointer
Error at old_id 9888682: target 9336211 is a CGZ pointer
Error at old_id 9893031: target 9669466 is a CGZ pointer
Error at old_id 9999644: target 11634650 is a CGZ pointer
Error at old_id 10052102: target 9965886 is a CGZ pointer
Error at old_id 10054845: target 1112946 is a CGZ pointer
Error at old_id 10054868: target 1112946 is a CGZ pointer
Error at old_id 10197208: target 9292452 is a CGZ pointer
Error at old_id 10197385: target 9292452 is a CGZ pointer
Error at old_id 10198524: target 9669466 is a CGZ pointer
Error at old_id 10233693: can't find main text row old_id 3513154
Error at old_id 10233694: can't find main text row old_id 3513154
Error at old_id 10233765: can't find main text row old_id 6135871
Error at old_id 10263396: target 10263414 is a CGZ pointer
Error at old_id 10263423: target 10263434 is a CGZ pointer
Error at old_id 10263457: target 10263474 is a CGZ pointer
Error at old_id 10263461: target 10263474 is a CGZ pointer
Error at old_id 10263480: target 10263494 is a CGZ pointer
Error at old_id 10263482: target 10263494 is a CGZ pointer
Error at old_id 10263483: target 10263494 is a CGZ pointer
Error at old_id 10263485: target 10263494 is a CGZ pointer
Error at old_id 10263533: target 10263534 is a CGZ pointer
Error at old_id 10263562: target 10263574 is a CGZ pointer
Error at old_id 10263563: target 10263574 is a CGZ pointer
Error at old_id 10263564: target 10263574 is a CGZ pointer
Error at old_id 10263587: target 10263594 is a CGZ pointer
Error at old_id 10263590: target 10263594 is a CGZ pointer
Error at old_id 10263591: target 10263594 is a CGZ pointer
Error at old_id 10263592: target 10263594 is a CGZ pointer
Error at old_id 10263596: target 10263614 is a CGZ pointer
Error at old_id 10263604: target 10263614 is a CGZ pointer
Error at old_id 10277636: target 9669466 is a CGZ pointer
Error at old_id 10294659: can't find main text row old_id 2805844
Error at old_id 10294660: can't find main text row old_id 2805844
Error at old_id 10294923: can't find main text row old_id 2750975
Error at old_id 10294939: can't find main text row old_id 8893895
Error at old_id 10295112: can't find main text row old_id 4892554
Error at old_id 10295113: can't find main text row old_id 4892554
Error at old_id 10295114: can't find main text row old_id 4892554
Error at old_id 10295246: can't find main text row old_id 7239783
Error at old_id 10295247: can't find main text row old_id 7239783
Error at old_id 10295248: can't find main text row old_id 7239783
Error at old_id 10295249: can't find main text row old_id 7239783
Error at old_id 10295251: can't find main text row old_id 9184072
Error at old_id 10295336: can't find main text row old_id 7241593
Error at old_id 10352923: target 10105048 is a CGZ pointer
Error at old_id 10357238: target 10186985 is a CGZ pointer
Error at old_id 10415676: target 10404042 is a CGZ pointer
Error at old_id 10415797: target 10404042 is a CGZ pointer
Error at old_id 10415915: target 10404042 is a CGZ pointer
Error at old_id 10477917: target 9669466 is a CGZ pointer
Error at old_id 10500216: target 9292452 is a CGZ pointer
Error at old_id 10540048: can't find main text row old_id 2782896
Error at old_id 10595714: can't find main text row old_id 8100986
Error at old_id 10595715: can't find main text row old_id 8100986
Error at old_id 10595716: can't find main text row old_id 8100986
Error at old_id 10595717: can't find main text row old_id 8100986
Error at old_id 10595718: can't find main text row old_id 8100986
Error at old_id 10595719: can't find main text row old_id 8100986
Error at old_id 10595720: can't find main text row old_id 8100986
Error at old_id 10595721: can't find main text row old_id 8100986
Error at old_id 10595722: can't find main text row old_id 8100986
Error at old_id 10595723: can't find main text row old_id 8100986
Error at old_id 10595724: can't find main text row old_id 8100986
Error at old_id 10595725: can't find main text row old_id 8100986
Error at old_id 10595726: can't find main text row old_id 8100986
Error at old_id 10595727: can't find main text row old_id 8100986
Error at old_id 10595728: can't find main text row old_id 8100986
100 stubs processed
Error at old_id 10596643: target 9336211 is a CGZ pointer
Error at old_id 10608188: target 1112946 is a CGZ pointer
Error at old_id 10616763: target 9669466 is a CGZ pointer
Error at old_id 10617074: target 9669466 is a CGZ pointer
Error at old_id 10618709: target 9669466 is a CGZ pointer
Error at old_id 10622039: target 9292452 is a CGZ pointer
Error at old_id 10643383: can't find main text row old_id 2486471
Error at old_id 10679761: target 10322521 is a CGZ pointer
Error at old_id 10709985: target 10322521 is a CGZ pointer
Error at old_id 10836962: target 9669466 is a CGZ pointer
Error at old_id 10859653: target 11634650 is a CGZ pointer
Error at old_id 10863971: can't find main text row old_id 8422913
Error at old_id 10863972: can't find main text row old_id 8422913
Error at old_id 10863973: can't find main text row old_id 8422913
Error at old_id 10863974: can't find main text row old_id 8422913
Error at old_id 10863975: can't find main text row old_id 8422913
Error at old_id 10863976: can't find main text row old_id 8422913
Error at old_id 10863977: can't find main text row old_id 8422913
Error at old_id 10879642: can't find main text row old_id 8420692
Error at old_id 10879643: can't find main text row old_id 8420692
Error at old_id 10897670: target 9669466 is a CGZ pointer
Error at old_id 11034777: target 11034219 is a CGZ pointer
Error at old_id 11035336: target 11034219 is a CGZ pointer
Error at old_id 11036031: target 11034219 is a CGZ pointer
Error at old_id 11036990: target 11034219 is a CGZ pointer
Error at old_id 11040229: target 11034219 is a CGZ pointer
Error at old_id 11040386: target 11034219 is a CGZ pointer
Error at old_id 11040908: target 11034219 is a CGZ pointer
Error at old_id 11041411: target 11034219 is a CGZ pointer
Error at old_id 11042226: target 11034219 is a CGZ pointer
Error at old_id 11042412: target 11034219 is a CGZ pointer
Error at old_id 11042488: target 11034219 is a CGZ pointer
Error at old_id 11042715: target 11034219 is a CGZ pointer
Error at old_id 11042853: target 11034219 is a CGZ pointer
Error at old_id 11042876: target 11034219 is a CGZ pointer
Error at old_id 11100117: target 9669466 is a CGZ pointer
Error at old_id 11100224: target 9336211 is a CGZ pointer
Error at old_id 11100437: target 9669466 is a CGZ pointer
Error at old_id 11183828: target 11050404 is a CGZ pointer
Error at old_id 11243046: target 9336211 is a CGZ pointer
Error at old_id 11307944: target 1112946 is a CGZ pointer
Error at old_id 11307949: target 1112946 is a CGZ pointer
Error at old_id 11322282: target 11050404 is a CGZ pointer
Error at old_id 11378361: target 9336211 is a CGZ pointer
Error at old_id 11383576: target 11634650 is a CGZ pointer
Error at old_id 11386856: target 9336211 is a CGZ pointer
Error at old_id 11397897: target 9336211 is a CGZ pointer
Error at old_id 11397986: target 9336211 is a CGZ pointer
Error at old_id 11408322: target 10999000 is a CGZ pointer
Error at old_id 11418154: target 11358354 is a CGZ pointer
Error at old_id 11418162: target 11358354 is a CGZ pointer
Error at old_id 11446604: target 11358354 is a CGZ pointer
Error at old_id 11480354: target 11358354 is a CGZ pointer
Error at old_id 11488919: can't find main text row old_id 10303
Error at old_id 11488920: can't find main text row old_id 10303
Error at old_id 11488922: can't find main text row old_id 10303
Error at old_id 11488923: can't find main text row old_id 10303
Error at old_id 11488924: can't find main text row old_id 10303
Error at old_id 11488925: can't find main text row old_id 10303
Error at old_id 11488926: can't find main text row old_id 10303
Error at old_id 11488927: can't find main text row old_id 10303
Error at old_id 11488928: can't find main text row old_id 10303
Error at old_id 11488929: can't find main text row old_id 10303
Error at old_id 11488930: can't find main text row old_id 10303
Error at old_id 11488931: can't find main text row old_id 10303
Error at old_id 11488932: can't find main text row old_id 10303
Error at old_id 11488933: can't find main text row old_id 10303
Error at old_id 11488934: can't find main text row old_id 10303
Error at old_id 11488935: can't find main text row old_id 10303
Error at old_id 11488936: can't find main text row old_id 10303
Error at old_id 11488937: can't find main text row old_id 10303
Error at old_id 11488938: can't find main text row old_id 10303
Error at old_id 11488940: can't find main text row old_id 537123
Error at old_id 11488941: can't find main text row old_id 537123
Error at old_id 11488942: can't find main text row old_id 537123
Error at old_id 11488943: can't find main text row old_id 537123
Error at old_id 11488944: can't find main text row old_id 537123
Error at old_id 11488945: can't find main text row old_id 537123
Error at old_id 11488946: can't find main text row old_id 537123
Error at old_id 11488947: can't find main text row old_id 537123
Error at old_id 11488948: can't find main text row old_id 537123
Error at old_id 11488949: can't find main text row old_id 537123
Error at old_id 11488950: can't find main text row old_id 537123
Error at old_id 11488951: can't find main text row old_id 537123
Error at old_id 11488952: can't find main text row old_id 537123
Error at old_id 11488953: can't find main text row old_id 537123
Error at old_id 11488954: can't find main text row old_id 537123
Error at old_id 11488955: can't find main text row old_id 537123
Error at old_id 11488956: can't find main text row old_id 537123
Error at old_id 11488957: can't find main text row old_id 537123
Error at old_id 11488958: can't find main text row old_id 537123
Error at old_id 11488960: can't find main text row old_id 791185
Error at old_id 11488961: can't find main text row old_id 791185
Error at old_id 11488962: can't find main text row old_id 791185
Error at old_id 11488963: can't find main text row old_id 791185
Error at old_id 11488964: can't find main text row old_id 791185
Error at old_id 11488965: can't find main text row old_id 791185
Error at old_id 11488966: can't find main text row old_id 791185
Error at old_id 11488967: can't find main text row old_id 791185
Error at old_id 11488968: can't find main text row old_id 791185
200 stubs processed
Error at old_id 11488969: can't find main text row old_id 791185
Error at old_id 11488970: can't find main text row old_id 791185
Error at old_id 11488971: can't find main text row old_id 791185
Error at old_id 11488972: can't find main text row old_id 791185
Error at old_id 11488973: can't find main text row old_id 791185
Error at old_id 11488974: can't find main text row old_id 791185
Error at old_id 11488975: can't find main text row old_id 791185
Error at old_id 11488976: can't find main text row old_id 791185
Error at old_id 11488977: can't find main text row old_id 791185
Error at old_id 11488978: can't find main text row old_id 791185
Error at old_id 11488980: can't find main text row old_id 1372275
Error at old_id 11488981: can't find main text row old_id 1372275
Error at old_id 11488982: can't find main text row old_id 1372275
Error at old_id 11488983: can't find main text row old_id 1372275
Error at old_id 11488984: can't find main text row old_id 1372275
Error at old_id 11488985: can't find main text row old_id 1372275
Error at old_id 11488986: can't find main text row old_id 1372275
Error at old_id 11488987: can't find main text row old_id 1372275
Error at old_id 11488988: can't find main text row old_id 1372275
Error at old_id 11488989: can't find main text row old_id 1372275
Error at old_id 11488990: can't find main text row old_id 1372275
Error at old_id 11488991: can't find main text row old_id 1372275
Error at old_id 11488992: can't find main text row old_id 1372275
Error at old_id 11488993: can't find main text row old_id 1372275
Error at old_id 11488994: can't find main text row old_id 1372275
Error at old_id 11488995: can't find main text row old_id 1372275
Error at old_id 11488996: can't find main text row old_id 1372275
Error at old_id 11488997: can't find main text row old_id 1372275
Error at old_id 11488998: can't find main text row old_id 1372275
Error at old_id 11489000: can't find main text row old_id 1830185
Error at old_id 11489001: can't find main text row old_id 1830185
Error at old_id 11489002: can't find main text row old_id 1830185
Error at old_id 11489003: can't find main text row old_id 1830185
Error at old_id 11489004: can't find main text row old_id 1830185
Error at old_id 11489005: can't find main text row old_id 1830185
Error at old_id 11489006: can't find main text row old_id 1830185
Error at old_id 11489007: can't find main text row old_id 1830185
Error at old_id 11489008: can't find main text row old_id 1830185
Error at old_id 11489009: can't find main text row old_id 1830185
Error at old_id 11489010: can't find main text row old_id 1830185
Error at old_id 11489011: can't find main text row old_id 1830185
Error at old_id 11489012: can't find main text row old_id 1830185
Error at old_id 11489013: can't find main text row old_id 1830185
Error at old_id 11489014: can't find main text row old_id 1830185
Error at old_id 11489015: can't find main text row old_id 1830185
Error at old_id 11489016: can't find main text row old_id 1830185
Error at old_id 11489017: can't find main text row old_id 1830185
Error at old_id 11489018: can't find main text row old_id 1830185
Error at old_id 11489020: can't find main text row old_id 2280063
Error at old_id 11489021: can't find main text row old_id 2280063
Error at old_id 11489022: can't find main text row old_id 2280063
Error at old_id 11489023: can't find main text row old_id 2280063
Error at old_id 11489024: can't find main text row old_id 2280063
Error at old_id 11489025: can't find main text row old_id 2280063
Error at old_id 11489026: can't find main text row old_id 2280063
Error at old_id 11489027: can't find main text row old_id 2280063
Error at old_id 11489028: can't find main text row old_id 2280063
Error at old_id 11489029: can't find main text row old_id 2280063
Error at old_id 11489030: can't find main text row old_id 2280063
Error at old_id 11489031: can't find main text row old_id 2280063
Error at old_id 11489032: can't find main text row old_id 2280063
Error at old_id 11489033: can't find main text row old_id 2280063
Error at old_id 11489034: can't find main text row old_id 2280063
Error at old_id 11489035: can't find main text row old_id 2280063
Error at old_id 11489036: can't find main text row old_id 2280063
Error at old_id 11489037: can't find main text row old_id 2280063
Error at old_id 11489038: can't find main text row old_id 2280063
Error at old_id 11489040: can't find main text row old_id 3168470
Error at old_id 11489041: can't find main text row old_id 3168470
Error at old_id 11489042: can't find main text row old_id 3168470
Error at old_id 11489043: can't find main text row old_id 3168470
Error at old_id 11489044: can't find main text row old_id 3168470
Error at old_id 11489045: can't find main text row old_id 3168470
Error at old_id 11489046: can't find main text row old_id 3168470
Error at old_id 11489047: can't find main text row old_id 3168470
Error at old_id 11489048: can't find main text row old_id 3168470
Error at old_id 11489049: can't find main text row old_id 3168470
Error at old_id 11489050: can't find main text row old_id 3168470
Error at old_id 11489051: can't find main text row old_id 3168470
Error at old_id 11489052: can't find main text row old_id 3168470
Error at old_id 11489053: can't find main text row old_id 3168470
Error at old_id 11489054: can't find main text row old_id 3168470
Error at old_id 11489055: can't find main text row old_id 3168470
Error at old_id 11489056: can't find main text row old_id 3168470
Error at old_id 11489057: can't find main text row old_id 3168470
Error at old_id 11489058: can't find main text row old_id 3168470
Error at old_id 11489060: can't find main text row old_id 3936324
Error at old_id 11489061: can't find main text row old_id 3936324
Error at old_id 11489062: can't find main text row old_id 3936324
Error at old_id 11489063: can't find main text row old_id 3936324
Error at old_id 11489064: can't find main text row old_id 3936324
Error at old_id 11489065: can't find main text row old_id 3936324
Error at old_id 11489066: can't find main text row old_id 3936324
Error at old_id 11489067: can't find main text row old_id 3936324
Error at old_id 11489068: can't find main text row old_id 3936324
Error at old_id 11489069: can't find main text row old_id 3936324
Error at old_id 11489070: can't find main text row old_id 3936324
Error at old_id 11489071: can't find main text row old_id 3936324
Error at old_id 11489072: can't find main text row old_id 3936324
Error at old_id 11489073: can't find main text row old_id 3936324
300 stubs processed
Error at old_id 11489074: can't find main text row old_id 3936324
Error at old_id 11489075: can't find main text row old_id 3936324
Error at old_id 11489076: can't find main text row old_id 3936324
Error at old_id 11489077: can't find main text row old_id 3936324
Error at old_id 11489078: can't find main text row old_id 3936324
Error at old_id 11489080: can't find main text row old_id 4412580
Error at old_id 11489081: can't find main text row old_id 4412580
Error at old_id 11489082: can't find main text row old_id 4412580
Error at old_id 11489083: can't find main text row old_id 4412580
Error at old_id 11489084: can't find main text row old_id 4412580
Error at old_id 11489085: can't find main text row old_id 4412580
Error at old_id 11489086: can't find main text row old_id 4412580
Error at old_id 11489087: can't find main text row old_id 4412580
Error at old_id 11489088: can't find main text row old_id 4412580
Error at old_id 11489089: can't find main text row old_id 4412580
Error at old_id 11489090: can't find main text row old_id 4412580
Error at old_id 11489091: can't find main text row old_id 4412580
Error at old_id 11489092: can't find main text row old_id 4412580
Error at old_id 11489093: can't find main text row old_id 4412580
Error at old_id 11489094: can't find main text row old_id 4412580
Error at old_id 11489095: can't find main text row old_id 4412580
Error at old_id 11489096: can't find main text row old_id 4412580
Error at old_id 11489097: can't find main text row old_id 4412580
Error at old_id 11489098: can't find main text row old_id 4412580
Error at old_id 11489100: can't find main text row old_id 5321268
Error at old_id 11489101: can't find main text row old_id 5321268
Error at old_id 11489102: can't find main text row old_id 5321268
Error at old_id 11489103: can't find main text row old_id 5321268
Error at old_id 11489104: can't find main text row old_id 5321268
Error at old_id 11489105: can't find main text row old_id 5321268
Error at old_id 11489106: can't find main text row old_id 5321268
Error at old_id 11489107: can't find main text row old_id 5321268
Error at old_id 11489108: can't find main text row old_id 5321268
Error at old_id 11489109: can't find main text row old_id 5321268
Error at old_id 11489110: can't find main text row old_id 5321268
Error at old_id 11489111: can't find main text row old_id 5321268
Error at old_id 11489112: can't find main text row old_id 5321268
Error at old_id 11489113: can't find main text row old_id 5321268
Error at old_id 11489114: can't find main text row old_id 5321268
Error at old_id 11489115: can't find main text row old_id 5321268
Error at old_id 11489116: can't find main text row old_id 5321268
Error at old_id 11489117: can't find main text row old_id 5321268
Error at old_id 11489118: can't find main text row old_id 5321268
Error at old_id 11489120: can't find main text row old_id 6679388
Error at old_id 11489121: can't find main text row old_id 6679388
Error at old_id 11489122: can't find main text row old_id 6679388
Error at old_id 11489123: can't find main text row old_id 6679388
Error at old_id 11489124: can't find main text row old_id 6679388
Error at old_id 11489125: can't find main text row old_id 6679388
Error at old_id 11489126: can't find main text row old_id 6679388
Error at old_id 11489127: can't find main text row old_id 6679388
Error at old_id 11489128: can't find main text row old_id 6679388
Error at old_id 11489129: can't find main text row old_id 6679388
Error at old_id 11489130: can't find main text row old_id 6679388
Error at old_id 11489131: can't find main text row old_id 6679388
Error at old_id 11489132: can't find main text row old_id 6679388
Error at old_id 11489133: can't find main text row old_id 6679388
Error at old_id 11489134: can't find main text row old_id 6679388
Error at old_id 11489135: can't find main text row old_id 6679388
Error at old_id 11489136: can't find main text row old_id 6679388
Error at old_id 11489137: can't find main text row old_id 6679388
Error at old_id 11489138: can't find main text row old_id 6679388
Error at old_id 11489140: can't find main text row old_id 7504489
Error at old_id 11489141: can't find main text row old_id 7504489
Error at old_id 11489142: can't find main text row old_id 7504489
Error at old_id 11489143: can't find main text row old_id 7504489
Error at old_id 11489144: can't find main text row old_id 7504489
Error at old_id 11489145: can't find main text row old_id 7504489
Error at old_id 11489146: can't find main text row old_id 7504489
Error at old_id 11489147: can't find main text row old_id 7504489
Error at old_id 11489148: can't find main text row old_id 7504489
Error at old_id 11489149: can't find main text row old_id 7504489
Error at old_id 11489150: can't find main text row old_id 7504489
Error at old_id 11489151: can't find main text row old_id 7504489
Error at old_id 11489152: can't find main text row old_id 7504489
Error at old_id 11489153: can't find main text row old_id 7504489
Error at old_id 11489154: can't find main text row old_id 7504489
Error at old_id 11489155: can't find main text row old_id 7504489
Error at old_id 11489156: can't find main text row old_id 7504489
Error at old_id 11489157: can't find main text row old_id 7504489
Error at old_id 11489158: can't find main text row old_id 7504489
Error at old_id 11489160: can't find main text row old_id 8161101
Error at old_id 11489161: can't find main text row old_id 8161101
Error at old_id 11489162: can't find main text row old_id 8161101
Error at old_id 11489163: can't find main text row old_id 8161101
Error at old_id 11489164: can't find main text row old_id 8161101
Error at old_id 11489165: can't find main text row old_id 8161101
Error at old_id 11489166: can't find main text row old_id 8161101
Error at old_id 11489167: can't find main text row old_id 8161101
Error at old_id 11489168: can't find main text row old_id 8161101
Error at old_id 11489169: can't find main text row old_id 8161101
Error at old_id 11489170: can't find main text row old_id 8161101
Error at old_id 11489171: can't find main text row old_id 8161101
Error at old_id 11489172: can't find main text row old_id 8161101
Error at old_id 11489173: can't find main text row old_id 8161101
Error at old_id 11489174: can't find main text row old_id 8161101
Error at old_id 11489175: can't find main text row old_id 8161101
Error at old_id 11489176: can't find main text row old_id 8161101
Error at old_id 11489177: can't find main text row old_id 8161101
Error at old_id 11489178: can't find main text row old_id 8161101
400 stubs processed
Error at old_id 11489180: can't find main text row old_id 8969614
Error at old_id 11489181: can't find main text row old_id 8969614
Error at old_id 11489182: can't find main text row old_id 8969614
Error at old_id 11489183: can't find main text row old_id 8969614
Error at old_id 11531205: target 11358354 is a CGZ pointer
Error at old_id 11592485: can't find main text row old_id 5527282
Error at old_id 11592486: can't find main text row old_id 5527282
Error at old_id 11592487: can't find main text row old_id 5527282
Error at old_id 11592488: can't find main text row old_id 5527282
Error at old_id 11592489: can't find main text row old_id 5527282
Error at old_id 11739394: target 11486778 is a CGZ pointer
Error at old_id 11744323: target 11410144 is a CGZ pointer
Error at old_id 11857669: target 11706686 is a CGZ pointer
Error at old_id 11868410: target 9336211 is a CGZ pointer
Error at old_id 11870703: target 9336211 is a CGZ pointer
Error at old_id 11930082: target 11846474 is a CGZ pointer
Error at old_id 11930353: target 11846474 is a CGZ pointer
Error at old_id 11930365: target 11846474 is a CGZ pointer
Error at old_id 11956921: target 11786320 is a CGZ pointer
Error at old_id 12083487: target 11786320 is a CGZ pointer
Error at old_id 12206538: target 12206433 is a CGZ pointer
Error at old_id 12231896: target 11855347 is a CGZ pointer
Error at old_id 12242468: target 12206433 is a CGZ pointer
Error at old_id 12243080: target 12206433 is a CGZ pointer
Error at old_id 12368040: target 9336211 is a CGZ pointer
Error at old_id 12386544: target 9336211 is a CGZ pointer
Error at old_id 12435365: target 12300587 is a CGZ pointer
Error at old_id 12490063: target 12300587 is a CGZ pointer
Error at old_id 12612214: target 12562863 is a CGZ pointer
Error at old_id 12612553: target 12562863 is a CGZ pointer
Error at old_id 12673727: target 12562863 is a CGZ pointer
Error at old_id 12691748: target 12407416 is a CGZ pointer
Error at old_id 12702096: target 9773231 is a CGZ pointer
Error at old_id 12702104: target 9773231 is a CGZ pointer
Error at old_id 12722888: target 9773231 is a CGZ pointer
Error at old_id 12724776: target 12407416 is a CGZ pointer
Error at old_id 12737593: target 12737115 is a CGZ pointer
Error at old_id 12755203: target 12737115 is a CGZ pointer
Error at old_id 12769921: can't find main text row old_id 2833258
Error at old_id 12769922: can't find main text row old_id 2833258
Error at old_id 12769923: can't find main text row old_id 5969568
Error at old_id 12769924: can't find main text row old_id 5969568
Error at old_id 12769931: can't find main text row old_id 1289120
Error at old_id 12769934: can't find main text row old_id 2051700
Error at old_id 12769937: can't find main text row old_id 1278977
Error at old_id 12769938: can't find main text row old_id 1278977
Error at old_id 12769968: can't find main text row old_id 4926843
Error at old_id 12769970: can't find main text row old_id 2833257
Error at old_id 12770684: can't find main text row old_id 1303914
Error at old_id 12770685: can't find main text row old_id 1303914
Error at old_id 12770686: can't find main text row old_id 1303914
Error at old_id 12770689: can't find main text row old_id 1303977
Error at old_id 12770756: can't find main text row old_id 4287010
Error at old_id 12770813: can't find main text row old_id 4286450
Error at old_id 12771015: can't find main text row old_id 2493432
Error at old_id 12771017: can't find main text row old_id 4000416
Error at old_id 12771246: can't find main text row old_id 6258845
Error at old_id 12771248: can't find main text row old_id 4893290
Error at old_id 12795624: can't find main text row old_id 8659755
Error at old_id 12811185: target 12785152 is a CGZ pointer
Error at old_id 12811296: target 12785152 is a CGZ pointer
Error at old_id 12814603: target 12785152 is a CGZ pointer
Error at old_id 12828773: target 12785152 is a CGZ pointer
Error at old_id 12836288: target 12785152 is a CGZ pointer
Error at old_id 12837505: target 12785152 is a CGZ pointer
Error at old_id 12842670: can't find main text row old_id 7453369
Error at old_id 12844199: target 12785152 is a CGZ pointer
Error at old_id 12896368: target 12785152 is a CGZ pointer
Error at old_id 12896649: target 12785152 is a CGZ pointer
Error at old_id 12898934: target 12785152 is a CGZ pointer
Error at old_id 12899293: target 12785152 is a CGZ pointer
Error at old_id 12917906: target 12785152 is a CGZ pointer
Error at old_id 12919497: target 12785152 is a CGZ pointer
Error at old_id 12951682: target 12785152 is a CGZ pointer
Error at old_id 12951837: target 12785152 is a CGZ pointer
Error at old_id 12969455: target 12785152 is a CGZ pointer
Error at old_id 12969497: target 12785152 is a CGZ pointer
Error at old_id 12979456: target 12785152 is a CGZ pointer
Error at old_id 12985105: target 12785152 is a CGZ pointer
Error at old_id 13085616: target 12908748 is a CGZ pointer
Error at old_id 13108539: target 13108402 is a CGZ pointer
Error at old_id 13127538: target 13108402 is a CGZ pointer
Error at old_id 13154301: target 13108402 is a CGZ pointer
Error at old_id 13154700: target 13108402 is a CGZ pointer
Error at old_id 13161355: target 13108402 is a CGZ pointer
Error at old_id 13161453: target 13108402 is a CGZ pointer
Error at old_id 13167169: target 13108402 is a CGZ pointer
Error at old_id 13167430: target 13108402 is a CGZ pointer
Error at old_id 13206354: target 13108402 is a CGZ pointer
Error at old_id 13218670: target 13108402 is a CGZ pointer
Error at old_id 13220961: target 13108402 is a CGZ pointer
Error at old_id 13224811: target 13108402 is a CGZ pointer
Error at old_id 13228013: target 13108402 is a CGZ pointer
Error at old_id 13228071: target 13108402 is a CGZ pointer
Error at old_id 13228252: target 13108402 is a CGZ pointer
Error at old_id 13257974: target 13108402 is a CGZ pointer
Error at old_id 13257995: target 13108402 is a CGZ pointer
Error at old_id 13267742: target 13108402 is a CGZ pointer
Error at old_id 13267815: target 13108402 is a CGZ pointer
Error at old_id 13359121: target 12999283 is a CGZ pointer
500 stubs processed
Error at old_id 13390261: target 13371051 is a CGZ pointer
Error at old_id 13414559: target 13400990 is a CGZ pointer
Error at old_id 13414858: target 13400990 is a CGZ pointer
Error at old_id 13459752: target 13400990 is a CGZ pointer
Error at old_id 13473436: target 10322521 is a CGZ pointer
Error at old_id 13517695: target 13503846 is a CGZ pointer
Error at old_id 13581490: target 13503846 is a CGZ pointer
Error at old_id 13655440: target 13621429 is a CGZ pointer
Error at old_id 13923519: target 10999025 is a CGZ pointer
Error at old_id 15838085: can't find main text row old_id 6070967

I'd guess this task caused this onwiki report, just a headsup since I see the get-backups task.

yeah, I think that's T128150#8921891 and onwards, I'm fixing most of them right now but I'm trying to figure out what to do with the rest. Probably tackle them in the next week

yeah, I think that's T128150#8921891 and onwards, I'm fixing most of them right now but I'm trying to figure out what to do with the rest. Probably tackle them in the next week

For transparency, the bug caused ~0.5M revisions to become inaccessible in enwiki (around 10K in nlwiki, svwiki and 2k in other legacy encoding wikis). Using iconv on the corrupted binary, I managed to bring back 340K edits, for the 160K left, my plan is to recover them from backups early next week and then run the fixed script on those rows again and hopefully put all of this mess of legacy encoding into rest.

Moving binaries around (from the backup to production) is tricky, pro-tip: Use hex:

sudo db-mysql db1133 -e "use nlwiki; select hex(old_text), old_flags from text where old_id = 1216415;"

update text set old_text = unhex('*Really large hex text*'), old_flags = 'gzip' where old_id = 1216415;

Marked any row left as bad row. Now it's ready to close.

Change 931306 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Stop setting wgLegacyEncdoing

https://gerrit.wikimedia.org/r/931306

Change 931306 merged by jenkins-bot:

[operations/mediawiki-config@master] Stop setting wgLegacyEncdoing

https://gerrit.wikimedia.org/r/931306

Mentioned in SAL (#wikimedia-operations) [2023-06-20T10:22:31Z] <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:931306|Stop setting wgLegacyEncdoing (T128150 T128151)]]

Mentioned in SAL (#wikimedia-operations) [2023-06-20T10:23:54Z] <ladsgroup@deploy1002> ladsgroup: Backport for [[gerrit:931306|Stop setting wgLegacyEncdoing (T128150 T128151)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-06-20T10:30:37Z] <ladsgroup@deploy1002> Finished scap: Backport for [[gerrit:931306|Stop setting wgLegacyEncdoing (T128150 T128151)]] (duration: 08m 06s)

Ladsgroup moved this task from Ready to Done on the DBA board.