Page MenuHomePhabricator

Migrate all old DB rows from windows-1252 to UTF-8 on svwiki
Closed, ResolvedPublic

Event Timeline

Change 383012 had a related patch set uploaded (by Zoranzoki21; owner: Zoranzoki21):
[operations/mediawiki-config@master] Migrate all old DB rows from windows-1252 to UTF-8 on several wikis:

https://gerrit.wikimedia.org/r/383012

Change 383012 abandoned by Zoranzoki21:
Migrate all old DB rows from windows-1252 to UTF-8 on several wikis:

Reason:
I will abandon this change. Told me please to restore this patch, if it will be need. Sorry for much emails about this.

https://gerrit.wikimedia.org/r/383012

Before run:

mysql:research@s2-analytics-replica.eqiad.wmnet [svwiki]> select old_flags, count(*) from text group by old_flags limit 50;
+---------------------+----------+
| old_flags           | count(*) |
+---------------------+----------+
|                     |     7390 |
| error               |      146 |
| external,gzip       |      173 |
| external,object     |       64 |
| external,utf-8      |  5005902 |
| gzip                |     9232 |
| object              |   148494 |
| utf-8,gzip          |   571498 |
| utf-8,gzip,external | 47239078 |
+---------------------+----------+
9 rows in set (2 min 5.181 sec)

After run:

mysql:research@s2-analytics-replica.eqiad.wmnet [svwiki]> select old_flags, count(*) from text group by old_flags limit 50;
+---------------------+----------+
| old_flags           | count(*) |
+---------------------+----------+
| error               |      146 |
| external            |      902 |
| external,gzip       |      173 |
| external,object     |       64 |
| external,utf-8      |  5005902 |
| gzip,utf-8,external |     9232 |
| object,external     |      120 |
| utf-8,gzip,external | 47965826 |
+---------------------+----------+
8 rows in set (1 min 42.813 sec)

And result of moving half a million rows:

root@dbstore1007:/srv/sqldata.s2/svwiki# ls -Ssh | grep -i text
3.2G text.ibd
4.0K text.frm
root@dbstore1007:/srv/sqldata.s2/svwiki# ls -Ssh | grep -i text
1.9G text.ibd
4.0K text.frm

Mentioned in SAL (#wikimedia-operations) [2023-06-08T10:49:25Z] <Amir1> mwscript maintenance/storage/moveToExternal.php --wiki=svwiki --iconv DB cluster27 (T128153)

Ladsgroup added a project: DBA.
Ladsgroup moved this task from Triage to In progress on the DBA board.

Mentioned in SAL (#wikimedia-operations) [2023-06-08T11:03:04Z] <Amir1> mwscript maintenance/storage/moveToExternal.php --wiki=dawiki --iconv DB cluster27 (T128153)

Change 928516 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Remove svwiktionary, svwiki and dawiki from legacy encoding

https://gerrit.wikimedia.org/r/928516

Change 928516 merged by jenkins-bot:

[operations/mediawiki-config@master] Remove svwiktionary, svwiki and dawiki from legacy encoding

https://gerrit.wikimedia.org/r/928516

Mentioned in SAL (#wikimedia-operations) [2023-06-08T13:49:43Z] <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:928516|Remove svwiktionary, svwiki and dawiki from legacy encoding (T128156 T128152 T128153)]]

Mentioned in SAL (#wikimedia-operations) [2023-06-08T13:51:26Z] <ladsgroup@deploy1002> ladsgroup: Backport for [[gerrit:928516|Remove svwiktionary, svwiki and dawiki from legacy encoding (T128156 T128152 T128153)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-06-08T13:58:56Z] <ladsgroup@deploy1002> Finished scap: Backport for [[gerrit:928516|Remove svwiktionary, svwiki and dawiki from legacy encoding (T128156 T128152 T128153)]] (duration: 09m 13s)

Ladsgroup moved this task from In progress to Done on the DBA board.