Migrate all old DB rows from windows-1252 to UTF-8 on svwiktionary
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Jdforrester-WMF
	Feb 26 2016, 1:32 AM

Details

	Subject	Repo	Branch	Lines +/-
	Remove svwiktionary, svwiki and dawiki from legacy encoding	operations/mediawiki-config	master	+0 -4

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T128149 Remove wgLegacyEncoding feature of Revision/BlobStore
Resolved	Ladsgroup	T128150 Stop needing to use wgLegacyEncoding in Wikimedia cluster production
Resolved	Ladsgroup	T128156 Migrate all old DB rows from windows-1252 to UTF-8 on svwiktionary

Event Timeline

Jdforrester-WMF created this task.Feb 26 2016, 1:32 AM

Danny_B moved this task from Unsorted to Migrate / Replace on the Technical-Debt board.Feb 26 2016, 1:53 AM

Jdforrester-WMF mentioned this in T155529: Get rid of UTF-8 encoded as latin-1.Jan 17 2017, 8:45 PM

Change 383012 had a related patch set uploaded (by Zoranzoki21; owner: Zoranzoki21):
[operations/mediawiki-config@master] Migrate all old DB rows from windows-1252 to UTF-8 on several wikis:

https://gerrit.wikimedia.org/r/383012

gerritbot added a project: Patch-For-Review.Oct 7 2017, 10:27 PM

Change 383012 abandoned by Zoranzoki21:
Migrate all old DB rows from windows-1252 to UTF-8 on several wikis:

Reason:
I will abandon this change. Told me please to restore this patch, if it will be need. Sorry for much emails about this.

https://gerrit.wikimedia.org/r/383012

Marostegui removed a project: DBA.Nov 17 2017, 2:07 PM

Marostegui subscribed.

EddieGP moved this task from Backlog to Blocked on development on the Wikimedia-Site-requests board.Dec 9 2017, 11:35 AM

MarcoAurelio moved this task from Blocked on development to Done on the Wikimedia-Site-requests board.Feb 22 2018, 11:16 AM

MarcoAurelio moved this task from Done to Blocked on development on the Wikimedia-Site-requests board.Feb 22 2018, 11:21 AM

Umherirrender removed a project: Patch-For-Review.Apr 5 2022, 6:12 PM

I'll run this tomorrow:

ladsgroup@mwmaint1002:~$ mwscript maintenance/storage/moveToExternal.php --wiki=svwiktionary --end 31287 DB cluster27

The end_id cam from this:

mysql:research@s3-analytics-replica.eqiad.wmnet [svwiktionary]> select max(old_id) from text where not old_flags like '%external%';
+-------------+
| max(old_id) |
+-------------+
|       31287 |
+-------------+
1 row in set (1.623 sec)

And

+------------+----------+
| old_flags  | count(*) |
+------------+----------+
|            |     6756 |
| gzip       |     2506 |
| object     |     8202 |
| utf-8,gzip |    13750 |
+------------+----------+

Some test revs to make sure things are alright:

mysql:research@s3-analytics-replica.eqiad.wmnet [svwiktionary]> select old_id from text where old_flags = '' limit 5;
+--------+
| old_id |
+--------+
|    235 |
|    236 |
|    237 |
|    238 |
|    239 |
+--------+
5 rows in set (0.001 sec)

mysql:research@s3-analytics-replica.eqiad.wmnet [svwiktionary]> select * from content where content_address in ('tt:235', 'tt:236', 'tt:238', 'tt:239');
+------------+--------------+---------------------------------+---------------+-----------------+
| content_id | content_size | content_sha1                    | content_model | content_address |
+------------+--------------+---------------------------------+---------------+-----------------+
|       7500 |         7543 | f01ffbkivurqnf6rmuhsl0c6m2wfa1o |             1 | tt:235          |
|       7501 |           32 | qku8x3tytyyegqwbyawrs9ecs9hgjos |             1 | tt:236          |
|       7503 |          113 | 71lez11a2u9tau2ij6izzsdhdkhq93p |             1 | tt:238          |
|       7504 |          111 | 8sz749bmctynpl48hr67vripwjgvesj |             1 | tt:239          |
+------------+--------------+---------------------------------+---------------+-----------------+
4 rows in set (1.002 sec)

mysql:research@s3-analytics-replica.eqiad.wmnet [svwiktionary]> select * from slots where slot_content_id in (7500, 7501, 7503, 7504);
+------------------+--------------+-----------------+-------------+
| slot_revision_id | slot_role_id | slot_content_id | slot_origin |
+------------------+--------------+-----------------+-------------+
|              235 |            1 |            7500 |         235 |
|              236 |            1 |            7501 |         236 |
|              238 |            1 |            7503 |         238 |
|              239 |            1 |            7504 |         239 |
+------------------+--------------+-----------------+-------------+
4 rows in set (2.512 sec)

Which is confirmed by https://sv.wiktionary.org/w/index.php?oldid=235 vs.:

mysql:research@s3-analytics-replica.eqiad.wmnet [svwiktionary]> select * from text where old_id = 235;
+--------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+
| old_id | old_text                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | old_flags |
+--------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+
|    235 | <center>
[[Idag]] �r det [[{{CURRENTDAYNAME}}]], den {{CURRENTDAY}} [[{{CURRENTMONTHNAME}}]] {{CURRENTYEAR}} ([[UTC]])<br>P� ISO 8601-format: {{CURRENTYEAR}}-{{CURRENTMONTH}}-{{CURRENTDAY}} '''T''' {{CURRENTTIME}} <br>
</center>

'''Wiktionary''' �r ett projekt, �ppet f�r alla att delta i, med m�ls�ttningen att skapa en ordlista f�r alla spr�k. All text i Wiktionary publiceras under '''[[GNU FDL]]''', vilket betyder att du f�r anv�nda texterna fritt, s� l�nge som de f�rblir fria under samma licens. 

Wiktionary �r l�tt att bidra till: pr�va dig g�rna fram p� [[Wiktionary:Sandl�dan|testsidan]] tills du k�nner att du vet hur man g�r. 

Den svenska wiktionaryn startade den 2:a maj 2004, och har just nu {{NUMBEROFARTICLES}} artiklar. (R�knaren verkar f.n. vara ur funktion...)

De [[Special:Newpages|senaste artiklarna]].


Alla artiklar som b�rjar med: 
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=A A]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=B B]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=C C] 
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=D D]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=E E]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=F F]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=G G]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=H H]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=I I]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=J J] 
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=K K]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=L L]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=M M]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=N N]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=O O]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=P P]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=Q Q]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=R R]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=S S]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=T T]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=U U]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=V V]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=W W]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=X X]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=Y Y]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=Z Z]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=� �]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=� �]
[http://sv.wiktionary.org/w/wiki.phtml?title=Special:Allpages&from=� �]


==Wiktionary p� andra spr�k==

[http://en.wiktionary.org Engelska (English)] - [http://fi.wiktionary.org Finska (Suomi)] - [http://fr.wiktionary.org Franska (Fran�ais)] - 
[http://nl.wiktionary.org Holl�ndska] - [http://pl.wiktionary.org Polska] - [http://ro.wiktionary.org Rum�nska (Rom�n&#259;)] - [http://ru.wiktionary.org Ryska (&#1056;&#1091;&#1089;&#1089;&#1082;&#1080;&#1081;)] - [http://de.wiktionary.org Tyska (Deutsch)] - [http://hu.wiktionary.org Ungerska]

<!-- These links are not needed as long as the other language situation is not settled or at least explained somewhere, it is only confusing: 

<small>[[Wiktionary:Multilingual coordination|Wiktionary Language List]] - [http://ar.wiktionary.org (Araby) coming soon] - [http://ay.wikipedia.org Aymara (coming soon] - [http://bg.wiktionary.org (Balgarski) (coming soon)] - [http://ca.wiktionary.org Catal&agrave; (coming soon)] - [http://co.wikipedia.org Corsu] (WWW coming soon) - [[Hlavn� strana|&#268esky]] [http://cs.wiktionary.org &#268esky coming soon] - [[Hauptseite|German (Deutsch)]]  [http://de.wiktionary.org Deutsch]  (WWW coming soon) - [[&#922;&#973;&#961;&#953;&#945; &#931;&#949;&#955;&#943;&#948;&#945;|(Ellenika)]] [http://el.wikipedia.org (Ellenika)] (WWW coming soon)] - [[Portada|Espa&ntilde;ol]] [http://es.wiktionary.org Espa&ntilde;ol] (WWW coming soon) - 
[[Cxefpagxo|Esperanto]] [http://eo.wiktionary.org Esperanto] (WWW coming soon) - [http://et.wiktionary.org Eesti (coming soon)] - [http://eu.wiktionary.org Euskara (coming soon)] - [http://fa.wikipedia.org Farsi] (WWW coming soon) - [http://fi.wiktionary.org Suomeksi] (WWW coming soon) - [[Accueil|Fran&ccedil;ais]] [http://fr.wiktionary.org Fran&ccedil;ais] (WWW coming soon) - [http://fy.wiktionary.org] (WWW coming soon) - [http://ga.wiktionary.org Gaelige (coming soon)] - [http://gl.wikipedia.org Galego (coming soon)] - [http://gn.wiktionary.org Guaran&iacute; (coming soon)] - [http://gu.wiktionary.org Gujarati (coming soon)] - [http://hr.wiktionary.org Hrvatski] (WWW coming soon) - [http://ia.wiktionary.org Interlingua] (coming soon) - [http://id.wiktionary.org Bahasa Indonesia (coming soon)] - [http://is.wikipedia.org &Iacute;slenska (coming soon)] - [http://it.wiktionary.org Italiano] (website coming soon) - 
[[&#12513;&#12452;&#12531;&#12506;&#12540;&#12472;|&#26085;&#26412;&#35486; (Nihongo)]] [http://ja.wiktionary.org (Nihongo) (Japanese) coming soon] - [http://ka.wikipedia.org (Kartuli-ena) (coming soon)] - [http://kn.wiktionary.org (Kannada) coming soon] - [http://ko.wiktionary.org (Hangukeo)] (website coming soon) - [http://ku.wiktionary.org Kurd&iacute;] - [http://la.wiktionary.org Latina] (coming soon) - [http://ml.wiktionary.org Malayalam] (WWW coming soon) - [http://mr.wiktionary.org Marathi] (WWW coming soon) - [http://ms.wiktionary.org Bahasa Melayu] (WWW coming soon) - 
[[onthaalpagina|Nederlands]] [http://nl.wiktionary.org (coming soon)] - [http://no.wiktionary.org Norsk (coming soon)] - [http://oc.wikipedia.org Occitan (coming soon)] - [http://pa.wiktionary.org Punjabi (coming soon)] - [http://pt.wiktionary.org Portugu&ecirc;s] (coming soon) - [[Pagina principal&#259;|Rom�n&#259;]]  [http://ro.wiktionary.org Rom�n&#259;] (WWW coming soon) - [http://ru.wiktionary.org (Russkiy)] (coming soon) - [http://sa.wikipedia.org (Samskrta)] (coming soon) - [http://sl.wikipedia.org Slovensko] (website coming soon) - [http://sv.wiktionary.org Svenska (coming soon)] - [http://th.wiktionary.org Thai (coming soon)] - [http://tr.wikipedia.org Turk&ccedil;e] (website coming soon)] -  [http://ur.wiktionary.org (Urdu)] (coming soon) - [[Trang Ch�nh|Ti&#7871;ng Vi&#7879;t]] [http://vi.wiktionary.org Ti&#7871;ng Vi&#7879;t (coming soon)] - [[&#39318;&#39029;|&#20013;&#25991; (Zhongwen)]] [http://zh.wiktionary.org] (coming soon) - [[Wikipedia:Multilingual Statistics|Statistics]]</small>
-->

==Andra Wikimediaprojekt==

<small>[http://meta.wikipedia.org Meta-Wikipedia] - [http://sv.wikipedia.org Wikipedia] - '''Wiktionary''' - [http://wikibooks.org Wikibooks] - [http://wikiquote.org Wikiquote] - [http://wikisource.org Wikisource]<!-- Saved for future reactivation: - [http://wikipediatlas.org (coming soon) Wikipediatlas (coming soon)]</small>
-->

== Se �ven ==
* [[Wiktionary:Bybrunnen]]
* [[Wiktionary:Beg�ran om administrat�rsskap]]
* [[Wiktionary:Hj�lp]]
* [[Wiktionary:FAQ]]
* [[Wiktionary:Sidor som b�r raderas]]
* [[Wiktionary:Svenskt spr�kindex]]
* [[Wiktionary:Statistik]] |           |
+--------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+
1 row in set (0.001 sec)

Ran the script on these revisions:

ladsgroup@mwmaint1002:~$ mwscript maintenance/storage/moveToExternal.php --wiki=svwiktionary --start 234 --end 237 DB cluster27
Moving text rows from 234 to 237 to external storage
oldid=234, moved=0

The revision works (the link given) and it's moved:

mysql:research@s3-analytics-replica.eqiad.wmnet [svwiktionary]> select * from text where old_id = 235;
+--------+-----------------------+---------------+
| old_id | old_text              | old_flags     |
+--------+-----------------------+---------------+
|    235 | DB://cluster27/264386 | gzip,external |
+--------+-----------------------+---------------+
1 row in set (0.001 sec)

Well, it didn't fix the encoding to utf-8. Here is the current ids:

mysql:research@s3-analytics-replica.eqiad.wmnet [svwiktionary]> select * from text order by old_id desc limit 5;
+---------+-----------------------+---------------------+
| old_id  | old_text              | old_flags           |
+---------+-----------------------+---------------------+
| 3907731 | DB://cluster27/265417 | utf-8,gzip,external |
| 3907730 | DB://cluster27/265416 | utf-8,gzip,external |
| 3907729 | DB://cluster27/265415 | utf-8,gzip,external |
| 3907728 | DB://cluster26/264858 | utf-8,gzip,external |
| 3907727 | DB://cluster27/265414 | utf-8,gzip,external |
+---------+-----------------------+---------------------+
5 rows in set (0.001 sec)

le sigh

Running it with --iconv would fix some but not the ones that are external but with legacy encoding. Fixing it is not that hard though.

Ran it, only 1K rows left:

mysql:research@s3-analytics-replica.eqiad.wmnet [svwiktionary]> select old_flags, count(*) from text group by old_flags limit 50;
+---------------------+----------+
| old_flags           | count(*) |
+---------------------+----------+
| external,utf-8      |   315803 |
| gzip,external       |      991 |
| gzip,utf-8,external |     2493 |
| utf-8,gzip,external |  3585718 |
+---------------------+----------+
4 rows in set (7.268 sec)

Umherirrender mentioned this in T337700: Exception: "Malformed UTF-8 characters" in Parser\MagicWordArray (via LqtVIew).May 31 2023, 9:55 PM

Mentioned in SAL (#wikimedia-operations) [2023-06-07T12:46:43Z] <Amir1> mwscript maintenance/storage/moveToExternal.php --iconv DB cluster27 on dawiktionary and svwiktionary (T128155 and T128156)

Ladsgroup claimed this task.Jun 8 2023, 10:51 AM

Ladsgroup added a project: DBA.

Ladsgroup moved this task from Triage to In progress on the DBA board.

Via the script I made in T282734, I moved the 1K lines. An undo sql file is prepared just in case.

Change 928516 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Remove svwiktionary from legacy encoding

https://gerrit.wikimedia.org/r/928516

gerritbot added a project: Patch-For-Review.Jun 8 2023, 12:17 PM

Change 928516 merged by jenkins-bot:

[operations/mediawiki-config@master] Remove svwiktionary, svwiki and dawiki from legacy encoding

https://gerrit.wikimedia.org/r/928516

Mentioned in SAL (#wikimedia-operations) [2023-06-08T13:49:43Z] <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:928516|Remove svwiktionary, svwiki and dawiki from legacy encoding (T128156 T128152 T128153)]]

Mentioned in SAL (#wikimedia-operations) [2023-06-08T13:51:26Z] <ladsgroup@deploy1002> ladsgroup: Backport for [[gerrit:928516|Remove svwiktionary, svwiki and dawiki from legacy encoding (T128156 T128152 T128153)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-06-08T13:58:56Z] <ladsgroup@deploy1002> Finished scap: Backport for [[gerrit:928516|Remove svwiktionary, svwiki and dawiki from legacy encoding (T128156 T128152 T128153)]] (duration: 09m 13s)

Ladsgroup closed this task as Resolved.Jun 8 2023, 2:01 PM

Ladsgroup moved this task from In progress to Done on the DBA board.

Maintenance_bot removed a project: Patch-For-Review.Jun 8 2023, 2:11 PM

	F37082322: grafik.png
	May 29 2023, 11:11 AM

Migrate all old DB rows from windows-1252 to UTF-8 on svwiktionaryClosed, ResolvedPublicActions

Details

Related ObjectsSearch...

Event Timeline

Migrate all old DB rows from windows-1252 to UTF-8 on svwiktionary
Closed, ResolvedPublic
Actions

Related Objects
Search...