Page MenuHomePhabricator

Address semantic differences from enabling nativeGalleryEnabled
Open, MediumPublic

Description

Last week, nativeGalleryEnabled was enabled in production https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/889257

Running rt testing this week reported a number semantic differences when serializing from html instead of from data-mw,
https://parsoid-rt-tests.wikimedia.org/regressions/between/eced3e2/798eef3b

Note that selser will prevent any dirtying for unedited galleries and galleries edited by VE would drop the data-mw properties so there's no change to production. This is just about reporting during roundtrip testing but might yield some issues worth fixing.


|thumb media option dropped

ex. https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/jawiki/%E5%A4%A7%E5%88%86%E7%A9%BA%E6%B8%AF

This was known in T214601 and felt fine to normalize away since it isn't a valid media option in galleries. It seems like we can do better than the patch that was merged for that by having another argument to renderMedia that prevents other formats from rendering (first one wins T305628),
https://github.com/wikimedia/mediawiki-services-parsoid/blob/master/src/Ext/ParsoidExtensionAPI.php#L938-L946

In any case, the reason this is classified as a semantic difference is that the alt attribute on the img is suppressed for thumbs and not for plain floated images because the caption is visible for thumbs.


  • Todo

|left media option dropped. Presumably, this is true for all horizontal and vertical alignment options because the classes aren't transferred. Some of which is captured in T275074

Maybe build on https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/909340 to resolve.


  • Todo

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/enwiki/MESSENGER_probe

------
ages of the spacecraft">
File:MESSENGER - Sonda.png|alt=Diagram of ''MESSENGER''|Diagram of ''MESSENGER''.
File:MESSENGER - installa
++++++
ages of the spacecraft">
File:MESSENGER - Sonda.png|alt=Diagram of MESSENGER|Diagram of ''MESSENGER''.
File:MESSENGER - installa

Here |alt formatting is lost. This can probably be fixed by T252246. The alt attribute is text though so the normalization seems fine.

The reason this is classified as a semantic difference is because the formatting in an attribute forces an mw:ExpandedAttrs typeof, which isn't there after roundtripping.


  • Todo

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/frwiki/Cormery

This seems to be because we're trimming the alt for the purpose of comparison with the caption but then using that for serialization. Should be fixable.

------
="packed" heights="150">
Cormery moulin6.jpg|alt= photograpgie en noir et blanc d'un bâtiment au dessus d'une rivière|<center>Moulin avant l'incendie de 1908.</center>
Cormery moulin3.jpg|alt=P
++++++
="packed" heights="150">
Cormery moulin6.jpg|alt=photograpgie en noir et blanc d'un bâtiment au dessus d'une rivière|<center>Moulin avant l'incendie de 1908.</center>
Cormery moulin3.jpg|alt=P

Another whitespace trimming difference, though it requires a different fix,
https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/itwiki/Famiglia_Banchieri


  • Todo

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/frwiki/Citro%D1%91n

The difference here seems to be that WTUtils::textContentFromCaption needs to strip non-breaking spaces before doing the comparison with the alt. Other entities probably need checking as well.

------
ains et ruraux|AMTUIR]].
Fichier:Citroën Type 23 - Coupe Florio 2013 (avant).jpg|thumb|Citroën Type 23.
Fichier:Belfégor.JPG|Camion Citroën type [[Citroën Belphégor|Belphégor]] 600 de 1966 : Poids vide : 4,610 T, PTAC 10,300 T.
</gallery>

==== Concept 
++++++
ains et ruraux|AMTUIR]].
Fichier:Citroën Type 23 - Coupe Florio 2013 (avant).jpg|Citroën Type 23.
Fichier:Belfégor.JPG|alt=Camion Citroën type Belphégor 600 de 1966 : Poids vide : 4,610 T, PTAC 10,300 T.|Camion Citroën type [[Citroën Belphégor|Belphégor]] 600 de 1966 : Poids vide : 4,610 T, PTAC 10,300 T.
</gallery>

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/hewiki/%D7%A1%D7%A4%D7%A8%D7%98%D7%A7_%D7%9E%D7%95%D7%A1%D7%A7%D7%91%D7%94_(%D7%9B%D7%93%D7%95%D7%A8%D7%92%D7%9C)

Similar to thumb above, dimensions are dropped but, considering T305628, the gallery implementation should be updated to add its dimension last so that they are assured to be selected.

------
חק נגד צסק"א, עונת 2008.
קובץ:Spartak-Dinamo (7).jpg|250px|אוהד ספרטק — אוטו פישר, בן ה-102.
קובץ:Fratria perf.jpg|ארג
++++++
חק נגד צסק"א, עונת 2008.
קובץ:Spartak-Dinamo (7).jpg|אוהד ספרטק — אוטו פישר, בן ה-102.
קובץ:Fratria perf.jpg|ארג

On a few pages, media options like upright, thumbtime, page, etc. are dropped. This is captured in T214603.

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/nlwiki/1952

------
[[Lineair B]].
<gallery>
Voorschip van de War Diwan bij Vlissingen gelicht.ogv|thumbtime=0:52|Bioscoopjournaal uit mei 1952: In 1944 zonk de Engelse tanker "War Diwan" tgv een mijnontploffing in het water van de Wielingen bij Vlissingen. Het schip brak daarbij in tweeën. Het voorschip wordt nu gelicht door met perslucht het water uit het wrak te blazen zodat het weer gaat drijven. Deze nieuwe manier van schepen lichten is bedacht door de Nederlandse ingenieur W. Van Wienen die zelf leiding geeft aan het gebeuren.

++++++
[[Lineair B]].
<gallery>
Voorschip van de War Diwan bij Vlissingen gelicht.ogv|Bioscoopjournaal uit mei 1952: In 1944 zonk de Engelse tanker "War Diwan" tgv een mijnontploffing in het water van de Wielingen bij Vlissingen. Het schip brak daarbij in tweeën. Het voorschip wordt nu gelicht door met perslucht het water uit het wrak te blazen zodat het weer gaat drijven. Deze nieuwe manier van schepen lichten is bedacht door de Nederlandse ingenieur W. Van Wienen die zelf leiding geeft aan het gebeuren.

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/hiwiki/%E0%A4%AF%E0%A5%8B%E0%A4%9C%E0%A4%A8%E0%A4%BE%E0%A4%A4%E0%A5%8D%E0%A4%AE%E0%A4%95_%E0%A4%86%E0%A4%B0%E0%A5%87%E0%A4%96

------
ियरी]] का व्यवस्था चित्र

चित्र:Gear pump exploded.svg|lang=hi|3D-[[इंजीनियरी ड्राइंग]] का विस्तारित दृष्य
चित्र:US 2000 census popu
++++++
ियरी]] का व्यवस्था चित्र
चित्र:Gear pump exploded.svg|link=चित्र:Gear_pump_exploded.svg?lang=hi|3D-[[इंजीनियरी ड्राइंग]] का विस्तारित दृष्य
चित्र:US 2000 census popu

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/enwiki/Cathal%20O'Shannon

==Gallery==
<gallery>
Piece 207-176; Cathal Shannon (1922).pdf|page=5|British Army Military Intelligence file for Cathal O'Shannon
Cathal_O'Shannon_Election
++++++
 

==Gallery==
<gallery>
Piece 207-176; Cathal Shannon (1922).pdf|link=File:Piece_207-176;_Cathal_Shannon_(1922).pdf?page=5|British Army Military Intelligence file for Cathal O'Shannon
Cathal_O'Shannon_Election

  • Todo

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/commonswiki/Ivan%20Aivazovsky

------
bzon|pl=Trabzon}}, 1875.
File:DSC07418b.jpg|{{title|fr=Souvenir de Nice : le cuirassé l’Océan dans la rade de Villefranche en 1874}}, 1875.
File:Облака над тихим мор
++++++
bzon|pl=Trabzon}}, 1875.
File:DSC07418b.jpg|alt=French:  Souvenir de Nice : le cuirassé l’Océan dans la rade de Villefranche en 1874label QS:Lfr,"Souvenir de Nice : le cuirassé l’Océan dans la rade de Villefranche en 1874"
, 1875.|{{title|fr=Souvenir de Nice : le cuirassé l’Océan dans la rade de Villefranche en 1874}}, 1875.
File:Облака над тихим мор

  • Todo

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/metawiki/Goats

Nowiki is lost from the caption

------
KINGDOM (2184300375).jpg
File:This Picture turned out better than I thought (14331397205).jpg|Do you have the g<nowiki>[oa|u]</nowiki>ts?
File:¿Hola Mundo?.jpg
Fil
++++++
KINGDOM (2184300375).jpg
File:This Picture turned out better than I thought (14331397205).jpg|Do you have the g[oa|u]ts?
File:¿Hola Mundo?.jpg
Fil

  • Todo

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/ukwiki/%D0%96%D1%83%D0%BF%D0%B0%D0%BD%D1%96%D0%B2%D0%BA%D0%B0

Empty caption is dropped

------
 війни 2014—2015 років.

<gallery mode="packed" heights="160" caption="">
Zhupanivka 01.JPG||Кав'ярня у Жупанівці
Zhupanivka 05.JPG||Сільський пейзаж
Zhupanivka 08.JPG|міні|Братська могила 
</gallery>

== Примітки =
++++++
 війни 2014—2015 років.

<gallery mode="packed" heights="160">
Zhupanivka 01.JPG|Кав'ярня у Жупанівці
Zhupanivka 05.JPG|Сільський пейзаж
Zhupanivka 08.JPG|Братська могила 
</gallery>

==Примітки==

  • Todo

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/commonswiki/Prekmur

Nowiki added

------
 moje po'selejnye, 1853)
File:Dár Zétek Ferenczá 1910.jpg|Present of Ferenc Zétek from [[Category:Orfalu|Orfalu]] in Apátistvánfalva (in the Baptist Case of the Church). Inscription ''Dár Zétek Ferenc''
File:Binkostne pesmi na p
++++++
 moje po'selejnye, 1853)
File:Dár Zétek Ferenczá 1910.jpg|Present of Ferenc Zétek from  [[Category:Orfalu|Orfalu]] <nowiki> </nowiki>in Apátistvánfalva (in the Baptist Case of the Church). Inscription ''Dár Zétek Ferenc''
File:Binkostne pesmi na p

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/commonswiki/AUS

WTUtils::textContentFromCaption needs to be aware of metadata tags?

==Other==
<gallery>
File:HMAS_Canberra_and_Hobart_in_2020.jpg|Royal Australian Navy {{maincat|Royal Australian Navy}}
File:Lam_and_Marcia_2015-
++++++
ry>

==Other==
<gallery>
File:HMAS_Canberra_and_Hobart_in_2020.jpg|alt=<nowiki>Royal Australian Navy .mw-parser-output .hatnote{font-style:italic}.mw-parser-output .hatnote i,.mw-parser-output .hatnote.noitalic{font-style:normal}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output div.hatnote+div.hatnote{margin-top:-0.5em}@media print{.mw-parser-output .hatnote{display:none}}
Main category: Royal Australian Navy</nowiki>|Royal Australian Navy {{maincat|Royal Australian Navy}}
File:Lam_and_Marcia_2015-

  • Todo

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/798eef3b/kawiki/%E1%83%93%E1%83%98%E1%83%A1%E1%83%A0%E1%83%90%E1%83%94%E1%83%9A%E1%83%98

Alt is lost?

------
ghts="150px" perrow="4">
File:Fall of caesar.jpg|alt=A depiction of the murder of Julius Caesar, with Robert Peel portrayed as Caesar, and British political rivals depicted as his assassins|[[რობერტ პილი|პილი]] ნაჩვენებია როგორც მკვლელებით გარშემორტყმული[[იულიუს კეისარი]], მარცხენა მხარეს მდგარი დიზრაელით, 1846
File:Cartoon-The-Dispatch-of-Business".jpg|alt=A caricature showing Gladstone on a donkey, marked "reform", being held back by Disraeli and a working-class man, who are holding the donkey's tail|ვირზე მჯდომი რეფორმის განმასახიერებელ [[უილიამ ევარტ გლადსტონი|გლადსტონი]]ს უკან ექაჩება დიზრაელი, რომელსაც ეხმარება ინგლისელი მუშა, 1866
File:Dishing-the-Whigs-18
++++++
ghts="150px" perrow="4">
File:Fall of caesar.jpg|[[რობერტ პილი|პილი]] ნაჩვენებია როგორც მკვლელებით გარშემორტყმული[[იულიუს კეისარი]], მარცხენა მხარეს მდგარი დიზრაელით, 1846
File:Cartoon-The-Dispatch-of-Business".jpg|ვირზე მჯდომი რეფორმის განმასახიერებელ [[უილიამ ევარტ გლადსტონი|გლადსტონი]]ს უკან ექაჩება დიზრაელი, რომელსაც ეხმარება ინგლისელი მუშა, 1866
File:Dishing-the-Whigs-18

  • Todo

From https://parsoid-rt-tests.wikimedia.org/resultFlagNew/eced3e2/42033752/commonswiki/S%C3%A9bastien%20Bourdon

------
"210px" heights="200px">
File:Bourdon, Sébastien - Le Camp de Bohémiens.jpg|{{title|en=Gypsy camp|fr=Le Camp de Bohémiens|pl=Obóz cygański}} <!-- ? ''Halte de bohémiens et de soldats'' -->  c. ? <!-- 1636 - 1638 -->, {{oil on canvas}}, [[Montpellier]], Musée Fabre.<sup>[http://www.oberlin.edu/amam/Bourdon_Encampment.htm similar]</sup>
File:Sébastien Bourdon-Christina of Sweden 1653.jpg|{{title|en=Queen Christine of Sweden on horseback|fr=La Reine Christine de Suède à cheval|pl=Królowa Krystyna Waza na koniu}}<br/>1653, {{Oil on canvas}}, {{size|cm|383|291}}, {{Madrid}}, {{Prado}}.
File:Bourdon, Sébastien - Le crucifiement de Saint Pierre - 1643.JPG|{{title|en=Crucifixion of Saint Peter|fr=Le crucifiement de Saint Pierre|pl=Ukrzyżowanie św. Piotra}}<br/>1643, {{Paris}}, Cathédrale Notre-Dame.
File:Sébastien Bourdons-Karl X Gustav.jpg|{{title|en=King Charles X Gustavus|fr=Le Roi Charles X Gustave|pl=Król Karol X Gustaw}}<br/>1652 {{LangSwitch|en=or|pl=lub}} 1653, [[Stockholm]], Nationalmuseum.
File:Bourdon, Sébastien - La Chute de Simon le Magicien - 1657.jpg|{{title|en=The Fall of Simon the Magician|fr=La Chute de Simon le Magicien|pl=Upadek Szymona Maga}}<br/>1657, {{Oil on canvas}}, [[Montpellier]], Cathédrale Saint-Pierre.<!-- engraving with the same title: http://www.culture.gouv.fr/Wave/image/joconde/0195/m503501_d0020464-000_p.jpg -->
</gallery>
===D===
<galle
++++++
"210px" heights="200px">
File:Bourdon, Sébastien - Le Camp de Bohémiens.jpg|alt=Gypsy camplabel QS:Len,"Gypsy camp"
label QS:Lpl,"Obóz cygański"
label QS:Lfr,"Le Camp de Bohémiens"   c. ? , oil on canvasmedium QS:P186,Q296955;P186,Q12321255,P518,Q861259, Montpellier, Musée Fabre.similar|{{title|en=Gypsy camp|fr=Le Camp de Bohémiens|pl=Obóz cygański}} <!-- ? ''Halte de bohémiens et de soldats'' -->  c. ? <!-- 1636 - 1638 -->, {{oil on canvas}}, [[Montpellier]], Musée Fabre.<sup>[http://www.oberlin.edu/amam/Bourdon_Encampment.htm similar]</sup>
File:Sébastien Bourdon-Christina of Sweden 1653.jpg|{{title|en=Queen Christine of Sweden on horseback|fr=La Reine Christine de Suède à cheval|pl=Królowa Krystyna Waza na koniu}}<br />1653, {{Oil on canvas}}, {{size|cm|383|291}}, {{Madrid}}, {{Prado}}.
File:Bourdon, Sébastien - Le crucifiement de Saint Pierre - 1643.JPG|{{title|en=Crucifixion of Saint Peter|fr=Le crucifiement de Saint Pierre|pl=Ukrzyżowanie św. Piotra}}<br />1643, {{Paris}}, Cathédrale Notre-Dame.
File:Sébastien Bourdons-Karl X Gustav.jpg|{{title|en=King Charles X Gustavus|fr=Le Roi Charles X Gustave|pl=Król Karol X Gustaw}}<br />1652 {{LangSwitch|en=or|pl=lub}} 1653, [[Stockholm]], Nationalmuseum.
File:Bourdon, Sébastien - La Chute de Simon le Magicien - 1657.jpg|{{title|en=The Fall of Simon the Magician|fr=La Chute de Simon le Magicien|pl=Upadek Szymona Maga}}<br />1657, {{Oil on canvas}}, [[Montpellier]], Cathédrale Saint-Pierre.<!-- engraving with the same title: http://www.culture.gouv.fr/Wave/image/joconde/0195/m503501_d0020464-000_p.jpg -->
</gallery>
===D===
<galle

Event Timeline

Arlolra triaged this task as Medium priority.
Arlolra moved this task from Needs Triage to Testing on the Parsoid board.

Change 908671 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] [WIP] Lint away format being used in gallery media options

https://gerrit.wikimedia.org/r/908671

Change 909340 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] Suppress media formats when used on a gallery line

https://gerrit.wikimedia.org/r/909340

Change 909398 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] [WIP] Serialize untrimmed alt attr on gallery images

https://gerrit.wikimedia.org/r/909398

Arlolra updated the task description. (Show Details)

Change 909340 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Suppress media formats when used on a gallery line

https://gerrit.wikimedia.org/r/909340

Arlolra updated the task description. (Show Details)

Change 910040 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] Ensure the correct dimensions are applied to gallery lines

https://gerrit.wikimedia.org/r/910040

Change 910040 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Ensure the correct dimensions are applied to gallery lines

https://gerrit.wikimedia.org/r/910040

Change 911253 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/vendor@master] Bump parsoid to 0.18.0-a6

https://gerrit.wikimedia.org/r/911253

Change 911253 merged by jenkins-bot:

[mediawiki/vendor@master] Bump parsoid to 0.18.0-a6

https://gerrit.wikimedia.org/r/911253

Running rt testing this week reported a number semantic differences when serializing from html instead of from data-mw,
https://parsoid-rt-tests.wikimedia.org/regressions/between/eced3e2/798eef3b

Re-running rt this week with the above merged patches produces a much shorter list, all of which is captured in the remaining todos,
https://parsoid-rt-tests.wikimedia.org/regressions/between/eced3e2/42033752

Arlolra updated the task description. (Show Details)

Change 923413 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] Sync parserTests with core+extensions

https://gerrit.wikimedia.org/r/923413

Change 923413 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Sync parserTests with core+extensions

https://gerrit.wikimedia.org/r/923413

Change 923686 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@master] Bump parsoid to 0.18.0-a12

https://gerrit.wikimedia.org/r/923686

Change 923686 merged by jenkins-bot:

[mediawiki/vendor@master] Bump parsoid to 0.18.0-a12

https://gerrit.wikimedia.org/r/923686

Change 967323 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] [WIP] Reuse LinkHandlerUtils::figureHandler in gallery

https://gerrit.wikimedia.org/r/967323

Change 967323 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Reuse LinkHandlerUtils::figureHandler in gallery

https://gerrit.wikimedia.org/r/967323

Change 969688 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.19.0-a4

https://gerrit.wikimedia.org/r/969688

Change 969688 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.19.0-a4

https://gerrit.wikimedia.org/r/969688