TemplateData: Normalisation should not assume paramOrder
Open, HighPublic

Description

VisualEditor shuffles template parameters order in articles on cswiki even if there is no paramOrder specified in template's TemplateData. A lot of cswiki users complained. Please, see

  1. https://cs.wikipedia.org/w/index.php?title=Milan_K%C5%88a%C5%BEko&curid=144065&diff=13844322&oldid=13643045
  2. https://cs.wikipedia.org/w/index.php?title=Ji%C5%99%C3%AD_Kov%C3%A1%C5%99&type=revision&diff=13849308&oldid=13810361
  3. https://cs.wikipedia.org/w/index.php?title=Michael_Koc%C3%A1b&diff=prev&oldid=13849130

and some other edits by this user for example.

This is a bug in TemplateData extension which is initializing paramOrder even when one has not been provided in the <templatedata> markup.

Urbanecm created this task.Jun 20 2016, 8:43 AM
Restricted Application added subscribers: Zppix, JEumerus, Matanya, Aklapper. · View Herald TranscriptJun 20 2016, 8:43 AM
Urbanecm renamed this task from Turn off shuffling params order in cswiki to Turn off shuffling params order in cswiki (VE).Jun 20 2016, 9:22 AM
Urbanecm updated the task description. (Show Details)
Dvorapa updated the task description. (Show Details)Jun 20 2016, 9:40 AM
JAnD added a subscriber: JAnD.
Esanders edited projects, added Parsoid; removed VisualEditor.Jun 20 2016, 11:07 AM
Restricted Application added a project: VisualEditor. · View Herald TranscriptJun 20 2016, 11:07 AM

This is a Parsoid issue - but I suspect they won't want to change this behaviour - or make it site configurable.

And could you provide me reason why? I think that better will be to don't allow VE change order of params when a template was already inserted, but tagging is possible and TemplateData'll be used only for inserting a new template.

@Esanders In my opinion, VE should sort template parameters according to paramOrder option in template's TemplateData and not sort them if there isn't this option set. Or there could be at least some switcher to turn off this sorting (like there is format option, there could be sort option).

One bad aspect of this behavior is that user doesn't see his/her change, because VE adds its own changes (sometimes really large) to diff.

Recording from IRC conversation.

<subbu> Dvorapa, hi .. quick qn ... are you saying Parsoid is reordering parameters even if it does not have 'paramOrder' field set in templatedata?
<subbu> if so, that sounds like a bug, but, my gut sense is that paramOrder is present because otherwise parsoid wouldn't know how to order the params.
<Dvorapa> yes
<subbu> ok, let me look at some sample diffs and look at the affected templates and templatedata for them.
<subbu> 1. for https://cs.wikipedia.org/w/index.php?title=V%C3%ADt%C4%9Bzslav_Jand%C3%A1k&type=revision&diff=13849145&oldid=13642729 .. https://cs.wikipedia.org/w/api.php?action=templatedata&titles=Template:Infobox%20Politik&format=json specifies the paramOrder.
<subbu> https://cs.wikipedia.org/w/index.php?title=Michael_Koc%C3%A1b&diff=prev&oldid=13849130 is Infobox Politik as well
<subbu> and so is https://cs.wikipedia.org/w/index.php?title=Helena_T%C5%99e%C5%A1t%C3%ADkov%C3%A1&diff=prev&oldid=13849139 ..
<subbu> Dvorapa, ^ so .. all of them use Infobox Politik and templatedata for it does provide a paramOrder field.
<Dvorapa> I added paramOrder after those incorrect behavior happenned (see template history x diffs time)
<subbu> oh, hmm ..
<Dvorapa> We thought Parsoid uses order of parameters in *params* and we also thought it is "by design" (not a bug)
<subbu> no, it should use the order specified by paramOrder only .. otherwise, leave param order unchanged.
ssastry renamed this task from Turn off shuffling params order in cswiki (VE) to Parsoid should not reorder transclusion parameters if paramOrder field is absent in TemplateData.Jun 20 2016, 2:36 PM
ssastry triaged this task as High priority.
Restricted Application added a subscriber: Dereckson. · View Herald TranscriptJun 20 2016, 2:36 PM
Mvolz added a subscriber: Mvolz.Jun 20 2016, 2:45 PM
Dvorapa added a comment.EditedJun 20 2016, 3:11 PM

This is the former revision of the Infobox Politik template and its TemplateData, which caused the incorrect behavior described above. Maybe there could be some issue with the TemplateData block? (@ssastry).

Update: current revision also makes this buggy behavior

https://github.com/wikimedia/parsoid/blob/8fd4c9bbee79ab0fe30f86d6ba44dc73b11b62a1/lib/html2wt/WikitextSerializer.js#L460-L477 is the code that determines order in which args are serialized. So, if paramOrder is not present, data-parsoid & data-mw info is used .. so, I suspect something else unrelated to templatedata affected these edits.

params is currently a keyed object and therefore has no order however I have suggested getting rid of this: T127958

Vojtech.dostal added a subscriber: Vojtech.dostal.

This is a serious issue that can undermine the trust of the Czech community that we generally put into VisualEditor. Several editors who use Wikimarkup are frustrated by the behaviour.

Adding Elitre on this (please feel free to unsubscribe if you believe this is of no concern to you - in that case I am sorry for subscribing you :-)) .

I am still investigating the bug .. it doesn't seem tied to paramOrder at this time.

Urbanecm removed Elitre as the assignee of this task.Jun 20 2016, 4:16 PM
Urbanecm added a subscriber: Elitre.

I think that if Elitre isn't working on this, she should be added as a subscriber, not as an assignee.

Dvorapa updated the task description. (Show Details)Jun 20 2016, 4:19 PM

@Vojtech.dostal if anybody wants to be assigned to this issue, (s)he make it her/himself. Currently ssastry is investigating this issue and there is also a conversation between me and him on IRC (#mediawiki-parsoid, most important points were cited above).

Dvorapa added a comment.EditedJun 20 2016, 6:11 PM

It seems that API contains paramOrder, but template doesn't. Therefore paramOrder is automatically added by TemplateData (?), which causes this bug. Compare API output with TemplateData of an example template.

ssastry renamed this task from Parsoid should not reorder transclusion parameters if paramOrder field is absent in TemplateData to TemplateData: Do not initialize paramOrder to paramNames if paramOrder has not been provided in the <templatedata> markup.Jun 20 2016, 6:14 PM
ssastry removed a project: Parsoid.
ssastry updated the task description. (Show Details)Jun 20 2016, 6:18 PM
ssastry removed a project: VisualEditor.
ssastry added subscribers: Krenair, Krinkle.
Restricted Application added a project: VisualEditor. · View Herald TranscriptJun 20 2016, 6:19 PM
Krenair edited subscribers, added: AlexMonk-WMF; removed: Krenair.Jun 20 2016, 7:05 PM

Can we make any progress on this? There are more and more erroneous edits on cswiki and users complain :/

Krinkle renamed this task from TemplateData: Do not initialize paramOrder to paramNames if paramOrder has not been provided in the <templatedata> markup to TemplateData: Normalisation should not assume paramOrder.Jun 29 2016, 3:42 PM
Krinkle added a comment.EditedJun 29 2016, 3:45 PM

LGTM. Let's remove the auto-creation of this property from the parser normalisation. I added it originally because property order enumeration happens to be stable in PHP (and originally the wikitext was only manually modified). However made it impossible to leave the order unspecified.

The TemplateData spec does not describe paramOrder as required, and for back-compat both our authored content and consumers may not assume the presence of this property.

Let's verify (or fix) VisualEditor and Parsoid to indeed not assume the existence of paramOrder, then we can go ahead and remove the auto-creation of it in the parser.

Let's verify (or fix) VisualEditor and Parsoid to indeed not assume the existence of paramOrder and function properly when absent.

Parsoid doesn't assume the existence of paramOrder. See tests that spec this and the mock API that supports those tests.

Krinkle moved this task from Backlog to Next-up on the TemplateData board.Jun 29 2016, 3:48 PM
This comment was removed by Urbanecm.
Restricted Application added a project: User-Urbanecm. · View Herald TranscriptNov 28 2016, 12:50 PM
Urbanecm moved this task from Backlog to Watching on the User-Urbanecm board.Nov 28 2016, 12:58 PM

Any progress on it? This is a high priority problem but last comment was made before almost half a year!

Any progress?

Any progress? High priority and last comment was almost one year ago!

Adding current PM.

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptNov 9 2017, 2:39 PM

Change 389852 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/services/parsoid@master] Preserve original transclusion's parameter order

https://gerrit.wikimedia.org/r/389852