Page MenuHomePhabricator

Template transclusion of a few list items extends to the entire list in VE, making it impossible to edit visually
Open, NormalPublic

Description

Case: https://de.wikipedia.org/wiki/Fürst_(Begriffsklärung)

Steps to reproduce:

  1. Open https://de.wikipedia.org/wiki/Fürst_(Begriffsklärung) with VE
  2. Try to edit line "Paula Fürst"
  3. Unable because the complete block from "Paul Fürst" to "Werner Fürst" is marked as template, see screenshot
  4. If you change to Wikitext edting you see that "{{:Paul Fürst}}" is an included page

Event Timeline

Raymond created this task.Jan 17 2018, 12:01 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 17 2018, 12:01 PM

You actually can edit Paula Fürst even in VE: Just click "edit" for the transclusion, the wikitext will be there.
Given that this is a list where the items a partially defined in "templates" and partially on the page itself, I think the fact that the transclusion block extends over the actual transclusion is expected: If the (implicite) <ul> is inside a transclusion, the </ul> can't be outside, because otherwise stuff would be unbalanced. OTOH, given that each transclusion starts a new block, it could actually be possible to change the behavior to make single list items after the transclusions directly editable.

VE should be able to handle transclusions like that, with some caveats:

  • Changing the structure might behave unpredictably, like changing the list from bulleted to numbered. Some of the tools might not be able to handle lists where some of the list items are not really list items.
  • The transclusion might display incorrectly in the editor after it is edited (but it should go back to normal if saved). We don't have the "context" of the whole list when updating a transclusion rendering. This is the same kind of problem that affects template-generated table cells.

In this case I think the problem is with the Parsoid output. Here's a fragment from the HTML it generates (https://de.wikipedia.org/api/rest_v1/page/html/Fürst_(Begriffsklärung)), reformatted a bit for clarify:

<ul id="mwKg">
...
<li id="mwXQ"> <a rel="mw:WikiLink" href="./Gerhard_Fürst" title="Gerhard Fürst" id="mwXg">Gerhard Fürst</a> (1897–1988), erster Präsident des Statistischen Bundesamtes</li>
<li id="mwXw"> <a rel="mw:WikiLink" href="./Gustav_Gerson_Fürst" title="Gustav Gerson Fürst" id="mwYA">Gustav Gerson Fürst</a> (1840–1918), deutscher Maler und Dekorationskünstler</li>
</ul>
<span about="#mwt1" typeof="mw:Transclusion" data-mw='{"parts":[{"template":{"target":{"wt":":Hans Fürst","href":"./Hans_Fürst"},"params":{},"i":0}},"\n* [[Heinrich Fürst (Oberstudienrat)]] (1861–1944), Oberstudienrat, fränkischer Kunstmaler, 1885–1926 Zeichenlehrer und Turnlehrer an der Kgl. Kreisrealschule in Nürnberg\n* [[Heinrich Fürst]] (1907–2001), deutscher Schauspieler und Regisseur &lt;!-- GND=18947209X -->\n* [[Helena Fürst]] (* 1974), deutsche Fernsehdarstellerin\n* [[Irmgard Fürst]] (* 1964), deutsche Politikerin (SPD)\n* [[János Fürst]] (1935–2007), ungarischer Dirigent"]}' id="mwYQ">
</span>
<ul about="#mwt1">
<li><a rel="mw:WikiLink" href="./Hans_Fürst_(Politiker)" title="Hans Fürst (Politiker)">Hans Fürst (Politiker)</a> (1902–1968), Schweizer Jurist und Politiker (FDP)</li>
<li><a rel="mw:WikiLink" href="./Hans_Fürst_(Chemiker)" title="Hans Fürst (Chemiker)">Hans Fürst (Chemiker)</a> (1909–2001), deutscher Chemiker</li>
<li> <a rel="mw:WikiLink" href="./Heinrich_Fürst_(Oberstudienrat)" title="Heinrich Fürst (Oberstudienrat)" class="new">Heinrich Fürst (Oberstudienrat)</a> (1861–1944), Oberstudienrat, fränkischer Kunstmaler, 1885–1926 Zeichenlehrer und Turnlehrer an der Kgl. Kreisrealschule in Nürnberg</li>
...
<li> <a rel="mw:WikiLink" href="./János_Fürst" title="János Fürst">János Fürst</a> (1935–2007), ungarischer Dirigent</li>
</ul>

It actually ends the first <ul> just before the template, then inserts a mysterious <span>, and then another <ul> begins – with these two elements being marked as generated from this transclusion (about="#mwt1"), rather than from normal wikitext. And then it ends the <ul> when another template appears.

I would expect output more like this:

<ul id="mwKg">
...
<li id="mwXQ"> <a rel="mw:WikiLink" href="./Gerhard_Fürst" title="Gerhard Fürst" id="mwXg">Gerhard Fürst</a> (1897–1988), erster Präsident des Statistischen Bundesamtes</li>
<li id="mwXw"> <a rel="mw:WikiLink" href="./Gustav_Gerson_Fürst" title="Gustav Gerson Fürst" id="mwYA">Gustav Gerson Fürst</a> (1840–1918), deutscher Maler und Dekorationskünstler</li>
<li about="#mwt1" typeof="mw:Transclusion" data-mw='{"parts":[{"template":{"target":{"wt":":Hans Fürst","href":"./Hans_Fürst"},"params":{},"i":0}}]}' id="mwYQ"><a rel="mw:WikiLink" href="./Hans_Fürst_(Politiker)" title="Hans Fürst (Politiker)">Hans Fürst (Politiker)</a> (1902–1968), Schweizer Jurist und Politiker (FDP)</li>
<li about="#mwt1"><a rel="mw:WikiLink" href="./Hans_Fürst_(Chemiker)" title="Hans Fürst (Chemiker)">Hans Fürst (Chemiker)</a> (1909–2001), deutscher Chemiker</li>
<li id="..."> <a rel="mw:WikiLink" href="./Heinrich_Fürst_(Oberstudienrat)" title="Heinrich Fürst (Oberstudienrat)" class="new">Heinrich Fürst (Oberstudienrat)</a> (1861–1944), Oberstudienrat, fränkischer Kunstmaler, 1885–1926 Zeichenlehrer und Turnlehrer an der Kgl. Kreisrealschule in Nürnberg</li>
...
<li id="..."> <a rel="mw:WikiLink" href="./János_Fürst" title="János Fürst">János Fürst</a> (1935–2007), ungarischer Dirigent</li>
...
</ul>

I guess this is another problem caused by the newline insertion, and a slight inconsistency with PHP Parser. PHP Parser adds a newline before templates whose contents begin with *, like this one (T14974: The newline added to a template, magic word, variable, or parser function that returns line-start wikicode formatting (*#:; {|) causes unexpected parsing), except if the template is already placed at the beginning of a line – I guess Parsoid does not have this exception, and that's why Parsoid output has the extra <span> containing only a newline and the list is split.

Here's a simplified test case:

== case 1 (normal, this should work) ==
* item1
* item2
{{1x|* templated1
* templated2}}
* item3
* item4

== case 2 (abusing newline insertion) ==
* item1
* item2 {{1x|* templated1
* templated2}}
* item3
* item4

== case 3 (abusing empty element removal) ==
* item1
* item2
* {{1x|* templated1
* templated2}}
* item3
* item4

You can test here:

All three of these lists render the same under the PHP Parser, but case 1 renders wrong under Parsoid.

VisualEditor in fact handles case 2 and 3 just fine:

matmarex renamed this task from By VE Identified template too big. Unable to edit page content to Template transclusion of a few list items extends to the entire list in VE, making it impossible to edit visually.

I guess this is another problem caused by the newline insertion, and a slight inconsistency with PHP Parser. PHP Parser adds a newline before templates whose contents begin with *, like this one (T14974: The newline added to a template, magic word, variable, or parser function that returns line-start wikicode formatting (*#:; {|) causes unexpected parsing), except if the template is already placed at the beginning of a line – I guess Parsoid does not have this exception, and that's why Parsoid output has the extra <span> containing only a newline and the list is split.

Ugh .. yes. Parsoid aims for independent parsing of transclusions and wants to push wikitext towards that going forward. So, ideally we would not rely on these exceptions that break that rule. But, we'll see if there is some post-processing we can do to deal with this.

How common is this kind of usage?

ssastry triaged this task as Normal priority.Mar 15 2018, 11:17 PM
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptFri, Jun 14, 11:33 AM
cscott added a subscriber: cscott.Fri, Jun 14, 3:55 PM

This *might* be related to Parsoid's slightly-incorrect handling of initial newlines in wikitext (T175421: Parsoid doesn't insert same spacers when article text starts with two or more newlines.) -- I think the "insert leading newline if the contents begin with *" part is actually done by the preprocessor, and so the result Parsoid gets back from the legacy preprocessor should have this hack (T14974, T2529) already applied.

This points out an interesting "type" for subbu's typed-wikitext-2.0 thought-experiment: "list items", which should be a forest of <li>s, or maybe a tree with <ul> at the top which should be merged into (instead of nested inside) any surrounding list of the same type.