Page MenuHomePhabricator

mw.text.jsonDecode throws Syntax Error exception on previously accepted TemplateData
Closed, ResolvedPublic1 Estimated Story Points

Description

On Vorlage:Literatur, a heavily used citation template, as of this morning the TemplateData JSON configuration is not decoded anymore. Instead, the error message "INTERNAL: mw.text.jsonDecode: Syntax-Fehler" is shown. This prevents the VisualEditor from automatically showing the fields and descriptions, making it very inconvenient to use the template.

The bug was first reported here on January 11th at 11 am CET. There have been no recent changes to the template configuration (as I'm reporting this on January 11th, the last change is from January 7th). Note that also past revisions show the error, even though they are certain to have been functional.

According to this validator, the JSON is not according to standard (probably due to umlauts), but has worked in the past. Has there been any change to the strictness of the JSON decoder?

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Related to T128029.

Not that I know of but IMO it's a good thing- we shouldn't be allowing invalid JSON :). If you just run the JSON through the validator and paste it back in, it should work.

Hmm, it looks like it's using a special template Vorlage:TemplateData which allows HTML to be in the template data. https://de.wikipedia.org/wiki/Vorlage:TemplateData

So the 'JSON' isn't actually JSON. (So what I suggested would work but cause it to lose that formatting .)

(Adding PerfektesChaos as de-wp's TemplateData expert.)

As far as I understand, Vorlage:TemplateData is only used to display the description on the page. Since the problem also affects the VisualEditor's capability to determine the default parameters and read out their description, there has to be an underlying problem.

To verify this, go to a random article on the German-language Wikipedia, open it in the VE, add a reference and choose "Literatur". It then claims that there is no TemplateData for this template, while in fact there is and it used to work just fine until today.

(I'm not advocating keeping invalid JSON, but I believe we should try to understand why the parsers suddenly fails.)

Might have been https://gerrit.wikimedia.org/r/#/c/398911/

This was a security fix that was backdated to several branches. Since it was parsing fix related to the description field, that seems most likely.

Anyway, the syntax was fixed on the template and citoid seems to be working again for Lituatur so I'm going to mark as resolved :).

Mvolz triaged this task as Medium priority.
Mvolz edited projects, added Security-Extensions; removed MediaWiki-General.

(Started writing before realising the task was closed, might as well share now)

That patch changes how the description value, after being parsed from the JSON, is shown on the page. For example, it meant that the following valid JSON "description":"<strong>hi</strong>", which was previously rendered as <strong>hi</strong> is now rendered as &lt;strong>hi&lt;/strong>.

The issue here however was that the template was using a template to produce the JSON itself, and was using another template within the JSON string, without applying <nowiki> to it (or otherwise ensuring it won't be interpreted as a template, before passing to TemplateData as JSON).

Before the edit that fixed it, it was using "description": "For example {{DOI|10.1000/182}}" from a place that was not directly inside <templatedata>, but as parameter to something else. This means that the value is first parsed as wikitext before the end result is seen as JSON by TemplateData. So what TemplataData saw was actually:

"description": "For example <span class="uri-handle">[//doi.org/10.1000%2F182 10.1000/182]</span>" which is invalid because the first quote in the <span> attribute would end the description string, and various other issues, resulting in invalid JSON. Even if it did work, it was not intended for the template to be parsed here, they wanted to put {{DOI}} literally in the description, which means it should either use <templatedata> directly (so that the JSON isn't first seen as wikitext), or by escaping it on some way, by using <nowiki>, or (as they ended up doing) by cleverly putting <code></code> in between the curly braces and the template name so that the parser gets confused :D

Post-resolving explanation.

First of all: sorry for confusionn.

The problem has been caused by a recent change in w:de:Template:DOI.

  • The URI is now wrapped into <span class="uri-handle"> supporting local and browser resolving tools rather than global URL target.
  • That brought " into JSON, as Krinkle remarked correctly.
  • Those " broke JSON.
  • Yes, template transclusions and markup are used here, rather than being escaped.

Yes, JSON is evolved into two formats:

  1. A plain text version, as common, for VisualEditor etc., with brief descriptions.
  2. A markup version with clickable links, as required on doc page, partially with exhaustive descriptions.