I isolated the problem under https://de.wikipedia.org/w/index.php?title=Benutzer:Kallichore/T429670&action=parsermigration-edit :

{{#tag:nowiki|de&params=54}}

Parsoid changes this to "de¶ms=54".

This is likely because the HTML spec doesn't require the entity to be ¶ but also recognizes &para as the pillcrow sign. This affects all other such named HTML entities without a trailing ";" and I tested a couple others and verified this.

There are about 106 entities that are affected by this, and if you create a HTML file with contents like &params=54 &quotes=54, it will show up in the browser as de¶ms=54 de"es=54 . So, this is definitely an expected outcome of the HTML5 parsing algorithm. The reason this is showing up now and not before is because Parsoid creates DOM fragments for parts of a page and in doing so, it parses HTML strings like these to DOM via the HTML5 parsing algorithm (the Remex library), and triggers this scenario. The legacy parser treats everything as string replacements and doesn't trip over it. That said, there is the Tidy pass that does run in legacy parser output that invokes Remex, so curious why this issue didn't trigger there. To be invesigated further.

And, looks like this URL shows up in two places in the document. Once in the indicator at the top of the page, and another time in the infobox. The URL in the indicator (Koordinaten: 50° 8′ 7,1″ N, 8° 39′ 16,9″ O) points to the link correctly. And, it appears that the way this issue is being worked around is by encoding the "&" as "&" ... so, the URL is https://geohack.toolforge.org/geohack.php?pagename=Europaturm&language=de&params=50.1353_N_8.654681_E_dim:250_region:DE-HE_type:building. So, it is &params there and not &params which effectively prevents the problem.

And, curious why Parsoid didn't trip over this in both places. In any case, it looks like Parsoid should preserve the & and not decode it.

Anyway, this transcript shows that this is just a bug in the Parsoid fragment handling because in all other contexts, Parsoid properly escapes the & entity.

ssastry@parsoidtest1001:/srv/parsoid-testing$ echo "<nowiki>de&params=5</nowiki> de&params=5 {{1x|1=de&params=5}} {{#tag:nowiki|de&params=5}}" | sudo -u www-data php /srv/mediawiki/multiversion/MWScript.php /srv/parsoid-testing/bin/parse.php --wiki=enwiki --integrated
<p data-parsoid='{"dsr":[0,89,0,0]}'><span typeof="mw:Nowiki" data-parsoid='{"dsr":[0,28,8,9]}'>de&amp;params=5</span> de&amp;params=5 <span about="#mwt1" typeof="mw:Transclusion" data-parsoid='{"pi":[[{"k":"1","named":true}]],"dsr":[41,61,null,null]}' data-mw='{"parts":[{"template":{"target":{"wt":"1x","href":"./Template:1x"},"params":{"1":{"wt":"de&amp;params=5"}},"i":0}}]}'>de&amp;params=5</span> <span typeof="mw:Nowiki mw:Transclusion" about="#mwt2" data-parsoid='{"pi":[[{"k":"1"},{"k":"2"}]],"dsr":[62,89,null,null]}' data-mw='{"parts":[{"template":{"target":{"wt":"#tag:nowiki","function":"tag"},"params":{"1":{"wt":"de&amp;params=5"}},"i":0}}]}'>de¶ms=5</span></p>

I briefly poked around in the background while watching soccer (!), but, it looks like the difference between building a DOM while building fragments and building a DOM while building the top level page is the HTML tokenizer. Both otherwise use Remex to "parse the HTML" and build the DOM.

While building fragments, we use the standard HTML5 tokenizer (which is subject to the HTML5 spec's named entity list tokenization algorithm), but while building the top level DOM, we use Parsoid's wikitext tokenizer and feed the tree builder tokens, and that tokenizer doesn't parse named character entities per the HTML5 spec. Specifically, that tokenizer has this grammar rule for entities which requires a trailing semicolon.

I think this difference in tokenization is why we run into this issue when creating DOM fragments out of wikitext content. So, we may need a hack / workaround when we convert parsoid-fragment-tokens to DOM fragments. Depending on where the fragment-tokens are coming from, in some cases, we may need to massage the "HTML string" to prevent the HTML5 named character recognition from kicking in.

Fragments from extensions (ex: cite, syntaxhighlight, etc.) can be considered HTML5 and don't need special handling, but fragments coming from templates and modules should be treated as "wikitext-flavored HTML" and might need this special handling.

<tangent>The section "Errors involving fragile syntax constructs" in https://html.spec.whatwg.org/#syntax-errors says There are syntax constructs that, for historical reasons, are relatively fragile. To help reduce the number of users who accidentally run into such problems, they are made non-conforming. So, this support for legacy web html content in the HTML5 parsing sepc is what is biting us here.</tangent>

@Thgoiter @Kallichore While we figure out how to cleanly handle this, you could handle it on the dewiki end right away by updating these templates to use &amp;params instead of &params in the url.

@ssastry thanks for watching soccer (and poking around). I can implement the workaround on WP:de.
This is not about only &para, but all known &xxx html special character sequences.

I now modified de:Vorlage:CoordinateSimple, de:Vorlage:CoordinateLink and de:Vorlage:CoordinateLINK. the other occurences in these templates are either just comments or do not use params.

Yeah, this is complicated:

• "wikitext entities" are different from "w3c html entities" in some corner cases. There are some comments about this in Sanitizer.php, but basically *HTML* allows some (not all) entities to omit the trailing semicolon, but *wikitext* requires a trailing semicolon for all entities. This is probably the shortest explanation of this issue: we used to parse that section using "wikitext" entity rules, and are now (for some reason) using "html" entity rules. (There are some other minor differences between wikitext entities and HTML entities that aren't relevant here; see Sanitizer::MW_ENTITY_ALIASES.)
• HTML5 has special rules for parsing urls in attributes (!) (mainly href attributes, but there's also src etc so the spec just applies this to any attribute): https://html.spec.whatwg.org/multipage/parsing.html#named-character-reference-state: "If the character reference was consumed as part of an attribute, and the last character matched is not a U+003B SEMICOLON character (;), and the next input character is either a U+003D EQUALS SIGN character (=) or an ASCII alphanumeric, then, for historical reasons, flush code points consumed as a character reference and switch to the return state." That means that &blahblahblah= is *always* parsed as literal text *in an attribute*. This only applies in an attribute, and only to those entities which HTML allows to be semicolon-less, for historical reasons.

I believe "officially" the & in a URL is "always" supposed to be escaped as &amp; in an attribute but the above rules are part of the backward compatibility machinery added to the spec to handle the fact that this is rarely done in practice. But in general we don't implement these exceptions for wikitext because instead we just insist that all entities be semicolon-terminated *in wikitext*.

As subbu points out, though, the output of some extensions is "half parsed wikitext" not really HTML, and so we need to be careful about using DocumentFragment to represent these, since the meaning of the entities is different in wikitext and HTML.