In T51904#1406296, @Krenair wrote:

I'm not sure the parsoid API from T73161 is quite what we were looking for. The idea had been to allow the user to mark a template to be substituted upon save, but with this we'd have to substitute an entire transclusion (immediately?)

The new performance counters integrated into Parsoid supersede this.

A little status update:

This also happened in the Catalan Wikipedia, there's a transcript of the error at https://ca.wikipedia.org/wiki/Topic:Shssmsofmcgkqhmv .

Deployed in https://www.mediawiki.org/wiki/Parsoid/Deployments#Monday.2C_May_18.2C_2015_around_1:10pm_PST:_Y_Deployed_8ed3e503

In T86782#1300693, @santhosh wrote:

Note that there are usecases where we need to display the references in two languages in same wiki page. CX has to show source language content and target language content with two different references style. So if we load the style based on content language, we get style for only one language. How to solve this issue?

We could just concatenate the target and the arguments in wikitext, or even just serialize it back to wikitext at that point (which would take more time).

Are we sure about the second case (&lt;a&gt;…)? Given that explicit <a> tags aren't allowed in wikitext, Parsoid already encodes it correctly:

echo '<a rel="mw:ExtLink" href="http://www.google.com"&g t;Google</a>' | node parse --html2html
…
<p data-parsoid='{"dsr":[0,59,0,0]}'>&lt;a rel="mw:ExtLink" href="http://www.google.com">Google&lt;/a></p>

What's the special type for it? mw:DisplaySpace?

I'm unable to reproduce. I suppose this was fixed in the round of nowiki work last quarter. Please reopen if seen again.

Investigating…

On IRC discussions, it was argued that mw:Entity wasn't the best typeof to use in this case. A fer other possibilities were discussed, including:

• Create a new typeof like mw:DisplayHack, mw:SpaceBeforeColon, etc.
• Add a new typeof to an exiting one, i.e. typeof="mw:Placeholder mw:DisplayHack".
• Just have VE not treat this case as a mw:Placeholder and allow the user to delete them.

Preliminary patches are available in gerrit. They are all considered WIP, but pushing them to get early feedback on the general approach.

Seems related to T97155.

For completeness, here's a screenshot with VE editing the new HTML and CSS, patched so that it copies the new HTML over:

A little update after the latest patch available at https://gerrit.wikimedia.org/r/#/c/170936/, which includes the new stuff done in master such as using data-mw.body.id, removing state from the Cite extension, etc.

Deployed in https://www.mediawiki.org/wiki/Parsoid/Deployments#Wednesday.2C_April_22.2C_2015_around_1:05pm_PST:_Y_Deployed_3311936a.

In T95794#1215710, @ssastry wrote:

It just needs to replace "\n" with "\n " and tokenize it.

After a little investigation: in wts.escapeWikitext.js, WEHP.hasWikitextTokens calls the tokenizer with the right SOL state, but the second line is tokenized to a listItem TagTk, which provokes it being surrounded with <nowiki>s. Maybe there should be a way for the tokenizer to act differently if the whole text is in pre?

I just tried using normal whitespaces and (besides a few changing selser blacklist results) everything seems to work OK. There's even one test for T71950 that now passes.
Changing the mw:Placeholder to mw:Entity should also work, but it triggers a lot of false failing tests due to differences in whitespace, which we don't normalize out yet.

Placing up for grabs as the Cite part is being dealt with at T63165.

I'm investigating the Cite refs part of the issue. It seems that the changes are in the ids for the <reference> entries, which changes the reflinks and the data-mw.body.id fields.

Reproducible in Parsoid:

$ echo -e "<gallery>\nFile:foo.jpg|A <ref group="n">ref one</ref>\n</gallery>" | node parse
…
<ul class="gallery mw-gallery-traditional" typeof="mw:Extension/gallery" data-mw='{"name":"gallery","attrs":{},"body":{"extsrc":"\nFile:foo.jpg|A &lt;ref group=n>ref one&lt;/ref>\n"}}' data-parsoid='{"dsr":[0,62,2,2]}' about="#mwt3">
<li class="gallerybox" style="width: 155px">
<div style="width: 155px">
<div class="thumb" style="width: 150px;">
<div style="margin:35.5px auto;"><a href="/wiki/File:Foo.jpg" class="image"><img alt="" src="//upload.wikimedia.org/wikipedia/commons/thumb/0/06/Foo.jpg/120px-Foo.jpg" width="120" height="79" srcset="//upload.wikimedia.org/wikipedia/commons/thumb/0/06/Foo.jpg/180px-Foo.jpg 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/0/06/Foo.jpg/240px-Foo.jpg 2x" data-file-width="300" data-file-height="197"/></a></div>
</div>
<div class="gallerytext">
<p>A <sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span>[</span>n 1<span>]</span></a></sup></p>
</div>
</div>
</li>
</ul><span about="#mwt3">
</span><p about="#mwt3"><br/>
<strong class="error mw-ext-cite-error">Cite error: There are <code>&lt;ref group=n></code> tags on this page, but the references will not show without a <code>{{reflist|group=n}}</code> template (see the <a href="/wiki/Help:Cite_errors/Cite_error_group_refs_without_references" title="Help:Cite errors/Cite error group refs without references">help page</a>).</strong>
<link rel="mw:PageProp/Category" href="./Category:Pages_with_missing_references_list#Main%20Page"/></p>

Sorry, wrong duplicate.

This issue is unrelated to extension parameters being parsed as wikitext, which was the original issue here. I see two problems with that section in http://parsoid-lb.eqiad.wikimedia.org/enwiki/Klondike_Gold_Rush?oldid=651957368, though, so I've opened two new bugs for them:

• T93580: The <ref> in the image's caption isn't detected.
• T93578: The <ref group=n> inside the <gallery> produces an error from Cite.php.

This change was deployed on March 19.

In T73708#1126551, @Krenair wrote:

What about files?

The problem comes from the escaping of template arguments, which sees the '[' separately from the ']' due to the italicized text in the middle and thus wraps both sides in nowikis:

I'm also unable to reproduce, I always get ''', project 'RESTBase'''' at the end of the line. There's been a lot of nowiki work in Parsoid since this change, so I'm closing as resolved, but feel free to reopen if it shows up again.

With the fix for T71950, the whole <ref> text is wrapped in a single <nowiki>. The single <nowiki/> doesn't appear, so I'm closing this one as resolved.

Closing this one as it should have been fixes with T71950.

Deployed in https://www.mediawiki.org/wiki/Parsoid/Deployments#Monday.2C_Mar_9.2C_2015_around_1pm_PST:_Y_Deployed_c8370a48.

In T88623#1109590, @Catrope wrote:

Given that T90374 will almost certainly be done before T78676, it would be nice for RESTbase to offer recombined HTML. In VE, we may want to load data-mw separately for encoding efficiency, but given that reference data has been moved out already, that doesn't seem very urgent, nor is it even obvious that it's a good idea.

I haven't been able to reproduce the described issue. In any case, patch https://gerrit.wikimedia.org/r/#/c/193842/ for T71950 should improve the positioning of nowikis around placeholders like the one produced before the colon in 'intitulé : "Plan'.

Sorry, the correct URL for the updated Parsoid instance is http://parsoid-lb.eqiad.wikimedia.org/mediawikiwiki/Extension%3AGraph%2FDemo. The results are the same, though.

@Yurik: That labs instance is the Parsoid we use for editing, and the page should have been produced directly from the graph demo page. Looking at the page source, I see that the example named "Embedded directly" has the data inside <graph>, and that's the one I'm saying that Parsoid passes along to VE. It doesn't really matter how the extension later renders that page into HTML, Parsoid only looks at the wikitext source.

Parsoid already includes the graph data JSON in the metadata for editing. If you go to http://parsoid.wmflabs.org/mediawikiwiki/Extension%3AGraph%2FDemo and look at the source, you'll see

There is indeed a non-breaking space between intitulé and the colon, but I'm unable to reproduce the bug. With a current Parsoid (and without having the actual change HTML), the whole sentence gets put inside a <nowiki> but the one around the &nbsp; doesn't appear.

This is a WIP that should fix the https://fr.wikipedia.org/w/index.php?title=Grand_Line&curid=4571652&diff=111992974&oldid=111360543 case, but I want to make sure it doesn't break anything else.

The current patch creates a problem for pages that have multiple <ref>s with the same name but different content. For an example, see the <ref> named "DeVries2007" in https://en.wikipedia.org/wiki/History_of_weapons.

After the last patch, there are two cases of data-parsoid.src in Barack_Obama:

• mw:Entities (), which need src to distinguish between different representations (&nbsp; and &#xA0, for example)
• Page properties (1, the page's DEFAULTSORT), which need src to distinguish between different valid spellings (like {{DEFAULTSORT}} and {{defaultsort}}.

In T86633#1069583, @yuvipanda wrote:

https://gerrit.wikimedia.org/r/#/c/193082/ has basically killed the current beta role and just picked up the prod role, and testing on deployment-parsoid01-test seems to work fine, but I *think* it is configured to use prod. I see that beta used to have /srv/deployment/parsoid/deploy/conf/wmf/betalabs.localsettings.js as its settings file but I have no idea what puts it there / where it comes from.

Somehow this disappeared, but there's a Patch-For-Review at https://gerrit.wikimedia.org/r/191593.

In T88017#1066495, @gerritbot wrote:

Change 192838 had a related patch set uploaded (by Marcoil):
T88017: Remove more cases of data-parsoid.src from mw:Extensions

https://gerrit.wikimedia.org/r/192838

Waiting on feedback from VE team.

In T88290#1053317, @marcoil wrote:

The current patch can't remove the duplication for <ref>s inside <references>, so the actual size reduction in [[Barack_Obama]] is lower: ~884K. I'll keep investigating to find a way to remove those cases too.

The current patch can't remove the duplication for <ref>s inside <references>, so the actual size reduction in [[Barack_Obama]] is lower: ~884K. I'll keep investigating to find a way to remove those cases too.

In the (admittedly extreme) case of [[Barack_Obama]], the removal of duplicates reduces the HTML size by a little over 1M ;)

In T88017#1044070, @ssastry wrote:

I think it would be good to get rid of the <meta typeof="mw:Extension/ref/Marker"../> tags from the output. I think a similar fix as for T88019 should do it after making sure they are just something that evaded cleanup.

Once this is fixed, we should revisit the test "References: 9." to create some manual changes and ensure that new <references /> are not created for changes unrelated to orphan <ref>s. See T88660.

In T88383#1036809, @GWicke wrote:

Unfortunately, abouts change the subject of nested RDF statements, so can't generally omitted without changing the RDF graph.