A post-deploy test seems to preserve headline spacing,
https://office.wikimedia.org/w/index.php?title=User:ABreault_(WMF)/sandbox&type=revision&diff=253119&oldid=253118&diffmode=source

As @cscott says,

The JS mock API and the ApiHelper.php don't exactly play nice together, in that they have different ideas of which formatversion to use. There's also some redundancy with the JS baseconfigs and the ApiHelper.php cached data.

Only on private wikis.

Since templated regions might expand or contract between parses, the encoding should probably include both dsr[0] and dsr[1]. Contractions that expose nodes could result in additional dirty diff'ing, but shouldn't produce corruption.

I guess the next steps are,

Right, I think I've exhausted my usefulness here, thanks.

If there's a solution to assign stable ids to the nested content, why not just use that for the entire page?

dsr isn't applied in templated regions (all data-parsoid is dropped in there as is), it only corresponds to the top-level page.

Basically, this amounts to, if we have valid dsr info, use that for generating an id. If not, discard data-parsoid for that node.

For the most part, this corresponds with the code here,
https://github.com/wikimedia/parsoid/blob/master/lib/wt2html/pp/handlers/CleanUp.js#L194-L214

See T123446#5098558

Is the process.exit going to be removed? It's causing issues in MWOffliner

Post deploy testing,
https://en.wikipedia.org/w/index.php?title=User:Arlolra/sandbox&type=revision&diff=889591137&oldid=884316233&diffmode=source

Since about a week ago, ...

It would be a good idea to add some parser tests which exercise both modes of StringUtils::explode

The final result, I41a45266d266195aa6002d3854e018cacf052ca6, is the one we want but since they all stack on top of each other, you might have to do all three. It's fine to just rebase the last one though.

Hold up, there's another bug here with ExplodeIterator we're discussing.

@zeljkofilipin Looking, thanks. Maybe it should be an iterator_count ...

Since https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/497320 merged, I'm going to assume this is done.

We've been looking for alternate DOM libraries and they don't seem to be common, alas.

We should remove the column that says "Nested wikilinks in external links that need to be fixed" since it doesn't seem to be used.

but there doesn't appear to be good dsr info to act on,

For the record, this is what was done from T214099#4981608,

This category is now being populated, but there doesn't appear to be good dsr info to act on,
https://en.wikipedia.org/wiki/Special:LintErrors/wikilink-in-extlink

T217597 looks related

Hmm, so the file was renamed recently on 2018-02-10,
https://commons.wikimedia.org/w/index.php?title=File%3ACoat_of_arms_of_Seychelles.svg&type=revision&diff=338463956&oldid=338257332

RESTBase provides fake Main_Page title to transform endpoint. I'm not sure if that's even needed any more, but seems like just a workaround of a bug in Parsoid request validation.

In any case, @cscott, this is a testing gap currently.

Why is num_workers set to 1 there? I'm reading the git commit message that changed it, but I don't understand the rationale.

This wasn't an issue deployed today.

Pretty sure the following patch is the problem of why Parsoid's maxConcurrentCalls isn't being respected.

Pretty sure the following patch is the problem of why Parsoid's maxConcurrentCalls isn't being respected.

Is there a phab task somewhere for "Parsoid is nondeterministic" generally? I seem to recall creating one for the "reproducible parses" issue...

I think you're confused between the test failure and the blacklist change.

From https://github.com/wikimedia/parsoid/commit/05f0f9b92314f64e567ef2fd4a04b4c9e8ee4692 when it was ignored in CI,

That's from T204608

https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/490654 identifies where all the docs are currently created.

Tests for content-negotiation rely on Parsoid being able to produce 2 different major content versions. Since a change in Parsoid 1.x content version is not supported anymore.

https://en.wikipedia.org/api/rest_v1/page/html/User:ESanders_(WMF)%2Fsandbox/877762681 now has,

http://localhost:8000/nl.wikipedia.org/v3/page/html/De_Meern/53147456 give this example,

https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-2019.02.07/parsoid?id=AWjIsGyznlBds_JjA_-S&_g=h@44136fa
http://localhost:8000/fr.wikipedia.org/v3/page/html/Utilisateur%3AChico75%2Fpages_avec_pourcent/156246032

No longer seems to be an issue with http://localhost:8000/sr.wikisource.org/v3/page/html/%D0%9A%D0%BE%D1%81%D0%BE%D0%B2%D1%81%D0%BA%D0%B8_%D0%B1%D0%BE%D1%98_(%D1%80%D1%83%D0%BA%D0%BE%D0%BF%D0%B8%D1%81)/60110

See the review at https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/488100/6//COMMIT_MSG

Note that this also happens when unpacking dom fragments,
https://github.com/wikimedia/parsoid/blob/master/lib/wt2html/pp/handlers/unpackDOMFragments.js#L200-L204

@ssastry Parsoid needs an equivalent patch that removes,
https://github.com/wikimedia/parsoid/blob/master/lib/wt2html/pp/processors/PWrap.js#L35-L36