First off, the output of both Parsoid as well as Parser.php are both not HTML5 compliant. https://www.mediawiki.org/wiki/Parsing/Notes/HTML5_Compliance has some notes I had started compiled from when we were replacing Tidy with RemexHtml. As that page makes clear, it is non-trivial to generate HTML5-compliant output.

There is no reason to keep this bug open for a specific page, especially for Parsoid/JS. Parsoid/PHP OOMs are tackled separately.

The above gerrit patches have moved the needle on this by doing a cleanup of extension code in the Parsoid repo by reducing exposure of Parsoid internals. But more work is needed. See commit message of gerrit 565563

To be clear, supporting arbitrary HTML is part of Parsoid's long-term HTML -> wikitext roadmap ( we originally antiicpated having that functionality in place well before now, but there was no pressing demand for it for us to prioritize that work ), and so eventually Parsoid will generate valid serialized output for HTML like this by switching to native HTML tags as required. But, that approach won't help here.

I am not sure there is an easy fix for this on the Parsoid end. The crux of the matter can be demonstrated with this snippet:

[subbu@earth:~/work/wmf/parsoid] cat /tmp/x.html
<h2>"native wikitext" representation exists for the table-in-list embedding below</h2>
<dl><dd><table><tr><td>x</td></tr></table></dd></dl>

This is a known issue. We'll be bumping the memory limit a bit more ( https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/564805 ).

In T242513#5814720, @Pols12 wrote:

I still reproduce on en.wikipedia.beta.wmflabs.org

In T240280#5782645, @cscott wrote:

From david: “The ability to enter :{| into any page and have it strip all newlines the next time Parsoid interacts with said page” isn’t great!
This is relevant to talk pages, since : is auto-inserted by the post-a-reply tool to get proper indentation. So the user "just" made a little typo when trying to create a table in the reply, and suddenly the whole page gets corrupted.

In T140606#5795256, @Jonesey95 wrote:

Is there any progress on this feature request? Signatures are still being added that cause Linter errors, e.g. this one from 11 January 2020:
https://en.wikipedia.org/w/index.php?title=Wikipedia%3AAdministrators%27_noticeboard%2FIncidents&type=revision&diff=935325391&oldid=935325138

In T242440#5804004, @Jdforrester-WMF wrote:

With https://www.mediawiki.org/wiki/Parsoid/Releases#0.11.0_(released_Jan_14,_2020) is this Resolved?

In T242757#5802816, @Dzahn wrote:

I'll assume it was literally the "Mirror sync in progress" (103.102.166.224 is eqsin).

In T242513#5802309, @JTannerWMF wrote:

We believe this is something that would potentially be changed in Parsoid, so we are adding the Parsing team to this task.

In T114432#5793209, @cscott wrote:

@Anomie brings up some interesting corner cases over in T230683: New syntax for multiline list items / talk page comments for extensions of this syntax:
...

I ran a rt test on scandium, and it came back clean when compared to the last JS rt test run on Nov 30. So, we can prepare the deb and release it.

In T230683#5792972, @Anomie wrote:

In T230683#5791876, @ssastry wrote:

I was saying: heredoc is NOT new syntax.

It doesn't actually exist yet in MediaWiki, even in master.

Let us wait for T241207 because we do have at least one more Parsoid/JS rt test run to do on scandium -- so that our final Parsoiid/JS deb release is tested with all the minor code cleanup since we stopped traffic to it in production.

T134469 is related but as @cscott noted, it is not going to be a simple thing to fix and rollout since it will probably require a whole lot of linting to clean up pages to prevent breakage.

But, see T110909: Parsoid tripped up by extensions that process wikitext as well. This year, we are going to start working on the parser integration work, and last year, we thought that by now, we would have a first draft of the new proposed parser hooks / Parsoid extension API, but we are a bit behind at this point since the Parsoid PHP port took longer and we still have some post-port followup work to finish ( T239660 being the big one ). So, it might still be a quarter before we have the Parsoid extension API proposal for review. We also have to get a Parser API resolved as well and that will go hand in hand with the extension API.

In T151308#5787348, @Izno wrote:

I honestly see the implementation of citation content (and other extensions) in Parsoid as technical debt to be cleaned up/removed eventually by the 1-parser-march. @ssastry should probably comment on the general plan for those extension reimplementations, if that has been considered previously (and if not, maybe it should be discussed somewhere more general).

In T230683#5792177, @Jc86035 wrote:

This is a comparison table of the various proposals and existing constructs that I'm currently aware of, with a focus on the syntax differences. (n is the nesting level; ⏎ indicates a line break; the "Compatible?" column indicates whether the proposal would allow mixing of old and new syntax.) There (understandably) aren't a lot of editors putting out fully realized proposals, so I've had to guess how Wnt and Jeblad's proposals would have worked syntax-wise.

In T230683#5791184, @Anomie wrote:

In T230683#5790881, @ssastry wrote:

Actually, this discussion is exactly why I suggested reframing the syntax-choice discussion as: "what syntactical support is needed to proceed that doesn’t foreclose future cleanup options”. Multi-line list items have always been a big pain point, especially in talk pages. Instead of trying to propose totally new syntax that are narrowly targeted to talk pages, or having to bikeshed choice of what kind of new list syntax we want, or if we want new syntax at all... all of which are going to bog down the talk page reboot project unnecessarily, I feel it is useful to refocus our attention narrowly on what is needed to help that project succeed for everyone's sake (not just for our own convenience as developers).

New syntax is being proposed on all sides. We should take the time to consider the best new syntax, rather than arbitrarily choosing one for expediency.

Actually, this discussion is exactly why I suggested reframing the syntax-choice discussion as: "what syntactical support is needed to proceed that doesn’t foreclose future cleanup options”. Multi-line list items have always been a big pain point, especially in talk pages. Instead of trying to propose totally new syntax that are narrowly targeted to talk pages, or having to bikeshed choice of what kind of new list syntax we want, or if we want new syntax at all... all of which are going to bog down the talk page reboot project unnecessarily, I feel it is useful to refocus our attention narrowly on what is needed to help that project succeed for everyone's sake (not just for our own convenience as developers). Certainly, there are arguments to be made for better syntax for talk pages or "indent-pre" (I dislike that syntax personally, for the record) or whatever else we are unhappy about in wikitext. The question there is if we should be making piecemeal decisions or think about new syntax more holistically. I don't know what approach will succeed, but what I do know is that we should avoid getting the first minimal talk page reboot project stuck on new syntax proposals for list items or only talk pages.

@jlinehan, are you able to work on this this month?

This means that VE (or Parsoid) understands that a footnote is not supposed to appear in a link, and quietly truncates what the user highlighted. This is reasonable behavior, and it should also happen when {{refn}} is used.

In T241202#5788891, @Joe wrote:

This happens because parsoid/js is still deployed in production, so it also gets monitoring.
Whenever we're allowed to remove it from production, we'll be happy to turn all of that off.
@ssastry can we set a dismissal date?

See T214649#5745042 for a discussion about which this was a real regression.

Reopening it temporarily to close this separately as a real Parsoid-PHP regression. This was not a duplicate.

In T237267#5769177, @Legoktm wrote:

I definitely think it's worth revisiting this since T39042 was decided 8 years ago since our stance on what changes to wikitext we're willing to make is rather different. I would suggest getting the input of the Parsing-Team and then probably writing an RfC for it.

I don't think this is related to Parosid/PHP deployment. Looking at output from Parsoid/JS & Parsoid/PHP, output is identical .. the HTML comes from a template, so probably worth looking there:

data-mw='{"parts":[{"template":{"target":{"wt":"ﺺﻧﺩﻮﻗ ﻙﺭﺓ ﻕﺪﻣ\n","href":"./ﻕﺎﻠﺑ:ﺺﻧﺩﻮﻗ_ﻙﺭﺓ_ﻕﺪﻣ"},"params":{"ﺕﺍﺮﻴﺧ":{"wt":"30-09-2551"},"ﻮﻘﺗ":{"wt":"20:30"},"ﻑﺮﻴﻗ1":{"wt":"{{ﻙ ﻕ-ﻲﻣ|BRA}}"},"ﻦﺘﻴﺟﺓ":{"wt":"12 –  1"},"ﺖﻗﺮﻳﺭ":{"wt":""},"ﻑﺮﻴﻗ2":{"wt":"{{ﻊﻠﻣ|ﺎﻠﻳﺎﺑﺎﻧ}}"},"ﺄﻫﺩﺎﻓ1":{"wt":""},"ﺄﻫﺩﺎﻓ2":{"wt":""},"ﻢﻠﻌﺑ":{"wt":"บราซิเลีย"},"ﺢﺿﻭﺭ":{"wt":""},"ﺢﻜﻣ":{"wt":""}},"i":0}}]}

So, this piece of wikitext on the page:

In T235307#5770013, @ssastry wrote:

...
So, overall, this refactoring might not be too hard.

In T235307#5770013, @ssastry wrote:

With some git grep, sed, sort, uniq, etc, here is how Env is being used:
Content version functionality:

$env->getInputContentVersion()
$env->getOutputContentVersion()
$env->setInputContentVersion( ... )
$env->setOutputContentVersion( ... )
$env->resolveContentVersion( ... )

With some git grep, sed, sort, uniq, etc, here is how Env is being used:

Yes, that is indeed the plan. Parsoid-PHP is a subproject of Parsoid right now and was created just for the duration of the porting. We'll keep it around till we are done dealing with post-port tech debt work which is actually an explicit goal of ours for this quarter.

Parsoid/JS is not in use any more.

Parsoid/JS is not in use any more.

Parsoid/JS is not in use any more.

Sorry, what is this about? Parsoid is still an active project, just that it is in PHP.

Parsoid/JS is not in use any more.

Parsoid/JS is not in use any more.

Need to revisit this phab task and update the description for parse.php now.

This port to PHP resulted in some natural split up and refactoring. We won't do this for the JS codebase.

Parsoid/JS is not in use any more and coveralls is JS-specific.

Parsoid/JS is not in use any more.

Parsoid/JS is not in use any more.

We need to retarget this phab task for Parsoid/PHP.

For the Parsoid/PHP port, we built a benchmarking script which can potentially be repurposed for this as necessary. Can reopen / create a new ticket if we ever find a need for doing this and our current solution isn't good enough.

Parsoid/JS is not in use any more and the call stack profile is different for the PHP version without the async callbacks.

Parsoid/JS is not in use any more.

Parsoid/JS is not in use any more.

Parsoid/JS is not in use any more.

Parsoid/JS is not in use any more.