and others.

http://localhost:8000/en.wikipedia.org/v3/page/html/Wikipedia_talk%3AWikiProject_LGBT_studies%2FArchive_15/791871517
http://localhost:8000/en.wikipedia.org/v3/page/html/Talk%3AJean_Guyon/799612539
http://localhost:8000/en.wikipedia.org/v3/page/html/Talk%3AList_of_homeopaths/809401915

Change 469684 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] [WIP] Throws from TSP re-tokenizing

https://gerrit.wikimedia.org/r/469684

While the above patch fixes the crasher and improves the rendering from before it was exposed, there's still some work to do to match the php parser's output.

It seems like the token stream patcher needs to buffer all the tokens until the end of the line that are in table start tag attribute position and then stringify them if we're going to be retokenizing the start tag.

Just noting the original reduced test case as,

<noinclude>
<!--
--></noinclude>{|

There's a space after the table start tag.

Without digging into the details, is this lintable and classifiable as 'unsupported' behavior?

Change 469684 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Accept spaces at the end of table_start_tag

https://gerrit.wikimedia.org/r/469684

The source on that fiwiki page is pretty much,

<noinclude>
<!-- 
--></noinclude{| <includeonly>border="0" style="background: none" </includeonly><noinclude>class="wikitable"
</noinclude>

We probably support that kind of attribute tokenizing in other circumstances where we don't end up the token stream patcher.

TBH, I'm not really sure what use cases the token stream patchers in handling or why we end up there at all in this case.

Going to repurpose this task now that the fix for the crasher is merged.

Mentioned in SAL (#wikimedia-operations) [2018-10-29T20:45:28Z] <arlolra> Updated Parsoid to b9fa661 (T100841, T186965, T167349, T198618, T206040, T207956)