In T317857#8238863, @Aklapper wrote:

Hi @Seudo, thanks for taking the time to report this!
Do you imply that this still worked a few days ago? Do you have a link to a test case?
Does it work when using three instead of two headings?

Here is a test page: https://fr.wikisource.org/wiki/Wikisource:TestTOC
You may play with it as you wish (I will remove it one of these days).

The bug was reported today at 08:46 (UTC) on the French Wikisource by the main contributor to this page: https://fr.wikisource.org/wiki/Wikisource:Facsimil%C3%A9_%C3%A0_int%C3%A9grer

He says that the TOC still existed a few hours before. It's a very large TOC with a lot of entries (they are created with a template that inserts a h2 HTML tag).

Here is a web.archive link to the same page in last January: http://web.archive.org/web/20220129143737/https://fr.wikisource.org/wiki/Wikisource:Facsimil%C3%A9_%C3%A0_int%C3%A9grer (notice the TOC on the top right corner)

I believe the TOC has disappeared in the Wikisource: and Help: namespaces with all skins except Vector 2022.

Also broken on Meta as @Mykola7 has noticed, for example no ToC on https://meta.wikimedia.org/wiki/Steward_requests/Global , so in fact this is not about non-main ns only.

This appears to be impacting all skins.
I'm seeing <meta property="mw:PageProp/toc"> in the article so presumably it's not being replaced.

Guessing, this might relate to T311502 ?

I can't replicate this in user namespace (https://en.wikivoyage.org/wiki/User:Jdlrobson/draft https://www.wikidata.org/wiki/User:Jdlrobson/draft, https://fr.wikisource.org/wiki/User:Jdlrobson/draft)

I thought it was a project namespace issue, but since it's occurring in https://meta.wikimedia.org/wiki/Steward_requests/Global I'm a bit stumped at identifying what these pages have in common (works fine on https://meta.wikimedia.org/wiki/User:Jdlrobson/draft). Unable to replicate this locally on master.

~~~Does purging the page from the parser cache help? I wonder if this is a just a production caching issue?~~~

From my reading of the code, there are only two ways the <meta property="mw:PageProp/toc"/> to show up in the output, as it does for https://fr.wikisource.org/wiki/Wikisource:TestTOC:

1. Something is tidying or otherwise modifying the marker such that Parser::replaceTableOfContentsMarker() doesn't match. For example, DiscussionTools could be running tidy on the output, which changes whitespace in the meta tag in some way such that the regexp no longer matches.
2. ParserOutput is being called with allowTOC=true and injectTOC=false even though injectTOC should be true for the vector 2020 skin.

works fine on https://meta.wikimedia.org/wiki/User:Jdlrobson/draft

Not for me, I don't see a ToC there. But I do see those on the voy:en, d: and s:fr: pages.

If it is helpful, here are the headers that seem to be relevant to determine which cache I am using:

server
	mw1355.eqiad.wmnet
server-timing
	cache;desc="pass", host;desc="cp3060"
strict-transport-security
	max-age=106384710; includeSubDomains; preload
vary
	Accept-Encoding,Cookie,Authorization
x-cache
	cp3050 miss, cp3060 pass
x-cache-status
	pass

Not specific to that page content too (I was thinking that perhaps no toc magic word is sneaked in there somehow), for instance I don't see a ToC at https://meta.wikimedia.org/wiki/Reports/WMRU-Report-2020

I am thus not sure about

when __TOC__ magic word present

as I don't see an obvious relevance whether the magic word is there or not

It could be related to DiscussionTools, which is only enabled (ie only modifies the page HTML on) pages in certain namespaces or which "look like" talk pages. That might explain why its hard to reproduce locally.

You are right. All these pages have discussion tools.

https://fr.wikisource.org/w/api.php?action=parse&format=json&page=Wikisource%3ATestTOC shows the page without the space before the final /> in the <meta> tag. The regex in Parser::replaceTableOfContentsMarker() is a bit fragile; I didn't really expect other code to be mutating the page HTML.

In T317857#8239745, @Base wrote:

works fine on https://meta.wikimedia.org/wiki/User:Jdlrobson/draft

Not for me, I don't see a ToC there. But I do see those on the voy:en, d: and s:fr: pages.

I don't see a ToC either in this page.

But I see it at https://meta.wikimedia.org/wiki/User:Seudo/Test2 which contains exactly the same wiki code (I copied it)...

In T317857#8239778, @Seudo wrote:

In T317857#8239745, @Base wrote:

works fine on https://meta.wikimedia.org/wiki/User:Jdlrobson/draft

Not for me, I don't see a ToC there. But I do see those on the voy:en, d: and s:fr: pages.

I don't see a ToC either in this page.

But I see it at https://meta.wikimedia.org/wiki/User:Seudo/Test2 which contains exactly the same wiki code (I copied it)...

This adds strength to the DiscussionTools theory. When I first saved it the table of contents was present and discussion tools was not active. Now it is active. I assume that gets turned on via some kind of JobQueue?

But I see it at https://meta.wikimedia.org/wiki/User:Seudo/Test2 which contains exactly the same wiki code (I copied it)...

I don't.

For the record I don't have the Discussion tools beta feature activated on Meta, but I guess some things out of the functionality work for me anyway.

In T317857#8239783, @Base wrote:

But I see it at https://meta.wikimedia.org/wiki/User:Seudo/Test2 which contains exactly the same wiki code (I copied it)...

I don't.

Yes, it has disappeared somehow.

I just tried copying the same code to https://meta.wikimedia.org/wiki/User:Seudo/Test3 :

• the TOC appeared when previewing,
• it was still there after saving the page,
• then I purged (?action=purge) and the TOC disappeared.

Hey @Esanders @matmarex can you point us to the modifications DiscussionTools makes to parser cache that might be causing the issue @cscott is describing in T317857#8239775?

Our modifications to the page HTML happen in CommentFormatter

I don't think that we deliberately do anything to a meta tag, but we do:

• call ParserOutput's getTOCHTML method to test whether the TOC is going to be shown
• use Wikimedia\Parsoid\Utils\DOMUtils to parse the HTML and then Wikimedia\Parsoid\Wt2Html\XMLSerializer to serialize it back down

The latter seems more likely to be introducing an issue, really.

Change 832512 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@master] Use more permissive match for TOC_PLACEHOLDER in parser output

https://gerrit.wikimedia.org/r/832512

Yeah, I think Parsoid's XMLSerializer is munging some whitespace. The above patch makes the TOC replacement less strict in core, which is probably a good idea for eventual Parsoid read views anyway. I can test it locally to some degree, but I haven't managed to get discussion tools enabled on my local install yet (that is, it's installed but isn't enabling itself on my test page on my local wiki), so it would be helpful if DT devs could reproduce the bug and verify that the above patch fixes the issue.

Ok, https://en.wikipedia.beta.wmflabs.org/wiki/Talk:Abraham_Lincoln has enough headings that it is demonstrating this bug (be sure to turn off the DT features at the bottom of https://en.wikipedia.beta.wmflabs.org/wiki/Special:Preferences#mw-prefsection-editing ) to get the same experience as the bug submitter was getting on wikisource. So if https://gerrit.wikimedia.org/r/832512 is merged on master, it should deploy on beta automatically and we can verify that the bug goes away there before cherry picking to production.

UPDATE: managed to get this tested on my local install as well. Everything looks good.

Change 832512 merged by jenkins-bot:

[mediawiki/core@master] Use more permissive match for TOC_PLACEHOLDER in parser output

https://gerrit.wikimedia.org/r/832512

Looks good on https://en.wikipedia.beta.wmflabs.org/wiki/Talk:Abraham_Lincoln?useskin=vector now.

Yep, looks good to me, too. I'll schedule the backport deploy.

Change 832547 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@wmf/1.40.0-wmf.1] Use more permissive match for TOC_PLACEHOLDER in parser output

https://gerrit.wikimedia.org/r/832547

For backporters: once the patch is deployed, there should be an inline TOC visible at the https://fr.wikisource.org/wiki/Wikisource:TestTOC?useskin=vector
You shouldn't need to purge the page to make this appear after deploy.

https://fr.wikisource.org/wiki/Wikisource:TestTOC?useskin=vector-2022 shouldn't regress after deploy, it should have a ToC in the side bar both before and after deploy.

Change 832547 merged by jenkins-bot:

[mediawiki/core@wmf/1.40.0-wmf.1] Use more permissive match for TOC_PLACEHOLDER in parser output

https://gerrit.wikimedia.org/r/832547

Mentioned in SAL (#wikimedia-operations) [2022-09-15T18:29:24Z] <dancy@deploy1002> Started scap: Backport for [[gerrit:832547|Use more permissive match for TOC_PLACEHOLDER in parser output (T317857)]]

Mentioned in SAL (#wikimedia-operations) [2022-09-15T18:29:50Z] <dancy@deploy1002> dancy and cscott: Backport for [[gerrit:832547|Use more permissive match for TOC_PLACEHOLDER in parser output (T317857)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2022-09-15T18:39:18Z] <dancy@deploy1002> Finished scap: Backport for [[gerrit:832547|Use more permissive match for TOC_PLACEHOLDER in parser output (T317857)]] (duration: 09m 53s)

In T317857#8240143, @cscott wrote:

For backporters: once the patch is deployed, there should be an inline TOC visible at the https://fr.wikisource.org/wiki/Wikisource:TestTOC?useskin=vector
You shouldn't need to purge the page to make this appear after deploy.

https://fr.wikisource.org/wiki/Wikisource:TestTOC?useskin=vector-2022 shouldn't regress after deploy, it should have a ToC in the side bar both before and after deploy.

Confirmed.

Thanks for the quick response everyone and thanks to the help from volunteers for reporting and helping us debug <3

Thank you so much! When I reported this bug this morning, I had no idea it would be solved so quickly. Congratulations!