Remex enabled on all wikis in MW 1.30-wmf.30 exposing corruption (signatures coloring unrelated follow-up sections, etc.) on unfixed content
Closed, ResolvedPublic

Description

Hello,

Sorry for this ambiguous title, but several users now in ar.wiki complain from a problem on the ar.wikipedia pages that it appear with different colors! it just appear now for many users!

for example

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 23 2018, 10:00 PM

@alanajjar Can you check the page in incognito mode? and also check if someone changed MediaWiki:Common.js recently in local wiki.

@alanajjar Can you check the page in incognito mode? and also check if someone changed MediaWiki:Common.js recently in local wiki.

I revert the last edit on Common.js (here). Incognito mode of the browser?

@Jayprakash12345 the problem affect mainly users signature, as you see in the second picture the user signature on yellow color, after it all the page appear on color, and the first pic the user signature on purple color, and all the page after it on purple. But no new edits in these users signature

Can you talk a look for code of one of these user signature for example, here? it's affect other users and now appear almost on all discussion pages (include village pump) on ar.wiki.

alanajjar closed this task as Invalid.Apr 23 2018, 10:30 PM
alanajjar reopened this task as Open.

The problem appear today, that many signatures should be closed as here
so what is the problem? and why it appear only now?

Aklapper changed the task status from Open to Stalled.Apr 23 2018, 11:39 PM

@alanajjar: In general, please always follow https://mediawiki.org/wiki/How_to_report_a_bug and provide a link to reproduce the problem.
Nobody can try to reproduce a problem if nobody knows where to see the problem.

Going to https://ar.wikipedia.org/w/index.php?title=مستخدم:Anass_Badou/توقيعي&diff=prev&oldid=28347143 you can see yourself at https://ar.wikipedia.org/w/index.php?title=مستخدم:Anass_Badou/action=edit that the wiki markup explicitly says <b style=color:#FFBF00;.

So far I do not see any software bug here; only expected behavior.

@Aklapper I report it as fast as I can as many users complain and you should see to them that you try to do anything, because of this I post it with few ambiguous sentences.

If it as expected, why it just appear yesterday? Why the problem did not appear previously? see here and here and here

Maybe Remex (new HTML parser) was enabled for your wiki and the signature had invalid HTML for Tidy (old HTML parser) but Remex was able to parse it and display it.

@Urbanecm so what we should do?

Also see here the middle section (عني) became uncollapsed and we don't find any problem here

@ssastry Did Remex get turned on inadvertently? This is occurring on en.WP right now (when I thought the timeline was Q3 2018).

Also see here the middle section (عني) became uncollapsed and we don't find any problem here

That might be a different problem. Following https://www.mediawiki.org/wiki/Help:Locating_broken_scripts on https://ar.wikipedia.org/wiki/مستخدم:باسم?debug=true , you can see in your browser's developer tools that https://ar.wikipedia.org/w/index.php?title=MediaWiki:LoadingContent.js and https://ar.wikipedia.org/w/index.php?title=Mediawiki:CustomSideBarLinks.js are broken. Both show an error that mw.util is undefined. T164242: Find and fix undeclared dependencies to mw.util, mw.notify etc of on-wiki scripts and gadgets provides more information how you can fix this on arwiki.

Aklapper changed the task status from Stalled to Open.Apr 24 2018, 4:32 PM

Maybe Remex (new HTML parser) was enabled for your wiki and the signature had invalid HTML for Tidy (old HTML parser)

@ssastry Did Remex get turned on inadvertently? This is occurring on en.WP right now (when I thought the timeline was Q3 2018).

Hmm, I don't see arwiki explicitly mentioned in T175706: Progressively switch Wikimedia wikis from Tidy to RemexHTML. Clarification from the Parsing Team definitely welcome here.

Izno added a comment.Apr 24 2018, 4:33 PM

En.WP thread is currently at VPT.

Aklapper renamed this task from Strange problem on ar.wiki to On ar.wiki and en.wiki, badly closed markup exposes corruption (signatures coloring unrelated follow-up sections, etc.).Apr 24 2018, 4:37 PM

Hmm .. Not that I know of .. I am still planning to do progressive deployments. @demon @Legoktm @Jdforrester-WMF .. is this somehow related to mediawiki defaulting to remex?

Aklapper renamed this task from On ar.wiki and en.wiki, badly closed markup exposes corruption (signatures coloring unrelated follow-up sections, etc.) to On several Wikipedias (ar, en, fr), badly closed markup exposes corruption (signatures coloring unrelated follow-up sections, etc.).Apr 24 2018, 4:38 PM
Aklapper added a subscriber: Lofhi.
Aklapper triaged this task as Unbreak Now! priority.Apr 24 2018, 4:40 PM
Restricted Application added subscribers: Liuxinyu970226, TerraCodes. · View Herald TranscriptApr 24 2018, 4:40 PM

If it as expected, why it just appear yesterday?

@alanajjar: I obviously misunderstood/missed the "this only started recently" part. I am sorry. It's great and welcome that you report such problems quickly.

Jdforrester-WMF renamed this task from On several Wikipedias (ar, en, fr), badly closed markup exposes corruption (signatures coloring unrelated follow-up sections, etc.) to Remex enabled on all wikis in MW 1.30-wmf.30 exposing corruption (signatures coloring unrelated follow-up sections, etc.) on unfixed content.Apr 24 2018, 5:02 PM

https://gerrit.wikimedia.org/r/plugins/gitiles/operations/mediawiki-config/+/master/wmf-config/InitialiseSettings.php#19738 still has the Tidy and Remex block .. so something must have changed in core that interprets these values or something is overriding these settings wrt Tidy / Remex.

Krinkle added a subscriber: Krinkle.EditedApr 24 2018, 5:11 PM
wmf-config
'wgTidyConfig' => [
	'default' => [ 'driver' => 'RemexHtml' ],
	// Wikipedias [ ..]
	'arwiki' => null,
mwscript eval.php --wiki arwiki
> var_dump($wgTidyConfig);
array(1) {
  ["driver"]=>
  string(9) "RemexHtml"
}

Looks like it's not about run-time interpretation. Something in wmf-config or core is obscuring or changing the variable during initialisation.

FShbib added a subscriber: FShbib.Apr 24 2018, 5:21 PM

Also see here the middle section (عني) became uncollapsed and we don't find any problem here

That might be a different problem. Following https://www.mediawiki.org/wiki/Help:Locating_broken_scripts on https://ar.wikipedia.org/wiki/مستخدم:باسم?debug=true , you can see in your browser's developer tools that https://ar.wikipedia.org/w/index.php?title=MediaWiki:LoadingContent.js and https://ar.wikipedia.org/w/index.php?title=Mediawiki:CustomSideBarLinks.js are broken. Both show an error that mw.util is undefined. T164242: Find and fix undeclared dependencies to mw.util, mw.notify etc of on-wiki scripts and gadgets provides more information how you can fix this on arwiki.

The problem appeared today's morning, yesterday it wasn't there. Seems like it appeared at the same time with the above problem.

Change 428697 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[operations/mediawiki-config@master] Explcitly set driver for Tidy wikis to RaggettInternal*

https://gerrit.wikimedia.org/r/428697

Also see here the middle section (عني) became uncollapsed and we don't find any problem here

That might be a different problem. Following https://www.mediawiki.org/wiki/Help:Locating_broken_scripts on https://ar.wikipedia.org/wiki/مستخدم:باسم?debug=true , you can see in your browser's developer tools that https://ar.wikipedia.org/w/index.php?title=MediaWiki:LoadingContent.js and https://ar.wikipedia.org/w/index.php?title=Mediawiki:CustomSideBarLinks.js are broken. Both show an error that mw.util is undefined. T164242: Find and fix undeclared dependencies to mw.util, mw.notify etc of on-wiki scripts and gadgets provides more information how you can fix this on arwiki.

The problem appeared today's morning, yesterday it wasn't there. Seems like it appeared at the same time with the above problem.

While we'll fix this accidental rollout of Remex now, please note that Tidy will be replaced everywhere end of June, and so, you will see these problems reappear after that unless those pages and signatures are fixed -- https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2018-April/001836.html

TheDJ added a subscriber: TheDJ.Apr 24 2018, 6:24 PM

This gives me an idea btw. We should definetly consider to run a dbquery to check how many accounts produce an unbalanced HTML signature and potentially warn those users with a bot or inpage message, then at a later time reset their signature if the user doesnt follow up.

He7d3r added a subscriber: He7d3r.Apr 24 2018, 6:35 PM

A similar problem was reported at ptwiki on april 19. See pt:WP:Café dos programadores#EADs.

This gives me an idea btw. We should definetly consider to run a dbquery to check how many accounts produce an unbalanced HTML signature and potentially warn those users with a bot or inpage message, then at a later time reset their signature if the user doesnt follow up.

Yes please! I was considering creating a (temporary) default gadget for this, which would check the syntax of the custom signature and in case of errors reset it and show a warning to the user.

A similar problem was reported at ptwiki on april 19. See pt:WP:Café dos programadores#EADs.

Good spot. That's probably people experiencing this task during the two hours on 2018-04-19 that wmf.30 was deployed before being reverted due to T192609; it was then re-deployed yesterday 2018-04-23 at ~20:30 UTC.

I was considering creating a (temporary) default gadget for this, which would check the syntax of the custom signature and in case of errors reset it and show a warning to the user.

You can't run gadgets (or any other kind of script) on Special:Preferences, for security purposes. Anyway, that part of this discussion should be moved to a new task under T175706, please. :-)

You can't run gadgets (or any other kind of script) on Special:Preferences, for security purposes.

It could run when the user edits a talk page, and use the API to get/check/set the signature.

Anyway, that part of this discussion should be moved to a new task under T175706, please. :-)

Ok.

TheDJ added a comment.Apr 24 2018, 6:56 PM

You can't run gadgets (or any other kind of script) on Special:Preferences, for security purposes. Anyway, that part of this discussion should be moved to a new task under T175706, please. :-)

Created T192950: Find unbalanced HTML in current signature of users and inform the user and/or reset the signature

Change 428697 merged by jenkins-bot:
[operations/mediawiki-config@master] Fix wgTidyConfig expansion to not ignore null as value

https://gerrit.wikimedia.org/r/428697

Mentioned in SAL (#wikimedia-operations) [2018-04-24T22:53:20Z] <legoktm@tin> Synchronized wmf-config/CommonSettings.php: Fix wgTidyConfig and restore proper tidy & Remex config - T192855 (duration: 01m 16s)

ssastry closed this task as Resolved.Apr 24 2018, 11:01 PM
ssastry assigned this task to Krinkle.

I used https://fr.wikipedia.org/wiki/Utilisateur:Lofhi/Test to verify that the config change worked. Pages will probably need to be purged or edited for them to be rendered with the expected tidy...

Yes, fixed as see here also.

Thanks all

Also see here the middle section (عني) became uncollapsed and we don't find any problem here

That might be a different problem. Following https://www.mediawiki.org/wiki/Help:Locating_broken_scripts on https://ar.wikipedia.org/wiki/مستخدم:باسم?debug=true , you can see in your browser's developer tools that https://ar.wikipedia.org/w/index.php?title=MediaWiki:LoadingContent.js and https://ar.wikipedia.org/w/index.php?title=Mediawiki:CustomSideBarLinks.js are broken. Both show an error that mw.util is undefined. T164242: Find and fix undeclared dependencies to mw.util, mw.notify etc of on-wiki scripts and gadgets provides more information how you can fix this on arwiki.

This fixed also.