Page MenuHomePhabricator

[Bug] Simplified Talk Page Does Not Support Sections With Non-ascii Characters/Several ascii characters aren't supported either
Closed, ResolvedPublic5 Estimated Story PointsBUG REPORT

Description

I've mentioned this bug in standup a couple times as part of working on T230695, but here it is formalized as a ticket.

Steps to reproduce

  1. Visit https://en.m.wikipedia.beta.wmflabs.org/wiki/Talk:Test_ascii_toc
  2. Click on '存在' section

Expected results

  • TalkSectionOverlay opens

Actual results

  • TalkSectionOverlay does not open

Different browsers have different behavior depending on the section clicked:

Sectionchrome 78safari 13.0.3firefox 65.0Android chrome 78iOS Safari 12.0
存在doesn't opendoesn't opendoesn't opendoesn't opendoesn't open
`backtick testdoesn't openopensdoesn't opendoesn't openopens
"Quote" testdoesn't openopensdoesn't opendoesn't openopens
2 > 1doesn't openopensdoesn't opendoesn't openopens
😭doesn't opendoesn't opendoesn't opendoesn't opendoesn't open
abc123opensopensopensopensopens

Check any additional observations

Dev notes

Each section contains an .mw-headline element with an id attribute. Whatever is in that id attribute is registered as a path in OverlayManager. OverlayManager listens to hash changes and only opens up the overlay if the id exactly equals the hash path.

However, browsers can encode location addresses in different ways which can break this equality check. If they perform any encoding at all on the characters, the overlay will not open.

E.g. Try clicking on the 存在 section. And then in dev console enter:

location.hash

This gives:

"#%E5%AD%98%E5%9C%A8"

But OverlayManager is registered as 存在 so because "存在" !== "%E5%AD%98%E5%9C%A8", the equality check fails. Furthermore, because browsers encode different characters differently as evident from the table above, we can't simply register the route with OverlayManager in its percent encoded form - we would also need the browser to navigate to this percent encoded hash (e.g. The .mw-headline id would need to also be percent encoded).

A workaround which literally relies on how the browser performs the encoding was performed here: https://gerrit.wikimedia.org/r/#/c/mediawiki/skins/MinervaNeue/+/548945/

But there is more discussion to be had on whether the section id should be percent encoded to begin with (e.g. via the Parser) which would seem to avoid this problem. However, there might be other concerns associated with doing it this way (see T152540).

QA Steps

Because browsers can exhibit different behavior with this ticket, it is important to test the following steps across a range of browsers. Therefore, it would be good idea to test:

Android Chrome, Android Firefox, Android Edge, iOS Safari:

  1. Go to https://en.m.wikipedia.beta.wmflabs.org/wiki/Talk:Test_ascii_toc
  2. Click on each section. Verify that the Talk Overlay opens and that it corresponds to the section clicked.

QA Results

Event Timeline

nray renamed this task from [Bug] Simplified Talk Page Does Not Support Sections With Non-ascii Characters And Even Some Ascii characters to [Bug] Simplified Talk Page Does Not Support Sections With Non-ascii Characters/Several ascii characters aren't supported.Nov 14 2019, 7:45 PM

Change 548945 had a related patch set uploaded (by Nray; owner: Nray):
[mediawiki/skins/MinervaNeue@master] Add support for non-ascii/reserved characters in simplified talk page section ids

https://gerrit.wikimedia.org/r/548945

nray renamed this task from [Bug] Simplified Talk Page Does Not Support Sections With Non-ascii Characters/Several ascii characters aren't supported to [Bug] Simplified Talk Page Does Not Support Sections With Non-ascii Characters/Several ascii characters aren't supported either.Nov 14 2019, 8:13 PM

Is there a ticket for the table of contents links relating to the html5 section id work as well or do we want to fold those details in here?

Is there a ticket for the table of contents links relating to the html5 section id work as well or do we want to fold those details in here?

Was going to be the next ticket I create after I review your OverlayManager patch :)

nray updated the task description. (Show Details)
ovasileva triaged this task as Medium priority.Nov 20 2019, 9:04 PM
ovasileva subscribed.

moving this back to needs analysis as the conversation in T238385 develops

Change 556840 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Talk headings should work with ascii characters

https://gerrit.wikimedia.org/r/556840

Looking into this some more, I think the URIs that are not impacted by the table of contents bug can be fixed with a simple encodeURIComponent. That's what my patch does. AS for the other characters - that should be handled in the parser.

Jdlrobson raised the priority of this task from Medium to High.Dec 13 2019, 12:48 AM

Looking into this some more, I think the URIs that are not impacted by the table of contents bug can be fixed with a simple encodeURIComponent. That's what my patch does. AS for the other characters - that should be handled in the parser.

Commented on patch, I think the success of this is still browser dependent

Change 558232 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] OverlayManager can handle encoded and unencoded hash fragments

https://gerrit.wikimedia.org/r/558232

Change 556840 abandoned by Jdlrobson:
Talk headings should work with ascii characters

Reason:
Fix in overlay manager - https://gerrit.wikimedia.org/r/#/q/I9cdaf3b01c2e5fe25512b6c18dcf6787c4422abd

https://gerrit.wikimedia.org/r/556840

Change 548945 abandoned by Nray:
Add support for non-ascii/reserved characters in simplified talk page section ids

Reason:
Abandoning this because I think it's adding confusion and I think a better proposal is in https://gerrit.wikimedia.org/r/#/c/mediawiki/skins/MinervaNeue/ /556840/2/resources/skins.minerva.scripts/talk.js@103

https://gerrit.wikimedia.org/r/548945

ovasileva set the point value for this task to 5.Dec 18 2019, 5:12 PM

I just looked at my own enwiki talk page, is is saying there are NO discussions at all (https://en.m.wikipedia.org/wiki/User_talk:Xaosflux) is this the same problem being discussed here? In order to see anything a reader has to click on "Read as wiki page".

i guess we can special case those 5 characters, just for safari and older browsers...

I just looked at my own enwiki talk page, is is saying there are NO discussions at all (https://en.m.wikipedia.org/wiki/User_talk:Xaosflux) is this the same problem being discussed here? In order to see anything a reader has to click on "Read as wiki page".

Different problem. The issue there is that the mobile talk pages are optimised for pages with headings and sections. If the content is wrapped inside a div it cannot be parsed so unexpected things like this happen. In this situation we should eiither default to read as wiki page or display some kind of notice to explain. The page attempts to make talk pages more consistent for new users so possibly the latter.

The page attempts to make talk pages more consistent for new users so possibly the latter.

I think we should only ever do that if the user is told the SAME story on desktop.

Change 558232 abandoned by Jdlrobson:
OverlayManager can handle encoded and unencoded hash fragments

Reason:
For now. Not working on this but somebody should feel free to fork the patch if it's useful.

https://gerrit.wikimedia.org/r/558232

Change 565158 had a related patch set uploaded (by Nray; owner: Nray):
[mediawiki/skins/MinervaNeue@master] 🐛 Make Talk Page Support Sections With Any Valid Id

https://gerrit.wikimedia.org/r/565158

Change 565158 merged by Jdlrobson:
[mediawiki/skins/MinervaNeue@master] 🐛 Make Talk Page Support Sections With Any Valid Id

https://gerrit.wikimedia.org/r/565158

Change 565400 had a related patch set uploaded (by Nray; owner: Nray):
[mediawiki/extensions/MobileFrontend@master] Clarify How Routes Should Be Passed To OverlayManager's add method

https://gerrit.wikimedia.org/r/565400

nray updated the task description. (Show Details)

Change 565400 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Clarify How Routes Should Be Passed To OverlayManager's add method

https://gerrit.wikimedia.org/r/565400

Edtadros subscribed.

Test Result

Status: ✅ PASS
OS: macOS Catalina
Browser: Chrome
Device: MBP
Emulated Device: specified below

Test Artifact(s):

QA Steps

Because browsers can exhibit different behavior with this ticket, it is important to test the following steps across a range of browsers. Therefore, it would be good idea to test:

Android Chrome, Android Firefox, Android Edge, iOS Safari:

Go to https://en.m.wikipedia.beta.wmflabs.org/wiki/Talk:Test_ascii_toc
Click on each section. Verify that the Talk Overlay opens and that it corresponds to the section clicked.

✅ AC1: Android Chrome : Galaxy 10

T238364-1.gif (850×412 px, 432 KB)

✅ AC2: Android Firefox : Galaxy 10
T238364-1.gif (806×390 px, 421 KB)

✅ AC3:Android Edge : Pixel 3XL
T238364-1.gif (746×388 px, 387 KB)

✅ AC4: iOS Safari:
T238364-1.gif (802×416 px, 641 KB)