Page MenuHomePhabricator

Parsoid should perform unicode normalization on titles
Open, MediumPublic

Description

Parsoid re-opens T21451: links to Unicode combined forms should appear blue if they autoredirect to component forms, emitting links in non-normalized forms and (as a logical result) not doing proper redlink processing on them.

This is the parser test "(T21451) Links should refer to the normalized form".

Event Timeline

This mostly affects redlink processing, hence rendering. The actual link is redirected by mediawiki-core when followed, so the link ends up in the right place, just takes longer to get there. But it is marked incorrectly as a redlink.

ssastry triaged this task as Medium priority.Sep 11 2020, 5:31 PM

Now that we're in core, we can use the same unicode library which code uses for the normalization. But I think this is still more properly part of 'redlink processing' (aka, part of readviews post processing, even if we end up adjusting blue links), since for editing purposes I think we want to preserve the exact form the editor typed the link.

ihurbain subscribed.

Now that we're in core, we can use the same unicode library which code uses for the normalization. But I think this is still more properly part of 'redlink processing' (aka, part of readviews post processing, even if we end up adjusting blue links), since for editing purposes I think we want to preserve the exact form the editor typed the link.

We normalize to NFC form on input, and on output, so in general everything should really be consistently NFC.

Change #1171714 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/services/parsoid@master] Fix title-casing and NFC normalization of titles

https://gerrit.wikimedia.org/r/1171714

Change #1171714 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Fix title-casing and NFC normalization of titles

https://gerrit.wikimedia.org/r/1171714

Change #1181757 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.22.0-a18

https://gerrit.wikimedia.org/r/1181757

Change #1181757 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.22.0-a18

https://gerrit.wikimedia.org/r/1181757