Page MenuHomePhabricator

[APNT] Fullwidth/halfwidth colons
Open, Needs TriagePublic5 Estimated Story PointsFeature

Description

Parent task (please check): T432814: Umbrella: CJK-related automatic punctuation normalization in title resolution

Feature summary: Make U+FF1A FULLWIDTH COLON somewhat useful

Use case(s):

Chinese uses U+FF1A FULLWIDTH COLON. So, that namespace separator U+003A COLON is not normal for us. For many input method engines it requires swtiching to ASCII/English/Latin mode.

Currently Chinese Wikipedia has a site gadget to help, but that's indierct, and it does not fix red links.

I notice that TitleMatcher::getNearMatchInternal does not work on fullwidth colons, probably because of namespaces. Try search for Special: on zhwiki.

Benefits: It would really benefit Chinese users because we type in and search for colons very frequently.

Event Timeline

The core near match implementation already converts input to NFKC, where U+FF1A becomes U+003A. Therefore Wikipedia:首页 or 维基百科:首页 can currently resolve to Wikipedia:首页 with srwhat=nearmatch.

The completion suggester and fulltext search do not appear to work for the U+FF1A cases on surface, because by default they work in the article namespace. Dealing with this case can be tricky, as shown by the book title https://en.wikipedia.org/wiki/Wikipedia_–_The_Missing_Manual. It might not be appropriate to assume a query starting with 维基百科: is searching either the article namespace or the Wikipedia namespace. It might need to recognize both interpretations, and probably needs some refactoring on the CirrusSearch side.

If this ambiguity is dismissed by assuming 维基百科: to be a legitimate namespace prefix for search purposes, it should be simple to implement.

The core near match implementation already converts input to NFKC, where U+FF1A becomes U+003A. Therefore Wikipedia:首页 or 维基百科:首页 can currently resolve to Wikipedia:首页 with srwhat=nearmatch.

The completion suggester and fulltext search do not appear to work for the U+FF1A cases on surface, because by default they work in the article namespace. Dealing with this case can be tricky, as shown by the book title https://en.wikipedia.org/wiki/Wikipedia_–_The_Missing_Manual. It might not be appropriate to assume a query starting with 维基百科: is searching either the article namespace or the Wikipedia namespace. It might need to recognize both interpretations, and probably needs some refactoring on the CirrusSearch side.

If this ambiguity is dismissed by assuming 维基百科: to be a legitimate namespace prefix for search purposes, it should be simple to implement.

For that "Wikipedia – The Missing Manual" case, the English Wikipedia has a redirect at "Wikipedia:The Missing Manual". We should create that kind of redirects if possible when we encounter those technical restrictions, so I guess searching only in the Wikipedia namespace is fair enough.