Page MenuHomePhabricator

[APNT] Middle dot normalization in title resolution
Open, Needs TriagePublic3 Estimated Story PointsFeature

Description

Parent task (please check): T432814: Umbrella: CJK-related automatic punctuation normalization in title resolution

Feature summary: When resolving titles with one of the listed characters, check the title substituting them with U+00B7 if the page is not found.

The preferred Chinese name-separator character on Chinese Wikipedia and in W3C CLREQ is U+00B7 · MIDDLE DOT. Big5/CNS 11643 A1 50 maps to U+00B7. Input-method limitations mean that all of the characters listed below are encountered in real-world use:

UnicodeCharacterNameNotes
U+FF0E.FULLWIDTH FULL STOPUsed by Taiwan Ministry of Education as a Chinese name separator in some guidance. Big5/CNS 11643 A1 44 maps to this character.
U+2027‧HYPHENATION POINTReportedly used as a substitute middle dot in Chinese text. Blacklisted on Chinese Wikipedia.
U+30FB・KATAKANA MIDDLE DOTJapanese middle dot. Sometimes seen in Chinese text as a substitute. Blacklisted on Chinese Wikipedia.
U+FF65・HALFWIDTH KATAKANA MIDDLE DOTHalfwidth form of U+30FB. Sometimes used as a substitute. Blacklisted on Chinese Wikipedia.
U+002E.FULL STOPOrdinary ASCII full stop. Occasionally used in place of a middle dot.
U+2022•BULLETCommonly mistaken for a middle dot. Blacklisted on Chinese Wikipedia. Easy to enter from Gboard punctuation panels.

Post-script

P.S. Resolving that in the search bar (during NFKC?) would be great, too.

Previous discussions: 2007 January, 2024 1, 2

Event Timeline

Change #1342104 had a related patch set uploaded (by Srapoj; author: Srapoj):

[mediawiki/extensions/CirrusSearch@master] Normalize middle dot variants in Chinese title near matches

https://gerrit.wikimedia.org/r/1342104

I think U+002E is more often used correctly in titles (e.g. in a domain, in numbers like 3.14, or some sort of stylistic name that does not have a Chinese translation). Folding U+002E also mess with the extension suffix of files. Therefore I did not put it into the patch. The rest get folded into U+00B7.

It could be reasonable considering cases like "J.K.罗琳" (J. K. Rowling) and "约翰.F.肯尼迪" (John F. Kennedy, should be 约翰·F.肯尼迪 per Chinese standard). However it would need regex like (\p{sc=Han})\.(\p{sc=Latin})\.(?=\p{sc=Han}) for the latter case. If the former case should also be supported, an example can be

{
  "han_on_right": {
    "type": "pattern_replace",
    "pattern": "(\\p{sc=Latn}|\\p{sc=Han})\\.(?=(?:\\p{sc=Latn}\\.)*\\p{sc=Han})",
    "replacement": "$1·"
  }
}

Brought this up on wiki https://zh.wikipedia.org/wiki/Wikipedia:互助客栈/技术#c-Srapoj-20260916133000-Srapoj-20260916044800.


Made a search insource:/([a-zA-Z\u4E00-\u9FFF])\.([a-zA-Z]\.)*[\u4E00-\u9FFF)]/ on zhwiki. In articles, U+002E appears most commonly used inside citations, where it may be correct if the citation style uses it to separate fields. Nevertheless, it sometimes appears at places where a mid dot should be used.