Page MenuHomePhabricator

Han Unification in users's language
Open, LowPublic

Description

tl;dr: We should pick the fonts based on language for Chinese, Japanese, and Korean.

The full reasoning is fairly technical and dense, so bear with me...

When I speak of Chinese, Japanese, and Korean, I'm referring to the languages or speakers of the languages, not the countries.

Because unicode has unified Han ideograms, using a single codepoint for different glyphs, this causes problems for maps. Native speakers expect to see language-specific glyphs and feel that other glyph forms are a foreign language, but this is not possible looking solely at Unicode text, because the same string could be in different languages. Chapter 5.10 of Unicode 9.0 describe scenarios where the reader is either Japanese or Chinese and has the corresponding fonts. This works for web pages because they have knowledge of the language and mechanisms to describe it (e.g. HTML language tags). When rendering server-side, we supply the fonts, and when rendering name tags we don't have knowledge of the language. The OpenStreetMap Carto team, including myself, have done some research into this and we're currently following the recommended approach of rendering in Japanese. Their approach is not ideal, getting the wrong glyphs in some regions, but is considered to be the best possible for a single map rendering of name tags not targeted at any specific region or language.

With internationalized maps, we can do better at picking the right glyphs. We still don't have a specific region we're targeting, but we now have a language, and in many cases, we know the language of the name. The target language comes from the query parameter, and the language of the name comes from knowing which name:* tag is being used. I don't propose doing anything with the latter knowledge, but the former lets us pick if we want Chinese, Japanese, or Korean characters based on the surrounding content of the page.

When doing this, we would have a Chinese language wiki displaying a map of China or Japan with Chinese glyphs, and a Japanese language wiki displaying a map of Japan or China with Japanese glyphs.

A better solution, but not a practical one is to incorporate knowledge of the source of the name into the font selection. So for a ja viewer seeing a name from a name:ko tag, it would be rendered preferring the Korean font/glyph. If the name came from a name tag, it would prefer the Japanese font/glyph, because that's their language.

Event Timeline

Pnorman created this task.Apr 11 2018, 11:50 PM
Restricted Application added subscribers: revi, Aklapper. · View Herald TranscriptApr 11 2018, 11:50 PM
Pnorman updated the task description. (Show Details)Apr 12 2018, 12:03 AM

@SBisson, can you please have a look at this? How hard is this to fix?

@SBisson, can you please have a look at this? How hard is this to fix?

We would need a conversation with @Pnorman to figure out the technical solution to this.

Mholloway triaged this task as Low priority.Jul 19 2018, 5:48 PM
Mholloway added a subscriber: Mholloway.

This is a big project in itself. We probably don't have the resources to work on it at this time. However it does implicate knowledge equity, so let's leave it open. Patches welcome, and if they come in, we'll have to figure out the product manager issue.

See Also https://github.com/gravitystorm/openstreetmap-carto/issues/2208
And this also affect some other languages, see the link for further details