Page MenuHomePhabricator

Korean characters rendered as Unicode boxes in the SVG Translate Tool preview (due to lack of fonts)
Closed, ResolvedPublic3 Estimated Story PointsBUG REPORT

Description

List of steps to reproduce (step by step, including full links if applicable):

What happens?:
Korean characters are misrendered in the online preview. I think it could be because the font used in the preview lacks support for Korean characters. The downloaded version seems fine, so I think the issue is only related to the preview tool.

The arrangement of the characters in the box is as follows:

 __
|D5|
|5C|
 --

This looks similar to this StackExchange question about misrendering of Thai characters (https://superuser.com/questions/772626/why-do-some-unicode-characters-appear-as-boxes)

What should have happened instead?:
The preview tool should have shown the proper Korean characters as they were typed. I was expecting to see 한국

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc.:
Here is a screenshot of the preview version showing mis-rendered Korean text in 3 spots.

image.png (1×2 px, 468 KB)

Here is the downloaded version showing Korean text rendering correctly in all 3 spots.

I got the same results from

  • Chrome 102 on macOS (Monterey)
  • Safari 15.5 on macOS (Catalina)

If more info is required, please let me know. I hope this helps.

Event Timeline

Restricted Application added subscribers: revi, Aklapper. · View Herald Transcript
Aklapper renamed this task from Korean characters are not rendered properly in the SVG Translate Tool preview to Korean characters rendered as Unicode boxes in the SVG Translate Tool preview (due to lack of fonts).Jun 13 2022, 9:51 PM
Aklapper added a project: I18n.

Thanks @Aklapper for renaming the task and adding it to a project!

Found this thread (luckily its cached as the original doesn't seem to exist, so I've archived it as well..) — of note is:

For anyone else who comes across this, I ended up figuring out that calling rsvg directly would load fonts from the local home directory. Example: /var/www/.fonts/truetype/google/anton.tff would be automatically loaded. Then you can run exec("rsvg-convert $svgFile > $pngFile");. You have to manually download the fonts from Google, but at least it works. On Heroku I was able to just add rsvg-convert to the application and call that version. It did not need any other libraries to be loaded.

Are the missing fonts due to them not being in the Toolforge php7.4 image? Because perhaps the fix here is to make sure that that image matches whatever Wikimedia production has (i.e. what is able to be rendered by rsvg in prod)?

Hi @bd808 ! I'm currently trying to fix a font issue with our SVG Translate tool and I came across a patch related to this that was created a couple of years ago. What is the best way to address this font issue? The tool is not currently using grid, should we run the service using the grid or should we submit a patch so that the image include the needed fonts? Your help is greatly appreciated.

Hi @bd808 ! I'm currently trying to fix a font issue with our SVG Translate tool and I came across a patch related to this that was created a couple of years ago. What is the best way to address this font issue? The tool is not currently using grid, should we run the service using the grid or should we submit a patch so that the image include the needed fonts? Your help is greatly appreciated.

This is basically the same issue as T213669: [BUG] Fonts missing for some languages on SVG Translate tool was when this same tool tried to use Kubernetes before. @Samwilson had proposed a patch to add the missing fonts to the php7.2 image, and I eventually convinced him to just revert to running the tool on the grid engine instead.

Time passes, and people can learn & grow. ;) I would like to test to see how much bigger the php7.4 image is with the same font packages installed as are installed on the grid engine, but I think that we can afford to add the fonts to support this tool.

If a quick fix is needed, I think reverting to running on the grid engine is your best bet. That will make @komla and @nskaggs sad though so if you can give me a day I'll try to unblock you with php7.4 on the Kubernetes cluster.

Change 838939 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/docker-images/toollabs-images@master] php74: add many TTF fonts

https://gerrit.wikimedia.org/r/838939

Change 838939 merged by jenkins-bot:

[operations/docker-images/toollabs-images@master] php74: add many TTF fonts

https://gerrit.wikimedia.org/r/838939

Mentioned in SAL (#wikimedia-cloud) [2022-10-06T00:04:21Z] <bd808> Building new php74-sssd-base & web images (T310435)

Mentioned in SAL (#wikimedia-cloud) [2022-10-06T00:14:10Z] <wm-bot> <root> Restart to pick up new php7.4 container with added fonts (T310435)

This issue was happening in the Japanese text too. This fix helps us so much. Thank you!