Page MenuHomePhabricator
Create Task
Maniphest T335271

SVG rasterizer renders non Latin text as tofu glyph randomly (as thumbor-k8s lack noto fonts)
Closed, ResolvedPublicBUG REPORT
Actions

Description

Some SVG images with non Latin text such as Chinese or Japanese when uploaded to Wikimedia, the non Latin text in the PNG render turn into tofu glyph completely. This issue happens very randomly. Sometimes it fixed itself after I had re-uploaded the SVG with non-essential changes; the other time it reverts to tofu glyph again. It doesn't matter whether we specify the exact typeface for the non Latin character or not.

SVG files observed with this glitch are File:Tokyu Railways topological diagram.svg and File:Min Kingdom, 945 v2 (zh-hans).svg.

Details

SubjectRepoBranchLines +/-
thumbor: new image versionoperations/deployment-chartsmaster+1 -1
Install noto fonts by defaultoperations/software/thumbor-pluginsmaster+10 -0
Customize query in gerrit

Related Objects

Event Timeline

Sameboat created this task.Apr 24 2023, 10:28 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 24 2023, 10:28 AM
Snaevar added projects: Thumbor, SRE-swift-storage.Apr 24 2023, 10:38 AM
hnowlan added a project: Platform Team Workboards (Platform Engineering Reliability).Apr 26 2023, 9:38 AM
hnowlan subscribed.
gerritbot added a comment.Apr 26 2023, 11:02 AM

Change 912258 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/software/thumbor-plugins@master] Install noto fonts by default

https://gerrit.wikimedia.org/r/912258

gerritbot added a project: Patch-For-Review.Apr 26 2023, 11:02 AM
hnowlan added a comment.Apr 26 2023, 11:04 AM

The inconsistency here is between old Thumbor and thumbor-k8s - older hosts had some noto fonts installed but the (much more minimal) docker images used in k8s don't. Installing more font packages fixes this issue.

TheDJ subscribed.Apr 26 2023, 11:11 AM
Aklapper renamed this task from SVG rasterizer renders non Latin text as tofu glyph randomly to SVG rasterizer renders non Latin text as tofu glyph randomly (as thumbor-k8s lack noto fonts).Apr 26 2023, 11:27 AM
hnowlan moved this task from Backlog to Doing 🔨 on the Platform Team Workboards (Platform Engineering Reliability) board.Apr 26 2023, 11:28 AM
gerritbot added a comment.Apr 26 2023, 1:54 PM

Change 912258 merged by jenkins-bot:

[operations/software/thumbor-plugins@master] Install noto fonts by default

https://gerrit.wikimedia.org/r/912258

Maintenance_bot removed a project: Patch-For-Review.Apr 26 2023, 2:10 PM
gerritbot added a comment.Apr 27 2023, 10:06 AM

Change 912801 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] thumbor: new image version

https://gerrit.wikimedia.org/r/912801

gerritbot added a project: Patch-For-Review.Apr 27 2023, 10:06 AM
gerritbot added a comment.Apr 27 2023, 10:52 AM

Change 912801 merged by jenkins-bot:

[operations/deployment-charts@master] thumbor: new image version

https://gerrit.wikimedia.org/r/912801

Maintenance_bot removed a project: Patch-For-Review.Apr 27 2023, 11:10 AM
hnowlan added a comment.Apr 27 2023, 11:19 AM

Looks like this issue is fixed in new thumbs:

I'll work on getting those invalid thumbs removed. Thanks for the report!

hnowlan added a comment.Apr 28 2023, 11:06 AM

These images have been purged. Closing for now, please reopen if you see more encoding issues.

hnowlan closed this task as Resolved.Apr 28 2023, 11:07 AM
hnowlan claimed this task.
TheDJ added a comment.Apr 28 2023, 12:42 PM

FYI, SVG authors consider this to be the official list of supported fonts:

https://noc.wikimedia.org/conf/fc-list

akosiaris subscribed.Apr 28 2023, 12:49 PM
In T335271#8813201, @TheDJ wrote:

FYI, SVG authors consider this to be the official list of supported fonts:

https://noc.wikimedia.org/conf/fc-list

Obligatory insertion of T335271. This ^ URL is going to become increasingly out of date and thus not useful pretty soon. On the plus side, we might now be able to generate and publish that file more easily.

kamila mentioned this in T335681: Text in PNG thumbnails of SVG files slightly wider than before.May 2 2023, 9:50 AM
Aklapper mentioned this in T228591: Document how to request installing additional fonts for SVG thumbnails and generated PDF files on Wikimedia servers.May 24 2023, 10:09 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits