Page MenuHomePhabricator

Upgrade Thumbor servers to Stretch
Closed, ResolvedPublic

Description

This should fix issues with a number of SVG files.

Update: We can upgrade to stretch, using librsvg 2.40.20-3 and later on upgrade to buster so to use librsvg >= 2.43

Additional/Related tasks:

  • T209886 Assess Thumbor upgrade options

Related Objects

StatusSubtypeAssignedTask
Resolvedjijiki
DeclinedKrenair
Resolved Gilles
Resolved Gilles
Resolved Gilles
Resolved Gilles
Resolved Gilles
Resolved Gilles
Resolved Gilles
Resolved Gilles
Resolvedjijiki
Resolved Gilles
Resolved Gilles
Resolvedjijiki
Resolvedjijiki
Resolvedjijiki
Resolvedfgiunchedi
DuplicateNone
Resolvedjijiki
DuplicateNone

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

It's probably a minor difference in rsvg rendering. 98.8% is very good similarity. Let's double check if the rendering difference is significant.

This is the reference thumbnail, identical to what Thumbor generates on Jessie at the moment:

400px-Map_of_the_Beboid_languages.svg.png (249×400 px, 255 KB)

This is what rsvg-convert generates on Stretch (ran on deployment-imagescaler02):

foo.png (249×400 px, 253 KB)

It's a text rendering difference. Not that it's great in the original, but it definitely gets worse on Stretch. Does the machine you're building Thumbor on and deployment-imagescaler02 have all the fonts we normally install on imagescalers?

I'm building on copper, so gsfonts definitely needs to be in thumbor's build deps

Change 370969 merged by Filippo Giunchedi:
[operations/puppet@production] mediawiki: clean up deprecated fonts packages

https://gerrit.wikimedia.org/r/370969

Change 371034 merged by Filippo Giunchedi:
[operations/puppet@production] thumbor: do not hardcode jessie-backports

https://gerrit.wikimedia.org/r/371034

It's a text rendering difference. Not that it's great in the original, but it definitely gets worse on Stretch. Does the machine you're building Thumbor on and deployment-imagescaler02 have all the fonts we normally install on imagescalers?

I'm building on copper, so gsfonts definitely needs to be in thumbor's build deps

Fixed in python-thumbor-wikimedia 1.2-2 and tests now pass in both stretch and jessie. gsfonts is now a runtime and build time dependency so no puppet fixes are needed.

To recap, deployment-imagescaler02 now has thumbor and stretch and all puppet patches required to adjust fonts for stretch are merged. Any other tests we could run with production-like files @Gilles ? Next step would be to swap deployment-imagescaler02 with 01 in beta and then reimage one production box with stretch.

Mentioned in SAL (#wikimedia-operations) [2017-08-16T10:03:56Z] <godog> copy ubuntu-font-family-sources to stretch-wikimedia - T170817

Change 372125 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] mediawiki: fix ttf-ubuntu-font-family handling

https://gerrit.wikimedia.org/r/372125

Change 372125 merged by Filippo Giunchedi:
[operations/puppet@production] mediawiki: fix ttf-ubuntu-font-family handling

https://gerrit.wikimedia.org/r/372125

I've compared deployment-imagescaler02 again and I see rendering differences for kochi fonts. Isn't it the same issue as the ttf ubuntu fonts?

They are installed on thumbor1001:

gilles@thumbor1001:~$ fc-list | grep Kochi
/usr/share/fonts/truetype/kochi/kochi-mincho-subst.ttf: Kochi Mincho,東風明朝:style=Regular,標準
/usr/share/fonts/truetype/kochi/kochi-mincho.ttf: Kochi Mincho,東風明朝:style=Regular,標準
/usr/share/fonts/truetype/kochi/kochi-gothic-subst.ttf: Kochi Gothic,東風ゴシック:style=Regular,標準
/usr/share/fonts/truetype/kochi/kochi-gothic.ttf: Kochi Gothic,東風ゴシック:style=Regular,標準

Similarly, the MgOpenCanonica fonts are found on thumbor1001 and not on deployment-imagescaler01 and render very differently. Those are also from a package you've removed from the common ones in https://gerrit.wikimedia.org/r/#/c/370969/ (fonts-mgopen):

gilles@thumbor1001:~$ fc-list | grep Canonica
/usr/share/fonts/truetype/mgopen/MgOpenCanonicaBoldItalic.ttf: MgOpen Canonica:style=Bold Italic
/usr/share/fonts/truetype/mgopen/MgOpenCanonicaRegular.ttf: MgOpen Canonica:style=Regular
/usr/share/fonts/truetype/mgopen/MgOpenCanonicaBold.ttf: MgOpen Canonica:style=Bold
/usr/share/fonts/truetype/mgopen/MgOpenCanonicaItalic.ttf: MgOpen Canonica:style=Italic

fonts-mgopen was removed from Debian after the jessie release: Turns out the removal request came from @faidon :-) https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819026
So we can either pick a Greek replacement font and alternatively import/build the old package from jessie.

ttf-kochi was also removed after jessie: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=726382 (replacement seems to be fonts-ipa-font)

OK, so if I'm following that means people are now advised to use other fonts than these ones, right? Meaning it's ok if those specific fonts don't render "right" on https://commons.wikimedia.org/wiki/File:MediaWiki_SVG_fonts.svg? And I'm guessing that this reference SVG will have to be updated to use the new "reference" fonts that are replacing those?

@Gilles That's my understanding, yes. I have no idea about the origin of that reference SVG, though (or who's "maintaining" it)

I wouldn't recommend reviving MgOpen for basically the reasons I described in #819026. TL;DR is that it had serious unresolved issues to begin with (hinting, missing Euro sign) and has been abandoned upstream for years. Meanwhile, there are plenty of good and free (as in OFL) fonts nowadays with Greek glyphs, including DejaVu, Liberation, the Google fonts (Droid, Roboto, CrOS), the Adobe fonts (Source Sans/Serif).

The reason I requested the removal of the package from Debian (despite the minimal maintenance burden) was to discourage new users and encourage existing users to switch to better fonts. I hadn't thought of Wikimedia at the time, but it feels pretty applicable here :)

OK, so if I'm following that means people are now advised to use other fonts than these ones, right? Meaning it's ok if those specific fonts don't render "right" on https://commons.wikimedia.org/wiki/File:MediaWiki_SVG_fonts.svg? And I'm guessing that this reference SVG will have to be updated to use the new "reference" fonts that are replacing those?

I think we should simply update the reference SVG with the new font. The best available fonts are not static, but are/were evolving/improving over the years, so it seems reflect fine to reflect that with an updated version of MediaWiki_SVG_fonts.svg from time to time. It's unlikely that this file has a particular maintainer per se, but I think the correct path forward is to be bold, send an email with the intent to update the file to wikitech-l asking for objections and then switch to the new version unless someone objects with a technical reason?

@fgiunchedi - Are you still working on this or should it be unassigned? It looks like this is blocking T36947 (i.e. upgrading librsvg to fix SVG rendering problems), which I need fixed for T201207.

fgiunchedi subscribed.

@fgiunchedi - Are you still working on this or should it be unassigned? It looks like this is blocking T36947 (i.e. upgrading librsvg to fix SVG rendering problems), which I need fixed for T201207.

Unassigning now, I won't have time to work on it this quarter for sure. Happy to assist on Phab though.

jijiki triaged this task as High priority.Oct 23 2018, 12:11 PM

Change 477796 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Use webp -exact option on Stretch

https://gerrit.wikimedia.org/r/477796

jijiki moved this task from St to Inbox 🐅 on the User-jijiki board.

Change 477796 merged by Gilles:
[operations/software/thumbor-plugins@master] Use webp -exact option on Stretch

https://gerrit.wikimedia.org/r/477796

jijiki claimed this task.

All servers have been upgraded to stretch, next episode on T216815 🍾