Page MenuHomePhabricator

Upgrade Thumbor servers to Stretch
Closed, ResolvedPublic

Description

This should fix issues with a number of SVG files.

Update: We can upgrade to stretch, using librsvg 2.40.20-3 and later on upgrade to buster so to use librsvg >= 2.43

Additional/Related tasks:

  • T209886 Assess Thumbor upgrade options

Related Objects

StatusAssignedTask
Resolvedjijiki
DeclinedKrenair
ResolvedGilles
ResolvedGilles
ResolvedGilles
ResolvedGilles
ResolvedGilles
ResolvedGilles
ResolvedGilles
ResolvedGilles
Resolvedjijiki
ResolvedGilles
ResolvedGilles
Resolvedjijiki
Resolvedjijiki
Resolvedjijiki
Resolvedfgiunchedi
DuplicateNone
Resolvedjijiki
DuplicateNone

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

It's probably a minor difference in rsvg rendering. 98.8% is very good similarity. Let's double check if the rendering difference is significant.
This is the reference thumbnail, identical to what Thumbor generates on Jessie at the moment:


This is what rsvg-convert generates on Stretch (ran on deployment-imagescaler02):

It's a text rendering difference. Not that it's great in the original, but it definitely gets worse on Stretch. Does the machine you're building Thumbor on and deployment-imagescaler02 have all the fonts we normally install on imagescalers?

I'm building on copper, so gsfonts definitely needs to be in thumbor's build deps

Change 370969 merged by Filippo Giunchedi:
[operations/puppet@production] mediawiki: clean up deprecated fonts packages

https://gerrit.wikimedia.org/r/370969

Change 371034 merged by Filippo Giunchedi:
[operations/puppet@production] thumbor: do not hardcode jessie-backports

https://gerrit.wikimedia.org/r/371034

It's a text rendering difference. Not that it's great in the original, but it definitely gets worse on Stretch. Does the machine you're building Thumbor on and deployment-imagescaler02 have all the fonts we normally install on imagescalers?

I'm building on copper, so gsfonts definitely needs to be in thumbor's build deps

Fixed in python-thumbor-wikimedia 1.2-2 and tests now pass in both stretch and jessie. gsfonts is now a runtime and build time dependency so no puppet fixes are needed.

To recap, deployment-imagescaler02 now has thumbor and stretch and all puppet patches required to adjust fonts for stretch are merged. Any other tests we could run with production-like files @Gilles ? Next step would be to swap deployment-imagescaler02 with 01 in beta and then reimage one production box with stretch.

fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.Aug 16 2017, 9:28 AM

Mentioned in SAL (#wikimedia-operations) [2017-08-16T10:03:56Z] <godog> copy ubuntu-font-family-sources to stretch-wikimedia - T170817

Change 372125 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] mediawiki: fix ttf-ubuntu-font-family handling

https://gerrit.wikimedia.org/r/372125

Change 372125 merged by Filippo Giunchedi:
[operations/puppet@production] mediawiki: fix ttf-ubuntu-font-family handling

https://gerrit.wikimedia.org/r/372125

I've compared deployment-imagescaler02 again and I see rendering differences for kochi fonts. Isn't it the same issue as the ttf ubuntu fonts?

They are installed on thumbor1001:

gilles@thumbor1001:~$ fc-list | grep Kochi
/usr/share/fonts/truetype/kochi/kochi-mincho-subst.ttf: Kochi Mincho,東風明朝:style=Regular,標準
/usr/share/fonts/truetype/kochi/kochi-mincho.ttf: Kochi Mincho,東風明朝:style=Regular,標準
/usr/share/fonts/truetype/kochi/kochi-gothic-subst.ttf: Kochi Gothic,東風ゴシック:style=Regular,標準
/usr/share/fonts/truetype/kochi/kochi-gothic.ttf: Kochi Gothic,東風ゴシック:style=Regular,標準

Similarly, the MgOpenCanonica fonts are found on thumbor1001 and not on deployment-imagescaler01 and render very differently. Those are also from a package you've removed from the common ones in https://gerrit.wikimedia.org/r/#/c/370969/ (fonts-mgopen):

gilles@thumbor1001:~$ fc-list | grep Canonica
/usr/share/fonts/truetype/mgopen/MgOpenCanonicaBoldItalic.ttf: MgOpen Canonica:style=Bold Italic
/usr/share/fonts/truetype/mgopen/MgOpenCanonicaRegular.ttf: MgOpen Canonica:style=Regular
/usr/share/fonts/truetype/mgopen/MgOpenCanonicaBold.ttf: MgOpen Canonica:style=Bold
/usr/share/fonts/truetype/mgopen/MgOpenCanonicaItalic.ttf: MgOpen Canonica:style=Italic

fonts-mgopen was removed from Debian after the jessie release: Turns out the removal request came from @faidon :-) https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819026
So we can either pick a Greek replacement font and alternatively import/build the old package from jessie.

ttf-kochi was also removed after jessie: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=726382 (replacement seems to be fonts-ipa-font)

OK, so if I'm following that means people are now advised to use other fonts than these ones, right? Meaning it's ok if those specific fonts don't render "right" on https://commons.wikimedia.org/wiki/File:MediaWiki_SVG_fonts.svg? And I'm guessing that this reference SVG will have to be updated to use the new "reference" fonts that are replacing those?

@Gilles That's my understanding, yes. I have no idea about the origin of that reference SVG, though (or who's "maintaining" it)

fgiunchedi moved this task from Doing to Radar on the User-fgiunchedi board.Aug 23 2017, 9:42 AM
faidon added a comment.Nov 7 2017, 9:46 AM

I wouldn't recommend reviving MgOpen for basically the reasons I described in #819026. TL;DR is that it had serious unresolved issues to begin with (hinting, missing Euro sign) and has been abandoned upstream for years. Meanwhile, there are plenty of good and free (as in OFL) fonts nowadays with Greek glyphs, including DejaVu, Liberation, the Google fonts (Droid, Roboto, CrOS), the Adobe fonts (Source Sans/Serif).

The reason I requested the removal of the package from Debian (despite the minimal maintenance burden) was to discourage new users and encourage existing users to switch to better fonts. I hadn't thought of Wikimedia at the time, but it feels pretty applicable here :)

Jeff_G added a subscriber: Jeff_G.
Gilles moved this task from Backlog to Radar on the Thumbor board.Mar 14 2018, 7:35 AM

OK, so if I'm following that means people are now advised to use other fonts than these ones, right? Meaning it's ok if those specific fonts don't render "right" on https://commons.wikimedia.org/wiki/File:MediaWiki_SVG_fonts.svg? And I'm guessing that this reference SVG will have to be updated to use the new "reference" fonts that are replacing those?

I think we should simply update the reference SVG with the new font. The best available fonts are not static, but are/were evolving/improving over the years, so it seems reflect fine to reflect that with an updated version of MediaWiki_SVG_fonts.svg from time to time. It's unlikely that this file has a particular maintainer per se, but I think the correct path forward is to be bold, send an email with the intent to update the file to wikitech-l asking for objections and then switch to the new version unless someone objects with a technical reason?

kaldari added a subscriber: kaldari.Oct 4 2018, 6:31 PM

@fgiunchedi - Are you still working on this or should it be unassigned? It looks like this is blocking T36947 (i.e. upgrading librsvg to fix SVG rendering problems), which I need fixed for T201207.

fgiunchedi removed fgiunchedi as the assignee of this task.Oct 5 2018, 8:53 AM
fgiunchedi added a subscriber: fgiunchedi.

@fgiunchedi - Are you still working on this or should it be unassigned? It looks like this is blocking T36947 (i.e. upgrading librsvg to fix SVG rendering problems), which I need fixed for T201207.

Unassigning now, I won't have time to work on it this quarter for sure. Happy to assist on Phab though.

mark added a subscriber: mark.Oct 11 2018, 3:23 PM
jijiki added a subscriber: jijiki.Oct 23 2018, 11:36 AM
jijiki triaged this task as High priority.Oct 23 2018, 12:11 PM
jijiki moved this task from Backlog/Radar to In Progress on the User-jijiki board.Nov 19 2018, 9:25 AM

Change 477796 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Use webp -exact option on Stretch

https://gerrit.wikimedia.org/r/477796

jijiki moved this task from Backlog to Incoming on the serviceops board.
fgiunchedi moved this task from Radar to Backlog on the User-fgiunchedi board.Jan 2 2019, 10:37 AM
jijiki moved this task from In Progress to Backlog/Radar on the User-jijiki board.Jan 9 2019, 3:15 PM
jijiki moved this task from Backlog/Radar to St on the User-jijiki board.Jan 9 2019, 3:18 PM
jijiki moved this task from St to Backlog/Radar on the User-jijiki board.
jijiki changed the status of subtask T209886: Assess Thumbor upgrade options from Open to Stalled.Jan 9 2019, 3:42 PM

Change 477796 merged by Gilles:
[operations/software/thumbor-plugins@master] Use webp -exact option on Stretch

https://gerrit.wikimedia.org/r/477796

jijiki updated the task description. (Show Details)Feb 12 2019, 11:46 AM
jijiki closed this task as Resolved.Feb 26 2019, 12:31 PM
jijiki claimed this task.

All servers have been upgraded to stretch, next episode on T216815 🍾