Page MenuHomePhabricator

Upgrade Thumbor to bullseye
Open, Needs TriagePublic

Description

Things to look for

  • Python going from 3.7.3 to 3.9.2. Python dependencies aren't a major concern here now that we are using requirements.txt and not relying on system packages.
  • librsvg2 going to 2.50.3 (T265549)
  • Similar changes in imagemagick and other binaries that we shell out to
  • ~Changes required in tools like 3d2png~
  • Potential HAProxy improvements we could make use of (particularly around metric granularity, maybe?)

Known blockers

  • xcftools has been removed from bullseye and the upstream is dead. We need to make a decision about whether to shift our support for this format somehow. We currently receive a very very small number of requests for this format, but we would be breaking support for something we have historically supported. @AntiCompositeNumber has already done the work to move us towards using ImageMagick's support here
  • 3d2png's dependencies are very very old. They will need breaking version bumps for canvas at best, also at some point require updates to the three library. A Good Enough implementation of the canvas fixes is here.
  • The newly patched imagemagick dependencies will produce different images for successful tests - we need to resolve T334863 before moving forward as we won't be forward porting packages as a partial hack

Event Timeline

Change 920759 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[3d2png@master] wip: update dependencies for use with bullseye

https://gerrit.wikimedia.org/r/920759

Change 920760 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/software/thumbor-plugins@master] wip: upgrade container and dependencies for bullseye

https://gerrit.wikimedia.org/r/920760

hnowlan updated the task description. (Show Details)

In the meantime, Bookworm has been released. Shouldn’t we upgrade directly to that?

Bookworm is not really tested enough to be released to a massive service in production, so far we only deployed it, in people.wikimedia.org backend (and a couple more small services). It's probably a good half a year before we can call safely discuss moving to bookworm.

It's also always better to do upgrades in small steps rather than large jumps to avoid a lot of moving parts.

Bookworm is not really tested enough to be released to a massive service in production, so far we only deployed it, in people.wikimedia.org backend (and a couple more small services). It's probably a good half a year before we can call safely discuss moving to bookworm.

FWIW, I'm not convinced by this :-) Personally I think it makes sense to move right away to Bookworm: This isn't some flaky Ubuntu release, but a Debian stable release which has ripened over hundreds of thousand of testing installations. There are no known major issues with Bookworm one month after the release and we've added full support for our Debian base layer since the last half year. peopleweb isn't the only service, we are also running a bastion and the entire FastNetMon infra on Bookworm (and the puppet 7 puppet masters/puppetdbs/puppetboards systems are also getting built on it).

The changes which were made so far (font adaptions and the move away from xcftools) are equally beneficial for a bookworm setup and the new stack will have several bugs fixed, that are still present in Bullseye, simply because of the former including two additional years of upstream development.

I understand, my point is around priorities of what to upgrade first: i.e. the order of OS upgrade roll out in the production. Upgrading thumbor first thing (with some exceptions) in production can cause issues in larger scale than it needs to be. It's not necessarily with the bookworm itself but also support of the OS in production. For example, docker images, networking inside kubernetes, etc. etc.