Page MenuHomePhabricator

[XL] Upgrade Thumbor to bullseye
Closed, ResolvedPublic

Description

Things to look for

  • Python going from 3.7.3 to 3.9.2. Python dependencies aren't a major concern here now that we are using requirements.txt and not relying on system packages.
  • librsvg2 going to 2.50.3 (T265549)
  • Similar changes in imagemagick and other binaries that we shell out to
  • ~Changes required in tools like 3d2png~
  • Potential HAProxy improvements we could make use of (particularly around metric granularity, maybe?)

Known blockers

  • xcftools has been removed from bullseye and the upstream is dead. We need to make a decision about whether to shift our support for this format somehow. We currently receive a very very small number of requests for this format, but we would be breaking support for something we have historically supported. @AntiCompositeNumber has already done the work to move us towards using ImageMagick's support here
  • 3d2png's dependencies are very very old. They will need breaking version bumps for canvas at best, also at some point require updates to the three library. A Good Enough implementation of the canvas fixes is here.
  • The newly patched imagemagick dependencies will produce different images for successful tests - we need to resolve T334863 before moving forward as we won't be forward porting packages as a partial hack

Related Objects

StatusSubtypeAssignedTask
OpenBUG REPORTNone
Resolvedhnowlan
Resolvedhnowlan
Resolvedhnowlan
Resolvedhnowlan
Resolvedhnowlan
Resolvedhnowlan
OpenFeatureNone
OpenBUG REPORTNone
ResolvedBUG REPORThnowlan
StalledNone
DuplicateBUG REPORTNone
OpenNone
ResolvedBUG REPORThnowlan
Resolvedhnowlan
OpenNone
Resolvedhnowlan
Resolvedhnowlan
ResolvedAntiCompositeNumber

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptMay 17 2023, 3:59 PM

Change 920759 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[3d2png@master] wip: update dependencies for use with bullseye

https://gerrit.wikimedia.org/r/920759

Change 920760 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/software/thumbor-plugins@master] wip: upgrade container and dependencies for bullseye

https://gerrit.wikimedia.org/r/920760

hnowlan updated the task description. (Show Details)

In the meantime, Bookworm has been released. Shouldnโ€™t we upgrade directly to that?

Bookworm is not really tested enough to be released to a massive service in production, so far we only deployed it, in people.wikimedia.org backend (and a couple more small services). It's probably a good half a year before we can call safely discuss moving to bookworm.

It's also always better to do upgrades in small steps rather than large jumps to avoid a lot of moving parts.

Bookworm is not really tested enough to be released to a massive service in production, so far we only deployed it, in people.wikimedia.org backend (and a couple more small services). It's probably a good half a year before we can call safely discuss moving to bookworm.

FWIW, I'm not convinced by this :-) Personally I think it makes sense to move right away to Bookworm: This isn't some flaky Ubuntu release, but a Debian stable release which has ripened over hundreds of thousand of testing installations. There are no known major issues with Bookworm one month after the release and we've added full support for our Debian base layer since the last half year. peopleweb isn't the only service, we are also running a bastion and the entire FastNetMon infra on Bookworm (and the puppet 7 puppet masters/puppetdbs/puppetboards systems are also getting built on it).

The changes which were made so far (font adaptions and the move away from xcftools) are equally beneficial for a bookworm setup and the new stack will have several bugs fixed, that are still present in Bullseye, simply because of the former including two additional years of upstream development.

I understand, my point is around priorities of what to upgrade first: i.e. the order of OS upgrade roll out in the production. Upgrading thumbor first thing (with some exceptions) in production can cause issues in larger scale than it needs to be. It's not necessarily with the bookworm itself but also support of the OS in production. For example, docker images, networking inside kubernetes, etc. etc.

Change 920759 merged by Hnowlan:

[3d2png@master] update dependencies for use with bullseye

https://gerrit.wikimedia.org/r/920759

The review for moving Thumbor to bullseye is ready for review - we've had to make a variety of changes as patches to imagemagick lead to slightly different outputs for generated PNGs (similar to the changes required of https://gerrit.wikimedia.org/r/c/operations/software/thumbor-plugins/+/921255)

https://gerrit.wikimedia.org/r/920760

MarkTraceur renamed this task from Upgrade Thumbor to bullseye to [XL] Upgrade Thumbor to bullseye.Mar 20 2024, 4:44 PM
MarkTraceur subscribed.

n.b. the Structured Content team has estimated this as XL for our purposes (code review only), and from our perspective, sorry if that causes any confusion on anyone else's boards.

Can anyone update the ticket with the current state ?
I believe thumbor hasn't had work done since October again and I'm not sure what the foundation's plans are for this any longer. Are people waiting to complete MW k8s ?

Can anyone update the ticket with the current state ?
I believe thumbor hasn't had work done since October again and I'm not sure what the foundation's plans are for this any longer. Are people waiting to complete MW k8s ?

This needs to be reviewed and merged: https://gerrit.wikimedia.org/r/c/operations/software/thumbor-plugins/+/920760

After that, it should be easy to push it forward.

Change #920760 merged by jenkins-bot:

[operations/software/thumbor-plugins@master] Upgrade container and dependencies for bullseye

https://gerrit.wikimedia.org/r/920760

Change #1039738 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] thumbor: upgrade staging to bullseye

https://gerrit.wikimedia.org/r/1039738

Change #1039738 merged by jenkins-bot:

[operations/deployment-charts@master] thumbor: upgrade staging to bullseye

https://gerrit.wikimedia.org/r/1039738

Change #1039743 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] thumbor: use bullseye version everywhere

https://gerrit.wikimedia.org/r/1039743

Change #1039743 merged by jenkins-bot:

[operations/deployment-charts@master] thumbor: use bullseye version everywhere

https://gerrit.wikimedia.org/r/1039743

Jdforrester-WMF assigned this task to hnowlan.

Resolved enough?

Now it's time to resolve like twenty bugs at one go?

Now it's time to resolve like twenty bugs at one go?

Hopefully! See T265549#9867973.