- Mentioned In
- T280843: Find 8 machines (4 eqiad + 4 codfw) for Thumbor
T256959: Allow PDF's to be rendered at higher (or user specified DPI)
T216815: Upgrade Thumbor to Bullseye
T43426: hsl colors not supported by rsvg
- Mentioned Here
- T18491: Support for Chemical Markup Language
T40010: RFC: Re-evaluate librsvg as SVG renderer on Wikimedia wikis
T290759: Undeploy VipsScaler from Wikimedia wikis
T216815: Upgrade Thumbor to Bullseye
T233196: Migrate thumbor to Kubernetes
T252719: Upgrade thumbor to Thumbor 7 and python3
T267327: Run latest Thumbor on Docker with Buster + Python 3
- need to update python
- old hardware
Can we define a product owner?
- not clear on if any product team has owned this in the past
- originally owned by the multimedia team
- we plan to ask for a multimedia team for 2022 fiscal
Greg to have conversation with SRE on what a potential timeline is
From my perspective, the largest threats to the continued stability of Thumbor are:
1 and 2 are relatively simple, and it makes sense to fix them at the same time (preferably with T233196: Migrate thumbor to Kubernetes ). It's pretty much only blocked on resourcing. Some work toward this goal was done in T267327. Most of the tasks in Thumbor are upstream library issues, and many are fixed in the versions of the libraries packaged in Buster or Bullseye.
The Python 3 upgrade is trickier. Upstream released an alpha version with Python 3 support (Thumbor 7) in February 2020, but has had very little activity since (none in the last 6 months and no releases of any kind in the last 18 months). T267327 got the Wikimedia customizations working with Thumbor 7, at least well enough to pass the Wikimedia test suite. From my perspective, most of the work remaining there is testing and dealing with whatever upstream may decide to do.
To be clear, thumbor is just one part of the problem, the entire file management stack needs a dedicated owner. Other people have just been covering for issues when it gets really bad. File metadata had been causing significant problems until Tim and Amir moved it into external storage. MediaWiki had been uselessly creating thumbnails of all images that Thumbor served, until Tim disabled it after realizing this was happening during Shellbox work. I ended up doing all of the file-related Shellbox porting in MediaWiki even though it's really not "ServiceOps" responsibility.
There's legacy cruft like T290759: Undeploy VipsScaler from Wikimedia wikis, which is both unused and used at the same time. T40010: RFC: Re-evaluate librsvg as SVG renderer on Wikimedia wikis is a very sad story of people putting in a lot of work to the point of building a new SVG renderer that is really just blocked on someone packaging stuff up and getting it installed (and then dealing with ensuing bugs and updates, etc.).
For emphasis, let me add that the effects of the lack of ownership there is visible to the community / end users. Speaking from the perspective of a community member, having a Multimedia team again would be good but there also needs to be somewhere the buck stops on the entirety of the file management stack. That can certainly be a "Multimedia team", but then the scope needs to be very clear so that the necessary resources are allocated. Because it's not obvious just from the name that things like Thumbor, or file metadata storage, or rearchitecting file operations to not require a database lock, or… fall within the scope of a "multimedia" team.
This stuff is fundamentals, and can't be left to random acts of kindness by people on other teams.
March 11, 2022 Update
- As per internal discussions, the PET team will take ownership of Thumbor until we resolve the larger media support question with Product Ownership falling Platform PM team.
We have outlined the following items to be worked on:
- Upgrade Thumbor to Buster: https://phabricator.wikimedia.org/T216815
- Migrate thumbor to Kubernetes: https://phabricator.wikimedia.org/T233196
- Upgrade thumbor to Thumbor 7 and python3: https://phabricator.wikimedia.org/T252719
We are currently aligning resources to pick up this work and will provide update on when we expect to start.
For everyone's info, currently no Code-Stewardship-Reviews are taking place as there is no clear path forward and as this is not prioritized work.
(Entirely personal opinion: I also assume lack of decision authority due to WMF not having a CTO currently. However, discussing this is off-topic for this task.)