For each file type, if possible. I've noticed that PIL (Thumbor's stock engine) seems to perform quite terribly right now in comparison to Mediawiki, I think a full review is in order.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • Gilles | T121388 Service-based thumbnailing re-architecture in production with Thumbor | |||
| Resolved | • Gilles | T111718 Service-based thumbnailing re-architecture on Vagrant | |||
| Resolved | • Gilles | T125078 Review performance of Thumbor versus stock Mediawiki | |||
| Resolved | • Gilles | T125137 Investigate how much faster (if at all) Pillow's image resize runs under PyPy |
Event Timeline
Comment Actions
For JPG, Thumbor (with PIL) is 4 times slower than Mediawiki. This could be due to the sampling trick used by imagemagick, which PIL is probably incapable of. I'll have to assess whether we should just use IM/GM with the same parameters in Thumbor.
Comment Actions
For PNG, Thumbor with VIPS is twice as slow as Mediawiki with IM. Thumbor with PIL is 6 times as slow as Mediawiki with IM.
This might be another indicator that IM/GM as the default engine might make more sense than PIL for performance. Also, VIPS should only be used for large PNGs.
Comment Actions
Looking at video, I've discovered that TMH seems to store a "canonical" image of the frame's full resolution, for which it bases further resizes onto. Thumbor doesn't do such chaining and as a result performs slower.
However when it comes to generating the full-size frame, Thumbor is much faster thanks to its optimization of only fetching the frame it needs.
We'll have to think about whether or not it's worth introducing the same chaining, considering that it is destructive. In a way it explains why the slow fetching of the whole video wasn't a big problem until now. In a world where all thumbnails are kept forever in Swift, this was only happening once per file, at upload time.
Comment Actions
On XCF, Thumbor seems approximately twice as fast in terms of CPU. In this case Thumbor uses a dedicated tool (xc2png), while Mediawiki relies on IM.
However, some overhead remains, as despite less CPU usage the requests take about the same time for the end-user in both cases. Which makes me wonder how efficient Mediawiki and Thumbor are in terms of saving the thumbnail to Swift "after hanging up". I.e. the image should be served right away, and saving to Swift can be taken care of after closing the connection to the client.
Comment Actions
@Gilles, you should benchmark performance with PyPy as well; it may be substantially improved.
Comment Actions
Actually, I don't know how hard they've tried making that work, but Thumbor's PyPy build is marked as consistently failing in their CI: https://travis-ci.org/thumbor/thumbor/jobs/104486315 I'll give it a shot anyway.
Comment Actions
SVG is the first one I look at that uses the same command-line tool and there Thumbor is on par in processing time and actually wins the total request speed. I realize now, though, that in all the cases before PIL was involved. Because the XCF converter merely converts to PNG, and PIL does the actual resizing afterwards (this is something I do in most engines). The SVG command line tool, however, is able to resize as well.
This would suggest that the slowdown in all the formats I've seen so far might be attribute to PIL/Pillow doing the resizing (technically the SVG code goes through PIL, but by that point it has no processing to do).
The remaining engines I haven't tested yet all do their resizing in PIL, so I'm not going to waste time investigating those yet, I'm first going to look either making PIL run faster with PyPy or swapping it for another engine (there are a couple of options written by other folks).
Comment Actions
Now that everything is backed by the IM engine, let's do the comparison again. This is very casual testing with a single image on VM. I don't think that a large test would be a valuable way to spend my time, we'll get that "for free" when we get to a gradual roll out, this is just a sanity check.
JPG
Thumbor is now 4-5 times faster than stock Mediawiki. The target size involved some sharpening, which we know is a lot faster (and probably more crude) with Thumbor. Since the resulting look is different, some backlash from the Commons community is to be expected there. We can always use IM's sharpening in case Thumbor's is unpopular, which would probably bring the performance on par with the status quo.
GIF
I need to check if things are setup the same way in production, but Mediawiki's default handling of animated GIFs is abysmal compared to Thumbor. Downscaling my test animated GIF took 8+ seconds on Mediawiki, 250ms on Thumbor.
XCF, PNG, TIFF, DJVU, PDF
Thumbor and Mediawiki seem to be on par.
I'm closing this, I'm not seeing any big red flags anymore. A thorough perf investigation will be done during the gradual release.