Page MenuHomePhabricator

rsvg-convert times out while generating large thumbnails with heavy use of Gaussian blur
Open, Needs TriagePublic

Description

Gaussian blurs in librsvg 2.40.16 are largely not optimized, causing larger thumbnails for files that make heavy use of Gaussian blur to time out.

  • 2.42.3 added minor optimizations (T193352)
  • 2.43.3 optimized further
  • 2.43.4 parallelized Gaussian blur
  • 2.44.0 introduced more speed improvements
Known affected files

At least the first three are rendered in 10-15 seconds with librsvg 2.48 in the requested size mentioned above.


Reported on IRC regarding https://commons.wikimedia.org/wiki/File:Jupiter_diagram.svg which has relatively large dimensions in the SVG base image.

This 2000px thumbnail has already rendered fine:
https://upload.wikimedia.org/wikipedia/commons/thumb/b/b5/Jupiter_diagram.svg/2000px-Jupiter_diagram.svg.png

But the file: page links offer a 5000px thumbnail which does not render:
https://upload.wikimedia.org/wikipedia/commons/thumb/b/b5/Jupiter_diagram.svg/5000px-Jupiter_diagram.svg.png

It takes some seconds (30-ish?) then times out with an HTTP 500 error. Too many attempts hits rate limiting, returning a 429.

A sample request failure id:

Related Objects

Event Timeline

brion created this task.Aug 1 2018, 2:03 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 1 2018, 2:03 AM

Indeed Thumbor rate-limits failures to render a certain original after some tries, https://wikitech.wikimedia.org/wiki/Thumbor#Throttling
My guess would be that 5000px exceeds thumbor's memory limit for rendering

Usin rsvg-convert version 2.48.4, it takes 8-10 seconds to generate the 5000px thumbnail on my laptop. With rsvg-convert version 2.40.16, which is running in production, it takes 50-55 seconds. That's dangerously close to the 60-second timeout for the command. This file makes fairly heavy use of Gaussian blur, which was optimized in 2.42 and 2.43. Waiting for T193352 is the best solution here.

AntiCompositeNumber renamed this task from Timeout and HTTP 500 error on 5000px thumbnail of large SVG image to rsvg-convert times out while generating large thumbnails with heavy use of Gaussian blur.May 15 2020, 12:39 AM
AntiCompositeNumber updated the task description. (Show Details)
AntiCompositeNumber moved this task from Backlog to Upstream on the Thumbor board.
AntiCompositeNumber moved this task from Backlog to Patch merged upstream on the Upstream board.
JoKalliauer updated the task description. (Show Details)May 15 2020, 10:08 AM
toorich changed the status of subtask T193352: Update librsvg to ≥2.42.3 (2.44.10) from Stalled to Open.Jun 24 2020, 9:59 AM
Aklapper changed the status of subtask T193352: Update librsvg to ≥2.42.3 (2.44.10) from Open to Stalled.Jun 24 2020, 10:34 AM
JoKalliauer renamed this task from rsvg-convert times out while generating large thumbnails with heavy use of Gaussian blur to rsvg-convert times out while generating large thumbnails with heavy use of Gaussian blur aswell with heavy use of gradients.Jul 18 2020, 1:55 PM
JoKalliauer updated the task description. (Show Details)

@JoKalliauer, You added https://commons.wikimedia.org/wiki/File:Hematopoiesis_(human)_diagram_en.svg to this task, apparently because it was timing out because of gradients. However, I'm not able to reproduce the issue locally with rsvg 2.40.16:

$ time rsvg-convert -w 1405 -f png -u -o Hematopoiesis_\(human\)_diagram_en.svg.png Hematopoiesis_\(human\)_diagram_en.svg 

real	0m0.397s
user	0m0.288s
sys	0m0.023s

Generating a new, large thumbnail for that file also works just fine. Unless you have more information to show that this file is actually timing out, I'd ask you to revert your changes and file a new task if there are issues with that file.

JoKalliauer renamed this task from rsvg-convert times out while generating large thumbnails with heavy use of Gaussian blur aswell with heavy use of gradients to rsvg-convert times out while generating large thumbnails with heavy use of Gaussian blur.Jul 18 2020, 7:17 PM
JoKalliauer updated the task description. (Show Details)

@AntiCompositeNumber : Thanks for your answer, I think I was wrong.

This image was reported (by Sarang) on my talkpage: https://commons.wikimedia.org/wiki/User_talk:JoKalliauer#Strange_failure

If I open
https://upload.wikimedia.org/wikipedia/commons/thumb/1/1f/Hematopoiesis_%28human%29_diagram_en.svg/80px-Hematopoiesis_%28human%29_diagram_en.svg.png
I get

Our servers are currently under maintenance or experiencing a technical problem. Please try again in a few minutes.

See the error message at the bottom of this page for more information.
If you report this error to the Wikimedia System Administrators, please include the details below.

Request from 193.81.142.140 via cp3055 frontend, Varnish XID 231829017
Upstream caches: cp3055 int
Error: 429, Too Many Requests at Sat, 18 Jul 2020 19:17:49 GMT
blinry removed a subscriber: blinry.Jul 19 2020, 9:09 AM