Page MenuHomePhabricator

Pre-render thumbnails in all sizes on the back-end
Closed, ResolvedPublic

Description

Migrated from: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/301

Narrative

As a user, I want to see images faster, even if this requires them all to be pre-rendered in all sizes on the back-end.

Acceptance Criteria

  • Work with operations to implement of image pre-rendering server-side in coming weeks, starting with newly uploaded images.
  • Investigate if this pre-rendering on the back-end improves image load performance for newly uploaded images ( #975 )
  • If performance is significantly improved by this method, pre-render all images on the back-end (open a separate ticket for that task)

<u>Note</u>: Images continue to take too long to load for users with slow connections. If an image has not been previously viewed by another user, it can take 10 seconds or more to load the full image in some low-bandwidth situations. This doesn't match user expectations of 1-2 seconds per image, as typically provided on top multimedia sites.

Related Bugs

Related Stories

  • #8 Media Viewer
  • #975 Investigate if pre-rendering speeds up image load
  • #302 Load smaller images if needed
  • #600 Generate a reference thumbnail for JPEGs
  • #898 Gather real-world available screen real estate in MV
  • #899 Gather real-world available screen real estate in ImageMetrics
  • #900 Request thumbnails based on rectangular size buckets
  • #901 Prerender thumbnails based on rectangular buckets

Related Changesets

Event Timeline

MingleTerminator raised the priority of this task from to High.Dec 8 2014, 4:52 PM
MingleTerminator added a project: Multimedia.
In mingle on 2014-07-31 at 23:45:49, @Tgr wrote:

Are there any implementation plans for this? Add to job queue and have the job queue ping a scaler, or something more complicated?

In mingle on 2014-09-12 at 09:58:30, @Tgr wrote:

It would be nice to expose the bucket list in the repoinfo API. We probably don't want to maintain a separate bucket list in MediaViewer and the server.

In mingle on 2014-09-24 at 14:07:09, @Gilles wrote:

I don't think it's in Media Viewer's best interest to use every single bucket available. There's little advantage loading another bucket that is very "near" to one you've already loaded in Media Viewer, for example. So while Media Viewer's list will always be a subset of the server's bucket list, I don't think that a 1-to-1 match is optimal for what Media Viewer does.

In mingle on 2014-09-24 at 20:56:16, @Tgr wrote:

There are three separate use cases here:

  • when loading the first thumbnail for a fail, more buckets are always better - it means you might be able to load a smaller version of the file.
  • when loading a thumbnail and you already have a larger thumbnail, you can just skip loading and reuse the larger file. The number of buckets does not matter.
  • when loading a thumbnail and you already have a smaller thumbnail, you would have been better off if the smaller bucket did not exist (then you would have loaded the current size the previous time), so less buckets are an advantage in this case. This is, however, not typical in MV usage - mostly just affects users who go fullscreen, and not many do that. The benefit from bullet point 1 should be more significant.
In mingle on 2014-09-30 at 12:25:41, @Gilles wrote:

I'd rather start with the existing MV buckets. Because this will tell us what impact we can expect in regards to new uploads vs existing uploads. If we change the buckets now, we'll introduce noise by virtue of people hitting cache misses a lot more on older uploads.

We can revisit the list shortly after (based on the live canvas size measurements, presumably), once we've measured the impact/share of new uploads vs existing files.

In mingle on 2014-10-01 at 22:36:17, @Tgr wrote:

Tested by uploading http://commons.wikimedia.beta.wmflabs.org/wiki/File:MediaViewer_progress_bar.png

tgr@deployment-cache-text02:~$ ls /data/project/upload7/wikipedia/commons/thumb/f/f8/MediaViewer_progress_bar.png/
1024px-MediaViewer_progress_bar.png 320px-MediaViewer_progress_bar.png
120px-MediaViewer_progress_bar.png 640px-MediaViewer_progress_bar.png
1280px-MediaViewer_progress_bar.png 800px-MediaViewer_progress_bar.png

In mingle on 2014-10-01 at 22:37:51, @Tgr wrote:

Needs to be deployed to production, moving to in development.

In mingle on 2014-10-03 at 08:33:32, @Tgr wrote:

Need to ensure it doesn't try to generate thumbnails larger than or equal to the original image. (Except maybe
for images with an EXIF rotation where an equal-sized thumbnail might be useful.)

In mingle on 2014-10-14 at 15:10:12, @Tgr wrote:

Has no non-merged patches; should be moved to ready for development?

In mingle on 2014-10-15 at 15:44:16, @Gilles wrote:

Moved, I just need to reschedule a SWAT for it

In mingle on 2014-10-17 at 09:01:08, @Gilles wrote:

Deployment to all wikis except commons seems to have gone fine. Next step: deploy to commons.