Page MenuHomePhabricator

[wmf.1] MediaSearch page 429 error responses on initial search request
Closed, ResolvedPublicBUG REPORT

Description

  1. Perform any search (i.e. 'snow', 'vikings' etc) on commons wmf.1 Special:MediaSearch page
  2. The Console will display 429 errors and the image placeholders will be displayed in the UI:

Screen Shot 2020-07-24 at 11.58.18 AM.png (707×1 px, 1 MB)

  1. Scrolling down the search results will produce more errors and empty spaces for images.

Note:

  • 429 responses are present only during the first request for search. Reloading the page will fix most 429 responses and eliminate most empty image placeholders (e.g. from 18 errors it'd be down to 2).
  • clicking on Search button again won't have any effect on the result set since it does not re-send any request

Event Timeline

CBogen added a subscriber: egardner.

@egardner could you take a look at this one? Thanks!

Confirming that I can see this myself on production commons. Is WMF-1 a specific server you are hitting to test this?

429 error code means too many requests, which is not surprising for a UI that quickly loads dozens of images in an infinite-scrolling way. It would be good to know if a specific rate-limit is being set either in MediaWiki generally or for Commons specifically – ideas on where to look for this info would be appreciated if anyone has them.

Here's what I suspect might be happening:

We load results in batches of 40 (including the initial batch when a user begins a search). Once the API request for a search is complete, the new results are added to the Vuex store, and the MediaSearch UI attempts to render 40 new image elements to the page at once. This goes over some limit that the server has set and so we start seeing 429 errors. If the user reloads the page, all the successful images are now cached, meaning they don't have to request as many images from the server. But as you scroll down the page to load more results, you will eventually start requesting images for the first time again and the problem will return.

Potential solutions to this problem might include:

  • Loading smaller batches at once to see if we can stay under the limit
  • Figuring out how to stagger the addition of new image elements to the page; Vue's transition API might be useful here
  • Implement some kind of lazy-loading for image elements here

I'm happy to try out some of these approaches to get rid of this behavior. Personally I don't think this bug necessitates rolling back the deploy (this whole special page is still a work-in-progress that is not widely used) but if others feel differently let me know.

@egardner I don't think we have to roll back the deploy, but this should be fixed before we hit alpha.

@CBogen agreed.

I think that a complete solution to this problem will be a combination of 1) implementing lazy loading for images and 2) changing the "load more" behavior to require additional action from the user every so often (clicking a "load more" button every X batches, etc); the latter was something we wanted to address anyway as part of T258516. Together these two changes should help to prevent a flood of requests from overwhelming the server's rate-limit (something that it's very easy to do in the current UI if you just scroll down the page as fast as you can).

Change 616918 had a related patch set uploaded (by Eric Gardner; owner: Eric Gardner):
[mediawiki/extensions/WikibaseMediaInfo@master] Add a base Image.vue component with lazy loading & error handling

https://gerrit.wikimedia.org/r/616918

It looks like there's an extraneous background color, noted in the gerrit patch, causing this background color mismatch:

Screen Shot 2020-07-30 at 9.12.39 PM.png (922×1 px, 486 KB)

Checked along with T259642: Implement lazy-loading of image results in wmf.4 - no issues with loading images were found.