Page MenuHomePhabricator

Newcomer tasks: slow loading time for suggested edit module
Closed, ResolvedPublic


When the user initiates the suggested edits module, there is a delay in loading the module onto the page, and a longer delay in loading the first suggested article. See gif below.

We should either increase the speed, or add some kind of loading animation. Increasing the speed is preferable, and so the first look should be from the engineers.

newcomer_tasks_loading_time_2019-11-12.gif (818×1 px, 2 MB)

Also, the height of the module changes during loading. It starts out short, and then extends when the article cards show up. This causes the adjacent impact module to do the same.

Most of the slowness is due to the growthtasks API query. We can try several things to speed it up:

  • Rendering the first task on the server-side by performing a search query for 1 article ({T236738: Newcomer tasks: server-side rendered version of suggested edits module})
  • Use internal search instead of the server sending web requests to itself (T235717: Newcomer tasks: non-HTTP-based ConfigurationLoader and TaskSuggester)
  • Instead of making six different search queries, combine them into a single one (means we won't have much control over the ordering / ratio of different task types)
  • Alternatively, keep using web queries and use parallel HTTP requests (doesn't seem like a great long-term approach)
  • Reduce the number of pages in the request (either by reducing the total number of cards we are showing, or by loading them in multiple batches)
  • Reduce the work done in the API request: instead of doing a generator query, do a list query, and then use the received page IDs to fetch extra information in a separate request. (This is something we might want to do anyway because this way we can use query API continuation, while with a generator returning random results we cannot. Although other possible ways of fixing that exist.) The extra information can then again be batched. Alternatively, we might omit some things we already get from the PCS summary API (such as images).
  • Push things into the search phase. Especially useful for page protection, since we use that for filtering: if we pushed the data into ElasticSearch, we'd get that for free, and seems like a useful CirrusSearch capability in general. (T259346: Add page protection filter to CirrusSearch)
  • Decouple the rendering of various things (e.g. currently we wait for both the extract and the pageviews before displaying either). This is probably a micro-optimization with nontrivial code complexity so unlikely to be worth it.
  • Cache things at various levels of aggressiveness (the first search result, or the first batch of search results, or all), and possibly include the cached data in the page HTML/JS. This adds a fair amount of complexity, and it won't speed up reloading the suggested edits module when the search parameters change (ie. the user changes the filters), but for other reloads (e.g. opening the homepage) this will be the most effective approach by far.
  • Show the module while loading (e.g. Facebook shows blurred image and text). Doesn't affect actual speed but improves perceived one.

Related Objects

Event Timeline

Moving off sprint board in favor of Newcomer Tasks V1.0 tasks.

@Tgr said he would add a list of ideas to the task description for improving loading times.

Added. The most effective approach is of course caching but that comes with a lot of complexity and some side effects. The most effective relatively simple change is probably splitting the API request into batches, and doing a single search instead of six (but we should probably benchmark things instead of guessing).

Hi @Tgr - the last approach you mention about showing the mod whole loading is the approach I am referring to on T238231

Instead of making six different search queries, combine them into a single one (means we won't have much control over the ordering / ratio of different task types)

We are now making potentially over a hundred queries, so this is getting more problematic.
Combining queries would raise several problems, though:

On a somewhat different note, IMHO the next useful thing to do from a user perception standpoint is to provide a server-side rendered view of the suggested edits module that includes:

  • the user's preset topic and task type filters
  • the skeleton screen (T238231)

That way, the user sees content in the suggested edits module immediately when the homepage renders, rather than waiting several seconds.

It could be useful to analyze patterns of use among users -- how often are they adjusting the topic / task type filters? Do most people just adjust these in a single session and then leave them in place?

We are now making potentially over a hundred queries, so this is getting more problematic.

Can we cache the result set for each query for, say, one or two weeks? Maybe with some invalidation logic after a suggested edit is completed from a given result set?

kostajh claimed this task.

I believe this is resolved with the caching improvements we've made (specifically T236738). cc @MMiller_WMF