Page MenuHomePhabricator

Change threshold for related pages
Closed, ResolvedPublic0.5 Estimated Story Points


As a user, i would like to avoid abrupt insertion of related pages when I scroll down to the bottom of the article.


As a user, I would like related pages to be ready to be read when i reach the bottom of the article.

The current threshold for related pages is a complicated formula.

Change the threshold to the height of viewport.

  • Also make sure on page load Related Pages show up when the article fits in a screen height.

Acceptance Criteria for QA:

  • Related pages are loaded when user scrolls to the bottom of page, no significant lag is observed
  • Behavior upon scrolling down is smooth (no abrupt loading)

Event Timeline


ScrollThreshold = $( document ).height() / 2 - $window.height();

ScrollThreshold = $( document ).height();

ScrollThreshold = $window.height();

$(window).height() gets you an unit-less pixel value of the height of the (browser) window aka viewport. With respect to the web browsers the viewport here is visible portion of the canvas(which many times is smaller than the document being rendered).

$(document).height() returns an unit-less pixel value of the height of the document being rendered. If the actual document’s body height is less than the viewport height then it will return the viewport height instead.

The formula we're currently using is pretty insane...

Half the page height - window height.

So for this page, in a phone, the related pages get triggered about half the way down on the page...:

|  |
|  |
|==| < phone height
|  |
|  |
|  |
|  |
|  |
|  |
|  |
|  |
|  |
|  | < half document height
|  |
|  | 
|  | < half document height - phone height (related pages loading triggered)
|  |
|  |
|  |
|  |
|  |
|  |
|  |
|  |
|++| < Related pages

This is pretty wrong... It has the potential to cause a shit-ton of api requests given the articles with the most views are the longest usually.

oops my bad.. i meant window height and not document height :)

it should be equal to viewport aka visible area of the page. should be around 500px on average for most phones.

bmansurov removed the point value for this task.

Sorry about accidentally removing points. Phab merge conflict.

Change 310050 had a related patch set uploaded (by Bmansurov):
Change scroll threshold to viewport height

So the scroll threshold is window.height() ? This seems very low.

For lazy loaded images we used $( window ).height() * 1.5 and that took lots of tweaking. I would suggest using the same here or even more - especially given the request to download related pages is likely to take longer than the time taken to download an image.

That's actually a good idea. the current threshold is 5 times lower than this.

Maybe let's keep lazy loading threshold consistent?

That's actually a good idea. the current threshold is 5 times lower than this.

This is wrong. Current threshold may be bigger for long articles, and maybe lower for shorter ones.

This is wrong. Current threshold may be bigger for long articles, and maybe lower for shorter ones.

This is wrong. not the statement, the logic :) different articles with different threshold.

Change 310050 merged by jenkins-bot:
Change scroll threshold to viewport height * 1.5

Looks good. again. not many articles with related pages so we can only stress test this on beta

phuedx added a subscriber: phuedx.

@Nirzar: The change will be deployed to the Wikipedias on Thursday, 22nd, so we could wait until then… or we could add related pages on the Beta Cluster wiki ourselves 😀

That search fails a lot so here are the results:

Don't shoot the messenger.

I wanna say yes. but on fence for following reason ->

I tested it out on different network speeds.
i am on fence to increase the threshold even further. this worked very well 7/10 times. thinking if we can avoid that 3 [1]

one thing can help us be sure is to check the actual byte size of related pages. on average compared to image thumbnails. larger the size, greater the thershold.

if average image thumbnail size is X and we decided good threshold is (viewport * 1.5)
and if average related pages is 2X then threshold should be (viewport * 3) instead?

of course X and 2X are just placeholders here.

[1]take this anecdotally if you will, this was tested without any scientific methods. just trial and error.

cc @ovasileva

I agree - is there some way we can play around with sizes and change thresholds according to relative size?

one more note to everyone: i know this going back and forth from original task description but in this particular case we had to do it and test it. there was no way knowing if we had the correct number from beginning. the task is just the idea :)

from post standup: @bmansurov will add multiplier value as query parameter in the URL and I will test with different values to figure out the sweetspot.

Can we prototype this rather than using production as a prototyping platform?

@Jdlrobson i forgot to mention the environment. we will be adding this param thing only on reading web staging i suppose thus making that a prototype? we will test there and come back to the patch.

(but do consider that such an environment is likely to be much slower than production)

Change 310894 had a related patch set uploaded (by Bmansurov):
Configurable threshold

@Nirzar, @ovasileva here you go:

Notice the threshold parameter in the URL. That number is multiplied by the window height before deciding whether to load related articles. Let me know when you're done testing so that I can abandon the patch and clean up the staging server.

I tested this on safari responsive mode and a 4" inch iOS and 4.5" android.

Tried 2x and 2.5x as values. though 2.5x works better, it might increase downloading information that might not be seen by the reader. so we need to be cautious of that.

I think 2x works the best. Let's go with (Viewport * 2)

Change 310917 had a related patch set uploaded (by Bmansurov):
Increase threshold for loadign related articles in the footer

Change 310894 abandoned by Bmansurov:
Configurable threshold

The test has been carried out and we decided to use 2.

I just wanted to point out that this test page doesn't use the morelike api to generate the related list which is used on the majority of our pages. At a glance the morelike api adds 400ms-1.27 s delay to rendering (on WiFi).

I would suggest setting up a test page that checks articles which pull from the morelike api to get a better indication of how a threshold will behave in this wild.

Another approach maybe to think in terms time the user needs to view content that occupies two viewports. Is that time enough for us to load related articles? We should not penalize users by loading something that they may not even read.

It's true that some users don't read and scroll past to the bottom. In that case we may want to display a loading indicator to indicate something will be loaded in the footer.

@bmansurov I think if the user is 2 scrolls away, it is likely that they will reach the bottom. more likely than 3 scrolls away. we can't predict but we can certainly create a hypothesis of likelihood.

i think we are getting in weeds here by looking at 1.8 or 2.0 or 2.1. let's look at from perspective of lazy loading images. we happily use 1.5 for images and we are using 2 for RP because it is more in size. let's keep it at that? again, looking at the current implementation this is a huge improvement.

Change 310917 merged by jenkins-bot:
Increase threshold for loading related articles in the footer