Page MenuHomePhabricator

Provide infrastructure to compare first paint and load time with on demand images
Closed, ResolvedPublic5 Story Points

Description

Currently we support an images disabled mode in MobileFrontend.
We want to test a hybrid mode which loads images as required via JavaScript.

With a flag enabled e.g query string parameter
AND the disableImages cookie / mobileoptions feature enabled any disabled images should be pulled in via JS (when scrolled to - see getBoundingClientRect / debouncing). Ideally it shouldn't need to do API requests.

You may need to update MobileFormatter to do this.

Event Timeline

Jdlrobson updated the task description. (Show Details)
Jdlrobson raised the priority of this task from to High.
Jdlrobson set Security to None.Sep 2 2015, 4:54 PM
Jdlrobson edited a custom field.
Jdlrobson moved this task from Needs Analysis to To Do on the Reading-Web-Sprint-55-π board.
bmansurov added a comment.EditedSep 2 2015, 5:56 PM

Another idea is to swap out all images on the page with a light-weight generic placeholder SVG (which scales) that would reveal the real image on click. I think this is advantageous to the proposed hybrid because even though images are loaded on scroll in the proposed hybrid mode, the user may not even want to see images when scrolling. But if all images are replaced with a generic image, the user will selectively show images that are of interest to the user.

Change 236729 had a related patch set uploaded (by Jdlrobson):
Lazy load images via a service worker

https://gerrit.wikimedia.org/r/236729

Talking to @Krinkle and @Gilles we suspect we may be able to avoid breaking the web by doing the rewrite on the html level which is pretty cool :)

1:41 PM <jdlrobson> phedenskog: So I can run the tests locally and for image heavy pages scrubbing out all images bar the lead image makes some difference to first paint - the difference will depend on how image heavy the page is
1:42 PM <jdlrobson> For example Barack Obama (all images) starts render at 17.895s and just lead image starts render at 9.496s http://www.webpagetest.org/result/150908_J5_14NT/ http://www.webpagetest.org/result/150908_XX_14YX/
1:43 PM <jdlrobson> Facebook first renders at 16.499s with all images and 13.700s without (http://www.webpagetest.org/result/150908_4C_14PD/ and http://www.webpagetest.org/result/150908_PK_14Z1/)
1:43 PM James_F → James_F|Away
1:46 PM <jdlrobson> But One Direction renders roughly the same/worse at 11.194 with and 12.191s (http://www.webpagetest.org/result/150908_EA_14PX/ and http://www.webpagetest.org/result/150908_S7_154T/)
1:46 PM <jdlrobson> one direction = 26 images; Facebook = 38 images; Barack Obama = 102 images
1:47 PM <jdlrobson> So I'm not sure how best to test this at scale. As well as performance perspective, lazy loading images would use less data for our users which I would say is a good thing

@Peter - so my updated patch allows us to opt into a service worker on the cluster that allows rewriting of HTML to stub out images so they can be lazy loaded. Given it's opt in, it would allow us a way to run tests on a larger subset of pages. Given the results on static pages I'd love to try this in a production environment. Is that a bad idea? If so what do you suggest as a better way forward here?

phuedx added a subscriber: phuedx.Sep 10 2015, 8:07 AM

@Jdlrobson: How would you feel about deploying this to a production-like environment rather than production production – I'm guessing you mean the Wikipedias here, right? If we're going to be running a specific set of tests, then maybe all we need is another reading-web-staging with the test pages imported into it.

Peter added a comment.Sep 10 2015, 3:52 PM

@phuedx @Jdlrobson production-like environment is perfect, the important thing if we wanna use WebPageTest is that the environment is reachable from the internet. One cool thing would be to actually also measure the downloaded size, so we both measure firstpaint/speed index and downloaded size.

About testing a larger subset: We can either crawl a list of X or if we can get a list of most accessed pages on mobile? And/or we can test both.

Would be interesting to know the behavior of the user, what happens when the page loads etc.

I think this would be really interesting to do and get some stats and surely will help do it (I'll just check with Ori). Using ServiceWorker is just for the test or is it the final solution? If so the lack of support for ServiceWorker concerns me. I mean it's only Chrome right, how many % of total usage will we help with the fix? Firefox is behind a flag so I guess it will happen sometimes, but it would be cool to do something for Safari users also. But as a test is super :)

I hope that we're only using ServiceWorker for a test – though, honestly, it's a neat little solution.

Would be interesting to know the behavior of the user, what happens when the page loads etc.

Let's make an epic for this and try and wrangle a researcher to design one or more experiments!? (read: I'm in).

@Peter ServiceWorker would just be for a test for the time being. It seems like the most lightweight cheap way to do this right now. It will be opted in via a querystring parameter - image-experiments=true which will register the service worker. Subsequent page loads would then be routed via the service worker with images stripped.
Please take a look at https://gerrit.wikimedia.org/r/#/c/236729/ and see if you think it is sane.

I'd like to get some results and assess whether this is worth pursuing more and defining what is needed for production quality (including whether we should use ServiceWorker or something else)
Going forward I feel like it's the right fit even though its Chrome only (I'm actively chatting to Mozilla about their ServiceWorker work to get them on board so I think they will follow shortly)

Just incase y'all aren't following the commit – why not!? – here's what I've asked for:

I think I'd also like to see this feature flagged so that we can turn it off PDQ. That is, I think I'd be happier about this being merged if it were disabled by default, even if the user – us and anyone else who is aware of the commit – happens to add the flag to the query.

http://stats.grok.se/en/top gives us top articles in March. This may be a good enough starting point.

Yay got some fresh stats from @JAllemandou
pageviews only, mobile app + mobile web, users only (no known bot, or almost), view_count since sept 1st:
https://gist.github.com/jobar/53db2d87461cf3137d03

Jdlrobson added a subscriber: rmoen.

This might be considered cheating so I'm not going to sign off on it but will allow @rmoen @phuedx @bmansurov to make a decision - but it seems despite no code being merged the conversation generated has been worthwhile and the code as a proof of concept is useful.

Although this wasn't the original goal I'm happy to close this bug and continue the conversation on https://phabricator.wikimedia.org/T110615#1629618 until we have a more concrete idea of what we are trying to measure. Thoughts?

[1] https://phabricator.wikimedia.org/T110615#1629618

Change 236729 abandoned by Jdlrobson:
Lazy load images via a service worker

Reason:
Whilst doing this:
https://en.m.wikipedia.org/wiki/User:Jdlrobson/lazyloader

I noticed that because the service worker lives on /w/load.php it is scoped to '/w/' and thus won't work in '/wiki' - we'll have to rethink this approach.

https://gerrit.wikimedia.org/r/236729

From @Gilles which is good to know:
"The service worker can "break out" of its path, thanks to a dedicated header: https://gerrit.wikimedia.org/r/#/c/219960/8/includes/resourceloader/ResourceLoader.php,unified as well as passing the scope parameter: https://gerrit.wikimedia.org/r/#/c/219960/8/resources/src/mediawiki/mediawiki.js,unified"

phuedx closed this task as Resolved.EditedSep 16 2015, 5:17 PM

I'm happy to sign this off. I know we tried to put this through our process as a story because of the initial idea for the implementation but it actually ended up being more spike-ish/don't do it in MobileFrontend -ish.

@Jdlrobson: Could you send a brief email to mobile-l about what you attempted to do and what you've learned?