Page MenuHomePhabricator

Will lazy loading images make us faster on 2G?
Closed, ResolvedPublic

Description

Work in progress :)

Background

We want to know what kind of impact lazy loading images would have from a performance perspective. My first feeling was that the fully loaded metric (the fully loaded happens when all responses has been downloaded to the browser) should be much smaller if we downloads less images. But when we tested serving the Obama page on a 2G connection and a special version with much less image data (1 mb), the numbers are almost the same (we use SPDY). However during these tests we had some problems with the login requests (redirects and 404), so that could have had impacts.

The test

Lets test how it really works. @ori has prepared a couple of different static versions of the mobile version of the Obama page. Lets also test what can give us most impact, we have a couple of different pages:

We test the pages using WebPageTest. We will do 5 runs for each page and take the median run. We will test on Chrome desktop (emulate mobile, set user agent to Iphone 6) and test on Chrome on a Moto G so that we also tests on a real device.

The metrics we collect:

  • Start render - when something first happens on the screen.
  • Speed Index - simplified it explains when the content within the current viewport is ready for the user. Smaller values are better.
  • Fully loaded when everything has been downloaded to the browser.

Emulated mobile 2G Fast using SPDY

TestStart renderSpeedIndexFully loadedComment
Original16.97 s19676107 s
NetSpeed B16.87 s1786748,76 s
No images16.07 s1636429.5 s
No refs11.07 s17143108,8 s
Inline CSS3.775 s17790112 s
No CSS4.26 s4935110.3 sNote: the SpeedIndex is not ok since the page is missing CSS

Do we get the same numbers testing on a real device?

Motorola G 2G Fast using SPDY

TestStart renderSpeedIndexFully loadedComment
Original17.739 s22636111.9 s
NetSpeed B17.475 s2046152.0 s
No images17.418 s1782635.0 s
No refs11.504 s17845107.1 s
Inline CSS4.937 s1136997 sA lot of runs break, lets rerun this later
No CSS4.668s4673110 sNote: the SpeedIndex is not ok since the page is missing CSS

What do we see?

  • Lazy loading images will make the fully loaded happen much earlier. That's really good. It will not give any extras for start rendering or SpeedIndex (or only a small small difference).
  • To really make a difference for start rendering and SpeedIndex we need to serve less HTML and inline the CSS.
  • The start rendering between emulating a mobile browser using Chrome and a real device doesn't have a so big difference in this case. That's good to know.

However, there's a elephant in the room :) If you check the waterfall graphs you will see that when we do request a CSS file, it takes long time to actually get it (13 seconds). This example is from the original page:


Since we are doing a couple of requests/responses at the same time, it is not strange that the download time (the blue part) is longer, because we are on a limited connection. But we also have a long response time for the server (green). The time to first byte is 7614 ms and download time is 5601 ms.

When I test using same browser/same throttling but forces to use HTTP/1 it looks like this:


The time to first byte for the CSS is 1401 ms and download time is 356 ms. The download time could maybe be explained that we only do the CSS download at that time, but the first byte I don't know. @ori @Krinkle do you have an idea?

Lets test what it looks like using HTTP/1.

SPDY vs HTTP/1

Lets check what the numbers looks like when we use HTTP/1.1. We can configure Chrome to use HTTP/1 by setting the flag --use-spdy=false. Then the browser will do only X requests at a time to each domain. The fully loaded time will probably be longer. Lets measure it to know:

Emulated mobile 2G Fast using HTTP/1

TestStart renderSpeedIndexFully loadedComment
Original6.299 s21678118 s
NetSpeed B6.109 s1598956.6 s
No images6.678 s691429.7 s
No refs6.114 s12640112 s
Inline CSS4.560 s15426117.9 s
No CSS4.074 s4931117.3 sNote: the SpeedIndex is not ok since the page is missing CSS

It looks like serving with HTTP/1 we already got almost a reasonable start rendering time. Lets investigate the TTFB before we continue.

Do we test the "right" way

Could it be that I do the testing wrongly somehow? Please give feedback if see something that looks strange.

Related Objects

StatusAssignedTask
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
DeclinedNone
OpenNone
Resolveddr0ptp4kt
DuplicateJhernandez
Duplicatedr0ptp4kt
OpenNone
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedPeter

Event Timeline

Peter created this task.Jan 27 2016, 11:02 AM
Peter claimed this task.
Peter raised the priority of this task from to Needs Triage.
Peter updated the task description. (Show Details)
Peter added a project: Performance-Team.
Peter added subscribers: Peter, Krinkle, ori.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptJan 27 2016, 11:02 AM
Peter updated the task description. (Show Details)Jan 27 2016, 10:48 PM
Peter set Security to None.

Summary

Will lazy loading images make us faster on 2G

No but it will make the fully loaded time much better and will make the user download much less data by default and that is super good.

From a start render perspective there's two things that will make an impact:

  • Inlining the CSS is the most important thing to fix (as @GWicke also showed in T124966).
  • Minimize the HTML.

What's extra interesting about the HTML is that with the amount we send, we also see that the browser needs to spend quite some time to parse it to render the page if you are not on a state of the art computer. You can check out this example of the devtools timeline from WebPageTest. I've turned on stack traces so that takes som extra time (meaning the start render time is slower than usual. You can see the original test here. It would be interesting to make more tests on a slower computer, lets setup a task for that.

The last thing: SPDY vs HTTP/1 on slow connections, let me setup a different task for that.

ori moved this task from Inbox to Doing on the Performance-Team board.Feb 1 2016, 7:44 PM
Peter closed this task as Resolved.Feb 2 2016, 9:49 PM