Page MenuHomePhabricator

Avoid SEO performance and user experience penalties by addressing our handling of lazy loaded images using IntersectionObserver
Closed, ResolvedPublic5 Estimated Story Points

Description

NOTE: Please watch vs to understand how this impacts users.
NOTE: Tap to click will be implemented separately in T246767

Google is actively focusing on performance penalties on "slow events" for search results. We have a slow event every time a section is expanded and lazy loaded images loaded. It's not exactly clear when this will become enforced but the penalties are already present in Google betas. Best guess - April.

The performance team and reading web met to discuss solutions.

Proposed Solution

We reimplement lazy loading images using a MutationObserver or tap to show image pattern

This is the arguably the best and most complete solution. In an ideal solution we would use the MutationObserver and use a "tap to show" pattern on browsers which do not support MutationObserver (e.g. KaiOS). It however would likely be the most complex solution in terms of effort on our part.

Winners: smart phone users, feature phone users, designers
Losers: product, devs

More info: T225946#5916039

Developer notes

Mutation/Intersection Observer is not available in < IE11, Chrome < 26, Safari < 7

The tap to show pattern will be implemented separately as part of T246767. In the mean time we will load all images for KaiOS browser.

You can use the content provider to test this (but note the open bug here > https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MobileFrontend/+/585537)

Since lazy loading images is only required on the mobile domain this code can be moved to mobile.init and the old code relating to transforming burned to start from scratch. The lazyLoad code is still good.
https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/MobileFrontend/+/587893

Rejected solutions

Use native image loading on Chrome only

Since Chrome is likely the browser SEO penalties will apply in we could use this fact to our advantage.
This would lead to a loss of control in the experience on Chrome and would hopefully be the lowest cost solution. On the short term this would break images displaying in printed PDFs (to be confirmed) and it's possible that we may lose some control over when lazy images are loaded and the transitions for when they display.

If we take this option we'd need to weigh up the trade offs here.

Winners: Product (done quicker!), Chrome users
Losers: product (breaks print! upstream bug), devs, designers

Use native lazy loading everywhere

This is the most simple solution. It should be simple to implement and it would remove code maintenance.

However browser support for native lazy loading loading="lazy" is low so browsers other than Chrome and Firefox would now load bandwidth - most notably iOS would no longer load images lazily meaning additional data usage for many of our users.

Winners: Product, Chrome+Firefox users
Losers: devs, designers, iOS users

Developer notes

Meeting notes can be found here: https://etherpad.wikimedia.org/p/Desktop_Refresh_Performance

QA steps

Smoke test:

Repeat the smoke test on other articles using exploratory testing. Note this change only impacts articles which have images in subsections so make sure the articles you test on have images inside a collapsed subsection.

Disa

sign off steps

QA Results - Beta

ACStatusDetails
1T246838#6175234
2T246838#6175234
3T246838#6175234

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Jdlrobson updated the task description. (Show Details)
ovasileva lowered the priority of this task from High to Medium.Mar 4 2020, 10:06 AM

I think we miss Other browsers (Firefox 48 in Kai OS for example) should not load images by default and instead have click to load functionality (as suggested by @Gilles). Needs an product decision and I think if you try Wikipedia on a Kai OS device you will agree. from T246866.

I think the SEO thing is number 2, It's important for our users. Google highlights this because it is making the experience bad for some users.

If you haven't tested what slow JS for lazy loading looks like on a slow device, please look at my two videos in T246767. Another good thing to test is loading an article with images (desktop version) on your mobile phone. Most browser have some indication of when the page is finished downloading and that will continue to move/spin until everything is ready, giving a bad experience.

What I mean is that what we do today is good, and we should make sure every user gets a good experience and that I like @Gilles idea of clicking on images to downloading them on slow devices.

I'm proposing solution 1 which includes using MutationObserver as well as "click to tap". I'd like to check in with the team that this is the right solution before fleshing out some developer notes explaining how that would work and what it would entail.

Jdlrobson renamed this task from Avoid SEO performance penalties by addressing our handling of lazy loaded images to Avoid SEO performance and user experience penalties by addressing our handling of lazy loaded images.Mar 11 2020, 5:30 PM
Jdlrobson updated the task description. (Show Details)

I want to add one thing after looking more into our metrics from Google. I think our fear for SEO penalties for first input delay on mobile was too high. It started with the SEO console warning emails but now when I looked at how we are doing I think we are doing good on first input delay. However the videos I shared show that our current lazy loading is a real problem for users.

At the moment it seems like Google judge us on two metrics: First Contentful Paint (for us that is the same as first paint) and First Input Delay.
The metrics are divided into three buckets: Slow, moderate and fast. For First Contentful Paint fast means the metric happens between zero and one second, moderate means between 1-3 seconds and slower than three seconds mean slow. For First Input Delay the limits are 0-100ms, 100-300ms and above 300 ms.

Google use the 75th percentile for First Contentful Paint and the 95th percentile for First Input Delay.

Looking at the metrics for mobile for the whole English Wikipedia the numbers looks like this:

Checking one page (Barack Obama!) on mobile:

I also looked at the Chinese Wikipedia (let me know if I should collect metrics for another wiki) :

My conclusion is that First Input Delay (at the moment) is not a problem for SEO and long term we should focus on First (contentful) paint.

I've added the pie charts/graphs at a new dashboard I'm working on: https://grafana.wikimedia.org/d/cZgMg49Wz/performance-summary

Jdlrobson renamed this task from Avoid SEO performance and user experience penalties by addressing our handling of lazy loaded images to Avoid SEO performance and user experience penalties by addressing our handling of lazy loaded images using MutationObserver.Mar 25 2020, 5:09 PM
Jdlrobson updated the task description. (Show Details)

Some recordings from January when I noticed the same issue on the Nokia 8110 4G and JoiPhone devices (KaiOS feature phones).

This compares https://en.m.wikipedia.org/wiki/Banksy to https://en.m.wikipedia.org/speed-tests/Banksy.enwiki.872156204/ where the latter is a static snapshot frozen in time. The only difference between it and the real version is that the client-side pipeline stops after phase 2 (we init the inline scripts, startup and and base modules; but not the additional JS modules queued by core/skins/extensions).

This compares https://en.m.wikipedia.org/wiki/Grace_Hopper and its "View desktop" version.

Nothing new I think, but just wanted to share them for comparison to show that it happens on other pages/devices as well, and to perhaps help narrow down or re-confirm the common cause.

Change 587893 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] POC: Lazy loading with IntersectionObserver

https://gerrit.wikimedia.org/r/587893

ovasileva set the point value for this task to 5.Apr 20 2020, 5:32 PM
Jdlrobson renamed this task from Avoid SEO performance and user experience penalties by addressing our handling of lazy loaded images using MutationObserver to Avoid SEO performance and user experience penalties by addressing our handling of lazy loaded images using IntersectionObserver.EditedApr 20 2020, 10:48 PM
Jdlrobson updated the task description. (Show Details)
Jdlrobson added a subscriber: alexhollender.

@Gilles I talked through the solution with the team today and raised the media viewer overlay behaviour. We agreed that opening the multimedia viewer while not ideal was better than loading all images on page load (the alternative). We may tweak this behaviour later, but for now we (designer, product and engineers) have agreed that behaviour is fine.

My intuition was more that tapping the thumbnail once would load just that thumbnail and tapping it again would open the media viewer, but this is totally a product decision! As far as performance is concerned, the current implementation is everything we were hoping for.

Change 588028 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Cleanup: Remove unnecessary lazy loading wire up code

https://gerrit.wikimedia.org/r/588028

@ovasileva given the impact of this one, would it be okay to move this into code review now that the board is a little clearer?

@ovasileva given the impact of this one, would it be okay to move this into code review now that the board is a little clearer?

Sounds good

Change 587893 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Lazy loading with IntersectionObserver

https://gerrit.wikimedia.org/r/587893

@Peter @Krinkle @Gilles patch should land next week. Is there anything we can do to get a sense of the impact of the change? Can the metrics in T246838#5997542 be used?

👏

Once this is proven successful in production, how difficult do you think it would be to adapt this code for it to work on Desktop as well? There are huge savings to be had there, and now that we have an efficient method for it, I think it would be a good time to make it work on both platforms.

As for impact, our usual telemetry should capture it. It's hard to say if any global metrics will be impacted significantly, but I can look at slow devices in particular, as this is where I think there is likely to be a visible change. Let me know the exact date it's been deployed to enwiki and I'll do a one-off before/after comparison for mobile devices bucketed by effective processing speed.

Change 588028 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Cleanup: Remove unnecessary lazy loading wire up code

https://gerrit.wikimedia.org/r/588028

Edtadros added a subscriber: Edtadros.

Test Result - Beta

Status: ✅ PASS
Environment: beta
OS: macOS Catalina
Browser: Chrome
Device: MBP
Emulated Device: iPhoneX

Test Artifact(s):

QA steps

✅ AC1: Visit https://en.m.wikipedia.beta.wmflabs.org/wiki/Albert%20Einstein on mobile as an anonymous user
Scroll the article. Make sure that as the images scroll into view they appear and/or when you open sections.


✅ AC2: Disable JS, check the images appear also.

✅ AC3: Repeat the smoke test on other articles using exploratory testing. Note this change only impacts articles which have images in subsections so make sure the articles you test on have images inside a collapsed subsection.
Verified that this works on Dog, Obama, and Feral Cat. No regressions were observed.

Edtadros updated the task description. (Show Details)

As for impact, our usual telemetry should capture it. It's hard to say if any global metrics will be impacted significantly, but I can look at slow devices in particular, as this is where I think there is likely to be a visible change. Let me know the exact date it's been deployed to enwiki and I'll do a one-off before/after comparison for mobile devices bucketed by effective processing speed.

@Gilles this was deployed on 2nd June and should be live now.

x

Once this is proven successful in production, how difficult do you think it would be to adapt this code for it to work on Desktop as well? There are huge savings to be had there, and now that we have an efficient method for it, I think it would be a good time to make it work on both platforms.

There's an open ticket @ T148047 - I think the challenges around doing this in desktop are mostly around the many printed mediums and ensuring images continue to load in them e.g. printable=yes (also deciding what to do with responsive images - we disable those in mobile)

I've taken a tour of our metrics and can't see a visible change, but then again we don't have any metric that captures rendering smoothness directly. First Input Delay had a big move around May 28 in CruX, but that applied to other websites as well, so it's probably Google changing something on their end.

As for our own collection of FID, I don't see a visible difference around June 2 either:

https://grafana.wikimedia.org/d/9zeb91jWk/first-input-delay

This isn't to say that the UX hasn't improved, since those metrics are only indirectly affected by this improvement. There are likely less frame drops than there used to be, it's just that it would be very expensive to measure that currently. There are discussions within the W3C web perf working group about new metrics that would allow to capture that.