Page MenuHomePhabricator

[3 days] Interaction to Next Paint (INP) Core Web Vital is scored as "Needs Improvement" or "Poor" for Mobile users on Desktop
Closed, ResolvedPublic5 Estimated Story Points


User Story

  • Team-facing: As the Web Team at Wikimedia, I do not want our site to be unduly penalized in search rankings for a "Poor" Interaction to Next Paint (INP) score in the Google Search Console for users on mobile devices using our desktop site, when Google changes its search methodology on March 12, 2024.
    • User-facing: As a mobile user accessing the desktop Vector 2022 version of Wikipedia (via "desktop site" link in footer ), I want pages to load and become interactive more quickly after I perform an action, so I can enjoy a smoother and more responsive experience similar to desktop users.


In this ticket, we investigated reports that our Interaction to Next Paint score was rated as "Poor", a spike we prioritized because Google will be changing their methodology to preference the INP score for SEO purposes.

From this investigation, we concluded that the 8.45 million affected URLs flagged as "poor" for this metric came from MOBILE devices accessing the DESKTOP version of this site. There are repro steps in this comment on that ticket.

In addition:

  1. See attached video () for a runthrough of the repo steps as outlined in this comment on the other ticket. Note that you'll have to enable all extra preferences in the Web Vitals extension to get the console logging. you may also need to remove the ".m" portion of the URL and force reload the page.
  1. Consult at this URL to see the full report (note you may see Showing results for URL: below the search bar, in which case you need to select Run with original URL to get the DESKTOP site as run on a mobile device)
  1. You can also use this dashboard of INP scores of mobile devices on the desktop site over time.
  1. Finally, as noted in the original ticket, this link in the Google Search Console has the latest up-to-date information for mobile devices running the desktop site.

Testing Environment for QA

    • Note that the numbers reported by Core Web Vitals in the Search Console are reported by all users of the site and will not immediately reflect or be updated by localized changes on an individual machine
  • Instead, you can install the Web Vitals extension, turn on all optional logs in preferences, and verify, via steps in the above video, that interacting with the TOC or other page elements does not result in a red triangle / console logs.
  • We will need to wait a bit to verify that this has been fixed for the general population, but you should be able to do so by 1) hitting "validate fix" in the google search console , and 2) running at this URL against

Acceptance Criteria

  • We have profiled MOBILE devices accessing (NOTE: NOT using the Core Vitals extension, and reproduced the "needs improvement" / "poor" score for INP via specific user interactions.
  • We have made a change or changes that locally improves the INP metric for users on MOBILE devices accessing it should be rated within the "good" range.
  • The above change should hold for a variety of user interactions, including clicking TOC and interacting with various other on-page elements
  • Once this hits production, we should do "validate fix" in the google search console, then if this doesn't immediately yield results, make up a ticket to check back in on this in a month to ensure the Web Core Vitals statistics as reported in Google Search Console from real-world users reflect many fewer "needs improvement" and "poor" scores.
  • We should set up alerting or notifications in the Google Search Console to help us identify and catch these kinds of issues in the future.

Event Timeline

NBaca-WMF created this task.
Jdlrobson updated the task description. (Show Details)
NBaca-WMF renamed this task from Interaction to Next Paint (INP) Core Web Vital is "Poor" for Mobile users on Desktop to Interaction to Next Paint (INP) Core Web Vital is scored as "Needs Improvement" or "Poor" for Mobile users on Desktop.Feb 23 2024, 8:30 PM
ovasileva renamed this task from Interaction to Next Paint (INP) Core Web Vital is scored as "Needs Improvement" or "Poor" for Mobile users on Desktop to [3 days] Interaction to Next Paint (INP) Core Web Vital is scored as "Needs Improvement" or "Poor" for Mobile users on Desktop.Feb 26 2024, 6:42 PM
ovasileva assigned this task to bwang.
ovasileva set the point value for this task to 5.
Jdlrobson changed the subtype of this task from "Deadline" to "Task".Feb 26 2024, 6:46 PM
Jdlrobson updated Other Assignee, added: Mabualruz.

As a few concrete examples of what this might look like:

  1. There's probably something happening around this section of tableOfContents.js - scrollToActiveSection does a bunch of calculations which likely function quite differently on mobile. We could use the natively-supported scrollIntoView instead and this may just be enough to fix that particular issue, for instance.
  1. There's also const scrollBehavior = prefersReducedMotion() ? 'smooth' : undefined; which looks like it's the reverse of what we should be doing?

We should look wherever possible to target the mobile userAgent, even if it impacts desktop also, to avoid regressions until we can verify this all works correctly.

And we should bring in Performance / @Peter once he's back next week to help us verify and ensure we're on top of this kind of thing in the future.

Hi! I have a question:

Are we sure this is really an issue for users (any users) and not only Google crawling Wikipedia and reporting it in the search console? I'm thinking maybe the fix is to ask them to not crawl desktop as a mobile? Do you know about any reasons why you as a user should use desktop on mobile? We collect data from the Chrome User Experience report (collecting data from their API and storing it in our data storage) and we graph it in There you can choose domain and form factor (Googles way of saying if the user is on mobile/tablet or desktop). There we can see how the data that Google collects from Chrome users. When I check for phone and tablet, there's no data. That means the Google API do not have any data for those combinations and I'm thinking this affecting very few users. But maybe we can see that in our own statistics?

I think the Interaction To Next Paint is a little bit tricky since it depends so much on our JavaScript and the speed of the users device. With the Chrome User Experience report and the search console we don't see what kind of interaction the user is trying to do. To give us better understanding of how it's impacting our users and what they are doing we need to start collecting it from our real users (I started a task about it in T327246). I'm thinking you as the web team is most skilled team to take that on? Collecting it means work since it's different than the rest of the performance metrics. We beacon back those after the load event end but the INP should be sent back just when the user exits the page and also want to have data of what kind of interaction the user was doing so the can do buckets on what they actually do. That way we could potentially try to reproduce what users get and fix problems.

I'm not against making the desktop menu faster (that's cool) just thinking we may have bigger issues but I haven't digged into the data we have from the Chrome User Experience report in a while (let me know if you need a demo on how to use the dashboard!) and we do not collect the metrics ourselves.

For actually testing INP, I made a script that I used on Samsung A51, clicked on the desktop version and then clicked on the menus and collected some data. It worked but I think I need to tune the INP collection because I didn't get the exact same values as the Chrome extension (and I think its me that has not implemented the latest changes on how to collect INP correct, I'll fix that this week). For setting up repeated tests on mobile phones, we need the mobile device lab and best case that will be up and running maybe June (depending on when the contract is done). We can do it on emulated mobile until then but I also think it would be great if we could collect it from real users too!

Hi again, I couldn't fully understand the numbers, so I asked about it on the Web Performance slack channel today and got answer from Barry Pollard who is a devrel at Google. If you are on the channel you can read the conversation here:

According to the data they collect, almost 2% of the users on the desktop English Wikipedia is users on mobile:

image (1).png (1×1 px, 178 KB)

Looking at the search console, those 2% affect 85% of our URLs making them "Need improvement" for First Input Delay (FID) and later on Interaction to next paint (the high FID will be included in the INP).

I'm surprised that 2% of the devices on mobile uses the desktop site (isn't high?), I wonder if we can see the same in our own metrics?

I wanted to be able to reproduce it on a phone (so we easily can verify that a fix work) but I have hard time get the same values as in the console (tried on Samsung A51 and a Moto G5). The way I've done it is going to, click on the desktop link and then clicking on "Features" and the "Localisation by country". For my slowest phone (Moto G5) I get a FID of 8 ms and INP of 150-160 ms. @bwang when you reproduce it on your machine, is there anything special I should do? I'll go over how I measure INP tomorrow (it has changed a little bit the last releases maybe I haven't followed along) but FID should just work (that code is simple). In the Google console the FID is 200+. The problem is that on a old phone like Moto G5 I spend almost 6 seconds in long tasks (parsing HTML/JS/CSS and render) so if the user do something during that time, the FID will be higher. In my test I wait for the page to finish loading.

@Peter I'm not doing anything special. In fact I'm usually able to reproduce with any pointer interaction, i.e. opening the hamburger dropdown menu in the top left corner of the page. Heres an example trace from me clicking that dropdown menu

Screenshot 2024-03-05 at 12.05.31 PM.png (352×960 px, 72 KB)

I missed update the issue: So I can reproduce on my Moto G5 when connecting devtools to Chrome and manually clicking. For automation, I don't get the same Input delays on the interactions as if I do it manually (I added T359506 for that). Right now we are stuck in understanding the Input delay we get on the interactions.

I got some feedback from Gilberto Cocchi that the delay come from that we set the width=1000 (meaning it waits for a double tap see I tried with touch-action: manipulation; and that fixed it for me. I wonder if that has any other impact?

Change 1009391 had a related patch set uploaded (by Mabualruz; author: Mabualruz):

[mediawiki/skins/Vector@master] Interaction to Next Paint (INP) Core Web Vital Improvement

Change 1009391 merged by jenkins-bot:

[mediawiki/skins/Vector@master] Interaction to Next Paint (INP) Core Web Vital Improvement

I was asked for input by @Mabualruz.

See also T118509: Evaluate using `touch-action: manipulation;`, suggested as Wikimedia-Performance-recommendation for Vector and Minerva in 2015.

I tried with touch-action: manipulation; and that fixed it for me. I wonder if that has any other impact?

The default delay that touch screens use exists to understand user intent, such as double-tap to zoom. It's quite common on mobile web pages to touble tap on an area, table, infobox, sidebar, thumbnail, etc. to zoom in or out to the width of the tapped area. For example:

  • Double-tab on the content to zoom in to the width of the content area.
  • Double tab on a thumbnail, infobox, or sidebar, to zoom in making the width of the thumbnail the width of the screen (without opening a lightbox that would need closing again etc).
  • After pinching to zoom in to a few words on a page, you can double-tap on the paragraph to zoom back out to the full width of the content area.

I believe it might also affect certain touch-based hooks in JavaScript (perhaps for things like JS-based maps, games etc) as disabling the delay means click is immediately produced, which might interefere with the ability to override those events to prevent the default. As long as this can be opted out from by a widget, that should be okay (e.g. Kartographer maps, or certain gadgets) confirms the first part at least:

  • manipulation: Enable panning and pinch zoom gestures, but disable additional non-standard gestures such as double-tap to zoom. Disabling double-tap to zoom removes the need for browsers to delay the generation of click events when the user taps the screen. This is an alias for pan-x pan-y pinch-zoom.

In other words, panning left-right and up-down for scrolling, and pincing to zoom in/out will still work, but double tap couldn't work since the first tap is immediately consumed to produce a click event.

Coming back to the "opt out" this is not trivial. CSS pointer-events, is inherited and thus effectively works "inside out" (that is, the closest element to where you interact decides how it works). But CSS touch-action is not inherited.

From the W3C Pointer Events spec:

A direct manipulation […] is supported if it conforms to the touch-action property of each element between the hit tested element and its nearest […] scroll container […]

In other words, if even one element between the target element and the body (or a custom scroll container) disables the gesture, the gesture will not ocurr. That sounds complicated, but is more an internal implementation detail. The simple and more practical way to look at is: From the top element moving inwards, when you pass through an element that disables the gesture, the gesture is cancelled.

This makes it quite hard to opt in after setting something like touch-action: manipuation on the html selector. But, hard is not impossible. Notice the spec doesn't say "body" but "scroll container". So if you have an element with position: relative; overflow: scroll; then you can re-allow all touch gestures within there. That's not always equally practical, but not impossible either. Perhaps that's

Example: I've shaded the area with this CSS rule applied in pink. Those areas don't allow zoom with double-tap or pinch. I've then re-enabled this in the green/second figure which is nested within the pink area, where native pinch works again.

Change 1010215 had a related patch set uploaded (by Jdlrobson; author: Mabualruz):

[mediawiki/skins/Vector@wmf/1.42.0-wmf.21] Interaction to Next Paint (INP) Core Web Vital Improvement

Change 1010215 merged by jenkins-bot:

[mediawiki/skins/Vector@wmf/1.42.0-wmf.21] Interaction to Next Paint (INP) Core Web Vital Improvement

Mentioned in SAL (#wikimedia-operations) [2024-03-11T20:20:01Z] <urbanecm@deploy2002> Started scap: Backport for [[gerrit:1010215|Interaction to Next Paint (INP) Core Web Vital Improvement (T358380)]]

Mentioned in SAL (#wikimedia-operations) [2024-03-11T20:22:13Z] <urbanecm@deploy2002> urbanecm and jdlrobson: Backport for [[gerrit:1010215|Interaction to Next Paint (INP) Core Web Vital Improvement (T358380)]] synced to the testservers (

Mentioned in SAL (#wikimedia-operations) [2024-03-11T20:33:59Z] <urbanecm@deploy2002> Finished scap: Backport for [[gerrit:1010215|Interaction to Next Paint (INP) Core Web Vital Improvement (T358380)]] (duration: 13m 57s)

Thanks everyone for weighing in here and helping with this, in particular @Peter
for your assistance debugging and @Krinkle for your context.

I am no longer able to reproduce this locally in my browser with the Core Vitals extension, so I'm going to set this as resolved for now.

Formally speaking, I've started validation within the Google Search Console, it claims it will be completed in a few days.

I will create new tickets if we find further issues.

Adding this for future reference, here we can see how the Chrome data has changed since this rolled out.

Screenshot 2024-04-10 at 08.25.00.png (1×2 px, 292 KB)

Screenshot 2024-04-10 at 08.25.17.png (1×2 px, 287 KB)

Jdlrobson subscribed.

Logging in to the console on March 20th I was seeing "needs improvement" but there are now zero marked as "poor"

Screenshot 2024-03-20 at 4.57.29 PM.png (1×1 px, 136 KB)

For this it says validation was started: 3/12/24 and will complete within 28 days so we should circle back to this ticket on 10th April.

Now on April 16th:
0 poor URLs and 0 URLS need improvement (hurrah!)

Screenshot 2024-04-16 at 8.26.21 AM.png (1×1 px, 162 KB)