This was visible for all WebPagerReplay tests in enwiki and LCP och Italian wiki. Since it was a change in Chrome it's hard to know what's going. You couldn't see the regression all pages but on some. I'll post some graphs for future reference. Yesterday evening I made new baseline, so that we can pickup changed on Wikipedia. I need to update the documentation on how to handle the same thing in the future and make a bug in Chrome and see if someone there can have a look.
Feed Advanced Search
Today
Today
Peter added a comment to T363126: Investigate Chrome 124 first visual change and largest contentful paint alerts.
Tue, Apr 23
Tue, Apr 23
Mon, Apr 22
Mon, Apr 22
Update container wirth Chrome 124 and Firefox 125.
Wed, Apr 10
Wed, Apr 10
Peter added a comment to T358380: [3 days] Interaction to Next Paint (INP) Core Web Vital is scored as "Needs Improvement" or "Poor" for Mobile users on Desktop.
Adding this for future reference, here we can see how the Chrome data has changed since this rolled out.
Tue, Apr 9
Tue, Apr 9
Peter added a comment to T361906: Pixel show old content for some tests.
It seems the number of banners is increasing (I got 4 now when I checked). It looks ok on https://pixel-production.wmcloud.org/reports/index.html.
Mon, Apr 8
Mon, Apr 8
I could verify it locally and created: https://groups.google.com/a/chromium.org/g/chrome-ux-report/c/j1MO_-eQQh0
Peter added a comment to T362054: Chrome User Experience dashboard for en.m misses data.
In the log I found:
Revert "Revert the Chrome trace category change."
Peter added a reverting change for rPSMT7b9388731503: Revert the Chrome trace category change.: rPSMTaf525b9c0efc: Revert "Revert the Chrome trace category change.".
Revert the Chrome trace category change.
Add Obama on beta to fix readers dashboard.
Update container that uses Chrome 123 and Firefox 124.
Fix correct file name for base configuration.
Add https://vi.m.wikipedia.org/wiki/Vincent_van_Gogh
Peter committed rPSMT5293704ff0c8: Fix names for Wikidata..
Fix names for Wikidata.
For direct test, test per URL instead of file.
Peter committed rPSMT06e59d124707: Restructure direct tests to follow same structure as replay tests..
Restructure direct tests to follow same structure as replay tests.
Remove/change configuration to run as WebPageReplay tests.
Fixing old Visual Metrics config.
Add DNS fluch and use xvfb.
Move direct test to bare metal server.
Remove codex/legacy search.
Remove client preferences cookie test.
Update container with Chrome 122 and Firefox 123.
Remove local storage tests.
Set correct latency for emulated mobile tests.
Peter committed rPSMT383e4f76267b: Config file needs to be named as dir to follow the same pattern..
Config file needs to be named as dir to follow the same pattern.
Rename webpagereplay tests, unify latency and test it.
Peter committed rPSMTed5ec294ab4a: Remove testing on beta..
Remove testing on beta.
Remove tests for tech blog.
Run 21 runs for all WebPageReplay tests.
Update to latest sitespeed.io 33.0.0.
Remove comments that creates failure on the server.
All WebPageReplay tests should have a new baseline on Sundays.
Remove beta and querybuilder tests.
Run baseline tests for emulated mobile.
Remove baselining for group 0 and 1.
Remove comments in group 0 and 1 files
Add group 0 and group 1 tests for Firefox.
Run WPR tests against Banksy.
Peter committed rPSMT9ee45a5ba430: Change to network idle on when to end the tests for first views..
Change to network idle on when to end the tests for first views.
Peter committed rPSMTb5118c06355d: Cleanup tests to make time for running more Mann Whitney U on group 0 and 1..
Cleanup tests to make time for running more Mann Whitney U on group 0 and 1.
Disable loading the Firefox extension since it's not used.
Fix to run on correct day.
Fix correct container version.
Update container with Firefox 122.
Run new baseline tests all Sundays.
Skip collecting the baseline.
Use browser in config file.
Do not set the browser for WebPageReplay tests.
Update container with more baseline information.
Stop collecting a baseline.
Fix correct naming for Firefox tests.
Release with a fix for Interaction To Next Paint.
Update container to 31.1.0.
Setup WebPageReplay tests for Firefox.
Save a new baseline for desktop tests.
Remove storing baseline tests for the new webpagereplay test.
Peter committed rPSMTa701b551e457: Do not collect baseline..
Do not collect baseline.
Fix bug for running on instant 2.
Collect a baseline for the new URLs.
Update sitespeed.io that sends "isSignicant to Graphite".
Add more enwiki URLs for WebPageReplay tests.
Bump container that has less confusing Mann Whitney tests.
Update container that uses Cliffs delta.
Update container to use Chrome 120.
Disable creating a baseline.
Make sure WebPageReplay tests one page by one.
Peter committed rPSMT876267c60f8c: Update container to also collect the CPU becnhmark for baselines..
Update container to also collect the CPU becnhmark for baselines.
Lets use today last run as the baseline.
Peter committed rPSMT74a4fa231a68: Remove dark mode tests..
Remove dark mode tests.
Remove CPU throttling for dark mode tests.
Peter committed rPSMTf11a8922434d: Update container to support user journeys tests with WebPageReplay..
Update container to support user journeys tests with WebPageReplay.
Throttle the CPU for dark mode tests.
Test dark mode on a dedicated server.
Update Chrome to version 118.
Increase number of runs for instant tests.
Update container with a new Chromedriver version.
Decrease iterations on instant tests.
Update Docker container with Firefox 118.
Use Docker container with Chrome 117.
Run login tests using 4G.
Move second view (warm cache) tests to 4g.
Peter committed rPSMTc441edc74929: Remove Alexa benchmark..
Remove Alexa benchmark.
Change login iterations to 1 instead of default.
Use 4g instead of cable for Obama warm cache test.