I've briefly checked the TTFB and that seems to be stable so this looks like a legit alert, we should look into it:
I couldn't see anything with WebPageReplay or first paint in the navigation timing when I briefly looked, so need to dig deeper.
Description
Description
Event Timeline
Comment Actions
I can't see anything obvious comparing a run before and after. It look like network-wise everything is a little slower.
Comment Actions
No nothing of first visual change, however if look at fully loaded it started to be slower at the same time as first visual change on desktop:
No I cannot see anything either. I looked at commit log for WPT and no changes. I also run tests directly on sitespeed.io without WebPageReplay (on another AWS server) and looking at the graph for Obama it looks like it could be an increase(in first visual change) but those tests runs only every four hours so need more data:
Looking at the Facebook there's no diff:
What's interesting that for WebPageTest we can see the same for both Firefox and Chrome for all three URLs we test:
This smells that it something going on on that server. We also do some test on the Russian Wikipedia and its the same there:
I wonder if we can see something on the emulated mobile tests on the same server?
No nothing of first visual change, however if look at fully loaded it started to be slower at the same time as first visual change on desktop:
It looks like its just slower on that WebPageTest agent, let me try to just restart it and see if something happens.
Comment Actions
I rebooted the agent through the AWS GUI and added a annotation in Grafana to be able track when it happened.
Comment Actions
Looked at the total content size and pages had an increase around that time see:
Cannot see it in the other tools though.
Comparing the HARs it looks like this:
So it seems the change is in the images:
Comment Actions
All images got bigger at once? Can you check the headers to see if the ones that changed were WebP before and aren't anymore?
Comment Actions
Not all but some. Yes I checked for https://upload.wikimedia.org/wikipedia/en/thumb/f/f8/Thefacebook.png/220px-Thefacebook.png and that one went from webp to png.
Attaching the HAR files.
Comment Actions
I looked at a few that got bigger and they were all webp before, png or jpeg after. I think that explains the size difference for all those images.
The cache host being hit is different, it used to be cp1082 where those images were hot enough to be webp, now it's cp1078 where they aren't (yet?).
I think it's simply because WebPageTest is now having its traffic routed to cp1078 where a lot of thumbnails aren't hot enough to be turned into webp. That might change if the images get hot enough on that host. If they don't, that's the new normal for WPT.
Comment Actions
Yep, I can't understand why its changed first visual change? The last two runs the first change is back to normal for all URLs.