Yesterday I pushed Chrome 69 to my test server and it seems like for the Wikipedia pages I test the first visual change moved something like 100 ms. I'm gonna rollback and check if the timings go back and then roll forward again and collect more data.
We can see that on some wikis (Chinese, Japanese, Russian) first paint is 20-30% slower when upgrading to Chrome 69 (from 68).
Upstream: https://bugs.chromium.org/p/chromium/issues/detail?id=887205
I'm not 100%, it seems like some of the URLs change, I think the only way to be sure is to update on our WebPageReplay/Browsertime machines (the test server I'm using isn't super stable).
For Obama it's hard to be sure:
But for the other Sweden and Facebook it looks like first visual change is affected:
Let me push the change so we can see on more URLs what happens.
Pushed 69 and Firefox 62 now. WebPageTest was upgraded late 4/9, I've added a annotation to make it easier to see.
I think I was wrong. The metrics are more unstable now for Chrome but it has been that way for a while, and I think it's time to deploy on a new server.
Re-open, I've seen on other wikis that something really changed in 69. I'll add metrics later today.
Looking at other wikis it seems pretty clear that 69 introduced something. Only change in the Docker container at that moment was upgrading to 69 (and FF to 62). The blue annotation line in the graph is when we updated to 69:
I've made more tests now and made sure the only change is 69. It seems that 69 spends more time in creating the layout:
Layout : 1214.3 vs Layout : 1836.0
I collected trace logs and will create a upstream issue in Chrome first thing tomorrow.
I've looked at the RUM metrics and see the same thing there:
They seem to match pretty good.