Moving to the Linux seems to give us more unstable values. This is what start render looks like for Windows (Sweden):
The same page on Linux:
For Windows the diff is 200 ms (13%) but for Linux 500 ms (70%!!!!).
SpeedIndex for the same page looks like this (Windows):
And Linux:
We need to look into it before we do the change. Do we need a larger instance? One way could be to pick the median run instead of the fastest. We could also try to increase the number of runs.
Right now the pick of fastest is hardcoded in the WPT wrapper but we can move that out and configure it in Jenkins for Linux.