Move WebPageTest instance to Linux
We should test the Linux versions of WebPageTest now when they are available on AWS.
- May 17 2017 - We created this task with the intention to move to Linux
- May 18 2017 - Initial tests with Chrome and Firefox (Firefox "should" work). Issues created upstreams with bugs
- Aug 25, 2017 - Initial *stable* release for WebPageTest 17.08 release with full support for Chrome and Firefox.
- Sep 4 2017 - Reported a batch of upstream issues that make Firefox unusable
- Sep 18 2017 - Tests takes very long time to finish on Linux, but we don't know why :(
- Jan 17 2018 - WebPageTest.org moves to Linux
- Feb 1 2018 - Finally a break through with the slow test on Linux. The auto scaling functionality isn't working on Linux. It sometimes kills an agent before it has finished it's work and do not start a new agent immediately.
- Feb 4 2018 - Firefox TTFB is unrealistic high sometimes making Firefox unusable.
- Feb 10 2018 - There are a lot more variance in metrics on Linux than on Windows.
- Feb 26 2018 - Chrome sometimes gets a too early first visual change for authenticated users or second view
- Mar 2 2018 - New way of setting connectivity on Linux, making the variance in metrics better.
- Setup a Linux Instance
- Let it run for a couple of days and verify that the metrics is ok - they aren't perfect but at least ok to move one
- Decide a date when we will remove the Windows version (!)
- Update docs or add a task describe what needs to be changed and what should be expected when updating (speed changes, name changes etc)
- Inform reading, portal and wikidata that we will do the change (dashboards needs to be changed)
- Update our dashboards.
- Kill the windows instance, remove the Jenkins job
- Update https://wikitech.wikimedia.org/wiki/WebPageTest
- Cleanup Graphite and remove old keys (but keep the Windows ones until we get the yearly stats).