In the current setup with WebPageTest we continuously test a couple of pages on different hosts. But we don't test commits. The long term goal should be to continuously and automatically do performance tests on new commits.
A first step could be to do one of the things that [Facebook do](https://developers.facebook.com/videos/f8-2016/mobile-performance-tools-at-facebook/): Make it possible for a developer to easily test her/his changes and get back median metrics with a confident value.
To make that happen there's a lot we need to do:
- We should have the test infrastructure in house and not on Amazon to make sure we have full control, to be able to minimize variance.
- We should prepare for testing on real mobile devices.
- Dedicated hardware to run the tests (we should have both desktop and mobile testing)
- Run the tests on commits
- GUI driven, whatever tools we use, it must be super easy to start a test and easy to understand the result = should the code move on or not.