Think the online outstanding issues is just to decide which pages :)

• Lets use east (so we have some latency from our servers)
• cable
• IE11, Firefox & Chrome (and lets add Edge when it's available in WebPageTest, and we can keep IE11 as tester for not using SPDY).
• 11 runs per test (and lets evaluate the numbers after a while)
• Today we test 2 times a day using SpeedCurve, think we can start with every two hours.
• Start testing for not logged in users and add a task to add for logged in users (we have the functionality already, think it good just to start as simple as possible
• Decide three pages that we should test + the baseline page. Lets discuss it on our next meeting.

FYI: Yesterday I started running test on our instance without sending the stats to statsv to verify everything and find a good number for number of runs. It seems to very quite much even with 11 runs lets run this for a while + we also need to discuss which pages to start with.

Lets start with the following URL:s:
https://en.wikipedia.org/wiki/Main_Page
https://en.wikipedia.org/wiki/Facebook
https://en.wikipedia.org/w/index.php?title=Special:UserLogin&type=signup
https://en.wikipedia.org/wiki/Special:BlankPage

And for mobile:
https://en.m.wikipedia.org/wiki/San_Francisco
https://en.m.wikipedia.org/wiki/Facebook
https://en.m.wikipedia.org/w/index.php?title=Special:UserLogin&type=signup
https://en.m.wikipedia.org/wiki/Special:BlankPage

Then also add one article on https://test2.wikipedia.org/ that will be locked for editing.

I actually also want to add one more thing: I want to keep track of timings for pages when the browser cache is primed, hitting one URL first and then start collect timings for the second URL.

Started testing running the URL:s every hour but that will not finish before the next run, lets try to run it every two hours, I think Etsy also used that so seems ok.

Change 242121 had a related patch set uploaded (by Phedenskog):
Added tests to run for test2 and beta

https://gerrit.wikimedia.org/r/242121

Ok: We have had an increase in TTFB for our instance and one idea from @Gilles is to test other sites to verify that the problem is on our side. That's cool, when I used keynotes services, we used to compare with branch index sites running on the same agent. To make it simple for us, I think the easiest solution would be to test the desktop site from wpt.org (or later on another aws instance). Today we only test on mobile and there's hard to see if the in TTFB because we changed from 3G to 3GFast just before the high number and it' s hard to actually see any difference. Lets discuss it on Thursday.

Change 242121 merged by jenkins-bot:
Add test runs for test2, beta and iPhone

https://gerrit.wikimedia.org/r/242121