Hey @Addshore wanna check how we are doing with the WebPageTest wikidata tests? I'm doing a major cleanup with the goal to run the tests every hour (today once every 3 hours), only test production and adding Grafana alerts to catch regressions.

Today we test three pages for Wikidata for three browsers (Chrome, Firefox and IE 11):
https://www.wikidata.org/wiki/Wikidata:Main_Page
https://www.wikidata.org/wiki/Q64
https://wikidata.beta.wmflabs.org/wiki/Q15905

Do you use the results, are all relevant to you?

Hey @Jdlrobson I wanna redo the whole setup for running WebPageTest runs for mobile, I think we test to much on different connectivities today that is just confusing. Instead I wanna focus on just three pages (we'll do the same on Desktop) and then activate alerts in Grafana (when we have 4.0 installed). I'll update the graphs on https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest when do the do the push, but I wanna make sure you are with me before I do the change :)

Change 325779 had a related patch set uploaded (by Phedenskog):
Remove and cleanup URLs to make it easy to run tests every hour

https://gerrit.wikimedia.org/r/325779

Can you summarise the commit?
Since we've done the lazy loading images experiment it should be enough just to test the same page (preferably Barack Obama size) on desktop,mobile and beta cluster. In addition to this blank page is useful.

The important thing i use it for now is to compare beta cluster with production and production mobile with desktop.

@Peter Yes all 3 results are used and relevant to us! :)

The important thing i use it for now is to compare beta cluster with production and production mobile with desktop.

@Jdlrobson yes but do we need to do that continuously or can we make it easier to just fire away tests? or how do you use it now?

Right now I must confess we're not looking too closely at it, but periodically I'm asked to report how mobile performance compares to desktop (for example when we were working on lazy loading images). I think making easier to fire off comparison tests for that purpose would be great - some kind of compare pages script that given multiple URLs reports on differences between them e.g. bytes changed/first paint.

@Jdlrobson cool! I've updated the commit with the exact URLs that are removed for mobile. The change will make desktop/mobile focus on the same thing: three pages (two large, one small) that we will run every hour + login and "second" view for one page as before. Doing thing I hope we can automate the alerts so we can make sure we don't miss a regression.

I'll create a new issue tomorrow for the compare pages, maybe we can use the compare functionality in WPT, just make it easier/more understandable.

Change 325779 merged by jenkins-bot:
Remove and cleanup URLs to make it easy to run tests every hour

https://gerrit.wikimedia.org/r/325779

I've changed manually in Jenkins to run every hour (the job takes 45 min now). Lets keep it so for a while to verify that everything works ok and then change in git.

Change 328159 had a related patch set uploaded (by Phedenskog):
Run WebPageTest jobs every hour

https://gerrit.wikimedia.org/r/328159

Lets close this one when we have changed the runs in Jenkins.

Change 328159 merged by jenkins-bot:
Run WebPageTest jobs every hour

https://gerrit.wikimedia.org/r/328159