I've been testing out annotations for sitespeed and I think it is something we also could use to make it easier to go from Grafana to specific test run http://wpt.wmftest.org/result/160825_9N_97/
To do that, we should send one annotation event per URL we test and add then we can use the template variables we have in https://grafana.wikimedia.org/dashboard/db/webpagetest-drilldown and add them as tags in the event, then we can show annotations per URL depending on the template variables, so it will be super easy to find each run. I've tested it, it works.
If we then have a regression in our graph, we can turn on annotations, find the test where the regression happened and just click on the link in the annotation message (that we generated) and go directly to that test run in WebPageTest.
Two questions:
- Can we send annotation events directly from our jobs in Jenkins to Graphite?
- Would be a problem to send 33 annotations per hour?
