Page MenuHomePhabricator

Remove unused keys in Graphite for WebPageReplay/WebPageTest
Closed, ResolvedPublic

Description

We been changing URLs and namespaces a couple of times and we should clean up the metrics in Graphite (see for example https://phabricator.wikimedia.org/T205870#4808181).

Event Timeline

Restricted Application added a subscriber: Gilles. · View Herald Transcript

I've remove old WebPageTest run wikis that was only made for one off test. One thing with the WebPageTest metrics that it seems that we some time had problem and created keys with only lower case? For example we have one part that is Facebook but we also have facebook. I haven't cleaned those yet but will.

For WebPageReplay I removed all testing keys under browsertime.* and specific old pages under enwiki, so we can use the dropdowns and automatically pick the right page.

There are more cleaning that can be done but then I need to go into each level of keys.

Peter triaged this task as Medium priority.Dec 21 2018, 6:20 AM

We currently have these for WebPageTest in Graphite as second key:

enwiki			      
enwiki-mobile-beta-2gslow
enwiki-bc-mobile-2gslow       
enwiki-mobile-beta-2gslowest
enwiki-bc-mobile-beta-2gslow  
enwiki-mobile-beta-cookie-2gslow
enwiki-beta		     
portals
enwiki-beta-mobile	     
portals-beta
enwiki-mobile		      
wikidatawiki
enwiki-mobile-2gslow	      
wikidatawiki-beta
enwiki-mobile-2gslowest

But the ones generated by WebPageTest is:

portals
enwiki
portals
portals-beta
wikidatawiki-beta
wikidatawiki
enwiki-mobile
enwiki-beta-mobile

so I'll remove the old ones.

For browsertime we had one on the second key that was wrong *mobile* removed that too.

For WebPageTest we have pages with lower cases (that was touched the 20 of October), I'm removing those too.

barack_obama
Barack_Obama
BlankPage
Elizabeth-second
facebook
Facebook
Facebook-second
Metalloid
sweden
Sweden

And all URLs have been from these locations that we should not use:

  • eu-central-1
  • us-east-1-hosted-linux
  • us-east-1-linux

I removed them too from mobile/desktop enwiki.

I'll try to simplify the graphs now and if I found more I'll clean it up.

Change 483116 had a related patch set uploaded (by Phedenskog; owner: Phedenskog):
[performance/WebPageTest@master] Remove unused tests.

https://gerrit.wikimedia.org/r/483116

Change 483116 merged by jenkins-bot:
[performance/WebPageTest@master] Remove unused tests.

https://gerrit.wikimedia.org/r/483116

I've updated https://grafana.wikimedia.org/d/000000095/webpagetest-drilldown and it looks good for enwiki and enwiki-mobile. Let me have a look for the rest and when they are cleaned up we can close this!

Change 483363 had a related patch set uploaded (by Phedenskog; owner: Phedenskog):
[performance/WebPageTest@master] Enable tests on beta portal

https://gerrit.wikimedia.org/r/483363

Change 483363 merged by jenkins-bot:
[performance/WebPageTest@master] Enable tests on beta portal

https://gerrit.wikimedia.org/r/483363

I've cleaned up all old WebPageTest data (WebPageRelay most where ok). Two things: we have tested from different locations (most when we tried out the new Linux versions) and there must have been a glitch somewhere where metrics keys where sent in lower case only, created double metrics. All these are fixed now.