@Pchelolo is correct, idea is to have both restbase and mw-action-api hourly varnish-requests counts in graphite.
@Krinkle and @Addshore: After looking once more in graphite, would mw.api.varnish_requests be ok for you guys? This would allow us to keep the restbase metric and have the same naming scheme for both.
If you prefer, we can also rename both, but we'll need to convince @Pchelolo .
@ggellerman : I think we can close this task, just want to confirm with you.
Some code exists for converting XML dump in hadoop: https://gerrit.wikimedia.org/r/#/c/361440/
It is not yet merged, but could be soon.
@Krinkle: I'm happy to use a less functional approach, and have for instance analytics.varnish_requests.restbase and analytics.varnish_requests.mw_api.
Only concern is that it changes the existing metric for restbase. @mobrovac and @Pchelolo, is that a big deal?
Tue, Nov 21
Ping @mobrovac, @Pchelolo, @Gilles , @Anomie, @Krinkle for naming confirmation:
The existing metric is named restbase.requests.varnish_requests, and we plan to go with MediaWiki.api.varnish_request for the new one, reusing the existing graphite domain MediaWiki.api.
Check in which webrequext_source partition data lives (and in the mean time check for api/rest too):
Mon, Nov 20
Mon, Nov 13
Fri, Nov 3
Tue, Oct 31
Mon, Oct 30
Not a bug for me:
SELECT access_method, count(1) from wmf.webrequest WHERE is_pageview AND pageview_info['project'] = 'bn.wikipedia' AND year = 2017 AND month = 9 AND day = 30 AND webrequest_source = 'text' AND x_analytics_map['ns'] = '-1' AND x_analytics_map['special'] = 'AboutTopic' group by access_method;
access_method _c1 desktop 13 mobile web 448
which is coherent with the request @Addshore did that doesn't include bn.m.wikipedia.org.
I let you either close or ping us back :)
Thu, Oct 26
Oct 23 2017
Oct 20 2017
Oct 19 2017
Oct 18 2017
Oct 17 2017
Oct 16 2017
Declinig since we're moving away from pivot because of it being non open-source.
We'll be using supset instead, which will allow to une that at dashboard level.
Oct 13 2017
Oct 9 2017
The change above doesn't change the behavior of cookies, but at least removes wikimedia project-family from the ones available in Druid. The only place it'll still be visible is in hive (it was already removed from to-be-externaly-published datasets).
Interestingly, checking this patch allowed me to notice we don't currently use the new version of druid loading for banners ... Mwarf
Base script merged - Now it needs a puppet companion to lauh the script :)
Oct 6 2017
Sep 28 2017
URIs to be mocked are defined as swagger-config in Restbase pull request: https://github.com/wikimedia/restbase/pull/875
Sep 26 2017
Sep 25 2017
Sep 13 2017
Sep 12 2017
Sep 8 2017
I updated the datasources to contain interesting metrics and created a new user able to use and create visualisation, but not mess with the config: