In order to filter logged-in vs logged-out access to APIs, the webrequests_sampled_128 cube (or equivalent) should have a dimension that indicates the authentication status (and ideally, authentication method) of each request.
Implementation ideas:
- Dan said: the pipeline is varnish -> kafka -> hdfs -> druid -> turnilo where Turnilo is just a way to query the data but you can get to it straight from kafka, hdfs, or druid without Turnilo
- Valentin pointed out: The X-Analytics header already has a loggedIn flag. This could be updated to indicate the auth method (cookie, auth header, or centrlauth token, etc).
- Mikhail suggested: This flag could be processed by oozie to make this information available in Druid.
- Gergö added: Note that at least for the action API, we have three auth methods (cookies, OAuth and CentralAuth tokens). The mediawiki_api_request table contains the raw data (headers + API parameters) but you'd have to post-process it a bit.