Page MenuHomePhabricator

Create a script to extract request logs for query.wikidata.org for dashboards
Closed, ResolvedPublic4 Story Points

Description

Create a script that would produce raw data on usage of query.wikidata.org and query.wikidata.org/bigdata/namespace/wdq/sparql (SPARQL endpoint) for further inclusion on the dashboards and KPIs

Event Timeline

Smalyshev assigned this task to Ironholds.
Smalyshev raised the priority of this task from to Normal.
Smalyshev updated the task description. (Show Details)
Smalyshev added a subscriber: Smalyshev.
Restricted Application added projects: Wikidata, Discovery. · View Herald TranscriptAug 17 2015, 7:34 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Ironholds removed Ironholds as the assignee of this task.Aug 17 2015, 7:36 PM
Ironholds moved this task from Needs triage to Analysis on the Discovery board.
Ironholds set Security to None.
Ironholds edited a custom field.Aug 19 2015, 8:30 PM
Ironholds assigned this task to mpopov.Aug 19 2015, 8:34 PM

Need to transfer the logic from HiveQL query to UDF and then to run the script on previous days to fill in the backlog.

mpopov added a comment.Sep 1 2015, 6:23 PM

Script: https://gerrit.wikimedia.org/r/#/c/235137/1/data_retrieval/wdqs.R

Oliver added it to the scheduler and I ran it on the past 40 days to backfill the aggregate dataset that will be up-to-date going forward.

Smalyshev closed this task as Resolved.Sep 9 2015, 7:27 AM

Assuming this is resolved.