Page MenuHomePhabricator

Create a Superset "dataset" and basic chart exposing the scraper per-wiki results
Open, Needs TriagePublic

Description

Context

We eventually want to have a Dashboard in Superset to look at the Scraper data. A first step into that direction would be having a data set and chart configured that can be used inspect the raw data coming from the aggregations.

Task

This task is about creating a dataset which can be used in charts.

  • Create a new dataset based on the aggregation table wmde.wiki_page_cite_references_monthly ( Until this table is created you can also use wmde_fisch.wiki_page_cite_references_monthlyfrom T413954 )
  • Configure column types appropriately so that the dataset can be used for a linechart.
  • Build a test chart which includes a per-wiki filter (defaulting to dewiki perhaps) and shows a time series.
Outcome

I created a dataset and test chart using my local db as described above. It seems pretty straight forward. The snapshot date can be used as datetime for the chart and the lines can be configured on the chart using sum().

https://superset.wikimedia.org/explore/?slice_id=4487

Event Timeline

WMDE-Fisch renamed this task from Create a Superset "dataset" exposing the scraper per-wiki results to Create a Superset "dataset" and basic chart exposing the scraper per-wiki results.Wed, Jan 21, 9:14 AM
WMDE-Fisch updated the task description. (Show Details)
WMDE-Fisch updated the task description. (Show Details)
WMDE-Fisch subscribed.

Feel free to sanity review this also for knowledge sharing.

It's a little bit hard to see what's happening here since it targets a private table where the data doesn't change per month, but the basic idea is right! We'll iterate as actual data is available.