Maniphest T302079

Improvements to mediawiki_geoeditors_monthly dimensions
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	mpopov
	Feb 18 2022, 4:15 PM

Description

Background

The mediawiki_geoeditors_monthly data cube in Turnilo/Superset currently has the following dimensions:

Activity Level
Country Code (2-letter ISO)
Users Are Anonymous
Wiki Db

Request

Over the past year I've have observed that a lot of staff are immensely interested in this dataset but this limited set of dimensions makes it rather difficult for them to use the dataset to its fullest potential and explore it with ease.

Additions

I'd like to request the following (derivable) dimensions to be added:

Country Name (canonical_data.name in Data Lake)
Continent (canonical_data.maxmind_continent in Data Lake)
Economic Region (canonical_data.economic_region in Data Lake)
Wiki Db Group (canonical_data.database_group in Data Lake)
Wiki Language Name (canonical_data.language_name in Data Lake)
Wiki Language Code (canonical_data.language_code in Data Lake)
Wiki English Name (canonical_data.english_name in Data Lake)

Modifications

Furthermore, "Users Are Anonymous" dimension is currently "0" and "1" which makes for awkward visualization:

Turnilo	Superset

I'd like to propose changing the values/labels to "Anonymous" and "Logged-in", respectively.

Event Timeline

mpopov created this task.Feb 18 2022, 4:15 PM

mpopov updated the task description. (Show Details)Feb 18 2022, 4:20 PM

ldelench_wmf moved this task from Triage to Tracking on the Product-Analytics board.Feb 22 2022, 6:17 PM

• EChetty moved this task from Incoming (new tickets) to Datasets on the Data-Engineering board.Feb 24 2022, 5:28 PM

• EChetty edited projects, added Data-Engineering-Kanban; removed Data-Engineering.

• EChetty added a project: Data-Engineering.

I wonder whether the requested change should be done for the data in Druid only, or if it would be valuable to change the geoeditors tables on the cluster (see https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Geoeditors).
Depending on the above, we should plan on doing this when migrating the geoeditors jobs to aiflow (some of it already done), or when migrating the druid loading jobs to airflow.

Hi @JAllemandou ! I'm moving this task back to DE Workboard since we depend on Airflow migration (either druid or geoeditors). Once that has taken place, we'll be able to retake this one in Kanban.

JArguello-WMF removed a project: Data-Engineering-Kanban.Jun 23 2022, 9:02 PM

I wonder whether the requested change should be done for the data in Druid only, or if it would be valuable to change the geoeditors tables on the cluster.

If those are separate (Druid version not just an ingestion of geoeditors in Hive) I'd say Druid is higher priority. But yes, the changes would be valuable across the board.

JArguello-WMF moved this task from Datasets to Data Products & Metrics on the Data-Engineering board.Jun 29 2023, 11:19 PM

lbowmaker moved this task from Data Products & Metrics to Icebox (not considered in current quarter) on the Data-Engineering board.Nov 10 2023, 2:40 PM

SNowick_WMF subscribed.Fri, May 3, 4:36 PM

	F34956984: Screen Shot 2022-02-18 at 11.05.26 AM.png
	Feb 18 2022, 4:15 PM

	F34956991: Screen Shot 2022-02-18 at 11.11.33 AM.png
	Feb 18 2022, 4:15 PM

Improvements to mediawiki_geoeditors_monthly dimensionsOpen, Needs TriagePublicActions