Wikidata Analytics Request

This task was generated using the Wikidata Analytics request form. Please use the task template linked on our project page to create issues for the team. Thank you!

Purpose

Please provide as much context as possible as well as what the produced insights or services will be used for.

This task is supporting a strategic product metric in the context of T314137: [EPIC] [MOB EDIT] Metrics for mobile editing UI improvements .

Specific Results

Please detail the specific results that the task should deliver.

We would like to continuously monitor (ideally daily; alternatively weekly) the following metrics:

• Number of edits made in Wikidata's Item namespace via the mobile editing UI
• Number of edits made in Wikidata's Item namespace via the desktop editing UI
• Number of edits made in Wikidata's Item namespace via all other means

Definitions

See the list of tags here.

• Mobile editing UI: The edit was tagged with the following tags:
  • (wikidata-ui AND termbox) [for main desktop editing UI]
  • OR (mobile edit AND NOT mobile app edit) [for special pages]
    • Andrew: Checked and it's not mobile edit, but rather mobile app edit

• Desktop editing UI: The edit was tagged with the following tags:
  • (wikidata-ui AND NOT termbox AND NOT mobile edit) [for main desktop editing UI including special pages]

Notes

• We will add additional clients to the pipeline and dashboards at a later point.
• Infos about tagging:
  • T236893: Tag all edits made via Wikibase View and Wikibase Client
  • https://www.wikidata.org/wiki/Special:Tags
  • Tags are saved in the revision history (https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/MediaWiki_history)

Desired Outputs

Please list the desired outputs of this task.

• Airflow pipeline to monitor the above daily metrics

[] Output as CSV to https://analytics.wikimedia.org/published/datasets/wmde/analytics/

Deadline

Please make the time sensitivity of this task clear with a date that it should be completed by. If there is no specific date, then the task will be triaged based on its priority.

DD.MM.YYYY

---

Information below this point is filled out by the task assignee.

Assignee Planning

Sub Tasks

A full breakdown of the steps to complete this task.

• Review work and methods from prior task (T336361: [Analytics] [MOB EDIT M1] Identify access from mobile vs. desktop devices)
  • This work is sadly not really useful for what we're doing here as the focus is derivation through user agents
  • I'll need to derive where to find these tags and then leverage them
• Plan out all methods and define if further work is needed
• Check planned methods with stakeholders
• Write Airflow job queries
• Test job queries locally
• Write Airflow DAG
  • SQL Operator for query
  • Export CSV job
  • Move CSV to published datasets job
• Have table for outputs be made in a way that the analytics user has access to it
• Test Airflow DAG without CSV export step
• Deploy Airflow DAG without CSV export
• Rework DAG jobs and DAG to include edits on mobile UI and desktop UI on mobile devices
• Before any further steps: Get approval from WMF for public data export via new Phab task
  • Task id: T373332

[] Test Airflow DAG with CSV export step
[] Deploy Airflow DAG with CSV export

Estimation

Estimate: 3 days
Actual:

Data

The tables that will be referenced in this task.

• event.mediawiki_revision_create
  • All revisions
• mediawiki_revision_tags_change
  • Those revisions that have had tags applied to them

Notes

Things that came up during the completion of this task, questions to be answered and follow up tasks.

• Note