Page MenuHomePhabricator

Migrate 'Readers' Key Product Metrics notebooks to analytics/wmf-product/jobs
Closed, ResolvedPublic

Description

Migrate specifically ETL notebooks from the following repos:

to analytics/wmf-product/jobs using the instructions found here: https://gerrit.wikimedia.org/r/plugins/gitiles/analytics/wmf-product/jobs/+/refs/heads/master/movement_metrics/

(Visualization notebooks should be separate from ETL notebooks.)

Event Timeline

ldelench_wmf moved this task from Triage to Upcoming Quarter on the Product-Analytics board.

I removed https://github.com/wikimedia-research/Platform-metrics since there is no ETL (the queried data is saved locally)

As an MVP we're planning to migrate Readers ETLs for October 2021 metrics. Schedule will be contingent on data being available in mediawiki_history and AQS API.

Change 736583 had a related patch set uploaded (by Bearloga; author: Bearloga):

[analytics/wmf-product/jobs@master] movement_metrics: Migrate pageviews_corrected

https://gerrit.wikimedia.org/r/736583

Change 736583 merged by Bearloga:

[analytics/wmf-product/jobs@master] movement_metrics: Migrate pageviews tables and ETL

https://gerrit.wikimedia.org/r/736583

Mayakp.wiki renamed this task from Migrate Key Product Metrics notebooks to analytics/wmf-product/jobs to Migrate 'Readers' Key Product Metrics notebooks to analytics/wmf-product/jobs.Nov 8 2021, 9:58 PM
Mayakp.wiki updated the task description. (Show Details)
Mayakp.wiki moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.

Job ran as scheduled and without problems

hive -e "SELECT COUNT(DISTINCT day) FROM wmf_product.pageviews_corrected WHERE year = 2021 AND month = 10"
31 ✅
hive -e "SELECT COUNT(DISTINCT day) FROM wmf_product.global_markets_pageviews WHERE year = 2021 AND month = 10"
31 ✅

\o/
Subsequently ran calculation, report and viz notebooks with no issues!

Closing this task. Next Up: Editing metrics ETL T295332 which we look into next quarter