Page MenuHomePhabricator

Consolidate labs / production sqoop lists to a single list
Closed, ResolvedPublic

Description

As @JAllemandou commented on https://gerrit.wikimedia.org/r/c/analytics/refinery/+/680414, it appears we duplicate the sqoop list in labs_grouped_wikis.csv and prod_grouped_wikis.csv. We should confirm that they are in fact the same, and update puppet to only use a single csv grouped_wikis.csv.

Event Timeline

Ok, confirmed that the two csvs are currently the same, other than their header:

~/w/refinery 11:08 (master) $ diff static_data/mediawiki/grouped_wikis/{labs_grouped_wikis,prod_grouped_wikis}.csv
1c1
< # Labs wiki,group,edit size in 2016
---
> # Production wiki,group,edit size in 2016

Time to deduplicate!

Change 681496 had a related patch set uploaded (by Razzi; author: Razzi):

[analytics/refinery@master] Combine labs_grouped_wikis and prod_grouped_wikis to grouped_wikis

https://gerrit.wikimedia.org/r/681496

Change 681498 had a related patch set uploaded (by Razzi; author: Razzi):

[operations/puppet@production] sqoop: switch to single grouped_wikis.csv

https://gerrit.wikimedia.org/r/681498

fdans triaged this task as High priority.Apr 26 2021, 4:03 PM
fdans moved this task from Incoming to Operational Excellence on the Analytics board.

Change 681496 merged by Razzi:

[analytics/refinery@master] Combine labs_grouped_wikis and prod_grouped_wikis to grouped_wikis

https://gerrit.wikimedia.org/r/681496

Change 681498 merged by Razzi:

[operations/puppet@production] sqoop: switch to single grouped_wikis.csv

https://gerrit.wikimedia.org/r/681498