Refactor golden to be more robust and backfilling-friendly
Closed, ResolvedPublic5 Story Points

Description

We discussed refactoring the data retrieval scripts so that we can have a single log file (easier to check for errors) and make it easier to run the individual scripts to backfill the data.

The current backfilling system involves copy-pasting individual scripts into a remote R session and then backfilling, one script at a time. We can do better.

mpopov created this task.Oct 7 2015, 5:56 PM
mpopov updated the task description. (Show Details)
mpopov raised the priority of this task from to Needs Triage.
mpopov assigned this task to Ironholds.
mpopov added a subscriber: mpopov.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 7 2015, 5:56 PM
mpopov set Security to None.
mpopov edited a custom field.
mpopov edited a custom field.
Ironholds edited a custom field.Oct 13 2015, 7:38 PM

Change 246963 had a related patch set uploaded (by OliverKeyes):
Finish the automated backfilling script - hopefully!

https://gerrit.wikimedia.org/r/246963

Change 246963 merged by Bearloga:
Finish the automated backfilling script - hopefully!

https://gerrit.wikimedia.org/r/246963

Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptNov 20 2015, 5:24 AM
Deskana closed this task as Resolved.Nov 20 2015, 5:25 AM
Deskana moved this task from Done to Resolved on the Discovery-Analysis (Current work) board.
Deskana added a subscriber: Deskana.