During this month's run, some tables were not sqooped properly from mediawiki dbs into hdfs. The code that was responsible for listing which tables these were failed to run due to a simple bug and therefore we have to figure out what ran based on absence of _SUCCESS flags in the 2017-07 snapshot. This task has two parts:
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Fix failure in failed_jobs handling | analytics/refinery | master | +5 -2 |
Change 369988 had a related patch set uploaded (by Milimetric; owner: Milimetric):
[analytics/refinery@master] Fix failure in failed_jobs handling
Change 369988 merged by Ottomata:
[analytics/refinery@master] Fix failure in failed_jobs handling
The jobs that didn't run were identified in T165233#3498662 and the error seems to have to do with database access, so for now I'm moving this to done and deploying the new version of the sqoop script. Since I don't think it makes sense to wait and sqoop the remaining wikis, I'm going to manually add the success flags in order to trigger the snapshot for this month.