mediawiki-history-load job got stuck this month because of missing sqooped data for tables:
python-sqoop was ready but puppet was not updated.
Running manually the jobs this month also allowed to discover a bug.
Change 562322 had a related patch set uploaded (by Joal; owner: Joal):
[operations/puppet@production] Add tables to analytics regular sqoop list
Change 562325 had a related patch set uploaded (by Joal; owner: Joal):
[analytics/refinery@master] Fix sqoop script and add CLI parameter
@JAllemandou was an e-mail sent for this failure, i do not think i received it but i might have totally spaced out.
@Nuria: No email was sent as I discovered the problem before the SLA limit.
Current limit for mediawiki-history jobs is at 39 days (31 + 8), since sqooping was taking a lot longer before.
We probably could set it to 34 to be more reactive (sqoop has regularly finished in less than 24 hours in the past months).
We probably could set it to 34 to be more reactive (sqoop has regularly finished in less than 24 hours in the past months).
Sounds good
Change 562500 had a related patch set uploaded (by Joal; owner: Joal):
[analytics/refinery@master] Reduce mediawiki-history oozie accepted SLA delay
Change 562325 merged by Joal:
[analytics/refinery@master] Fix sqoop script and add CLI parameter
Change 562500 merged by Joal:
[analytics/refinery@master] Reduce mediawiki-history oozie accepted SLA delay
Change 562322 merged by Elukey:
[operations/puppet@production] Add tables to analytics regular sqoop list