I got cron mail about a run that was relying on the august 10th files, presumably writing to output files with day August 11th and that had been cleaned up because these jobs should finish in the same day.
On the snapshot host where these run, there were several old processes; I shot them. The current run seems hung on commonswiki; a look at an earlier run shows that the commonswiki job should complete in about a minute but it's been several hours now.
This is likely related to T260232
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| disable category rdf dumps for now | operations/puppet | production | +1 -1 |
I have shot a bunch of these hung processes again on snapshot1008. If a few more days go around without the relevant patch being merged, I'll disable the cron job in puppet.
Repeat of last week. I'll disable the cron job this week, to be re-enabled once this issue is resolved.
Change 623692 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/puppet@production] disable category rdf dumps for now
Change 623692 merged by ArielGlenn:
[operations/puppet@production] disable category rdf dumps for now
https://gerrit.wikimedia.org/r/c/operations/puppet/+/628607 I've re-enabled te category dumps, since both https://gerrit.wikimedia.org/r/c/mediawiki/core/+/620775/ and https://gerrit.wikimedia.org/r/c/operations/puppet/+/628607 are merged and deployed. I'll leave this task open and check tomorrow morning to see that the daily run went ok.