Page MenuHomePhabricator

Add alerting on suspiciously long mw-cron jobs
Open, MediumPublic

Description

Today, @Michael discovered there is a long-running mw-cron job for the Growth team:

migr@deploy2002:~$ kubectl logs -f job/growthexperiments-listtaskcounts-29513771 mediawiki-main-app
Error from server (BadRequest): container "mediawiki-main-app" in pod "growthexperiments-listtaskcounts-29513771-jrv9k" is waiting to start: trying and failing to pull image
migr@deploy2002:~$ kubectl get jobs -l team=growth,cronjob=growthexperiments-listtaskcounts
NAME                                        STATUS    COMPLETIONS   DURATION   AGE
growthexperiments-listtaskcounts-29513771   Running   0/1           33d        33d
migr@deploy2002:~$

This job should never take more than a couple of minutes to complete. If it takes more than an hour, we definitely have a problem.

Would it be possible to add something like max_expected_length = XX hours to the job definition, and trigger an alert if the expected duration was exceeded? This would help us notice this much quickly than manually after a month.


Update 2026-03-19 - One alternative to passive alerting that we may want to consider is functionality for setting activeDeadlineSeconds on the cronjob's job template. That would provide an active backstop on execution time. As a side effect, the job failure alert should also fire, as a result of the job being terminated. See T420354#11722534.

Event Timeline

Scott_French moved this task from Inbox to Needs Info / Blocked on the ServiceOps board.
Scott_French subscribed.

Interesting! Indeed, the job was referencing an image that does not appear to exist (event):

Failed to pull image "docker-registry.discovery.wmnet/restricted/mediawiki-multiversion-cli:2026-02-11-150202-publish-83": rpc error: code = NotFound desc = failed to pull and unpack image "docker-registry.discovery.wmnet/restricted/mediawiki-multiversion-cli:2026-02-11-150202-publish-83": failed to resolve reference "docker-registry.discovery.wmnet/restricted/mediawiki-multiversion-cli:2026-02-11-150202-publish-83": docker-registry.discovery.wmnet/restricted/mediawiki-multiversion-cli:2026-02-11-150202-publish-83: not found

Given the specific timestamp in the image, I suspect this is somehow an artifact of the rollback from the apus-backed registry to the swift-backed one for the restricted prefix (i.e., somehow, at the time the Job was created, the template in the CronJob referenced an image that never made it into either one).

Regardless of the cause, agreed that this is something that would be good to catch - i.e., a job wildly overrunning its expected duration.

@Urbanecm_WMF - Just to undertand the desired use case, is it essential that the alert passively surfaces this condition? The reason I ask is that there's also activeDeadlineSeconds, which would offer the option of terminating a job that exceeds the desired deadline. That would also result in an alert (i.e., for the failed job).

@Urbanecm_WMF - Just to undertand the desired use case, is it essential that the alert passively surfaces this condition? The reason I ask is that there's also activeDeadlineSeconds, which would offer the option of terminating a job that exceeds the desired deadline. That would also result in an alert (i.e., for the failed job).

Good question! My original thought was "let's give all jobs a somehow-sensible limit, allowing the maintainers to determine if a job should continue running". But, if k8s has a built-in way to terminate jobs that run for too long, I wouldn't be opposed to using that one instead. We just need to be a bit more careful in defining the value, but that sounds doable.

Thanks, @Urbanecm_WMF - Good to know the deadline option might also fit your use case. I also suspect it would have a much simpler implementation in the end.