MediaWiki periodic job update-special-pages-s5 failed
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	phaultfinder
	Apr 5 2026, 5:38 AM

Project Tags

Referenced Files

	F75165883: image.png
	Apr 6 2026, 4:39 PM

Subscribers

Description

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:update-special-pages-s5)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron eqiad; kubectl get jobs -l team=mediawiki-special-pages,cronjob=update-special-pages-s5 --field-selector status.successful=0; kubectl logs -l cronjob=update-special-pages-s5 --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job update-special-pages-s5 failed

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:update-special-pages-s5)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron eqiad; kubectl get jobs -l team=mediawiki-special-pages,cronjob=update-special-pages-s5 --field-selector status.successful=0; kubectl logs -l cronjob=update-special-pages-s5 --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job update-special-pages-s5 failed
alertname: MediaWikiCronJobFailed
label_cronjob: update-special-pages-s5
label_team: mediawiki-special-pages
prometheus: k8s
severity: task
site: eqiad
source: prometheus
team: mediawiki-special-pages
Source

Mentioned In: T431518: MediaWiki periodic job update-special-pages-s5 failed
T422580: MediaWiki periodic job update-special-pages-s5 failed
T422413: MediaWiki periodic job update-special-pages-s5 failed
Mentioned Here: T400133: When a database replica is depooled/rebooted, update-special-pages maintenance scripts connected to that replica seem to fail (instead of continuing with a different replica)

Restricted Application changed the subtype of this task from "Task" to "Production Error". · View Herald TranscriptApr 5 2026, 5:38 AM

Restricted Application added subscribers: A_smart_kitten, • Aklapper. · View Herald Transcript

jijiki closed this task as Resolved.Apr 6 2026, 4:39 PM

jijiki claimed this task.

copying down error from screenshot for future searchability

dewiki Error: 2006 MySQL server has gone away

FWICS, the time from that Logstash screenshot is 2026-04-05 05:32:22.969.
My first guess for a possible root-cause would probably have been T400133: When a database replica is depooled/rebooted, update-special-pages maintenance scripts connected to that replica seem to fail (instead of continuing with a different replica); however, looking at https://sal.toolforge.org/production?p=0&q=&d=2026-04-05, it seems like there wasn't any logged DB maintenance (or much logged anything) in prod-SAL yesterday.
Huh. Anyway, I guess we probably don't need to worry about it too much for now, unless something like this happens again :)