MediaWiki periodic job initsitestats failed
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	phaultfinder
	Tue, Jul 21, 2:44 AM

Project Tags

Referenced Files

None

Subscribers

Description

Common information

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:initsitestats)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron eqiad; kubectl get jobs -l team=mediawiki-special-pages,cronjob=initsitestats --field-selector status.successful=0; kubectl logs -l cronjob=initsitestats --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job initsitestats failed

alertname: MediaWikiCronJobFailed
label_cronjob: initsitestats
label_team: mediawiki-special-pages
prometheus: k8s
severity: task
site: eqiad
source: prometheus
team: mediawiki-special-pages

Firing alerts

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:initsitestats)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron eqiad; kubectl get jobs -l team=mediawiki-special-pages,cronjob=initsitestats --field-selector status.successful=0; kubectl logs -l cronjob=initsitestats --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job initsitestats failed
alertname: MediaWikiCronJobFailed
label_cronjob: initsitestats
label_team: mediawiki-special-pages
prometheus: k8s
severity: task
site: eqiad
source: prometheus
team: mediawiki-special-pages
Source

Related Objects
Search...

Status	Subtype	Assigned	Task
Open		None	T422489 rdbms errors in eqiad
In Progress		None	T422486 MediaWiki periodic job failures due to timeouts
Resolved	PRODUCTION ERROR	JMeybohm	T432675 MediaWiki periodic job initsitestats failed

Event Timeline

phaultfinder created this task.Tue, Jul 21, 2:44 AM

Restricted Application changed the subtype of this task from "Task" to "Production Error". · View Herald TranscriptTue, Jul 21, 2:44 AM

Restricted Application added subscribers: A_smart_kitten, Aklapper. · View Herald Transcript

Pppery moved this task from To triage to Maintenance reports on the MediaWiki-Special-pages board.Sun, Jul 26, 3:33 PM

jasmine_ moved this task from Inbox to Needs Info / Blocked on the ServiceOps board.Mon, Jul 27, 5:35 AM

JMeybohm added a parent task: T422486: MediaWiki periodic job failures due to timeouts.Wed, Jul 29, 10:56 AM

NAME                     STATUS   COMPLETIONS   DURATION   AGE
initsitestats-29745900   Failed   0/1           6d13h      6d13h
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181489050Z brwikimedia Fatal error: Uncaught MediaWiki\Config\ConfigException: Failed to load configuration from etcd: (curl error: 28) Timeout was reached Resolving timed out after 2000 milliseconds in /srv/mediawiki/php-1.47.0-wmf.12/includes/Config/EtcdConfig.php:207
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181503676Z brwikimedia Stack trace:
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181513627Z brwikimedia #0 /srv/mediawiki/php-1.47.0-wmf.12/includes/Config/EtcdConfig.php(130): MediaWiki\Config\EtcdConfig->load()
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181522103Z brwikimedia #1 /srv/mediawiki/wmf-config/CommonSettings.php(197): MediaWiki\Config\EtcdConfig->getModifiedIndex()
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181530150Z brwikimedia #2 /srv/mediawiki/php-1.47.0-wmf.12/LocalSettings.php(4): require('/srv/mediawiki/...')
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181550312Z brwikimedia #3 /srv/mediawiki/php-1.47.0-wmf.12/includes/Setup.php(212): require_once('/srv/mediawiki/...')
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181558492Z brwikimedia #4 /srv/mediawiki/php-1.47.0-wmf.12/maintenance/run.php(52): require_once('/srv/mediawiki/...')
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181566154Z brwikimedia #5 /srv/mediawiki/multiversion/MWScript.php(219): require_once('/srv/mediawiki/...')
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181581759Z brwikimedia #6 {main}
[pod/initsitestats-29745900-jvl5s/mediawiki-main-app] 2026-07-22T21:09:01.181589932Z brwikimedia   thrown in /srv/mediawiki/php-1.47.0-wmf.12/includes/Config/EtcdConfig.php on line 207
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.544056935Z zuwiktionary Counting total edits...15191
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.553647306Z zuwiktionary Counting number of articles...1248
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.555405955Z zuwiktionary Counting total pages...2130
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.558859875Z zuwiktionary Counting number of users...3961
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.560456666Z zuwiktionary Counting number of images...0
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.560467654Z zuwiktionary 
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.564557473Z zuwiktionary Updating site statistics...done.
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.564573247Z zuwiktionary 
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.564616512Z zuwiktionary Done.
[pod/initsitestats-29751660-l68zc/mediawiki-main-app] 2026-07-26T23:00:18.575230917Z initSiteStats.php: Finished run
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.268748521Z zuwiktionary Counting total edits...15191
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.278736168Z zuwiktionary Counting number of articles...1248
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.279737261Z zuwiktionary Counting total pages...2130
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.283424479Z zuwiktionary Counting number of users...3962
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.287241540Z zuwiktionary Counting number of images...0
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.287257016Z zuwiktionary 
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.305755691Z zuwiktionary Updating site statistics...done.
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.305775441Z zuwiktionary 
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.305780083Z zuwiktionary Done.
[pod/initsitestats-29753100-bsfdt/mediawiki-main-app] 2026-07-27T23:18:49.316106449Z initSiteStats.php: Finished run
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.106015201Z zuwiktionary Counting total edits...15191
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.111677496Z zuwiktionary Counting number of articles...1248
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.112332016Z zuwiktionary Counting total pages...2130
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.115844978Z zuwiktionary Counting number of users...3962
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.117698808Z zuwiktionary Counting number of images...0
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.117709709Z zuwiktionary 
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.123084574Z zuwiktionary Updating site statistics...done.
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.123095698Z zuwiktionary 
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.123098007Z zuwiktionary Done.
[pod/initsitestats-29754540-8b77d/mediawiki-main-app] 2026-07-28T23:56:03.127988521Z initSiteStats.php: Finished run

job.batch "initsitestats-29745900" deleted

MediaWiki periodic job initsitestats failedClosed, ResolvedPublicPRODUCTION ERRORActions