MediaWiki periodic job campaignevents-aggregateanswers-metawiki failed
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	phaultfinder
	Jun 5 2026, 12:06 PM

Project Tags

Referenced Files

None

Subscribers

Aklapper

Daimona

phaultfinder

Description

Common information

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:campaignevents-aggregateanswers-metawiki)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron eqiad; kubectl get jobs -l team=campaigns-product,cronjob=campaignevents-aggregateanswers-metawiki --field-selector status.successful=0; kubectl logs -l cronjob=campaignevents-aggregateanswers-metawiki --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job campaignevents-aggregateanswers-metawiki failed

alertname: MediaWikiCronJobFailed
label_cronjob: campaignevents-aggregateanswers-metawiki
label_team: campaigns-product
prometheus: k8s
severity: task
site: eqiad
source: prometheus
team: campaigns-product

Firing alerts

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:campaignevents-aggregateanswers-metawiki)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron eqiad; kubectl get jobs -l team=campaigns-product,cronjob=campaignevents-aggregateanswers-metawiki --field-selector status.successful=0; kubectl logs -l cronjob=campaignevents-aggregateanswers-metawiki --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job campaignevents-aggregateanswers-metawiki failed
alertname: MediaWikiCronJobFailed
label_cronjob: campaignevents-aggregateanswers-metawiki
label_team: campaigns-product
prometheus: k8s
severity: task
site: eqiad
source: prometheus
team: campaigns-product
Source

Related Objects

Mentioned In: T428301: MediaWiki periodic job campaignevents-aggregateanswers-metawiki failed
Mentioned Here: T346971: Uncaught ConfigException: Failed to load configuration from etcd

Event Timeline

phaultfinder created this task.Jun 5 2026, 12:06 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 5 2026, 12:06 PM

From the logs:

[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480816495Z Fatal error: Uncaught MediaWiki\Config\ConfigException: Failed to load configuration from etcd: (curl error: 28) Timeout was reached in /srv/mediawiki/php-1.47.0-wmf.5/includes/Config/EtcdConfig.php:207
[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480820675Z Stack trace:
[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480824424Z #0 /srv/mediawiki/php-1.47.0-wmf.5/includes/Config/EtcdConfig.php(130): MediaWiki\Config\EtcdConfig->load()
[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480827114Z #1 /srv/mediawiki/wmf-config/CommonSettings.php(199): MediaWiki\Config\EtcdConfig->getModifiedIndex()
[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480830114Z #2 /srv/mediawiki/php-1.47.0-wmf.5/LocalSettings.php(4): require('/srv/mediawiki/...')
[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480832587Z #3 /srv/mediawiki/php-1.47.0-wmf.5/includes/Setup.php(213): require_once('/srv/mediawiki/...')
[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480835031Z #4 /srv/mediawiki/php-1.47.0-wmf.5/maintenance/run.php(51): require_once('/srv/mediawiki/...')
[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480837458Z #5 /srv/mediawiki/multiversion/MWScript.php(219): require_once('/srv/mediawiki/...')
[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480839900Z #6 {main}
[pod/campaignevents-aggregateanswers-metawiki-29677680-rc28v/mediawiki-main-app] 2026-06-05T12:00:14.480842748Z   thrown in /srv/mediawiki/php-1.47.0-wmf.5/includes/Config/EtcdConfig.php on line 207

So an instance of T346971: Uncaught ConfigException: Failed to load configuration from etcd, for which no real actionable is possible here, hence closing.

Daimona mentioned this in T428301: MediaWiki periodic job campaignevents-aggregateanswers-metawiki failed.Jun 9 2026, 1:10 AM

MediaWiki periodic job campaignevents-aggregateanswers-metawiki failedClosed, ResolvedPublicActions