MediaWiki periodic job update-flaggedrev-stats failed
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	phaultfinder
	Thu, Apr 16, 12:17 AM

Description

Common information

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:update-flaggedrev-stats)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron eqiad; kubectl get jobs -l team=flaggedrevs,cronjob=update-flaggedrev-stats --field-selector status.successful=0; kubectl logs -l cronjob=update-flaggedrev-stats --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job update-flaggedrev-stats failed

alertname: MediaWikiCronJobFailed
label_cronjob: update-flaggedrev-stats
label_team: flaggedrevs
prometheus: k8s
severity: task
site: eqiad
source: prometheus
team: flaggedrevs

Firing alerts

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:update-flaggedrev-stats)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron eqiad; kubectl get jobs -l team=flaggedrevs,cronjob=update-flaggedrev-stats --field-selector status.successful=0; kubectl logs -l cronjob=update-flaggedrev-stats --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job update-flaggedrev-stats failed
alertname: MediaWikiCronJobFailed
label_cronjob: update-flaggedrev-stats
label_team: flaggedrevs
prometheus: k8s
severity: task
site: eqiad
source: prometheus
team: flaggedrevs
Source

Related Objects
Search...

Status	Subtype	Assigned	Task
Open		None	T422489 rdbms errors in eqiad
In Progress		None	T422486 MediaWiki periodic job failures due to timeouts
Resolved	PRODUCTION ERROR	• Clement_Goubert	T423538 MediaWiki periodic job update-flaggedrev-stats failed

Event Timeline

phaultfinder created this task.Thu, Apr 16, 12:17 AM

Restricted Application changed the subtype of this task from "Task" to "Production Error". · View Herald TranscriptThu, Apr 16, 12:17 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

MLechvien-WMF added a parent task: T422486: MediaWiki periodic job failures due to timeouts.Thu, Apr 16, 7:17 AM

[pod/update-flaggedrev-stats-29604968-r4d4n/mediawiki-main-app] 2026-04-16T00:10:58.775367153Z fawiki ValidationStatistics           Wikimedia\Rdbms\DBConnectionError from line 1125 of /srv/mediawiki/php-1.46.0-wmf.23/includes/libs/Rdbms/LoadBalancer/LoadBalancer.php: Cannot access the database: MySQL server has gone away (db1181)

I'm not finding a smoking gun for why that connection failed, either on the wikikube-worker or the db server.
Data-Persistence does LoadBalancer.php have a way to maybe retry on these, or is that undesireable?

Failed job deleted.

A_smart_kitten subscribed.Thu, Apr 16, 10:03 PM

MediaWiki periodic job update-flaggedrev-stats failedClosed, ResolvedPublicPRODUCTION ERRORActions