Maniphest T419140

MediaWiki periodic job purge-temporary-accounts failed
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	phaultfinder
	Mar 5 2026, 3:39 PM

Description

Common information

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:purge-temporary-accounts)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron codfw; kubectl get jobs -l team=mediawiki-platform,cronjob=purge-temporary-accounts --field-selector status.successful=0; kubectl logs -l cronjob=purge-temporary-accounts --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job purge-temporary-accounts failed

alertname: MediaWikiCronJobFailed
label_cronjob: purge-temporary-accounts
label_team: mediawiki-platform
prometheus: k8s
severity: task
site: codfw
source: prometheus
team: mediawiki-platform

Firing alerts

dashboard: https://logstash.wikimedia.org/app/dashboards#/view/d51552d0-e309-11ef-87d0-9371e01d3c68?_a=(filters:!((query:(match_phrase:(kubernetes.labels.cronjob.keyword:purge-temporary-accounts)))))&_g=(time:(from:now-24h,to:now))
description: Use kube-env mw-cron codfw; kubectl get jobs -l team=mediawiki-platform,cronjob=purge-temporary-accounts --field-selector status.successful=0; kubectl logs -l cronjob=purge-temporary-accounts --timestamps=true --prefix=true -c mediawiki-main-app to see failures and logs
runbook: https://wikitech.wikimedia.org/wiki/Periodic_jobs#Troubleshooting
summary: MediaWiki periodic job purge-temporary-accounts failed
alertname: MediaWikiCronJobFailed
label_cronjob: purge-temporary-accounts
label_team: mediawiki-platform
prometheus: k8s
severity: task
site: codfw
source: prometheus
team: mediawiki-platform
Source

Related Objects

Mentioned In: T419229: Periodic job alerts could use some more information on what to do
Mentioned Here: T419229: Periodic job alerts could use some more information on what to do
T419143: The wikis are currently read-only

Event Timeline

phaultfinder created this task.Mar 5 2026, 3:39 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 5 2026, 3:39 PM

Due to the ongoing incident, presumably?
The error is very unhelpfully truncated in the logs.

[pod/purge-temporary-accounts-29543907-bzbxz/mediawiki-main-app] 2026-03-04T18:03:43.316066884Z extensions/CentralAuth/maintenance/expireTemporaryAccounts.php: Finished run
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781553808Z frwikibooks #2 /srv/mediawiki/php-1.46.0-wmf.18/includes/Auth/AbstractTemporaryPasswordPrimaryAuthenticationProvider.php(270): MediaWiki\Extension\CentralAuth\CentralAuthTemporaryPasswordPrimaryAuthenticationProvider->setTemporaryPassword('~2025-38492-29', Object(MediaWiki\Password\InvalidPassword), NULL)
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781557445Z frwikibooks #3 /srv/mediawiki/php-1.46.0-wmf.18/includes/Auth/AbstractPrimaryAuthenticationProvider.php(73): MediaWiki\Auth\AbstractTemporaryPasswordPrimaryAuthenticationProvider->providerChangeAuthenticationData(Object(MediaWiki\Auth\TemporaryPasswordAuthenticationRequest))
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781561042Z frwikibooks #4 /srv/mediawiki/php-1.46.0-wmf.18/includes/Auth/AuthManager.php(2991): MediaWiki\Auth\AbstractPrimaryAuthenticationProvider->providerRevokeAccessForUser('~2025-38492-29')
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781564541Z frwikibooks #5 /srv/mediawiki/php-1.46.0-wmf.18/includes/Auth/AuthManager.php(1026): MediaWiki\Auth\AuthManager->callMethodOnProviders(6, 'providerRevokeA...', Array)
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781573683Z frwikibooks #6 /srv/mediawiki/php-1.46.0-wmf.18/maintenance/expireTemporaryAccounts.php(151): MediaWiki\Auth\AuthManager->revokeAccessForUser('~2025-38492-29')
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781577013Z frwikibooks #7 /srv/mediawiki/php-1.46.0-wmf.18/maintenance/expireTemporaryAccounts.php(192): ExpireTemporaryAccounts->expireTemporaryAccount(Object(MediaWiki\User\UserIdentityValue))
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781591061Z frwikibooks #8 /srv/mediawiki/php-1.46.0-wmf.18/maintenance/includes/MaintenanceRunner.php(694): ExpireTemporaryAccounts->execute()
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781595044Z frwikibooks #9 /srv/mediawiki/php-1.46.0-wmf.18/maintenance/run.php(53): MediaWiki\Maintenance\MaintenanceRunner->run()
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781597949Z frwikibooks #10 /srv/mediawiki/multiversion/MWScript.php(219): require_once('/srv/mediawiki/...')
[pod/purge-temporary-accounts-29545347-9vdqs/mediawiki-main-app] 2026-03-05T15:33:18.781601153Z frwikibooks #11 {main}

The log lines in the Logstash link are out of order for some reason, but one of them is:

frwikibooks The system administrator who locked it offered this explanation: The wikis are currently read-only

So yes, caused by the incident (T419143).

Presumably the next run of the script will expire the accounts? If so, no action should be needed.

NotKlea subscribed.Mar 5 2026, 7:56 PM

Yeah, the script can recover.

...but the alert cannot, the job needs to be deleted manually.

tgr@deploy2002:~$ kube-env mw-cron codfw
tgr@deploy2002:~$ KUBECONFIG=/etc/kubernetes/mw-cron-deploy-codfw.config
tgr@deploy2002:~$ kubectl get jobs -l team=mediawiki-platform,cronjob=purge-temporary-accounts --field-selector status.successful=0
NAME                                STATUS   COMPLETIONS   DURATION   AGE
purge-temporary-accounts-29545347   Failed   0/1           21h        21h
tgr@deploy2002:~$ kubectl delete job purge-temporary-accounts-29545347
job.batch "purge-temporary-accounts-29545347" deleted

• Tgr merged a task: T419185: MediaWiki periodic job purge-temporary-accounts failed.Mar 6 2026, 12:11 PM

Filed T419229: Periodic job alerts could use some more information on what to do about making this clearer.

MediaWiki periodic job purge-temporary-accounts failedClosed, ResolvedPublicActions