Page MenuHomePhabricator

PageImages/initImageData managed to load wmf.5 code, caused stacktrace
Closed, ResolvedPublicPRODUCTION ERROR

Description

The following happened when someone ran initImageData from PageImages:

Database::query: lost connection to 10.64.32.136; reconnected:
#0 /srv/mediawiki/php-1.29.0-wmf.5/includes/libs/rdbms/database/Database.php(1254): Database->query('SELECT  page_id...', 'InitImageData::...')
#1 /srv/mediawiki/php-1.29.0-wmf.5/extensions/PageImages/maintenance/initImageData.php(67): Database->select(Array, Array, Array, 'InitImageData::...', Array, Array)
#2 /srv/mediawiki/php-1.29.0-wmf.5/maintenance/doMaintenance.php(111): InitImageData->execute()
#3 /srv/mediawiki/php-1.29.0-wmf.5/extensions/PageImages/maintenance/initImageData.php(107): require_once('/srv/mediawiki/...')
#4 /srv/mediawiki/multiversion/MWScript.php(97): require_once('/srv/mediawiki/...')
#5 {main}

The stacktrace itself isn't so scary to me, although maybe queries could use a double check. It's the fact that we're currently running wmf.11 and wmf.10 in production (wmf.10 for simplewiki at time of filing this) but we somehow are loading code from wmf.5!! How is this possible?

Happened twice, both to simplewiki, at 2017-02-08T21:34:04 and 2017-02-08T18:52:09

Event Timeline

This looks to be a tmux session i forgot about. The code from wmf.5 had a bug where the init image data script would have jobs fail, and jobs that are abandoned would stick around and be counted as active for a week before the maintenance script would start injecting new jobs. I've killed this and you shouldn't see any wmf.5 error messages.

EBernhardson claimed this task.

Change 336730 had a related patch set uploaded (by Chad):
Scap clean: Rework --l10n-only into --keep-static

https://gerrit.wikimedia.org/r/336730

Ok, thanks for clarifying. This got me scared generally, so I'm adding some extra protections to our cleanup processes :)

Change 336730 merged by jenkins-bot:
Scap clean: Rework --l10n-only into --keep-static

https://gerrit.wikimedia.org/r/336730

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:10 PM