Page MenuHomePhabricator

Exception caught inside exception handler
Closed, ResolvedPublic

Assigned To
Authored By
Vituzzu
Oct 26 2015, 3:15 PM
Referenced Files
None
Tokens
"The World Burns" token, awarded by Sylvain_WMFr."The World Burns" token, awarded by Steinsplitter."The World Burns" token, awarded by Thibaut120094."The World Burns" token, awarded by Vituzzu.

Description

Any page request turns into:
MediaWiki internal error.

Exception caught inside exception handler.

Set $wgShowExceptionDetails = true; at the bottom of LocalSettings.php to show detailed debugging information.

Event Timeline

Vituzzu raised the priority of this task from to Unbreak Now!.
Vituzzu updated the task description. (Show Details)
Vituzzu subscribed.

Caused by https://gerrit.wikimedia.org/r/248877 rollback in progress

MWException from line 331 of /srv/mediawiki/php-1.27.0-wmf.3/includes/resourceloader/ResourceLoader.php: ResourceLoader duplicate registration error. Another module has already been registered as schema.Search

I'm not that familiar with the deploy process, but maybe the deployment script should have some sort of smoke test built in (e.g. Deploy to test.wikipedia.org, make sure its not fataling, before deploying to rest of cluster)

I think the incident report should have some sort of actionables on time to revert. 9 minutes is a very long time. Do our docs need to be modified about how to revert multiple patchsets quickly? Do we need a new script for returning the wikis to a previous known good state? Or something else?

Scap has some smoke testing and automated rollback functionality. It's not currently tailored to work with MediaWiki deployments but it is on our radar, specifically T114313: [EPIC] Migrate the MW weekly train deploy to scap3.

@mmodell beat me to everything I was going to say. I'll just throw these two at you for reference: T113073: Canary deploy for scap3 and T109514: Scap3 should support rollback as well.

See T112845 for improving that somewhat cryptic error message

This can be considered resolved, since the offending patches were reverted.