Page MenuHomePhabricator

Fix "LogicException from JobQueueGroup.php: Domain ... not recognized."
Closed, ResolvedPublic

Description

MediaWiki	1.30.0-alpha (439597c)
PHP	7.1.1 (apache2handler)
MariaDB	10.1.21-MariaDB
ICU	57.1
[exception] [637adf14a8e6c71e18811473] /mw-master/index.php?title=YetAgainAMediaWikiIssue&action=submit   LogicException from line 136 of ...\includes\jobqueue\JobQueueGroup.php: Domain 'mw?hmaster' is not recognized.
#0 ...\includes\jobqueue\JobQueueGroup.php(218): JobQueueGroup->push(array)
#1 ...\includes\deferred\MWCallableUpdate.php(30): JobQueueGroup::pushLazyJobs()
#2 ...\includes\deferred\DeferredUpdates.php(257): MWCallableUpdate->doUpdate()
#3 ...\includes\deferred\DeferredUpdates.php(210): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactorySimple, string, integer)
#4 ...\includes\deferred\DeferredUpdates.php(131): DeferredUpdates::execute(array, string, integer)
#5 ...\includes\MediaWiki.php(911): DeferredUpdates::doUpdates(string)
#6 ...\includes\MediaWiki.php(738): MediaWiki->restInPeace(string)
#7 ...\includes\MediaWiki.php(757): MediaWiki->{closure}()
#8 ...\includes\MediaWiki.php(553): MediaWiki->doPostOutputShutdown(string)
#9 ...\index.php(43): MediaWiki->run()
#10 {main}
[DBConnection] Connected to database 0 at 'localhost'.
[runJobs] recentChangesUpdate Special:RecentChanges type=cacheUpdate requestId=637adf14a8e6c71e18811473 (id=1146,timestamp=20170824125007) STARTING
[exception] [637adf14a8e6c71e18811473] /mw-master/index.php?title=YetAgainAMediaWikiIssue&action=submit   LogicException from line 136 of ...\includes\jobqueue\JobQueueGroup.php: Domain 'mw?hmaster' is not recognized.
#0 ...\includes\jobqueue\JobQueueGroup.php(218): JobQueueGroup->push(array)
#1 ...\includes\deferred\MWCallableUpdate.php(30): JobQueueGroup::pushLazyJobs()
#2 ...\includes\deferred\DeferredUpdates.php(257): MWCallableUpdate->doUpdate()
#3 ...\includes\deferred\DeferredUpdates.php(210): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactorySimple, string, integer)
#4 ...\includes\deferred\DeferredUpdates.php(131): DeferredUpdates::execute(array, string, integer)
#5 ...\includes\jobqueue\JobRunner.php(301): DeferredUpdates::doUpdates()
#6 ...\includes\jobqueue\JobRunner.php(193): JobRunner->executeJob(RecentChangesUpdateJob, Wikimedia\Rdbms\LBFactorySimple, BufferingStatsdDataFactory, integer)
#7 ...\includes\MediaWiki.php(978): JobRunner->run(array)
#8 ...\includes\MediaWiki.php(964): MediaWiki->triggerSyncJobs(integer, MediaWiki\Logger\LegacyLogger)
#9 ...\includes\MediaWiki.php(916): MediaWiki->triggerJobs()
#10 ...\includes\MediaWiki.php(738): MediaWiki->restInPeace(string)
#11 ...\includes\MediaWiki.php(757): MediaWiki->{closure}()
#12 ...\includes\MediaWiki.php(553): MediaWiki->doPostOutputShutdown(string)
#13 ...\index.php(43): MediaWiki->run()
#14 {main}

The test DB is stored as mw-master, somehow MW seems to assume Domain 'mw?hmaster' .

Event Timeline

I reckoned there was a similar issue [0] in the past and I would have expected that a unit test is in place that would prevent such a regression.

[0] https://phabricator.wikimedia.org/T145840

@aaron

I was patiently awaiting (two weeks+) this issue to be triaged since this is clearly a regression since 1.29!

Do we think that this is going to be fixed in 1.30, given that it has a valid exception stack trace?

@mwjames You know how to mark bugs as release blockers, so if there's a regression you've found and want to see it fixed, mark it as a release blocker. :)

In this case it looks like something is calling JobQueueGroup::singleton( $wiki )->lazyPush( $jobs ); with an DatabaseDomain object for $wiki instead of wfWikiId(). I'm not really sure how this is happening since lazyPush() has the exact same check as push() does. Can you figure out which job is causing this? Maybe then we can trace it back to the job object creation and figure out what's going wrong

You know how to mark bugs as release blockers, so if there's a regression you've found and want to see it fixed, mark it as a release blocker. :)

I know, but the last time ... anyway my experience on that issue has not an encouraging track record, so I only do the minimum now.

ith an DatabaseDomain object for $wiki instead of wfWikiId(). I'm not really sure how this is happening since lazyPush() has the exact same check as push() does.

Test instance is maintained with $wgDBname = "mw-master"; where DatabaseDomain::encode will encode - as ?h, it is reasonable to argue that the decode is messing up - given the "... JobQueueGroup.php: Domain 'mw?hmaster' is not recognized. ..." message.

I didn't look any further.

an you figure out which job is causing this?

A quick look in logs shows:

  • HTMLCacheUpdateJob, LinksUpdate, RecentChangesUpdateJob
2017-09-23 05:43:17 TAURUS mw-master: [f4ea3042ed26cd211d3e3154] /mw-master/index.php?title=Lorem_ipsum-1eab&action=submit   LogicException from line 138 of ...\mw-master\includes\jobqueue\JobQueueGroup.php: Domain 'mw?hmaster' is not recognized.
#0 ...\mw-master\includes\deferred\DeferredUpdates.php(252): JobQueueGroup->push(JobSpecification)
#1 ...\mw-master\includes\deferred\DeferredUpdates.php(210): DeferredUpdates::runUpdate(LinksUpdate, Wikimedia\Rdbms\LBFactorySimple, string, integer)
#2 ...\mw-master\includes\deferred\DeferredUpdates.php(131): DeferredUpdates::execute(array, string, integer)
#3 ...\mw-master\includes\MediaWiki.php(895): DeferredUpdates::doUpdates(string)
#4 ...\mw-master\includes\MediaWiki.php(719): MediaWiki->restInPeace(string, boolean)
#5 ...\mw-master\includes\MediaWiki.php(740): MediaWiki->{closure}()
#6 ...\mw-master\includes\MediaWiki.php(553): MediaWiki->doPostOutputShutdown(string)
#7 ...\mw-master\index.php(43): MediaWiki->run()
#8 {main}
2017-09-23 05:43:18 TAURUS mw-master: [f4ea3042ed26cd211d3e3154] /mw-master/index.php?title=Lorem_ipsum-1eab&action=submit   LogicException from line 138 of ...\mw-master\includes\jobqueue\JobQueueGroup.php: Domain 'mw?hmaster' is not recognized.
#0 ...\mw-master\includes\jobqueue\JobQueueGroup.php(220): JobQueueGroup->push(array)
#1 ...\mw-master\includes\deferred\MWCallableUpdate.php(30): JobQueueGroup::pushLazyJobs()
#2 ...\mw-master\includes\deferred\DeferredUpdates.php(257): MWCallableUpdate->doUpdate()
#3 ...\mw-master\includes\deferred\DeferredUpdates.php(210): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactorySimple, string, integer)
#4 ...\mw-master\includes\deferred\DeferredUpdates.php(131): DeferredUpdates::execute(array, string, integer)
#5 ...\mw-master\includes\MediaWiki.php(895): DeferredUpdates::doUpdates(string)
#6 ...\mw-master\includes\MediaWiki.php(719): MediaWiki->restInPeace(string, boolean)
#7 ...\mw-master\includes\MediaWiki.php(740): MediaWiki->{closure}()
#8 ...\mw-master\includes\MediaWiki.php(553): MediaWiki->doPostOutputShutdown(string)
#9 ...\mw-master\index.php(43): MediaWiki->run()
#10 {main}
2017-09-23 05:43:19 TAURUS mw-master: [f4ea3042ed26cd211d3e3154] /mw-master/index.php?title=Lorem_ipsum-1eab&action=submit   LogicException from line 138 of ...\mw-master\includes\jobqueue\JobQueueGroup.php: Domain 'mw?hmaster' is not recognized.
#0 ...\mw-master\includes\jobqueue\JobQueueGroup.php(220): JobQueueGroup->push(array)
#1 ...\mw-master\includes\deferred\MWCallableUpdate.php(30): JobQueueGroup::pushLazyJobs()
#2 ...\mw-master\includes\deferred\DeferredUpdates.php(257): MWCallableUpdate->doUpdate()
#3 ...\mw-master\includes\deferred\DeferredUpdates.php(210): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactorySimple, string, integer)
#4 ...\mw-master\includes\deferred\DeferredUpdates.php(131): DeferredUpdates::execute(array, string, integer)
#5 ...\mw-master\includes\jobqueue\JobRunner.php(301): DeferredUpdates::doUpdates()
#6 ...\mw-master\includes\jobqueue\JobRunner.php(193): JobRunner->executeJob(HTMLCacheUpdateJob, Wikimedia\Rdbms\LBFactorySimple, BufferingStatsdDataFactory, integer)
#7 ...\mw-master\includes\MediaWiki.php(962): JobRunner->run(array)
#8 ...\mw-master\includes\MediaWiki.php(948): MediaWiki->triggerSyncJobs(integer, MediaWiki\Logger\LegacyLogger)
#9 ...\mw-master\includes\MediaWiki.php(900): MediaWiki->triggerJobs()
#10 ...\mw-master\includes\MediaWiki.php(719): MediaWiki->restInPeace(string, boolean)
#11 ...\mw-master\includes\MediaWiki.php(740): MediaWiki->{closure}()
#12 ...\mw-master\includes\MediaWiki.php(553): MediaWiki->doPostOutputShutdown(string)
#13 ...\mw-master\index.php(43): MediaWiki->run()
#14 {main}

#10 {main}
2017-09-23 05:45:28 TAURUS mw-master: [f4ea3042ed26cd211d3e3154] /mw-master/index.php?title=Lorem_ipsum-1eab&action=submit   LogicException from line 138 of ...\mw-master\includes\jobqueue\JobQueueGroup.php: Domain 'mw?hmaster' is not recognized.
#0 ...\mw-master\includes\jobqueue\JobQueueGroup.php(220): JobQueueGroup->push(array)
#1 ...\mw-master\includes\deferred\MWCallableUpdate.php(30): JobQueueGroup::pushLazyJobs()
#2 ...\mw-master\includes\deferred\DeferredUpdates.php(257): MWCallableUpdate->doUpdate()
#3 ...\mw-master\includes\deferred\DeferredUpdates.php(210): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactorySimple, string, integer)
#4 ...\mw-master\includes\deferred\DeferredUpdates.php(131): DeferredUpdates::execute(array, string, integer)
#5 ...\mw-master\includes\jobqueue\JobRunner.php(301): DeferredUpdates::doUpdates()
#6 ...\mw-master\includes\jobqueue\JobRunner.php(193): JobRunner->executeJob(RecentChangesUpdateJob, Wikimedia\Rdbms\LBFactorySimple, BufferingStatsdDataFactory, integer)
#7 ...\mw-master\includes\MediaWiki.php(962): JobRunner->run(array)
#8 ...\mw-master\includes\MediaWiki.php(948): MediaWiki->triggerSyncJobs(integer, MediaWiki\Logger\LegacyLogger)
#9 ...\mw-master\includes\MediaWiki.php(900): MediaWiki->triggerJobs()
#10 ...\mw-master\includes\MediaWiki.php(719): MediaWiki->restInPeace(string, boolean)
#11 ...\mw-master\includes\MediaWiki.php(740): MediaWiki->{closure}()
#12 ...\mw-master\includes\MediaWiki.php(553): MediaWiki->doPostOutputShutdown(string)
#13 ...\mw-master\index.php(43): MediaWiki->run()
#14 {main}

Change 380471 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] Clean up wiki ID and DB domain ID handling

https://gerrit.wikimedia.org/r/380471

https://gerrit.wikimedia.org/r/380471

A quick checkout of the PR confirms that the system no longer produces above listed exceptions.

Krinkle renamed this task from LogicException from line 136 of ... JobQueueGroup.php: Domain ... recognized. to Fix "LogicException from JobQueueGroup.php: Domain ... not recognized.".Sep 25 2017, 5:56 PM
Krinkle added a project: MW-1.30-release.

Change 380471 merged by jenkins-bot:
[mediawiki/core@master] Clean up wiki ID and DB domain ID handling

https://gerrit.wikimedia.org/r/380471

Change 381703 had a related patch set uploaded (by Legoktm; owner: Aaron Schulz):
[mediawiki/core@REL1_30] Clean up wiki ID and DB domain ID handling

https://gerrit.wikimedia.org/r/381703

Change 381703 merged by jenkins-bot:
[mediawiki/core@REL1_30] Clean up wiki ID and DB domain ID handling

https://gerrit.wikimedia.org/r/381703

Legoktm assigned this task to aaron.

Merged into master and REL1_30.