Page MenuHomePhabricator

Merge triggered warning about unequal numbers of ids to be restored and records to be inserted into master
Closed, ResolvedPublic

Description

Error message
PHP Warning: [data-update-failed]: A data update callback triggered an exception (Fail-safe exception. Number of ids to be restored is not equal to the number of records that are about to be inserted into master. This should never happen, except for an edge-case that was not detected during development or due to a race-condition that is not covered by this implementation.) [Called from Wikibase\Repo\Content\DataUpdateAdapter::doUpdate in /srv/mediawiki/php-1.34.0-wmf.23/extensions/Wikibase/repo/includes/Content/DataUpdateAdapter.php at line 65]
Impact
Notes

Details

Request ID
XYTW9wpAICIAAJRvn8AAAACA
Request URL
https://www.wikidata.org/wiki/Special:MergeItems
Stack Trace
#0 [internal function]: MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.34.0-wmf.23/includes/debug/MWDebug.php(333): trigger_error(string, integer)
#2 /srv/mediawiki/php-1.34.0-wmf.23/includes/debug/MWDebug.php(188): MWDebug::sendMessage(string, array, string, integer)
#3 /srv/mediawiki/php-1.34.0-wmf.23/includes/GlobalFunctions.php(1079): MWDebug::warning(string, integer, integer, string)
#4 /srv/mediawiki/php-1.34.0-wmf.23/extensions/Wikibase/lib/includes/Reporting/LogWarningExceptionHandler.php(28): wfLogWarning(string, integer)
#5 /srv/mediawiki/php-1.34.0-wmf.23/extensions/Wikibase/repo/includes/Content/DataUpdateAdapter.php(65): Wikibase\Lib\Reporting\LogWarningExceptionHandler->handleException(Exception, string, string)
#6 /srv/mediawiki/php-1.34.0-wmf.23/includes/deferred/DeferredUpdates.php(383): Wikibase\Repo\Content\DataUpdateAdapter->doUpdate()
#7 /srv/mediawiki/php-1.34.0-wmf.23/includes/Storage/DerivedPageDataUpdater.php(1630): DeferredUpdates::attemptUpdate(Wikibase\Repo\Content\DataUpdateAdapter, Wikimedia\Rdbms\LBFactoryMulti)
#8 /srv/mediawiki/php-1.34.0-wmf.23/includes/Storage/DerivedPageDataUpdater.php(1446): MediaWiki\Storage\DerivedPageDataUpdater->doSecondaryDataUpdates(array)
#9 /srv/mediawiki/php-1.34.0-wmf.23/includes/deferred/MWCallableUpdate.php(38): MediaWiki\Storage\DerivedPageDataUpdater->MediaWiki\Storage\{closure}()
#10 /srv/mediawiki/php-1.34.0-wmf.23/includes/deferred/DeferredUpdates.php(383): MWCallableUpdate->doUpdate()
#11 /srv/mediawiki/php-1.34.0-wmf.23/includes/deferred/DeferredUpdates.php(281): DeferredUpdates::attemptUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti)
#12 /srv/mediawiki/php-1.34.0-wmf.23/includes/deferred/DeferredUpdates.php(226): DeferredUpdates::run(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti, Monolog\Logger, BufferingStatsdDataFactory, string)
#13 /srv/mediawiki/php-1.34.0-wmf.23/includes/deferred/DeferredUpdates.php(149): DeferredUpdates::handleUpdateQueue(array, string, integer)
#14 /srv/mediawiki/php-1.34.0-wmf.23/includes/MediaWiki.php(934): DeferredUpdates::doUpdates(string)
#15 /srv/mediawiki/php-1.34.0-wmf.23/includes/MediaWiki.php(758): MediaWiki->restInPeace(string, boolean)
#16 /srv/mediawiki/php-1.34.0-wmf.23/includes/MediaWiki.php(781): MediaWiki->{closure}()
#17 /srv/mediawiki/php-1.34.0-wmf.23/includes/MediaWiki.php(556): MediaWiki->doPostOutputShutdown(string)
#18 /srv/mediawiki/php-1.34.0-wmf.23/index.php(44): MediaWiki->run()
#19 /srv/mediawiki/w/index.php(3): require(string)
#20 {main}

Event Timeline

Michael created this task.Sep 20 2019, 2:49 PM
Restricted Application added a project: Wikidata. · View Herald TranscriptSep 20 2019, 2:49 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Michael updated the task description. (Show Details)Sep 20 2019, 2:52 PM

Seeing quite a bit of this in production logs at present:

brennen@mwlog1001:/srv/mw-log$ grep -c 'A data update callback triggered an exception' error.log
1018

Looks to still be happening

addshore@mwlog1001:/srv/mw-log$ grep -c 'A data update callback triggered an exception' error.log
304
addshore@mwlog1001:/srv/mw-log$ grep -c 'DataUpdateAdapter::doUpdate' error.log
304
Addshore triaged this task as Medium priority.Nov 19 2019, 12:56 PM
Addshore moved this task from Incoming to Ready to estimate on the Wikidata-Campsite board.
Maintenance_bot moved this task from incoming to in progress on the Wikidata board.Dec 3 2019, 1:15 PM

Still happening: 240

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptDec 11 2019, 11:50 PM
Ladsgroup removed Ladsgroup as the assignee of this task.Dec 13 2019, 11:44 AM

It's caused by this patch and that $recordsToInsert doesn't necessary have all of the $idsToRestore cases because some might be already in the table. Fixing it is not easy because $recordsToInsert is an associative array of terms properly mapped and $idsToRestore is just list of term ids. I leave this for anyone else to pick up for now.

Ladsgroup claimed this task.EditedJan 6 2020, 4:49 AM

okay since no one touched it until I came back from vacation (I took vacation to run away from this :D) it seems I should do it.

Macro challenge-accepted:

Change 562344 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Failing test for T233414

https://gerrit.wikimedia.org/r/562344

Change 562475 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Temporary add metrics of the need to reinsert in the new term store

https://gerrit.wikimedia.org/r/562475

Change 562477 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.11] Temporary add metrics of the need to reinsert in the new term store

https://gerrit.wikimedia.org/r/562477

Change 562475 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Temporary add metrics of the need to reinsert in the new term store

https://gerrit.wikimedia.org/r/562475

Change 562477 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.11] Temporary add metrics of the need to reinsert in the new term store

https://gerrit.wikimedia.org/r/562477

Change 562565 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Simply return ids to insert instead of failing

https://gerrit.wikimedia.org/r/562565

Change 562565 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Simply return ids to insert instead of failing

https://gerrit.wikimedia.org/r/562565

Addshore closed this task as Resolved.Jan 15 2020, 5:07 PM

Change 562344 abandoned by Ladsgroup:
Failing test for T233414

Reason:
not needed anymore. Fixed.

https://gerrit.wikimedia.org/r/562344