Page MenuHomePhabricator

Getting InvalidArgumentException when running a query on the API
Closed, ResolvedPublic

Description

I have a long running script traversing through the contributions of InternetArchiveBot, at some point the API will simply return [XXeLMgpAIDoAAEAw8J8AAABK] Caught exception of type InvalidArgumentException

Here's the GET string that triggers it.

action=query&format=json&list=allrevisions&arvprop=ids%7Ctimestamp%7Cflags%7Ccomment%7Cuser&arvlimit=max&arvuser=InternetArchiveBot&arvnamespace=0&arvstart=2019-08-06T00%3A00%3A00.000Z&arvdir=newer

Event Timeline

Cyberpower678 triaged this task as Unbreak Now! priority.Sep 10 2019, 11:45 AM
message
The given Title does not belong to page ID 6770541 but actually belongs to 61659441
trace
#0 /srv/mediawiki/php-1.34.0-wmf.21/includes/Revision/RevisionStore.php(1874): MediaWiki\Revision\RevisionStoreRecord->__construct(Title, User, CommentStoreComment, stdClass, MediaWiki\Revision\RevisionSlots, boolean)
#1 /srv/mediawiki/php-1.34.0-wmf.21/includes/Revision/RevisionStore.php(1800): MediaWiki\Revision\RevisionStore->newRevisionFromRowAndSlots(stdClass, NULL, integer, Title, boolean)
#2 /srv/mediawiki/php-1.34.0-wmf.21/includes/api/ApiQueryAllRevisions.php(225): MediaWiki\Revision\RevisionStore->newRevisionFromRow(stdClass)
#3 /srv/mediawiki/php-1.34.0-wmf.21/includes/api/ApiQueryRevisionsBase.php(59): ApiQueryAllRevisions->run()
#4 /srv/mediawiki/php-1.34.0-wmf.21/includes/api/ApiQuery.php(250): ApiQueryRevisionsBase->execute()
#5 /srv/mediawiki/php-1.34.0-wmf.21/includes/api/ApiMain.php(1596): ApiQuery->execute()
#6 /srv/mediawiki/php-1.34.0-wmf.21/includes/api/ApiMain.php(534): ApiMain->executeAction()
#7 /srv/mediawiki/php-1.34.0-wmf.21/includes/api/ApiMain.php(505): ApiMain->executeActionWithErrorHandling()
#8 /srv/mediawiki/php-1.34.0-wmf.21/api.php(86): ApiMain->execute()
#9 /srv/mediawiki/w/api.php(3): include(string)
#10 {main}
Aklapper lowered the priority of this task from Unbreak Now! to Needs Triage.Sep 10 2019, 12:10 PM

This isn't the first time I've run my script against IABot's contributions, so I would say it is a recent issue.

Of note, this error was seen 4 times in the last 7 days, of which 3 are today, and one yesterday. All of these were for IABot's contributions. So maybe there's an issue with a specific page on enwp?

I couldn't say. I can't even see which page it's trying to pull in that given batch.

Anomie subscribed.

The revision in question is rev_id = 909641830. The problem is that Special:MergeHistory was used and only updated revision.rev_page, without updating the denormalized revision_actor_temp.revactor_page to match.

To fix this we'll have to both fix MergeHistory to update revactor_page and write and run a maintenance script to clean up broken rows in the database.

Change 535625 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@master] MergeHistory: Update revactor_page too

https://gerrit.wikimedia.org/r/535625

@Anomie what's the latest about this bug? I see it's awaiting review, but when will it get merged and deployed? This is sadly holding up work on my end.

Current status is that it's waiting for review. Unfortunately that requires a person with the right knowledge to have time (and also see it).

After it gets reviewed and merged, we'll have to wait for it to be deployed (via train or SWAT), so no new instances of the problem are being created.

Then we'll have to run the maintenance scripts. I don't plan for there to be significant delay between the deploy and the start of this process, but to keep in mind that it may take some time to process the hundreds of millions of revisions on large wikis like enwiki. Note I do plan to run the maintenance scripts in parallel by section, so at least enwiki won't be waiting on dewiki or vice versa.

Change 535625 merged by jenkins-bot:
[mediawiki/core@master] MergeHistory: Update revactor_page too

https://gerrit.wikimedia.org/r/535625

Change 537436 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@wmf/1.34.0-wmf.22] MergeHistory: Update revactor_page too

https://gerrit.wikimedia.org/r/537436

Change 537436 merged by jenkins-bot:
[mediawiki/core@wmf/1.34.0-wmf.22] MergeHistory: Update revactor_page too

https://gerrit.wikimedia.org/r/537436

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:39:33Z] <anomie@deploy1001> Synchronized php-1.34.0-wmf.22/includes/MergeHistory.php: Backport MergeHistory fix for T232464 [[gerrit:537436]] (duration: 00m 54s)

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:48:29Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on test wikis and mediawikiwiki for T232464

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:50:30Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on section 1 wikis for T232464

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:50:33Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on section 2 wikis for T232464

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:50:35Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on remaining section 3 wikis for T232464

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:50:42Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on section 4 wikis for T232464

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:50:57Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on section 5 wikis for T232464

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:50:59Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on section 6 wikis for T232464

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:51:03Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on section 7 wikis for T232464

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:51:05Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on section 8 wikis for T232464

Mentioned in SAL (#wikimedia-operations) [2019-09-17T14:51:07Z] <anomie@mwmaint1002> Running cleanupRevActorPage.php on wikitech for T232464

Status: Patch is merged and backported. Maintenance scripts are running to clean up existing broken rows, but this may take a few hours or days on larger wikis such as enwiki.

Maintenance script runs completed.

Seems to be working now on my end. :-)