Page MenuHomePhabricator

Allow seeking in UsageLookup::getPagesUsing results
Open, HighPublic

Description

Needed to be able to iterate over all pages that use a certain entity id (which can be several hundred thousand) and do something with them in multiple consecutive jobs that schedule each other.

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

hoo created this task.Sep 27 2016, 11:35 PM
hoo added a comment.Sep 27 2016, 11:40 PM

See also EntityUsageTable::getPagesUsing.

hoo added a comment.Oct 30 2017, 12:07 PM

This is currently running into timeouts from time to time:

2017-10-30 08:20:44 [24fef4b5c21833a8949cf7b1] mw1302 ruwiki 1.31.0-wmf.4 DBQuery WARNING: Wikimedia\Rdbms\Database::query: lost connection to 10.64.32.114; reconnected:
#0 /srv/mediawiki/php-1.31.0-wmf.4/includes/libs/rdbms/database/Database.php(1362): Wikimedia\Rdbms\Database->query()
#1 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Usage/Sql/EntityUsageTable.php(309): Wikimedia\Rdbms\Database->select()
#2 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Usage/Sql/SqlUsageTracker.php(296): Wikibase\Client\Usage\Sql\EntityUsageTable->getPagesUsing()
#3 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/AffectedPagesFinder.php(211): Wikibase\Client\Usage\Sql\SqlUsageTracker->getPagesUsing()
#4 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/AffectedPagesFinder.php(115): Wikibase\Client\Changes\AffectedPagesFinder->getAffectedPages()
#5 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/ChangeHandler.php(120): Wikibase\Client\Changes\AffectedPagesFinder->getAffectedUsagesByPage()
#6 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/ChangeHandler.php(103): Wikibase\Client\Changes\ChangeHandler->handleChange()
#7 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/ChangeNotificationJob.php(99): Wikibase\Client\Changes\ChangeHandler->handleChanges()
#8 /srv/mediawiki/php-1.31.0-wmf.4/includes/jobqueue/JobRunner.php(294): Wikibase\Client\ChangeNotificationJob->run()
#9 /srv/mediawiki/php-1.31.0-wmf.4/includes/jobqueue/JobRunner.php(193): JobRunner->executeJob()
#10 /srv/mediawiki/rpc/RunJobs.php(47): JobRunner->run()
#11 {main}

This is probably because we run a limit-less query here which can return a lot of results.