Needed to be able to iterate over all pages that use a certain entity id (which can be several hundred thousand) and do something with them in multiple consecutive jobs that schedule each other.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T107722 [Task] Make client ChangeHandler scale by batching updates | |||
| Open | None | T146838 Allow seeking in UsageLookup::getPagesUsing results |
Event Timeline
Comment Actions
This is currently running into timeouts from time to time:
2017-10-30 08:20:44 [24fef4b5c21833a8949cf7b1] mw1302 ruwiki 1.31.0-wmf.4 DBQuery WARNING: Wikimedia\Rdbms\Database::query: lost connection to 10.64.32.114; reconnected:
#0 /srv/mediawiki/php-1.31.0-wmf.4/includes/libs/rdbms/database/Database.php(1362): Wikimedia\Rdbms\Database->query()
#1 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Usage/Sql/EntityUsageTable.php(309): Wikimedia\Rdbms\Database->select()
#2 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Usage/Sql/SqlUsageTracker.php(296): Wikibase\Client\Usage\Sql\EntityUsageTable->getPagesUsing()
#3 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/AffectedPagesFinder.php(211): Wikibase\Client\Usage\Sql\SqlUsageTracker->getPagesUsing()
#4 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/AffectedPagesFinder.php(115): Wikibase\Client\Changes\AffectedPagesFinder->getAffectedPages()
#5 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/ChangeHandler.php(120): Wikibase\Client\Changes\AffectedPagesFinder->getAffectedUsagesByPage()
#6 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/Changes/ChangeHandler.php(103): Wikibase\Client\Changes\ChangeHandler->handleChange()
#7 /srv/mediawiki/php-1.31.0-wmf.4/extensions/Wikidata/extensions/Wikibase/client/includes/ChangeNotificationJob.php(99): Wikibase\Client\Changes\ChangeHandler->handleChanges()
#8 /srv/mediawiki/php-1.31.0-wmf.4/includes/jobqueue/JobRunner.php(294): Wikibase\Client\ChangeNotificationJob->run()
#9 /srv/mediawiki/php-1.31.0-wmf.4/includes/jobqueue/JobRunner.php(193): JobRunner->executeJob()
#10 /srv/mediawiki/rpc/RunJobs.php(47): JobRunner->run()
#11 {main}This is probably because we run a limit-less query here which can return a lot of results.