Page MenuHomePhabricator

Number of requested IDs (15052) is too high
Closed, ResolvedPublic

Description

Saw this in Kibana. Looks like this job purges a potentially large number of pages in one go. I wonder if this floods too many UDP packets and if it should do batching with some sleep calls.

If we move to a kakfa purge relayer, the flood concern would at least be lessened. I'm also wondering why the UID limit was in place there (which I might remove).

{"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/utils/UIDGenerator.php","line":344,"function":"getSequentialPerNodeIDs","class":"UIDGenerator","type":"->","args":["string","integer","integer","integer"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/CdnCacheUpdate.php","line":182,"function":"newSequentialPerNodeIDs","class":"UIDGenerator","type":"::","args":["string","integer","integer","integer"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/CdnCacheUpdate.php","line":112,"function":"HTCPPurge","class":"CdnCacheUpdate","type":"::","args":["array"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/CdnCacheUpdate.php","line":79,"function":"purge","class":"CdnCacheUpdate","type":"::","args":["array"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/DeferredUpdates.php","line":151,"function":"doUpdate","class":"CdnCacheUpdate","type":"->","args":[]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/DeferredUpdates.php","line":84,"function":"execute","class":"DeferredUpdates","type":"::","args":["array","string"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/jobqueue/JobRunner.php","line":184,"function":"doUpdates","class":"DeferredUpdates","type":"::","args":[]}, {"file":"/srv/mediawiki/rpc/RunJobs.php","line":47,"function":"run","class":"JobRunner","type":"->","args":["array"]}
/rpc/RunJobs.php?wiki=ruwiki&type=ChangeNotification&maxtime=60&maxmem=300M

Event Timeline

aaron raised the priority of this task from to Needs Triage.
aaron updated the task description. (Show Details)
aaron added a project: Wikidata.
aaron subscribed.

Change 260416 had a related patch set uploaded (by Aaron Schulz):
Remove arbitrary newSequentialPerNodeIDs() $count limit

https://gerrit.wikimedia.org/r/260416

Change 260416 merged by jenkins-bot:
Remove arbitrary newSequentialPerNodeIDs() $count limit

https://gerrit.wikimedia.org/r/260416

This sounds like it would also be fixed with T107722, wouldn't it?

aaron claimed this task.

This sounds like it would also be fixed with T107722, wouldn't it?

It would help.