Page MenuHomePhabricator

Number of requested IDs (15052) is too high
Closed, ResolvedPublic

Description

Saw this in Kibana. Looks like this job purges a potentially large number of pages in one go. I wonder if this floods too many UDP packets and if it should do batching with some sleep calls.

If we move to a kakfa purge relayer, the flood concern would at least be lessened. I'm also wondering why the UID limit was in place there (which I might remove).

{"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/utils/UIDGenerator.php","line":344,"function":"getSequentialPerNodeIDs","class":"UIDGenerator","type":"->","args":["string","integer","integer","integer"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/CdnCacheUpdate.php","line":182,"function":"newSequentialPerNodeIDs","class":"UIDGenerator","type":"::","args":["string","integer","integer","integer"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/CdnCacheUpdate.php","line":112,"function":"HTCPPurge","class":"CdnCacheUpdate","type":"::","args":["array"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/CdnCacheUpdate.php","line":79,"function":"purge","class":"CdnCacheUpdate","type":"::","args":["array"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/DeferredUpdates.php","line":151,"function":"doUpdate","class":"CdnCacheUpdate","type":"->","args":[]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/deferred/DeferredUpdates.php","line":84,"function":"execute","class":"DeferredUpdates","type":"::","args":["array","string"]}, {"file":"/srv/mediawiki/php-1.27.0-wmf.9/includes/jobqueue/JobRunner.php","line":184,"function":"doUpdates","class":"DeferredUpdates","type":"::","args":[]}, {"file":"/srv/mediawiki/rpc/RunJobs.php","line":47,"function":"run","class":"JobRunner","type":"->","args":["array"]}
/rpc/RunJobs.php?wiki=ruwiki&type=ChangeNotification&maxtime=60&maxmem=300M

Event Timeline

aaron created this task.Dec 21 2015, 7:12 PM
aaron raised the priority of this task from to Needs Triage.
aaron updated the task description. (Show Details)
aaron added a project: Wikidata.
aaron added a subscriber: aaron.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptDec 21 2015, 7:12 PM

Change 260416 had a related patch set uploaded (by Aaron Schulz):
Remove arbitrary newSequentialPerNodeIDs() $count limit

https://gerrit.wikimedia.org/r/260416

Change 260416 merged by jenkins-bot:
Remove arbitrary newSequentialPerNodeIDs() $count limit

https://gerrit.wikimedia.org/r/260416

hoo added a subscriber: hoo.Dec 27 2015, 10:05 AM

This sounds like it would also be fixed with T107722, wouldn't it?

aaron closed this task as Resolved.Apr 12 2016, 11:04 PM
aaron claimed this task.

This sounds like it would also be fixed with T107722, wouldn't it?

It would help.