Page MenuHomePhabricator

Rewrite/modify $wgJobBackoffThrottling handling code to be less pessimistic
Closed, DeclinedPublic

Description

$wgJobBackoffThrottling bails out workers if the throttle is reached (which quickly happens after spawning a worker, esp. with 20 default workers on WMF). The problem is that another worker for that queue might not respawn for a while (relatively). The cycle then repeats. So the de facto rate is far lower than the nominal throttle rate.

This would probably help with the 5 million sized job queue, mostly wikidata (and some commons/enwiki) htmlcacheupdate jobs, which use $wgJobBackoffThrottling.

Event Timeline

aaron raised the priority of this task from to Needs Triage.
aaron updated the task description. (Show Details)
aaron added a project: MediaWiki-Core-JobQueue.
aaron subscribed.
aaron renamed this task from Rewrite $wgJobBackoffThrottling handling code to be less pessimistic to Rewrite/modify $wgJobBackoffThrottling handling code to be less pessimistic.Jan 21 2016, 1:35 AM
aaron claimed this task.

Change 265438 had a related patch set uploaded (by Ori.livneh):
Job runners: Add a dedicated htmlCacheUpdate runner

https://gerrit.wikimedia.org/r/265438

Change 265438 merged by Ori.livneh:
Job runners: Add a dedicated htmlCacheUpdate runner

https://gerrit.wikimedia.org/r/265438

ori triaged this task as Low priority.Jan 25 2016, 7:36 PM
ori subscribed.
aaron removed aaron as the assignee of this task.Feb 24 2016, 1:00 AM

This should be left off to the next jobqueue rewrite.