From P5240 when running the redisJobRunnerService with --verbose:
2017-04-11T14:18:34+0000 DEBUG: Redis cmd: hGetAll ["jobqueue:aggregator:h-ready-queues:v2"] 2017-04-11T14:18:34+0000 DEBUG: No jobs available... 2017-04-11T14:18:34+0000 DEBUG: No jobs available... 2017-04-11T14:18:34+0000 DEBUG: No jobs available... 2017-04-11T14:18:35+0000 DEBUG: No jobs available... 2017-04-11T14:18:35+0000 DEBUG: No jobs available... 2017-04-11T14:18:35+0000 DEBUG: No jobs available... 2017-04-11T14:18:35+0000 DEBUG: No jobs available... 2017-04-11T14:18:35+0000 DEBUG: No jobs available... 2017-04-11T14:18:35+0000 DEBUG: No jobs available... 2017-04-11T14:18:35+0000 DEBUG: No jobs available... 2017-04-11T14:18:35+0000 DEBUG: Redis cmd: hGetAll ["jobqueue:aggregator:h-ready-queues:v2"]
It shows we retrieve jobs every second but the loop runs every 100ms.
The loop with a 100ms sleep:
$pending =& $this->getReadyQueueMap(); if ( !count( $pending ) ) { $this->debug( "No jobs available..." ); $this->incrStats( "idle.$host", 1 ); usleep( 100000 ); // no jobs continue; }
getReadyQueueMap() result is cached for 1 second.
class RedisJobRunnerService extends RedisJobService { const AGGR_CACHE_TTL_SEC = 1; /** * @return array Cached map of (job type => domain => UNIX timestamp) */ private function &getReadyQueueMap() { if ( $age <= self::AGGR_CACHE_TTL_SEC ) { return $pendingDBs; // process cache hit }
Originally we we waited five seconds between iteration and that got lowered to 100ms but the cache hasn't been updated. Commit 19c880d1d62197a52fa66b09b03f4863a5fe5605
So I guess we should just sleep for a second, or even better check whether redis has a way for us to wait for new events.