Page MenuHomePhabricator

Job queue throws "RedisException: read error on connection" (Redis exception on server)
Closed, ResolvedPublic

Description

Redis exception on server "rdb1001.eqiad.wmnet"

Sub exception:

RedisException: read error on connection

{  
  "function":"checkConnection",
  "class":"Redis",
  "type":"->",
  "args":[  ]
},
{  
  "function":"sockReadLine",
  "class":"Redis",
  "type":"->",
  "args":[  ]
},
{  
  "function":"sockReadData",
  "class":"Redis",
  "type":"->",
  "args":[   null ]
},
{  
  "function":"processLongResponse",
  "class":"Redis",
  "type":"->",
  "args":[   ]
},
{  
  "function":"flushCallbacks",
  "class":"Redis",
  "type":"->",
  "args":[  ]
},
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/aggregator/JobQueueAggregatorRedis.php:66,
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/aggregator/JobQueueAggregator.php:76,
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/JobQueue.php:362,
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/JobQueueGroup.php:192,
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/JobRunner.php:154,
/srv/mediawiki/rpc/RunJobs.php:42

Happend about 10,000 times per minute after 1.27.0-wmf.6 was rolled out. Resolved by reverting rMWf1adea609aa6.

Event Timeline

Krinkle raised the priority of this task from to Needs Triage.
Krinkle updated the task description. (Show Details)
Krinkle subscribed.

Change 252587 had a related patch set uploaded (by Krinkle):
Revert "Avoid doNotifyQueueEmpty() race conditions for Redis"

https://gerrit.wikimedia.org/r/252587

Change 252587 merged by jenkins-bot:
Revert "Avoid doNotifyQueueEmpty() race conditions for Redis"

https://gerrit.wikimedia.org/r/252587

Krinkle claimed this task.