Page MenuHomePhabricator

Job queue throws "RedisException: read error on connection" (Redis exception on server)
Closed, ResolvedPublic

Description

Redis exception on server "rdb1001.eqiad.wmnet"

Sub exception:

RedisException: read error on connection

{  
  "function":"checkConnection",
  "class":"Redis",
  "type":"->",
  "args":[  ]
},
{  
  "function":"sockReadLine",
  "class":"Redis",
  "type":"->",
  "args":[  ]
},
{  
  "function":"sockReadData",
  "class":"Redis",
  "type":"->",
  "args":[   null ]
},
{  
  "function":"processLongResponse",
  "class":"Redis",
  "type":"->",
  "args":[   ]
},
{  
  "function":"flushCallbacks",
  "class":"Redis",
  "type":"->",
  "args":[  ]
},
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/aggregator/JobQueueAggregatorRedis.php:66,
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/aggregator/JobQueueAggregator.php:76,
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/JobQueue.php:362,
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/JobQueueGroup.php:192,
/srv/mediawiki/php-1.27.0-wmf.6/includes/jobqueue/JobRunner.php:154,
/srv/mediawiki/rpc/RunJobs.php:42

Happend about 10,000 times per minute after 1.27.0-wmf.6 was rolled out. Resolved by reverting f1adea609aa6a97c2bf0a59ad496f21508975882.

Event Timeline

Krinkle created this task.Nov 12 2015, 12:40 AM
Krinkle updated the task description. (Show Details)
Krinkle raised the priority of this task from to Needs Triage.
Krinkle added a subscriber: Krinkle.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 12 2015, 12:40 AM

Change 252587 had a related patch set uploaded (by Krinkle):
Revert "Avoid doNotifyQueueEmpty() race conditions for Redis"

https://gerrit.wikimedia.org/r/252587

Change 252587 merged by jenkins-bot:
Revert "Avoid doNotifyQueueEmpty() race conditions for Redis"

https://gerrit.wikimedia.org/r/252587

Krinkle closed this task as Resolved.Nov 17 2015, 2:04 AM
Krinkle claimed this task.