Page MenuHomePhabricator
Create Task
Maniphest T155150

Investigate Redis connection issues in GC audit parser
Closed, ResolvedPublic1 Estimated Story Points
Actions

Description

The GlobalCollect audit parser has been failing with Redis connection problems.

php: Predis\Connection\ConnectionException: Error while reading line from the server. [tcp://blah:6379] in
Predis\Connection\AbstractConnection->onConnectionError()

Not seeing this error from any other part of our infrastructure. Is there something unique about how the audit parser talks to Redis?

Details

SubjectRepoBranchLines +/-
Reconnect to Redis on idle timeoutwikimedia/fundraising/php-queuemaster+41 -0
Always reconnect before sending queue messageswikimedia/fundraising/crmmaster+1 -1
Customize query in gerrit

Event Timeline

Ejegg created this task.Jan 12 2017, 1:49 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 12 2017, 1:49 AM
• ggellerman triaged this task as High priority.Jan 17 2017, 10:07 PM
• ggellerman added a project: Spike.
• ggellerman set the point value for this task to 1.
• ggellerman moved this task from Triage to Sprint +1 on the Fundraising-Backlog board.
Ejegg claimed this task.Jan 20 2017, 8:47 PM
Ejegg added a project: Fundraising Sprint Baudelaire Bowdlerizer.
Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Baudelaire Bowdlerizer board.
gerritbot subscribed.Jan 20 2017, 11:00 PM

Change 333308 had a related patch set uploaded (by Ejegg):
Always reconnect before sending queue messages

https://gerrit.wikimedia.org/r/333308

gerritbot added a project: Patch-For-Review.Jan 20 2017, 11:00 PM
Ejegg moved this task from Doing to Review on the Fundraising Sprint Baudelaire Bowdlerizer board.Jan 20 2017, 11:01 PM
gerritbot added a comment.Jan 20 2017, 11:20 PM

Change 333308 merged by jenkins-bot:
Always reconnect before sending queue messages

https://gerrit.wikimedia.org/r/333308

Ejegg moved this task from Review to Pending Deployment on the Fundraising Sprint Baudelaire Bowdlerizer board.Jan 20 2017, 11:56 PM
Ejegg moved this task from Sprint +1 to Current Sprint and Completed in Q3 2016-17 on the Fundraising-Backlog board.Jan 21 2017, 6:57 AM
gerritbot added a comment.Jan 23 2017, 10:03 PM

Change 333793 had a related patch set uploaded (by Ejegg):
Reconnect to Redis on idle timeout

https://gerrit.wikimedia.org/r/333793

Ejegg moved this task from Pending Deployment to Done on the Fundraising Sprint Baudelaire Bowdlerizer board.Jan 25 2017, 10:38 PM
Ejegg closed this task as Resolved.Feb 1 2017, 8:02 PM

Seems to be working, no recurrence since deployment

gerritbot added a comment.Apr 20 2017, 12:53 AM

Change 333793 abandoned by Ejegg:
Reconnect to Redis on idle timeout

https://gerrit.wikimedia.org/r/333793

• DStrine added a project: Unplanned-Sprint-Work.Sep 27 2017, 10:46 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL