Page MenuHomePhabricator
Create Task
Maniphest T122641

Quit looking in pending queue for completion message information
Closed, ResolvedPublic2 Estimated Story Points
Actions

Description

We're processing Amazon messages in a fragile way. We send a message from DonationInterface to the pending queue with donor information and utm data. Then when the IPN message comes in, we send a sparse message to the donations queue with the amount, time, and ID of the message we assume is in the pending queue.

When the donations queue consumer sees a message with a completion_message_id, it tries to find that message in the pending queue. If it doesn't exist, it takes 5 seconds to time out, which can back the queue up like crazy if there are a lot of missing messages in the pending queue.

We should store the pending info someplace where it's a lot easier to look things up by ID and not as expensive when it fails.

Details

SubjectRepoBranchLines +/-
Stop using pending queue in RecordCaptureJobwikimedia/fundraising/SmashPigmaster+24 -27
Stop using pending queue in ProcessCaptureRequestJobwikimedia/fundraising/SmashPigmaster+86 -110
Don't pollute $msg with pending_idwikimedia/fundraising/crmmaster+3 -0
Check for duplicates before re-queueing MISSING_PREDECESSORwikimedia/fundraising/crmmaster+10 -0
Import info from pending DB, not AMQwikimedia/fundraising/crmmaster+200 -75
WIP tests for combining pending DB infowikimedia/fundraising/crmmaster+180 -2
Customize query in gerrit

Related Objects
Search...

Event Timeline

Ejegg created this task.Dec 30 2015, 4:35 AM
Ejegg raised the priority of this task from to High.
Ejegg updated the task description. (Show Details)
Ejegg added projects: Wikimedia-Fundraising-CiviCRM, Fundraising-Backlog.
Ejegg subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 30 2015, 4:35 AM
awight added a project: FR-Amazon.Jan 19 2016, 8:36 PM
awight subscribed.
• DStrine set Security to None.Jan 19 2016, 8:36 PM
• DStrine edited a custom field.
awight added a project: FR-dlocal.Jan 19 2016, 8:37 PM
• DStrine moved this task from Triage to Sprint +3 on the Fundraising-Backlog board.Jan 19 2016, 8:37 PM
• DStrine moved this task from Sprint +3 to Sprint +2 on the Fundraising-Backlog board.Jul 18 2016, 7:28 PM
Ejegg added a parent task: T130897: [Epic] Consolidate "pending" queue usages.Jul 18 2016, 9:46 PM
Ejegg added a parent task: T140484: CRM reads 'completion messages' from pending DB and ActiveMQ, scream if no match.Jul 18 2016, 9:49 PM
Ejegg changed the point value for this task from 4 to 2.
gerritbot subscribed.Jul 20 2016, 4:28 AM

Change 299938 had a related patch set uploaded (by Ejegg):
DO NOT MERGE: import info from pending DB, not AMQ

https://gerrit.wikimedia.org/r/299938

gerritbot added a project: Patch-For-Review.Jul 20 2016, 4:28 AM
Ejegg claimed this task.Jul 20 2016, 8:41 PM
Ejegg added a project: Unplanned-Sprint-Work.
Ejegg added a project: Fundraising Sprint Muggle Baiting.
Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Muggle Baiting board.Jul 20 2016, 11:13 PM
Ejegg edited projects, added Fundraising Sprint Nitpicking; removed Unplanned-Sprint-Work.
Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Nitpicking board.
gerritbot added a comment.Jul 22 2016, 5:52 PM

Change 300584 had a related patch set uploaded (by Ejegg):
WIP tests for combining pending DB info

https://gerrit.wikimedia.org/r/300584

Ejegg added a project: FR-ActiveMQ.Jul 27 2016, 9:30 PM
gerritbot added a comment.Jul 27 2016, 9:31 PM

Change 301313 had a related patch set uploaded (by Ejegg):
DO NOT MERGE: kill SmashPig pending queue code

https://gerrit.wikimedia.org/r/301313

Ejegg moved this task from Backlog to Pending consolidation on the FR-ActiveMQ board.Jul 27 2016, 9:32 PM
Ejegg added a project: Fundraising Sprint Octopus Untangling.Aug 16 2016, 8:48 PM
Ejegg moved this task from Backlog to Review on the Fundraising Sprint Octopus Untangling board.
• DStrine moved this task from Sprint +2 to Sprint +3 on the Fundraising-Backlog board.Aug 17 2016, 4:09 PM
• DStrine moved this task from Sprint +3 to Sprint +1 on the Fundraising-Backlog board.
• DStrine moved this task from Sprint +1 to Current Sprint & Completed in Q1 1617 on the Fundraising-Backlog board.
• DStrine added a project: Fundraising Sprint Pretending This Isn't Happening.Aug 17 2016, 11:12 PM
Ejegg moved this task from Backlog to Review on the Fundraising Sprint Pretending This Isn't Happening board.Aug 17 2016, 11:25 PM
gerritbot added a comment.Aug 18 2016, 8:40 PM

Change 305577 had a related patch set uploaded (by Ejegg):
Check for duplicates before re-queueing MISSING_PREDECESSOR

https://gerrit.wikimedia.org/r/305577

gerritbot added a comment.Aug 23 2016, 4:31 PM

Change 300584 abandoned by Ejegg:
WIP tests for combining pending DB info

Reason:
Folded in with I29477a5319180d54

https://gerrit.wikimedia.org/r/300584

gerritbot added a comment.Aug 29 2016, 7:55 PM

Change 307359 had a related patch set uploaded (by Ejegg):
Stop using pending queue in ProcessCaptureRequestJob

https://gerrit.wikimedia.org/r/307359

gerritbot added a comment.Aug 29 2016, 8:18 PM

Change 299938 merged by jenkins-bot:
Import info from pending DB, not AMQ

https://gerrit.wikimedia.org/r/299938

gerritbot added a comment.Aug 29 2016, 9:55 PM

Change 305577 merged by Ejegg:
Check for duplicates before re-queueing MISSING_PREDECESSOR

https://gerrit.wikimedia.org/r/305577

gerritbot added a comment.Aug 29 2016, 10:58 PM

Change 307445 had a related patch set uploaded (by Ejegg):
Don't pollute $msg with pending_id

https://gerrit.wikimedia.org/r/307445

Ejegg moved this task from Review to Done on the Fundraising Sprint Pretending This Isn't Happening board.Aug 29 2016, 10:58 PM
gerritbot added a comment.Aug 30 2016, 12:11 AM

Change 307445 merged by jenkins-bot:
Don't pollute $msg with pending_id

https://gerrit.wikimedia.org/r/307445

gerritbot added a comment.Aug 30 2016, 9:15 PM

Change 307359 merged by jenkins-bot:
Stop using pending queue in ProcessCaptureRequestJob

https://gerrit.wikimedia.org/r/307359

gerritbot added a comment.Aug 30 2016, 9:15 PM

Change 301313 merged by jenkins-bot:
Stop using pending queue in RecordCaptureJob

https://gerrit.wikimedia.org/r/301313

• DStrine added a project: Fundraising Sprint Qwerty Thwacking.Aug 31 2016, 11:10 PM
Ejegg moved this task from Backlog to Done on the Fundraising Sprint Qwerty Thwacking board.Sep 2 2016, 5:20 PM
Ejegg closed this task as Resolved.Sep 13 2016, 10:14 PM
• mmodell removed a subscriber: awight.Jun 22 2017, 9:44 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL