Page MenuHomePhabricator

Quit looking in pending queue for completion message information
Closed, ResolvedPublic2 Estimated Story Points

Description

We're processing Amazon messages in a fragile way. We send a message from DonationInterface to the pending queue with donor information and utm data. Then when the IPN message comes in, we send a sparse message to the donations queue with the amount, time, and ID of the message we assume is in the pending queue.

When the donations queue consumer sees a message with a completion_message_id, it tries to find that message in the pending queue. If it doesn't exist, it takes 5 seconds to time out, which can back the queue up like crazy if there are a lot of missing messages in the pending queue.

We should store the pending info someplace where it's a lot easier to look things up by ID and not as expensive when it fails.

Event Timeline

Ejegg raised the priority of this task from to High.
Ejegg updated the task description. (Show Details)
Ejegg added a subscriber: Ejegg.
DStrine edited a custom field.

Change 299938 had a related patch set uploaded (by Ejegg):
DO NOT MERGE: import info from pending DB, not AMQ

https://gerrit.wikimedia.org/r/299938

Change 300584 had a related patch set uploaded (by Ejegg):
WIP tests for combining pending DB info

https://gerrit.wikimedia.org/r/300584

Change 301313 had a related patch set uploaded (by Ejegg):
DO NOT MERGE: kill SmashPig pending queue code

https://gerrit.wikimedia.org/r/301313

Change 305577 had a related patch set uploaded (by Ejegg):
Check for duplicates before re-queueing MISSING_PREDECESSOR

https://gerrit.wikimedia.org/r/305577

Change 300584 abandoned by Ejegg:
WIP tests for combining pending DB info

Reason:
Folded in with I29477a5319180d54

https://gerrit.wikimedia.org/r/300584

Change 307359 had a related patch set uploaded (by Ejegg):
Stop using pending queue in ProcessCaptureRequestJob

https://gerrit.wikimedia.org/r/307359

Change 299938 merged by jenkins-bot:
Import info from pending DB, not AMQ

https://gerrit.wikimedia.org/r/299938

Change 305577 merged by Ejegg:
Check for duplicates before re-queueing MISSING_PREDECESSOR

https://gerrit.wikimedia.org/r/305577

Change 307445 had a related patch set uploaded (by Ejegg):
Don't pollute $msg with pending_id

https://gerrit.wikimedia.org/r/307445

Change 307445 merged by jenkins-bot:
Don't pollute $msg with pending_id

https://gerrit.wikimedia.org/r/307445

Change 307359 merged by jenkins-bot:
Stop using pending queue in ProcessCaptureRequestJob

https://gerrit.wikimedia.org/r/307359

Change 301313 merged by jenkins-bot:
Stop using pending queue in RecordCaptureJob

https://gerrit.wikimedia.org/r/301313