Page MenuHomePhabricator
Create Task
Maniphest T133195

[Epic] All pending producers should write to a single pending queue
Closed, ResolvedPublic2 Estimated Story Points
Actions

Assigned To
None
Authored By
awight
Apr 20 2016, 6:46 PM
Tags
Referenced Files
None
Subscribers
Aklapper
Ejegg
gerritbot
TerraCodes

Description

This will have FIFO behavior, and only one consumer will read from it.

Unfortunately, we might want to use a mirroring approach, where we write to both the old pending thing and the new queue.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Configurable queue mirroringmediawiki/extensions/DonationInterfacemaster+10 -21
Write all limbo queues to new pending queuemediawiki/extensions/DonationInterfacemaster+37 -0
REALLY add order_id to queue messagesmediawiki/extensions/DonationInterfacemaster+3 -0
Kill 'limbo' queuemediawiki/extensions/DonationInterfacedeployment+5 -350
Kill 'limbo' queuemediawiki/extensions/DonationInterfacemaster+7 -9
Add order ID to pending queue messagesmediawiki/extensions/DonationInterfacemaster+3 -2
Customize query in gerrit

Related Objects
Search...

Event Timeline

awight created this task.Apr 20 2016, 6:46 PM
Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptApr 20 2016, 6:46 PM
awight created subtask T133197: Write pending queue consumer and schema.Apr 20 2016, 6:48 PM
awight removed a project: Epic.
awight updated the task description. (Show Details)Apr 20 2016, 6:52 PM
awight removed a subtask: T133197: Write pending queue consumer and schema.
awight moved this task from Unsorted to Needs refactor on the Technical-Debt board.Apr 21 2016, 8:42 PM
• DStrine moved this task from Triage to Current Sprint & Completed in Q4 1516 on the Fundraising-Backlog board.Apr 22 2016, 3:50 PM
awight claimed this task.Apr 25 2016, 9:15 PM
awight moved this task from Backlog to Doing on the Fundraising Sprint Hermit Crab Husbandry board.
• DStrine added a project: Fundraising Sprint Internet Exploring.Apr 27 2016, 10:25 PM
awight removed awight as the assignee of this task.Apr 29 2016, 12:34 AM
• DStrine added a project: Fundraising Sprint Jabberwock Slaying.May 11 2016, 10:22 PM
Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Jabberwock Slaying board.May 13 2016, 5:38 PM
gerritbot subscribed.May 13 2016, 5:43 PM

Change 288439 had a related patch set uploaded (by Ejegg):
Add order ID to pending queue messages

https://gerrit.wikimedia.org/r/288439

gerritbot added a project: Patch-For-Review.May 13 2016, 5:43 PM
gerritbot added a comment.May 25 2016, 9:10 PM

Change 288439 merged by jenkins-bot:
Add order ID to pending queue messages

https://gerrit.wikimedia.org/r/288439

• DStrine added a project: Fundraising Sprint Killing Time.May 25 2016, 10:18 PM
ReleaseTaggerBot added a project: MW-1.28-release (WMF-deploy-2016-06-07_(1.28.0-wmf.5)).May 31 2016, 10:01 PM
• DStrine added a project: Fundraising Sprint Licking Cookies.Jun 8 2016, 10:57 PM
• DStrine added a project: Fundraising Sprint Muggle Baiting.Jul 6 2016, 9:45 PM
awight renamed this task from All pending producers should write to a single pending queue to [Epic] All pending producers should write to a single pending queue.Jul 7 2016, 1:27 AM
Danny_B renamed this task from [Epic] All pending producers should write to a single pending queue to All pending producers should write to a single pending queue.Jul 7 2016, 8:15 PM
Danny_B added a project: Epic.
awight renamed this task from All pending producers should write to a single pending queue to [Epic\ All pending producers should write to a single pending queue.Jul 7 2016, 8:17 PM
awight renamed this task from [Epic\ All pending producers should write to a single pending queue to [Epic] All pending producers should write to a single pending queue.
awight removed the point value 2 for this task.Jul 7 2016, 8:44 PM
• DStrine moved this task from Current Sprint & Completed in Q4 1516 to Current Sprint & Completed in Q1 1617 on the Fundraising-Backlog board.Jul 14 2016, 5:25 PM
awight moved this task from Backlog to Pending consolidation on the FR-ActiveMQ board.Jul 15 2016, 8:29 PM
gerritbot added a comment.Jul 18 2016, 7:43 PM

Change 299608 had a related patch set uploaded (by Awight):
Kill 'limbo' queue

https://gerrit.wikimedia.org/r/299608

gerritbot added a comment.Jul 18 2016, 7:52 PM

Change 299608 merged by jenkins-bot:
Kill 'limbo' queue

https://gerrit.wikimedia.org/r/299608

ReleaseTaggerBot added a project: MW-1.28-release (WMF-deploy-2016-07-19_(1.28.0-wmf.11)).Jul 18 2016, 8:00 PM
gerritbot added a comment.Jul 19 2016, 5:22 PM

Change 299797 had a related patch set uploaded (by Cdentinger):
WIP: Write all limbo queues to new pending queue.

https://gerrit.wikimedia.org/r/299797

gerritbot added a comment.Jul 19 2016, 9:55 PM

Change 299873 had a related patch set uploaded (by Awight):
Kill 'limbo' queue

https://gerrit.wikimedia.org/r/299873

gerritbot added a comment.Jul 19 2016, 10:00 PM

Change 299873 merged by jenkins-bot:
Kill 'limbo' queue

https://gerrit.wikimedia.org/r/299873

• DStrine added a project: Fundraising Sprint Nitpicking.Jul 20 2016, 10:52 PM
gerritbot added a comment.Jul 22 2016, 10:50 PM

Change 300677 had a related patch set uploaded (by Ejegg):
REALLY add order_id to queue messages

https://gerrit.wikimedia.org/r/300677

gerritbot added a comment.Jul 22 2016, 11:13 PM

Change 300677 merged by jenkins-bot:
REALLY add order_id to queue messages

https://gerrit.wikimedia.org/r/300677

ReleaseTaggerBot added a project: MW-1.28-release (WMF-deploy-2016-07-26_(1.28.0-wmf.12)).Jul 23 2016, 12:00 AM
gerritbot added a comment.Jul 25 2016, 6:53 PM

Change 299797 merged by jenkins-bot:
Write all limbo queues to new pending queue

https://gerrit.wikimedia.org/r/299797

awight added a comment.Jul 25 2016, 6:54 PM

Deployment: please set the "queue" key so that the pending-new redis queue has actual name "pending":

$wgDonationInterfaceQueues = array(
    "pending-new" => array(
        "queue" => "pending",
        "expiry" => 3600, // Or a few days.
        "servers" => "tcp://localhost", // TBD
        "type" => "PHPQueue\\Backend\\Predis"
    ), 
);
awight moved this task from Backlog to Doing on the Fundraising Sprint Nitpicking board.Jul 25 2016, 10:32 PM
gerritbot added a comment.Jul 28 2016, 6:11 PM

Change 301656 had a related patch set uploaded (by Ejegg):
Configurable queue mirroring

https://gerrit.wikimedia.org/r/301656

gerritbot added a comment.Jul 28 2016, 10:32 PM

Change 301656 merged by jenkins-bot:
Configurable queue mirroring

https://gerrit.wikimedia.org/r/301656

ReleaseTaggerBot added a project: MW-1.28-release (WMF-deploy-2016-08-02_(1.28.0-wmf.13)).Jul 28 2016, 11:00 PM
Ejegg moved this task from Doing to Pending Deployment on the Fundraising Sprint Nitpicking board.Jul 28 2016, 11:41 PM
Ejegg subscribed.

Deployed, just needs config change to add pending-new queue pointing to predis and this mirroring setup:

$wgDonationInterfaceQueueMirrors = array(
	'pending' => 'pending-new',
	'globalcollect-cc-limbo' => 'pending-new'
);
• DStrine added a project: Fundraising Sprint Octopus Untangling.Aug 3 2016, 10:50 PM
• DStrine added a project: Fundraising Sprint Pretending This Isn't Happening.Aug 17 2016, 11:12 PM
• DStrine added a project: Fundraising Sprint Qwerty Thwacking.Aug 31 2016, 11:10 PM
• DStrine added a project: Fundraising Sprint Rocket Surgery 2016.Sep 13 2016, 11:02 PM
• DStrine added a project: Fundraising Sprint Stirring The Pot.Sep 28 2016, 11:09 PM
Ejegg closed this task as Resolved.Oct 12 2016, 10:34 PM
• DStrine moved this task from Backlog to Done on the Fundraising Sprint Stirring The Pot board.Oct 13 2016, 4:38 PM
• DStrine set the point value for this task to 2.Nov 1 2016, 9:18 PM
• mmodell removed a subscriber: awight.Jun 22 2017, 9:34 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits