Maniphest T133197

Write pending queue consumer and schema
Closed, ResolvedPublic2 Estimated Story Points
Actions

Description

The pending queue will have a single consumer job, which copies the data into a RDBMS, preferably in a new database.

This data will need various indexes, at least:

Transaction ID
Timestamp
Gateway

Details

	Subject	Repo	Branch	Lines +/-
	Consume pending queue to database	wikimedia/fundraising/SmashPig	master	+161 -4
	Pending table schema	wikimedia/fundraising/SmashPig	master	+14 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		None	T117466 Q3 GOALS! (January-March) Keep at top of Q3 column
		Resolved		None	T108229 [Epic] SPOF: Replace ActiveMQ donation queues with a more robust software stack
		Resolved		None	T133108 [Epic] Rewrite all queue clients to use a single shim library, improve library
		Resolved		awight	T133754 Upstream whatever we can to PHP-Queue
		Resolved		awight	T133190 Remove KeyValueStore from PHP-Queue
		Open		None	T134191 Write CiviCRM extension to view pending db records
		Resolved		None	T130897 [Epic] Consolidate "pending" queue usages
		Resolved		Ejegg	T133197 Write pending queue consumer and schema

Event Timeline

awight created this task.Apr 20 2016, 6:48 PM

awight removed a parent task: T133195: [Epic] All pending producers should write to a single pending queue.Apr 20 2016, 6:53 PM

awight added a parent task: T130897: [Epic] Consolidate "pending" queue usages.

awight claimed this task.Apr 20 2016, 8:33 PM

awight moved this task from Backlog to Doing on the Fundraising Sprint Hermit Crab Husbandry board.

awight moved this task from Doing to Backlog on the Fundraising Sprint Hermit Crab Husbandry board.Apr 21 2016, 12:57 AM

awight removed awight as the assignee of this task.Apr 21 2016, 12:59 AM

• DStrine moved this task from Triage to Current Sprint & Completed in Q4 1516 on the Fundraising-Backlog board.Apr 22 2016, 3:50 PM

• DStrine added a project: Fundraising Sprint Internet Exploring.Apr 27 2016, 10:25 PM

Ejegg claimed this task.May 2 2016, 7:01 PM

Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Internet Exploring board.

Change 287042 had a related patch set uploaded (by Ejegg):
WIP pending table schema

https://gerrit.wikimedia.org/r/287042

gerritbot added a project: Patch-For-Review.May 5 2016, 4:40 AM

Change 287147 had a related patch set uploaded (by Ejegg):
WIP pending queue consumer

https://gerrit.wikimedia.org/r/287147

Change 287042 merged by jenkins-bot:
Pending table schema

https://gerrit.wikimedia.org/r/287042

• DStrine added a project: Fundraising Sprint Jabberwock Slaying.May 11 2016, 10:22 PM

Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Jabberwock Slaying board.May 11 2016, 10:28 PM

Ejegg moved this task from Doing to Review on the Fundraising Sprint Jabberwock Slaying board.May 13 2016, 5:38 PM

• DStrine added a project: Fundraising Sprint Killing Time.May 25 2016, 10:18 PM

Ejegg moved this task from Backlog to Review on the Fundraising Sprint Killing Time board.May 26 2016, 8:33 PM

@Ejegg
Should this be linked (and taken from WIP?) https://gerrit.wikimedia.org/r/#/c/288037

• DStrine added a project: Fundraising Sprint Licking Cookies.Jun 8 2016, 10:57 PM

Ejegg moved this task from Backlog to Review on the Fundraising Sprint Licking Cookies board.Jun 8 2016, 11:25 PM

Abandoning the stomp popAtomic - the code was pretty gruesome anyway

Change 287147 merged by jenkins-bot:
Consume pending queue to database

https://gerrit.wikimedia.org/r/287147

Ejegg moved this task from Review to Pending Deployment on the Fundraising Sprint Licking Cookies board.Jun 13 2016, 3:58 PM

• DStrine added a project: Fundraising Sprint Muggle Baiting.Jul 6 2016, 9:45 PM

Ejegg moved this task from Backlog to Pending Deployment on the Fundraising Sprint Muggle Baiting board.Jul 6 2016, 10:34 PM

awight moved this task from Pending Deployment to Done on the Fundraising Sprint Muggle Baiting board.Jul 7 2016, 6:26 PM

• DStrine moved this task from Current Sprint & Completed in Q4 1516 to Current Sprint & Completed in Q1 1617 on the Fundraising-Backlog board.Jul 14 2016, 5:25 PM

Ejegg closed this task as Resolved.Jul 15 2016, 5:40 PM

• mmodell removed a subscriber: awight.Jun 22 2017, 9:50 PM