Maniphest T196009

Support timestamp based consumption in KafkaSSE and EventStreams
Closed, ResolvedPublic8 Estimated Story Points
Actions

Assigned To

Authored By

	Ottomata
	May 30 2018, 7:50 PM

Description

https://github.com/Blizzard/node-rdkafka/issues/336 is resolved, so we can do it!

Details

	Subject	Repo	Branch	Lines +/-
	Update deploy repo for multi stream subscription and since timestamp	mediawiki/services/eventstreams/deploy	master	+139 K -82 K
	Support timestamp based historical stream assignment via `since` query param	mediawiki/services/eventstreams	master	+76 -21

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	Ottomata	T196009 Support timestamp based consumption in KafkaSSE and EventStreams
Resolved	Ottomata	T196407 EventBus should produce messages to Kafka with event time set to meta.dt
Resolved	Ottomata	T196409 Consider increasing retention for mediawiki event topics

Event Timeline

Ottomata created this task.May 30 2018, 7:50 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 30 2018, 7:50 PM

Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.May 30 2018, 7:50 PM

Ottomata triaged this task as Low priority.May 30 2018, 8:03 PM

Ottomata added a subscriber: • Pchelolo.

• Nuria raised the priority of this task from Low to High.May 31 2018, 4:47 PM

• Nuria moved this task from Incoming to Smart Tools for Better Data on the Analytics board.

Ottomata moved this task from In Progress to In Code Review on the Analytics-Kanban board.Jun 1 2018, 3:46 PM

Ottomata mentioned this in rWKSE25935735308c: Support timestamp based consumption via Last-Event-ID header.Jun 4 2018, 4:33 PM

Change 437360 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[mediawiki/services/eventstreams@master] Support timestamp based historical stream assignment

https://gerrit.wikimedia.org/r/437360

gerritbot added a project: Patch-For-Review.Jun 4 2018, 8:56 PM

• fdans closed subtask T196407: EventBus should produce messages to Kafka with event time set to meta.dt as Resolved.Jun 11 2018, 4:10 PM

Change 437360 merged by Ottomata:
[mediawiki/services/eventstreams@master] Support timestamp based historical stream assignment via since query param

https://gerrit.wikimedia.org/r/437360

Change 439638 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[mediawiki/services/eventstreams/deploy@master] Update deploy repo for multi stream subscription and since timestamp

https://gerrit.wikimedia.org/r/439638

Change 439638 merged by Ottomata:
[mediawiki/services/eventstreams/deploy@master] Update deploy repo for multi stream subscription and since timestamp

https://gerrit.wikimedia.org/r/439638

Mentioned in SAL (#wikimedia-operations) [2018-06-11T20:17:47Z] <otto@deploy1001> Started deploy [eventstreams/deploy@6b013f9]: Enable composite stream and timestamp since param - T196009 , T187418

Stashbot mentioned this in T187418: Enable multiple topics in EventStreams URL.Jun 11 2018, 8:17 PM

Mentioned in SAL (#wikimedia-operations) [2018-06-11T20:27:38Z] <otto@deploy1001> Finished deploy [eventstreams/deploy@6b013f9]: Enable composite stream and timestamp since param - T196009 , T187418 (duration: 09m 52s)