[Event Platform] Flink Operations
Open, Needs TriagePublic
Actions

Assigned To

Authored By

	lbowmaker
	Feb 1 2023, 2:51 PM

Description

NOTE: This tasks need to be groomed, the purpose of this task is to capture all of the aspects a platform team needs to consider when operating a Flink application . Please add any subtasks for areas the need consideration.

User Story

As an event platform engineer, I need to consider and document the different aspects of operating a Flink application (for example: how to handle restarts, upgrades, etc)

Done is:

Subtasks complete

Related Objects
Search...

Status	Assigned	Task
Open	None	T317045 [Epic] Re-architect the Search Update Pipeline
Open	None	T340548 [EPIC] Deployment of the Search Update Pipeline on Flink / k8s
Open	lbowmaker	T328561 [Event Platform] Flink Operations
Resolved	gmodena	T328563 [Flink Operations] How to handle restarting a Flink application
Open	None	T328565 [Flink Operations] Automate Replay of Failed Events
Resolved	gmodena	T328569 [Flink Operation] How to handle app upgrades
Resolved	bking	T344614 Add Zookeeper config to 'rdf-streaming-updater' test service on DSE cluster
Resolved	bking	T345957 Restore dse-k8s' rdf-streaming-updater from savepoint/improve bootstrapping process
Resolved	bking	T346048 Troubleshoot rdf-streaming-updater/dse-k8s cluster

Event Timeline

lbowmaker created this task.Feb 1 2023, 2:51 PM

Restricted Application added a project: Data-Engineering. · View Herald TranscriptFeb 1 2023, 2:51 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

dcausse awarded a token.Feb 1 2023, 3:30 PM

dcausse added subscribers: dcausse, bking.

dcausse mentioned this in T326409: Migrate the wdqs streaming updater flink jobs to flink-k8s-operator deployment model.Feb 2 2023, 3:07 PM

• EChetty edited projects, added Data-Engineering-Planning; removed Data-Engineering.Feb 10 2023, 12:38 PM

• EChetty moved this task from Backlog to Event Platform on the Data-Engineering-Planning board.Feb 10 2023, 12:44 PM

lbowmaker moved this task from Backlog to Parent Tasks/Epics on the Event-Platform board.Mar 6 2023, 2:14 PM

JArguello-WMF closed subtask T328569: [Flink Operation] How to handle app upgrades as Resolved.Mar 13 2023, 2:12 PM

JArguello-WMF closed subtask T328563: [Flink Operations] How to handle restarting a Flink application as Resolved.

Gehel added a project: Discovery-Search (Current work).Jun 26 2023, 3:17 PM

Gehel moved this task from Incoming to Blocked/Waiting on the Discovery-Search (Current work) board.

Gehel added a project: Data-Platform-SRE.Jun 27 2023, 12:55 PM

Gehel moved this task from Incoming to Blocked / Waiting on the Data-Platform-SRE board.

Gehel added a parent task: T340548: [EPIC] Deployment of the Search Update Pipeline on Flink / k8s.Jun 27 2023, 1:25 PM

JArguello-WMF removed a project: Data-Engineering-Planning.Jun 29 2023, 9:47 PM

Restricted Application added a project: Data-Engineering. · View Herald TranscriptJun 29 2023, 9:47 PM

JArguello-WMF moved this task from Incoming (new tickets) to Event Platform Backlog on the Data-Engineering board.Jun 29 2023, 10:30 PM

JArguello-WMF removed a project: Data-Platform-SRE.Jun 29 2023, 10:56 PM

JArguello-WMF edited projects, added Data Engineering and Event Platform Team; removed Data-Engineering.Jun 30 2023, 4:15 PM

Restricted Application added a project: Data-Engineering. · View Herald TranscriptJun 30 2023, 4:15 PM

JArguello-WMF moved this task from Data Eng Backlog to Parent Tasks/Epics on the Data Engineering and Event Platform Team board.Jun 30 2023, 4:17 PM

JArguello-WMF removed a project: Data-Engineering.Jun 30 2023, 4:18 PM

Restricted Application added a project: Data-Engineering. · View Herald TranscriptJun 30 2023, 4:18 PM

Gehel removed a project: Discovery-Search (Current work).Jul 17 2023, 3:16 PM

pfischer subscribed.Jul 17 2023, 3:17 PM

bking mentioned this in T342149: Test common operations in the flink operator/k8s/Flink ZK environment.Jul 18 2023, 5:44 PM

bking added a subtask: T344614: Add Zookeeper config to 'rdf-streaming-updater' test service on DSE cluster.Aug 22 2023, 2:05 PM

bking mentioned this in T317045: [Epic] Re-architect the Search Update Pipeline.Sep 6 2023, 2:17 PM

Gehel closed subtask T344614: Add Zookeeper config to 'rdf-streaming-updater' test service on DSE cluster as Resolved.Sep 20 2023, 1:43 PM

lbowmaker moved this task from Parent Tasks/Epics to Event Platform Maintenance (current quarter) on the Data Engineering and Event Platform Team board.Oct 20 2023, 2:43 PM

Ahoelzl renamed this task from Flink Operations to [Event Platform] Flink Operations.Oct 20 2023, 5:31 PM

lbowmaker removed a project: Data Engineering and Event Platform Team.Nov 10 2023, 1:40 PM

[Event Platform] Flink OperationsOpen, Needs TriagePublicActions