Maniphest T331971

[dse-k8s] Deploy spark cli to submit jobs on DSE K8S cluster with K8S config
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	• nfraison
	Mar 14 2023, 10:31 AM

Description

DoD:

cli deployed on all stats/airflow/launcher instance
appropriate K8S Config deployed on stats/airflow/launcher instance based on spark-deploy svc account. Should only be accessible for root user and Data Engineering team as it will only be used for testing spark on K8S

Related Objects
Search...

		Status	Subtype	Assigned	Task
		In Progress		None	T318712 Enable spark jobs on the dse-k8s cluster via the spark-operator
		Declined		None	T331971 [dse-k8s] Deploy spark cli to submit jobs on DSE K8S cluster with K8S config

Event Timeline

• nfraison created this task.Mar 14 2023, 10:31 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 14 2023, 10:31 AM

JArguello-WMF moved this task from Backlog to To be discussed on the Shared-Data-Infrastructure board.Mar 14 2023, 3:53 PM

BTullis added a project: Data-Platform-SRE.Jun 9 2023, 11:56 AM

JArguello-WMF removed a project: Shared-Data-Infrastructure.Jun 29 2023, 1:41 PM

BTullis renamed this task from Deploy spak cli to submit jobs on DSE K8S cluster with K8S config to [dse-k8s] Deploy spark cli to submit jobs on DSE K8S cluster with K8S config.Jul 3 2023, 12:04 PM

BTullis triaged this task as Low priority.

BTullis added a parent task: T318712: Enable spark jobs on the dse-k8s cluster via the spark-operator.Jul 3 2023, 12:08 PM

BTullis merged a task: T318923: Add the sparkctl binary to the stat boxes.

BTullis added a subscriber: • EChetty.

Gehel moved this task from Incoming to Misc on the Data-Platform-SRE board.Dec 7 2023, 2:21 PM