Maniphest T380343

Move oozie/util/swift/upload/ out of the refinery oozie folder
Closed, ResolvedPublic2 Estimated Story Points
Actions

Assigned To

Authored By

	• dcausse
	Nov 20 2024, 8:47 AM

Tags

Referenced Files

None

Subscribers

Description

The script swift_upload.py is still being used by search jobs to push data from hadoop to elastic.
It is probably part of the oozie folder for historical reasons but is not related to oozie.
It appears that search is the sole user of this script and thus could possibly be moved to the discolytics project.

AC:

swift_upload.py is no longer in the refinery oozie folder

Event Timeline

• dcausse created this task.Nov 20 2024, 8:47 AM

Restricted Application added a project: Discovery-Search. · View Herald TranscriptNov 20 2024, 8:47 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Gehel triaged this task as Medium priority.Nov 25 2024, 1:16 PM

Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.

+1

Gehel set the point value for this task to 2.Nov 25 2024, 4:37 PM

Gehel moved this task from Incoming to Ready for Dev -- SWE on the Discovery-Search (Current work) board.

dcausse opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/937

search: swift_upload.py moved to refinery/bin/

https://gerrit.wikimedia.org/r/c/analytics/refinery/+/1092879 is moving the script to bin

• dcausse moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Mon, Dec 2, 4:13 PM

Ahoelzl edited projects, added Data-Engineering (Q2 2024 October 1st - December 31th); removed Data-Engineering.Wed, Dec 4, 9:10 PM

Ottomata assigned this task to • dcausse.Wed, Dec 4, 9:10 PM

Gehel closed this task as Resolved.Fri, Dec 6, 1:21 PM