Page MenuHomePhabricator

Create a script to ease the Oozie work while enabling kerberos in Hadoop
Closed, ResolvedPublic5 Estimated Story Points

Description

All Oozie coordinators and bundles will need to be restarted to get the new Hive server 2 credentials. Since this job is really tedious and error prone, we could create a script that uses the Oozie APIs to:

  1. get a list of states for all running coordinators and bundles (like coordinator X running hour Y, etc..). In case things will get bad and we'll need to rollback, it will be useful as starting point.
  2. get a list of oozie commands to kill and start oozie jobs (starting from the state outlined in 1)

The ideal goal for this task is to avoid manual work during the procedure, especially mistakes in restarting jobs with incorrect parameters.

Event Timeline

fdans assigned this task to JAllemandou.Nov 4 2019, 4:41 PM
fdans moved this task from Incoming to Operational Excellence on the Analytics board.

Change 549861 had a related patch set uploaded (by Joal; owner: Joal):
[analytics/refinery@master] [WIP] Add python oozie lib and oozie-dumper script

https://gerrit.wikimedia.org/r/549861

mforns moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Nov 21 2019, 4:00 PM
Nuria moved this task from Done to Ready to Deploy on the Analytics-Kanban board.Nov 22 2019, 5:05 PM

Change 549861 merged by Mforns:
[analytics/refinery@master] Add python oozie lib and oozie-dumper script

https://gerrit.wikimedia.org/r/549861

Nuria closed this task as Resolved.Dec 20 2019, 5:28 PM
Nuria set the point value for this task to 5.