Page MenuHomePhabricator

Create a script to ease the Oozie work while enabling kerberos in Hadoop
Closed, ResolvedPublic5 Estimated Story Points

Description

All Oozie coordinators and bundles will need to be restarted to get the new Hive server 2 credentials. Since this job is really tedious and error prone, we could create a script that uses the Oozie APIs to:

  1. get a list of states for all running coordinators and bundles (like coordinator X running hour Y, etc..). In case things will get bad and we'll need to rollback, it will be useful as starting point.
  2. get a list of oozie commands to kill and start oozie jobs (starting from the state outlined in 1)

The ideal goal for this task is to avoid manual work during the procedure, especially mistakes in restarting jobs with incorrect parameters.

Event Timeline

Change 549861 had a related patch set uploaded (by Joal; owner: Joal):
[analytics/refinery@master] [WIP] Add python oozie lib and oozie-dumper script

https://gerrit.wikimedia.org/r/549861

Change 549861 merged by Mforns:
[analytics/refinery@master] Add python oozie lib and oozie-dumper script

https://gerrit.wikimedia.org/r/549861

Nuria set the point value for this task to 5.