Page MenuHomePhabricator

Enable purging of old jobs and coordinators in oozie
Closed, ResolvedPublic5 Estimated Story Points

Description

Using oozie.service.PurgeService.* settings, oozie should clean out old data. This will help prune the WF_JOBS table which is now almost 40GB.

We should set oozie.service.PurgeService.purge.old.coord.action to true, and audit the other setting's default values (we probably want to keep completed workflow jobs around for more than 30 days).

Event Timeline

Milimetric moved this task from Incoming to Analytics Query Service on the Analytics board.

Change 274827 had a related patch set uploaded (by Ottomata):
Set oozie.service.PurgeService.purge.old.coord.action to true

https://gerrit.wikimedia.org/r/274827

Change 274827 merged by Ottomata:
Set oozie.service.PurgeService.purge.old.coord.action to true and parameterize purge_jobs_older_than_days

https://gerrit.wikimedia.org/r/274827

Change 274829 had a related patch set uploaded (by Ottomata):
Update cdh submodule with oozie purge change

https://gerrit.wikimedia.org/r/274829

Change 274829 merged by Ottomata:
Update cdh submodule with oozie purge change

https://gerrit.wikimedia.org/r/274829

Milimetric set the point value for this task to 5.Mar 10 2016, 5:14 PM