Page MenuHomePhabricator

Delete HDFS raw *_camus directories 60 days after July 12 (after 2021-09-10)
Closed, ResolvedPublic

Description

During the camus -> gobblin migration, @JAllemandou and I moved previously camus imported raw data into several /wmf/data/raw/*_camus directories. These are no longer operated on by any purging scripts. We could delete them now, but for safety, let's wait until all of their data would have been purged, 60 days after the latest migration we did.

After September 10th would suffice.

Event Timeline

odimitrijevic moved this task from Incoming to Smart Tools for Better Data on the Analytics board.
JAllemandou added a project: Analytics-Kanban.

I deleted the old files with Dan today, assuming that 30 days of waiting was enough (this is the time we keep raw webrequest data). Moving this task to done and closing after standup.