Page MenuHomePhabricator

Add HdfsArchiver job to hdfs-tools
Closed, ResolvedPublic1 Estimated Story Points


Move HdfsArchiver job to hdfs-tools repo on github.

  • Add dependency on refinery-tools
  • Add (non provided) scala dependencies to hdfs-tools.

Event Timeline

There were 2 fix made in this repo:

  • There was a bug with the hdfs-tools HdfsRsync job. The condition to update the Lastmodification time of the Destination file was wrong.
  • The HdfsRsync test were failing. To resolve this a thread delay was added to the job.
EChetty set the point value for this task to 1.Sep 6 2022, 3:23 PM
EChetty moved this task from Discussed (Radar) to Sprint 01 on the Data Pipelines board.
EChetty edited projects, added Data Pipelines (Sprint 01); removed Data Pipelines.