Page MenuHomePhabricator

back up data used to crunch numbers
Closed, ResolvedPublic

Event Timeline

Milimetric raised the priority of this task from to High.
Milimetric updated the task description. (Show Details)
Milimetric added a project: Analytics-Kanban.
Milimetric added a subscriber: Milimetric.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 25 2015, 10:02 PM
ggellerman moved this task from Next Up to In Progress on the Analytics-Kanban board.
ggellerman set Security to None.
ggellerman removed a subscriber: Milimetric.
Milimetric reassigned this task from Milimetric to Ottomata.Aug 27 2015, 3:53 PM
Ottomata added a comment.EditedAug 28 2015, 3:36 PM

We got 27th and the 30th backed up, but the 26th had already begun to be deleted by the time we started backing up. :/

hdfs@stat1002:/mnt/hdfs/wmf/data/archive$ hdfs dfs -du -s -h /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=*/year=2015/month=6/day=*
51.2 M  153.5 M  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=misc/year=2015/month=6/day=26
151.2 M  453.5 M  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=misc/year=2015/month=6/day=27
159.9 M  479.6 M  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=misc/year=2015/month=6/day=30
23.4 G  70.2 G  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=mobile/year=2015/month=6/day=26
77.0 G  231.0 G  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=mobile/year=2015/month=6/day=27
70.9 G  212.8 G  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=mobile/year=2015/month=6/day=30
358.7 G  1.1 T  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=text/year=2015/month=6/day=26
503.1 G  1.5 T  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=text/year=2015/month=6/day=27
558.4 G  1.6 T  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=text/year=2015/month=6/day=30
485.5 G  1.4 T  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=upload/year=2015/month=6/day=26
495.9 G  1.5 T  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=upload/year=2015/month=6/day=27
540.7 G  1.6 T  /wmf/data/archive/backup/wmf-vs-nsa/webrequest/webrequest_source=upload/year=2015/month=6/day=30
Ottomata moved this task from In Progress to Done on the Analytics-Kanban board.Aug 28 2015, 3:37 PM