Page MenuHomePhabricator

User maximilianklein is using 1.2TB out of 8TB available in NFS storage
Closed, ResolvedPublic

Description

/home/maximilianklein is currently using 1.2TB of storage.

Please investigate the possibility of cleaning any files that are not in use and having a file rotation process in place.

42G	/home/maximilianklein/Wikidata-Toolkit-Examples/dumpfiles/wikidatawiki/json-20181008/20181008.json.gz
41G	/home/maximilianklein/Wikidata-Toolkit-Examples/dumpfiles/wikidatawiki/json-20180924/20180924.json.gz
40G	/home/maximilianklein/Wikidata-Toolkit-Examples/dumpfiles/wikidatawiki/json-20180917/20180917.json.gz
40G	/home/maximilianklein/Wikidata-Toolkit-Examples/dumpfiles/wikidatawiki/json-20180910/20180910.json.gz
39G	/home/maximilianklein/Wikidata-Toolkit-Examples/dumpfiles/wikidatawiki/json-20180903/20180903.json.gz
39G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180827/20180827.json.gz
39G	/home/maximilianklein/Wikidata-Toolkit-Examples/dumpfiles/wikidatawiki/json-20180827/20180827.json.gz
38G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180820/20180820.json.gz
38G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180806/20180806.json.gz
37G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180730/20180730.json.gz
37G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180716/20180716.json.gz
36G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180709/20180709.json.gz
35G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180702/20180702.json.gz
35G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180625/20180625.json.gz
34G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180618/20180618.json.gz
34G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180611/20180611.json.gz
34G	/home/maximilianklein/Wikidata-Toolkit-Examples/dumpfiles/wikidatawiki/json-20181022/20181022.json.gz
34G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180604/20180604.json.gz
33G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180528/20180528.json.gz
32G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180521/20180521.json.gz
31G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180514/20180514.json.gz
31G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180507/20180507.json.gz
31G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180430/20180430.json.gz
30G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180423/20180423.json.gz
30G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180416/20180416.json.gz
30G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180409/20180409.json.gz
30G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180402/20180402.json.gz
30G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180326/20180326.json.gz
29G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180319/20180319.json.gz
29G	/home/maximilianklein/Wikidata-Toolkit/wdtk-examples/dumpfiles/wikidatawiki/json-20180312/20180312.json.gz
24G	/home/maximilianklein/Wikidata-Toolkit-Examples/dumpfiles/wikidatawiki/json-20181015/20181015.json.gz
17G	/home/maximilianklein/Wikidata-Toolkit-Examples/dumpfiles/wikidatawiki/json-20181001/20181001.json.gz

We are trying to clean up in the shared NFS storage and would really appreciate your help.

Event Timeline

GTirloni created this task.

Rather than having these in home dirs they should probably just be read from /public/dumps/public/wikidatawiki/entities
I guess they all exist there (unless old dumps are removed from there?)

Implemented a fix to delete all but the last 2 dumps which are all I need,
using, `$ ls -d1tr $WDTKDIR"/dumpfiles/wikidatawiki/json"* | head -n -2 |
xargs -d '\n' rm -rf --`

Make a great day,
Max Klein ‽ http://notconfusing.com/

@Addshore I think that's a great suggestion. It's up to the tool maintainers to adopt them so maybe there's a place in the documentation to make that more evident.

@Maximilianklein Thanks a lot!