Page MenuHomePhabricator

2020-08-26: tools NFS share cleanup
Closed, ResolvedPublic

Description

Once again, in the tradition of T156982: Cleanup tools nfs share on labstore1004/5 and T247315: 2020-03-10: tools and misc nfs share cleanup, we must clean up the NFS to prevent collapse

This time, just /srv/tools

/dev/drbd4      8.0T  6.9T  745G  91% /srv/tools
/dev/drbd3      5.0T  1.9T  2.9T  40% /srv/misc

Projects using the most storage in grafana

Event Timeline

Bstorm triaged this task as High priority.
Bstorm created this task.
Bstorm moved this task from Inbox to Clinic Duty on the cloud-services-team (Kanban) board.

Mentioned in SAL (#wikimedia-cloud) [2020-08-26T17:12:48Z] <bstorm> Running 'ionice -c 3 nice -19 find /srv/tools -type f -size +100M -printf "%k KB %p\n" > tools_large_files_20200826.txt' on labstore1004 T261336

cat tools_large_files_20200826.txt | sort -h > tools_large_files-sorted_20200826.txt

Top individual files are:

49303704 KB /srv/tools/shared/tools/project/splinetools/dumps/enwiki-20141106-pages-articles.xml
51288456 KB /srv/tools/shared/tools/project/persondata/dumps/dnb-all_lds.rdf
63967176 KB /srv/tools/shared/tools/project/magnus-toolserver/error.log
64778744 KB /srv/tools/shared/tools/project/wikidata-analysis/public_html_tmp/dumpfiles/json-20191125/20191125.json.gz
77725816 KB /srv/tools/shared/tools/project/persondata/logs/person_bkl2.out
85264568 KB /srv/tools/shared/tools/project/.shared/dumps/20200720.json.gz
85736004 KB /srv/tools/shared/tools/project/.shared/dumps/20200803.json.gz
86201908 KB /srv/tools/shared/tools/project/.shared/dumps/20200817.json.gz
101160884 KB /srv/tools/shared/tools/project/robokobot/virgule.err
410869488 KB /srv/tools/shared/tools/project/persondata/logs/wikidata_sitelinks.out

Truncated /srv/tools/shared/tools/project/robokobot/virgule.err

Andrew claimed this task.
Andrew added a subscriber: Andrew.

/dev/drbd4 8.0T 5.5T 2.1T 73% /srv/tools
/dev/drbd3 5.0T 2.0T 2.9T 41% /srv/misc

Seems ok for now