We have been filling storage on the tools NFS project share as of late, and it has grown 3% in less than 48 hours. We really need some historical perspective to track this over time.
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| labstore: Diamond collector to track directory sizes | operations/puppet | production | +156 -0 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • Bstorm | T126083 overhaul labstore setup [tracking] | |||
| Resolved | • madhuvishy | T126623 track growth of project and home directories |
Event Timeline
Comment Actions
Historical data is available in graphite.wm.o as servers.labstore1001.diskspace._srv_project_tools.byte_used:
The current growth rate is ~200GB/24h:
However, we probably want to know this on a user level. Could we enable quota on the NFS server to track per-user disk usage (but without actually enforcing quotas immediately)?
Comment Actions
Change 335855 had a related patch set uploaded (by Madhuvishy):
labstore: Diamond collector to track tools home and project dir sizes
Comment Actions
Change 335855 had a related patch set uploaded (by Madhuvishy):
labstore: Diamond collector to track directory sizes
Comment Actions
Change 335855 merged by Madhuvishy:
labstore: Diamond collector to track directory sizes
Comment Actions
Graphite now logs tools and misc NFS share directory sizes at labstore.labstore-secondary.DirectorySizeCollector on prod. Grafana dashboard based on this data - https://grafana.wikimedia.org/dashboard/db/labstore-nfs-directory-sizes