Monitoring alerts about deployment-tin disk space:
$ df -h / /srv Filesystem Size Used Avail Use% Mounted on /dev/vda3 19G 14G 4.0G 78% / /dev/mapper/vd-second--local--disk 40G 29G 9.6G 75% /srv
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | thcipriani | T166492 deployment-tin has disk space issues | |||
| Invalid | None | T167723 Create a new instance flavor for deployment-prep |
Event Timeline
Comment Actions
Mentioned in SAL (#wikimedia-releng) [2017-05-29T09:51:44Z] <hashar> deployment-tin: rm /var/lib/l10nupdate/caches/cache-master/*.json T166492
Comment Actions
@thcipriani you have clones of mediawiki repositories on deployment-tin in /home/thcipriani/mwclonetest. I guess that can be safely deleted.
Comment Actions
Mentioned in SAL (#wikimedia-releng) [2017-05-29T10:06:40Z] <hashar> deployment-tin rm -fR /usr/src/hhvm T166492
Comment Actions
/ looks better now, thank you!
$ df -h / /srv Filesystem Size Used Avail Use% Mounted on /dev/vda3 19G 12G 6.3G 65% / /dev/mapper/vd-second--local--disk 40G 29G 9.4G 76% /srv
Next candidate is /srv/mediawiki/.git which is 8 GBytes. I am not sure why the .git directory ends up being synced to the deployment target. Most of that is taken by a 7.0G pack file:
$ ls -lh /srv/mediawiki/.git/objects/pack/pack-09e0a9019c69f9efcfda2463a1a582a6e7a454b8.pack vvvv -r--r--r-- 1 mwdeploy mwdeploy 7.0G ^^^^ May 29 13:24 /srv/mediawiki/.git/objects/pack/pack-09e0a9019c69f9efcfda2463a1a582a6e7a454b8.pack
Comment Actions
Spoke with Tyler about the 7.0G, that is a git repository that contains all the other repos. Meant for scap3 usage.
Lets get an instance with more disk.
Comment Actions
We would need an OpenStack flavor with a larger disk. Filled as T167723: Create a new instance flavor for deployment-prep
Comment Actions
Hit this last week as well, just deleted /srv/mw/.git
Not a dire need, but we should rebuild -tin with a larger disk in the not too distant future.
Comment Actions
- created instance deployment-deploy-01 with stretch and flavor x-large (T192561)
- deployment-deploy1001 has been deleted by thcipriani
Comment Actions
deployment-tin does not exist anymore and deployment-deploy01 and 02 have 60GB disk space each.