Page MenuHomePhabricator

Add a volume for /srv on deployment-mediawiki12.deployment-prep
Closed, ResolvedPublic

Description

Today the beta-scap-sync-world job started failing due to rsync failure on deployment-mediawiki12.deployment-prep.eqiad1.wikimedia.cloud:
rsync: write failed on "/srv/mediawiki/php-master/.gitignore": No space left on device (28). We found a lingering 2.7 GiB /srv/mediawiki/php-master/cache/l10n/upstream/.~tmp~ directory left over from a prior rsync failure but the filesystem filled up again after deleting it.

To deal with this I created a new 20GB volume, attached it to the instance, created an ext4 filesystem on it, and moved the files from /srv onto it.

root@deployment-mediawiki12:/# df -h -t ext4
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        20G  8.8G   11G  47% /
/dev/sdb         20G  7.5G   12G  41% /srv

Event Timeline

dancy claimed this task.
dancy triaged this task as Medium priority.
dancy updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-releng) [2024-05-06T19:01:36Z] <dancy> Rebooting deployment-mediawiki12.deployment-prep to verify that it is reboot safe (T356692)