Today the beta-scap-sync-world job started failing due to rsync failure on deployment-mediawiki12.deployment-prep.eqiad1.wikimedia.cloud:
rsync: write failed on "/srv/mediawiki/php-master/.gitignore": No space left on device (28). We found a lingering 2.7 GiB /srv/mediawiki/php-master/cache/l10n/upstream/.~tmp~ directory left over from a prior rsync failure but the filesystem filled up again after deleting it.
To deal with this I created a new 20GB volume, attached it to the instance, created an ext4 filesystem on it, and moved the files from /srv onto it.
root@deployment-mediawiki12:/# df -h -t ext4 Filesystem Size Used Avail Use% Mounted on /dev/sda1 20G 8.8G 11G 47% / /dev/sdb 20G 7.5G 12G 41% /srv