Both cloudweb2001-dev.wikimedia.org-Monthly-1st-Mon-production-a-backup and labweb1001.wikimedia.org-Monthly-1st-Sat-production-a-backup are failing due to being setup to backup /a/backup, which are either non-existing or practically empty on those hosts. Please review backup configuration to make sure backups are running on the right directory so file are safe. Most likely this is either a puppet missconfiguration or those hosts are being setup up or being decommisioned. Please let us know how to proceed.
|Resolved||jcrespo||T229209 Strengthen backup infrastructure and support|
|Open||None||T247045 Migrate all of production metal and VMs to Buster or later|
|Resolved||jcrespo||T236406 Switchover backup director service from helium to backup1001|
|Resolved||Andrew||T237237 cloudweb2001-dev & labweb1001 backing up non-existing or empty directories|
I hope this is the last hidden reference to /a!
All of that moved to /srv long ago; I'm attaching a patch but I don't know much about bacula config so let me know if there are other pieces I need to update.
This will be fixed soon- I will resolve when I have confirmation the first backup ran correctly. Thanks for the quick patch and response!
@Andrew I have an additional question: I see there is 2 configured jobs, one on labweb1001.wikimedia.org-Monthly-1st-Sat-production-cloudweb-srv-backup and another on labweb1002.wikimedia.org-Monthly-1st-Wed-production-cloudweb-srv-backup (in addition, of course, to cloudweb2001-dev) this is not a big deal if the dataset is small. Also an expected occurrence if you are in the middle of replacing one server with the other. However, if these are redundant servers, please consider enabling backups only on 1? Please clarify, although that is independent of this task.