Page MenuHomePhabricator

Tools and Misc NFS weekly backups failing since 1/17
Closed, ResolvedPublic

Description

The backup crons have been failing with errors:

Traceback (most recent call last):
  File "/usr/local/sbin/block_sync", line 125, in <module>
    lock_file = open('/var/lock/{}_{}_backup.lock'.format(args.r_vg, args.r_lv, 'w+'))
FileNotFoundError: [Errno 2] No such file or directory: '/var/lock/misc_misc-project_backup.lock'

Event Timeline

Change 407460 had a related patch set uploaded (by Madhuvishy; owner: Madhuvishy):
[operations/puppet@production] block_sync: Fix lock file open with w mode

https://gerrit.wikimedia.org/r/407460

Change 407460 merged by Madhuvishy:
[operations/puppet@production] block_sync: Fix lock file open with w mode

https://gerrit.wikimedia.org/r/407460

Fixed with https://gerrit.wikimedia.org/r/407460. I'm running manual backup jobs now for both shares on screen. Will close after confirming that the scheduled crons run successfully next week.

Fixed with https://gerrit.wikimedia.org/r/407460. I'm running manual backup jobs now for both shares on screen. Will close after confirming that the scheduled crons run successfully next week.

Did this all work out? :)

@chasemp The crons are scheduled for tomorrow and day after :) My manual backups got done fine.

@madhuvishy seems settled now?

ping @madhuvishy to confirm and close :)

Yup looks good, backups have been running fine for the last 2 weeks.