Page MenuHomePhabricator

🟦️ Add validation of backups to wbaas-backup
Closed, ResolvedPublic

Description

We want to validate the backups that have been taken.

This requires adding some logic to the scripts in https://github.com/wmde/wbaas-backup.

We can check file existence before taring and encrypting. We can also check replica state with additional sql commands

The backup should be no older than 60s compared to the state of the data in the primary database.

A/C:

  • Check apidb file exists
  • Check seconds behind master in SHOW REPLICA STATUS; and be sure it is less than 60 (can be done either before or after the back up is taken)
  • Fail with a status code >0 and log an error to the console if either of the above is true

n.b. currently we cat the .metadata file which is a rough check: https://github.com/wmde/wbaas-backup/blob/main/src/backup.sh#L17

Event Timeline

toan removed toan as the assignee of this task.Apr 21 2022, 8:38 AM
toan added a subscriber: toan.
toan removed toan as the assignee of this task.Apr 21 2022, 2:36 PM
toan updated the task description. (Show Details)
dang removed dang as the assignee of this task.May 3 2022, 1:37 PM
dang added a subscriber: dang.
toan removed Rosalie_WMDE as the assignee of this task.
toan added a subscriber: Rosalie_WMDE.

This still very much wants to get reviewed.

merged https://github.com/wmde/wbaas-deploy/pull/249

and executed on staging and production primary sql:

GRANT SLAVE MONITOR ON *.* TO 'backup-manager'@'%';
FLUSH PRIVILEGES;
WMDE-leszek renamed this task from Add validation of backups to wbaas-backup to 🟦️ Add validation of backups to wbaas-backup.May 17 2022, 7:17 PM
Tarrow claimed this task.