Page MenuHomePhabricator

Backup Gerrit with LVM snapshots and Bacula pre/post job hooks
Open, MediumPublic

Description

Context

In T430775#12105890 @hashar raised the consistency of hot backups: the backup walks /srv/gerrit while Gerrit is writing. Analysis on the parent task narrowed the real exposure down to time skew across files in a multi-hour job (crash-consistent, not point-in-time, accurate = no), with one plausible loss scenario around refs being packed mid-cycle.

In T430775#12177927 @jcrespo endorsed snapshotting: create and free an LVM snapshot from Bacula pre and post job hooks. Incrementals take minutes now, so the snapshot lifetime and COW cost stay small. This task implements that.

A single snapshot of the srv LV covers /srv/gerrit/git and /srv/gerrit/data atomically, which gives every backup a true point-in-time view and closes the consistency question for good. This also feeds the tiering/snapshot follow-up left open from T411583: Gerrit backups are growing.

What is already in place

  • The gerrit hosts (gerrit[1-2]00[1-4]) install with partman standard.cfg + raid1-2dev.cfg: RAID1, one VG (vg0), /srv as an LV, guided_size 80%, so about 20% of the VG is unallocated by design. Free extents for the snapshot COW should be there; confirm with vgs on each host in case LVs were extended after provisioning.
  • Bacula pre-job hooks have in-house precedent: backup::mysqlset (predump method) and backup::openldapset ship ClientRunBeforeJob scripts, and backup::set / bacula::client::job already accept extras.
  • The fileset is gerrit-repo-data (modules/profile/manifests/backup/filesets.pp), an allowlist of /srv/gerrit/git and /srv/gerrit/data, scheduled Hourly-Tue-ReposEqiad (full Tue 02:05, hourly incrementals) from modules/profile/manifests/gerrit.pp.

Proposed shape

  • Pre-job (ClientRunBeforeJob): lvcreate -s on vg0/srv, mount the snapshot read-only at a stable path (e.g. /mnt/gerrit-backup-snap). Include a guard that removes any stale snapshot left by a previous failed job before creating the new one.
  • Post-job (ClientRunAfterJob, and the failure variant so cleanup also runs on aborted jobs): unmount and lvremove.
  • Point the gerrit-repo-data fileset at the snapshot mountpoint instead of /srv/gerrit. Restores are unaffected in practice since restore jobs choose their own where= target, but document the path change.
  • Bacula fails the job if ClientRunBeforeJob fails, which is the behavior we want: no silent fallback to backing up the live tree.

Open questions

  • Snapshot COW size: fixed size vs all free extents. It only has to absorb writes landing during the job, minutes for incrementals, longer for the Tuesday full.
  • Monitoring: alert if a gerrit-backup snapshot exists outside a backup window, or if COW usage crosses a threshold (a full COW invalidates the snapshot and the running job with it).
  • Whether the same pattern should be applied to the home set on the same hosts (probably not: /home lives on the root LV and is low-churn).

Acceptance criteria