Page MenuHomePhabricator

Netbox add hourly postgres backups
Closed, ResolvedPublic

Description

Current Netbox compressed postgres backup are merely ~1.1MB in size (from netbox-dev, including the imported data). Given the complexity of re-importing data from the CSV dumps, it would be very nice to have hourly DB backups keeping the last 24h.
It's not needed to replicate this into bacula, it's ok to have 1 per day there.

Event Timeline

Volans triaged this task as High priority.Sep 11 2020, 2:05 PM
Volans created this task.
Volans added a subscriber: crusnov.

Change 820463 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Netbox: add hourly postgres backups

https://gerrit.wikimedia.org/r/820463

Change 820463 merged by Ayounsi:

[operations/puppet@production] Netbox: add hourly postgres backups

https://gerrit.wikimedia.org/r/820463

Change 821171 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Netbox-dev add db::dump_interval

https://gerrit.wikimedia.org/r/821171

Change 821171 merged by Ayounsi:

[operations/puppet@production] Netbox-dev add db::dump_interval

https://gerrit.wikimedia.org/r/821171

Change 821173 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Netbox backup: only run on the primary node

https://gerrit.wikimedia.org/r/821173

Change 821175 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Postgres dumps: add hour and minute to filename

https://gerrit.wikimedia.org/r/821175

Change 821175 merged by Ayounsi:

[operations/puppet@production] Postgres dumps: add hour and minute to filename

https://gerrit.wikimedia.org/r/821175

Change 821191 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Netbox: move db::dump_interval for profile default

https://gerrit.wikimedia.org/r/821191

Change 821191 merged by Ayounsi:

[operations/puppet@production] Netbox: move db::dump_interval to profile default

https://gerrit.wikimedia.org/r/821191

Change 821173 merged by Ayounsi:

[operations/puppet@production] Netbox DB dump, hourly on secondary, daily on primary

https://gerrit.wikimedia.org/r/821173

ayounsi claimed this task.
ayounsi added a subscriber: ayounsi.

This is done, hourly on the secondary DB and daily on the primary (as suggested on T310615)
Impact on the DB seems to be fine: https://grafana.wikimedia.org/d/000000469/postgres?orgId=1&var-instance=netboxdb1002&var-instance=netboxdb2002
Confirmed that the dumps do show up as expected.

Change 828005 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Netbox: only keep 2 days of hourly DB dumps

https://gerrit.wikimedia.org/r/828005

Change 828005 merged by Ayounsi:

[operations/puppet@production] Netbox: only keep 2 days of hourly DB dumps

https://gerrit.wikimedia.org/r/828005