Page MenuHomePhabricator

Handle object metadata backups and compare it with stored database object inventory
Open, LowPublic


Aside from backups and backup_files, there is an unmaintained backup_objects table on the zarcillo backups metadata. The idea for such table is to store the link between generated files and original objects from the production databases:

CREATE TABLE `backup_objects` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `backup_id` int(10) unsigned NOT NULL,  -- which backup the object refers to
  `db` varchar(100) CHARACTER SET latin1 NOT NULL, -- which db the object refers to
  `name` varchar(100) CHARACTER SET latin1 DEFAULT NULL, -- which object (table, trigger, etc.) the objects refer to (NULL if it refers to a schema only)
  `size` bigint(20) DEFAULT NULL, -- total size of all compressed files that are part of the backup
  PRIMARY KEY (`id`),
  UNIQUE KEY `backup_id_db_name` (`backup_id`,`db`,`name`)
) ENGINE=InnoDB AUTO_INCREMENT=6352 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci

Because there is not active inventory system for production database server objects (such as table metadata), the backups_object is not filled in. When the database_table inventory is working, we could start filling it in and then setup extra icinga checks to make sure all objects are backed up and have an adequate individual size.

E.g. was the enwiki.revision table backed up? Does it have an expected size? Was the enwikivoyage schema backed up? etc.

Event Timeline

jcrespo triaged this task as Medium priority.Sep 27 2018, 2:43 PM
jcrespo created this task.
jcrespo lowered the priority of this task from Medium to Low.Oct 20 2020, 5:16 PM

Wishlist but not planned at the moment, we need first to work on object inventory- but we will want it eventually to check live data corruption/backup corruption.