Page MenuHomePhabricator

Make sure both `reviewdb-test` (used forgerrit upgrade testing) and `reviewdb` (formerly production) databases get torn down
Closed, ResolvedPublic

Description

In T254516 a database was requested to test gerrit upgrades with. Thanks again for the database.

The tests are done and reviewdb-test database is no longer needed. It can be deleted again.

(Not sure if a separate ticket is needed to get it torn down again, but wanted to err on the safe side)

Event Timeline

QChris created this task.Jun 17 2020, 8:50 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 17 2020, 8:50 PM
Marostegui moved this task from Triage to Next on the DBA board.
Marostegui added a subscriber: Marostegui.

Thanks for the new ticket.
As Jaime handled the initial DB and grants ticket, I am going to assign this to him as he has probably all the context still fresh in his mind :)

Change 606387 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] Revert "gerrit: add parameter for db_name, let gerrit1002 use test db"

https://gerrit.wikimedia.org/r/606387

jcrespo added a subscriber: Dzahn.EditedJun 18 2020, 8:17 AM

Waiting to proceed on @Dzahn's ok +private repo revert: https://gerrit.wikimedia.org/r/c/operations/puppet/+/606387

Dzahn added a comment.Jun 18 2020, 3:17 PM

Since we won't need a mysql/mariadb db anymore for Gerrit after the upgrade is complete, this ticket might as well be used to drop the actual prod "reviewdb" as well. Or we can make another ticket for that if you prefer.

It is all ok, let's rename the task to state so and wait for the upgrade + some time to act on it.

jcrespo changed the task status from Open to Stalled.Jun 18 2020, 3:40 PM

Change 606549 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] gerrit: remove all database parameters / support

https://gerrit.wikimedia.org/r/606549

Dzahn renamed this task from Make sure `reviewdb-test` database (used forgerrit upgrade testing) gets torn down to Make sure both `reviewdb-test` (used forgerrit upgrade testing) and `reviewdb` (formerly production) databases get torn down.Jun 19 2020, 7:18 PM

@Dzahn @QChris Can this start happening? Database backups will be available for some time already, but I wonder if the live databases can be deleted (they will have to be recovered from backups, anyway).

Dzahn added a comment.Jul 1 2020, 3:02 PM

@jcrespo Chris has asked to keep it for a couple more days.

Perfect, no problem.

Change 606549 merged by Dzahn:
[operations/puppet@production] gerrit: remove all database parameters / support

https://gerrit.wikimedia.org/r/c/operations/puppet/ /606549

Change 606387 abandoned by Dzahn:
[operations/puppet@production] Revert "gerrit: add parameter for db_name, let gerrit1002 use test db"

Reason:
done in https://gerrit.wikimedia.org/r/c/operations/puppet/ /606549

https://gerrit.wikimedia.org/r/606387

Change 609241 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[labs/private@master] remove gerrit db_pass, it does not use mysql anymore

https://gerrit.wikimedia.org/r/609241

Change 609241 merged by Dzahn:
[labs/private@master] remove gerrit db_pass, it does not use mysql anymore

https://gerrit.wikimedia.org/r/609241

Dzahn added a comment.Jul 2 2020, 10:06 PM

db_pass was removed from private hieradata, from private passwords module, from labs/private..

The parameters for db_ were all removed from the puppet classes, gerrit service has been restarted on all prod servers.

So this definitely does not use reviewdb and review-db test anymore now.

I went over the possible scenarios with @Dzahn.

How long do we keep DB backups?

If we can still keep the backups for a bit (say ... 1 month?), then I think we won't need the actual databases after the weekend any longer.

Does removing them need sign-off from releng? (I'm not sure, so I'll leave that for WMF employees to decide)

Does removing them need sign-off from releng?

cc: @thcipriani

If we can still keep the backups for a bit (say ... 1 month?),

We keep database backups of m2 currently for a bit over a month automatically.

QChris added a comment.Jul 3 2020, 9:02 AM

We keep database backups of m2 currently for a bit over a month automatically.

Perfect! Thanks.

Change 609884 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] mariadb: remove ferm firewall hole for gerrit servers

https://gerrit.wikimedia.org/r/609884

QChris added a comment.Jul 7 2020, 6:53 AM

[...] then I think we won't need the actual databases after the weekend any longer.

It's ok from my point of view to remove them now.

Does removing them need sign-off from releng? (I'm not sure, so I'll leave that for WMF employees to decide)

@thcipriani : Is it ok to remove Gerrit's now-no-longer-used reviewdb database, or do you still want to run some checks against it?

I would suggest we start by renaming the tables first, to make sure nothing really breaks.
Let me know when I can go ahead and do so.

Marostegui changed the task status from Stalled to Open.Jul 9 2020, 5:34 AM
Dzahn added a comment.Jul 9 2020, 5:57 AM

I would suggest we start by renaming the tables first, to make sure nothing really breaks.
Let me know when I can go ahead and do so.

Feel free to go ahead. We can close the firewall hole first per https://gerrit.wikimedia.org/r/c/operations/puppet/+/609884/ and then remove the grants or the other way around. And we can skip renaming tables, imho.

There seem to be two parallel conversations, one here and one on the patch :-)
I don't mind either way really, whatever is easier for you. If you want to close the FW holes, that's also good from my side. I +1ed the patch, so feel free to merge and deploy

Change 609884 merged by Dzahn:
[operations/puppet@production] mariadb: remove ferm firewall hole for gerrit servers

https://gerrit.wikimedia.org/r/609884

Change 611963 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] production-m2.sql: Remove gerrit related grants

https://gerrit.wikimedia.org/r/611963

Mentioned in SAL (#wikimedia-operations) [2020-07-13T06:16:19Z] <marostegui> Reverse gerrit password on m2 master - T255715

Marostegui moved this task from Next to In progress on the DBA board.Jul 13 2020, 7:14 AM

Change 611963 merged by Marostegui:
[operations/puppet@production] production-m2.sql: Remove gerrit related grants

https://gerrit.wikimedia.org/r/611963

Mentioned in SAL (#wikimedia-operations) [2020-07-14T07:04:30Z] <marostegui> Drop gerrit, gerritro, gerrittest users from m2 databases - T255715

@jcrespo once you have the last dump from these two databases done and saved, please assign the task back to me so I can schedule the databases deletion.

I have removed the grants from puppet and from production:

root@db1132.eqiad.wmnet[(none)]> select user,host from mysql.user where user like 'gerrit%';
Empty set (0.001 sec)

root@db1132.eqiad.wmnet[(none)]>

reviewdb is just backed up, but otrs backup on the same instance has yet to finish to make the backup fully consistent.

reviewdb is just backed up, but otrs backup on the same instance has yet to finish to make the backup fully consistent.

No rush, just assign the task back to me once you're fully done from your side

Bacula is running now.

jcrespo reassigned this task from jcrespo to Marostegui.Jul 14 2020, 2:05 PM
jcrespo added a subscriber: jcrespo.
db backup metadata
root@db2093.codfw.wmnet[zarcillo]> select file_name, size, file_date FROM backup_files where backup_id=6759 and file_name like 'reviewdb.%';
+--------------------------------------------------------+-----------+---------------------+
| file_name                                              | size      | file_date           |
+--------------------------------------------------------+-----------+---------------------+
| reviewdb.accounts-schema.sql.gz                        |       309 | 2020-07-14 03:06:44 |
| reviewdb.accounts.sql.gz                               |    120475 | 2020-07-14 03:05:53 |
| reviewdb.account_external_ids-schema.sql.gz            |       267 | 2020-07-14 03:06:44 |
| reviewdb.account_external_ids.sql.gz                   |    182528 | 2020-07-14 03:05:52 |
| reviewdb.account_groups-schema.sql.gz                  |       313 | 2020-07-14 03:06:44 |
| reviewdb.account_groups.sql.gz                         |     95643 | 2020-07-14 03:05:52 |
| reviewdb.account_group_by_id-schema.sql.gz             |       213 | 2020-07-14 03:06:44 |
| reviewdb.account_group_by_id.sql.gz                    |      2873 | 2020-07-14 03:05:52 |
| reviewdb.account_group_by_id_aud-schema.sql.gz         |       280 | 2020-07-14 03:06:44 |
| reviewdb.account_group_by_id_aud.sql.gz                |      6894 | 2020-07-14 03:05:52 |
| reviewdb.account_group_id-schema.sql.gz                |       207 | 2020-07-14 03:06:44 |
| reviewdb.account_group_id.sql.gz                       |      3821 | 2020-07-14 03:05:52 |
| reviewdb.account_group_members-schema.sql.gz           |       217 | 2020-07-14 03:06:44 |
| reviewdb.account_group_members.sql.gz                  |     10195 | 2020-07-14 03:05:52 |
| reviewdb.account_group_members_audit-schema.sql.gz     |       267 | 2020-07-14 03:06:44 |
| reviewdb.account_group_members_audit.sql.gz            |    138689 | 2020-07-14 03:05:52 |
| reviewdb.account_group_names-schema.sql.gz             |       211 | 2020-07-14 03:06:44 |
| reviewdb.account_group_names.sql.gz                    |     17107 | 2020-07-14 03:05:52 |
| reviewdb.account_id-schema.sql.gz                      |       202 | 2020-07-14 03:06:44 |
| reviewdb.account_id.sql.gz                             |     13141 | 2020-07-14 03:05:53 |
| reviewdb.bak_accounts-schema.sql.gz                    |       312 | 2020-07-14 03:06:44 |
| reviewdb.bak_accounts.sql.gz                           |    120478 | 2020-07-14 03:05:53 |
| reviewdb.bak_account_external_ids-schema.sql.gz        |       272 | 2020-07-14 03:06:44 |
| reviewdb.bak_account_external_ids.sql.gz               |    182532 | 2020-07-14 03:05:53 |
| reviewdb.bak_account_groups-schema.sql.gz              |       316 | 2020-07-14 03:06:44 |
| reviewdb.bak_account_groups.sql.gz                     |     95646 | 2020-07-14 03:05:53 |
| reviewdb.bak_account_group_by_id-schema.sql.gz         |       217 | 2020-07-14 03:06:44 |
| reviewdb.bak_account_group_by_id.sql.gz                |      2877 | 2020-07-14 03:05:53 |
| reviewdb.bak_account_group_by_id_aud-schema.sql.gz     |       284 | 2020-07-14 03:06:44 |
| reviewdb.bak_account_group_by_id_aud.sql.gz            |      6898 | 2020-07-14 03:05:53 |
| reviewdb.bak_account_group_id-schema.sql.gz            |       210 | 2020-07-14 03:06:44 |
| reviewdb.bak_account_group_id.sql.gz                   |      3825 | 2020-07-14 03:05:53 |
| reviewdb.bak_account_group_members-schema.sql.gz       |       222 | 2020-07-14 03:06:44 |
| reviewdb.bak_account_group_members.sql.gz              |     10202 | 2020-07-14 03:05:53 |
| reviewdb.bak_account_group_members_audit-schema.sql.gz |       271 | 2020-07-14 03:06:44 |
| reviewdb.bak_account_group_members_audit.sql.gz        |    138703 | 2020-07-14 03:05:53 |
| reviewdb.bak_account_group_names-schema.sql.gz         |       216 | 2020-07-14 03:06:44 |
| reviewdb.bak_account_group_names.sql.gz                |     17111 | 2020-07-14 03:05:53 |
| reviewdb.bak_account_id-schema.sql.gz                  |       206 | 2020-07-14 03:06:44 |
| reviewdb.bak_account_id.sql.gz                         |     13148 | 2020-07-14 03:05:53 |
| reviewdb.bak_changes-schema.sql.gz                     |       481 | 2020-07-14 03:06:44 |
| reviewdb.bak_changes.sql.gz                            |  30860272 | 2020-07-14 03:06:16 |
| reviewdb.bak_change_id-schema.sql.gz                   |       207 | 2020-07-14 03:06:44 |
| reviewdb.bak_change_id.sql.gz                          |    962494 | 2020-07-14 03:05:53 |
| reviewdb.bak_change_messages-schema.sql.gz             |       358 | 2020-07-14 03:06:44 |
| reviewdb.bak_change_messages.sql.gz                    | 120350002 | 2020-07-14 03:06:56 |
| reviewdb.bak_patch_comments-schema.sql.gz              |       448 | 2020-07-14 03:06:44 |
| reviewdb.bak_patch_comments.sql.gz                     |  22838083 | 2020-07-14 03:06:22 |
| reviewdb.bak_patch_sets-schema.sql.gz                  |       364 | 2020-07-14 03:06:44 |
| reviewdb.bak_patch_sets.sql.gz                         |  30303350 | 2020-07-14 03:06:28 |
| reviewdb.bak_patch_set_approvals-schema.sql.gz         |       354 | 2020-07-14 03:06:44 |
| reviewdb.bak_patch_set_approvals.sql.gz                |  17277891 | 2020-07-14 03:06:27 |
| reviewdb.bak_schema_version-schema.sql.gz              |       213 | 2020-07-14 03:06:44 |
| reviewdb.bak_schema_version.sql.gz                     |       155 | 2020-07-14 03:06:27 |
| reviewdb.bak_system_config-schema.sql.gz               |       334 | 2020-07-14 03:06:44 |
| reviewdb.bak_system_config.sql.gz                      |       192 | 2020-07-14 03:06:27 |
| reviewdb.changes-schema.sql.gz                         |       476 | 2020-07-14 03:06:44 |
| reviewdb.changes.sql.gz                                |  30859760 | 2020-07-14 03:06:36 |
| reviewdb.change_id-schema.sql.gz                       |       203 | 2020-07-14 03:06:44 |
| reviewdb.change_id.sql.gz                              |    962465 | 2020-07-14 03:06:27 |
| reviewdb.change_messages-schema.sql.gz                 |       354 | 2020-07-14 03:06:44 |
| reviewdb.change_messages.sql.gz                        | 120344717 | 2020-07-14 03:07:09 |
| reviewdb.patch_comments-schema.sql.gz                  |       444 | 2020-07-14 03:06:44 |
| reviewdb.patch_comments.sql.gz                         |  22837801 | 2020-07-14 03:06:36 |
| reviewdb.patch_sets-schema.sql.gz                      |       361 | 2020-07-14 03:06:45 |
| reviewdb.patch_sets.sql.gz                             |  30302851 | 2020-07-14 03:06:43 |
| reviewdb.patch_set_approvals-schema.sql.gz             |       351 | 2020-07-14 03:06:44 |
| reviewdb.patch_set_approvals.sql.gz                    |  17276788 | 2020-07-14 03:06:46 |
| reviewdb.schema_version-schema.sql.gz                  |       211 | 2020-07-14 03:06:45 |
| reviewdb.schema_version.sql.gz                         |       151 | 2020-07-14 03:06:43 |
| reviewdb.system_config-schema.sql.gz                   |       331 | 2020-07-14 03:06:45 |
| reviewdb.system_config.sql.gz                          |       189 | 2020-07-14 03:06:43 |
+--------------------------------------------------------+-----------+---------------------+
72 rows in set (0.037 sec)
check_bacula.py dbprov2001.codfw.wmnet-Monthly-1st-Wed-Databases-mysql-srv-backups-dumps-latest
...
2020-07-14 11:20:03: type: I, status: T, bytes: 919486035088
bacula metadata
root@backup1001:~$ echo "list files jobid=245901" | bconsole | grep 'reviewdb.'     
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb-schema-create.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_external_ids-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_by_id-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_by_id_aud-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_id-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_members-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_members_audit-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_names-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_groups-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_id-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.accounts-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_external_ids-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_by_id-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_by_id_aud-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_id-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_members-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_members_audit-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_names-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_groups-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_id-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_accounts-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_change_id-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_change_messages-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_changes-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_patch_comments-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_patch_set_approvals-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_patch_sets-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_schema_version-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_system_config-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.change_id-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.change_messages-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.changes-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.patch_comments-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.patch_set_approvals-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.patch_sets-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.schema_version-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.system_config-schema.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_external_ids.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_by_id.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_by_id_aud.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_id.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_members.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_members_audit.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_group_names.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_groups.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.account_id.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.accounts.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_external_ids.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_by_id.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_by_id_aud.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_id.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_members.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_members_audit.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_group_names.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_groups.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_account_id.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_accounts.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_change_id.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_change_messages.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_changes.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_patch_comments.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_patch_set_approvals.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_patch_sets.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_schema_version.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.bak_system_config.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.change_id.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.change_messages.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.changes.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.patch_comments.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.patch_set_approvals.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.patch_sets.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.schema_version.sql.gz |
| /srv/backups/dumps/latest/dump.m2.2020-07-14--03-05-42/reviewdb.system_config.sql.gz |

At the moment I can guarantee this being kept for around a month. More long term archival would require a different process.

jcrespo added a comment.EditedJul 14 2020, 2:06 PM

Last note: after db drop, grants for 'dump' users should be dropped too, so no backups are attempted of that db.

reviewdbtest was not being backed up (and never was) as per instructions.

reviewdb last modification made to the data files:

root@db1132:/srv/sqldata/reviewdb# ls -lhrt | tail -n5
-rw-rw---- 1 mysql mysql  96K Jun 28 02:20 bak_system_config.ibd
-rw-rw---- 1 mysql mysql  11M Jun 28 02:40 bak_account_external_ids.ibd
-rw-rw---- 1 mysql mysql  96K Jun 28 03:04 system_config.ibd
-rw-rw---- 1 mysql mysql  96K Jun 28 03:04 schema_version.ibd
-rw-rw---- 1 mysql mysql 160K Jun 28 03:04 account_group_id.ibd

The test one is even older:

root@db1132:/srv/sqldata/reviewdb@002dtest# ls -lhrt | tail -n5
-rw-rw---- 1 mysql mysql 148M Jun 15 14:07 patch_set_approvals.ibd
-rw-rw---- 1 mysql mysql 172M Jun 15 14:08 patch_sets.ibd
-rw-rw---- 1 mysql mysql  96K Jun 15 22:06 system_config.ibd
-rw-rw---- 1 mysql mysql  96K Jun 15 22:06 schema_version.ibd
-rw-rw---- 1 mysql mysql 160K Jun 15 22:06 account_group_id.ibd

Mentioned in SAL (#wikimedia-operations) [2020-07-16T07:25:44Z] <marostegui> Drop database reviewdb-test T255715

root@db1132.eqiad.wmnet[(none)]> drop database if exists `reviewdb-test`;

Query OK, 36 rows affected (23.864 sec)

root@db1132.eqiad.wmnet[(none)]>
root@db1132.eqiad.wmnet[(none)]>

I will delete reviewdb database on Monday.

Mentioned in SAL (#wikimedia-operations) [2020-07-20T07:19:08Z] <marostegui> Drop non used reviewdb database - T255715

Marostegui closed this task as Resolved.Mon, Jul 20, 7:20 AM

Dropped reviewdb after double checking nothing wrote to it again

root@db1132:/srv/sqldata/reviewdb# ls -lhrt | tail -n5
-rw-rw---- 1 mysql mysql  96K Jun 28 02:20 bak_system_config.ibd
-rw-rw---- 1 mysql mysql  11M Jun 28 02:40 bak_account_external_ids.ibd
-rw-rw---- 1 mysql mysql  96K Jun 28 03:04 system_config.ibd
-rw-rw---- 1 mysql mysql  96K Jun 28 03:04 schema_version.ibd
-rw-rw---- 1 mysql mysql 160K Jun 28 03:04 account_group_id.ibd
root@db1132.eqiad.wmnet[(none)]> drop database if exists reviewdb;
Query OK, 36 rows affected (12.542 sec)

root@db1132.eqiad.wmnet[(none)]>
Dzahn added a comment.Mon, Jul 27, 8:34 PM

If it hasn't already happened you can remove the grant for the "gerritro" user on m2 as well. That resolves a part of T244964. Same for the regular gerrit user that wasn't "ro".

If it hasn't already happened you can remove the grant for the "gerritro" user on m2 as well. That resolves a part of T244964. Same for the regular gerrit user that wasn't "ro".

This happened before we dropped the databases, see T255715#6303397
Thanks!