Page MenuHomePhabricator
Create Task
Maniphest T334947

ToolsDB: discard obsolete GTID domains
Open, Stalled, Needs TriagePublic
Actions

Description

The GTID in the ToolsDB servers (tools-db-1 and tools-db-2) contains two obsolete domain ids that can be safely discarded as explained at https://mariadb.com/kb/en/gtid/#deleting-unused-domains

Domain id 0 is from an old server that was deleted years ago and can be discarded.

Domain id 2886731673 was the id used by clouddb1001 before it was shut down in T333471, and can also be discarded.

2886731301 is the id of tools-db-1 and the one we want to keep because it's the one currently being used for replicating from tools-db-1 to tools-db-2.

//tools-db-1

MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE 'gtid%';
+-------------------------+-------------------------------------------------------------------------------------------+
| Variable_name           | Value                                                                                     |
+-------------------------+-------------------------------------------------------------------------------------------+
| gtid_binlog_pos         | 2886731301-2886731301-143124522,2886731673-2886731673-4887243158                          |
| gtid_binlog_state       | 2886731301-2886731301-143124522,2886731673-2886731673-4887243158                          |
| gtid_cleanup_batch_size | 64                                                                                        |
| gtid_current_pos        | 0-2886731673-33522724637,2886731301-2886731301-143124522,2886731673-2886731673-4887243158 |
| gtid_domain_id          | 2886731301                                                                                |
| gtid_ignore_duplicates  | OFF                                                                                       |
| gtid_pos_auto_engines   |                                                                                           |
| gtid_slave_pos          | 0-2886731673-33522724637,2886731673-2886731673-4887243158                                 |
| gtid_strict_mode        | OFF                                                                                       |
+-------------------------+-------------------------------------------------------------------------------------------+

//tools-db-2

MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE 'gtid%';
+-------------------------+-------------------------------------------------------------------------------------------+
| Variable_name           | Value                                                                                     |
+-------------------------+-------------------------------------------------------------------------------------------+
| gtid_binlog_pos         | 2886731301-2886731301-143124674,2886731673-2886731673-4887243158                          |
| gtid_binlog_state       | 2886731301-2886731301-143124674,2886731673-2886731673-4887243158                          |
| gtid_cleanup_batch_size | 64                                                                                        |
| gtid_current_pos        | 0-2886731673-33522724637,2886731301-2886731301-143124674,2886731673-2886731673-4887243158 |
| gtid_domain_id          | 2886730873                                                                                |
| gtid_ignore_duplicates  | OFF                                                                                       |
| gtid_pos_auto_engines   |                                                                                           |
| gtid_slave_pos          | 0-2886731673-33522724637,2886731301-2886731301-143124674,2886731673-2886731673-4887243158 |
| gtid_strict_mode        | OFF                                                                                       |
+-------------------------+-------------------------------------------------------------------------------------------+

Related Objects
Search...

Event Timeline

fnegri created this task.Apr 18 2023, 1:51 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 18 2023, 1:51 PM
fnegri updated the task description. (Show Details)Apr 18 2023, 2:14 PM
fnegri updated the task description. (Show Details)
fnegri updated the task description. (Show Details)Apr 18 2023, 2:24 PM
fnegri added a comment.Apr 18 2023, 2:34 PM

My plan is to run on both tools-db-1 (primary) and tools-db-2 (secondary):

FLUSH BINARY LOGS DELETE_DOMAIN_ID=(0,2886731673);

From the documentation, it looks like I will also have to manually update gtid_slave_pos in tools-db-2 (secondary):

SET gtid_slave_pos="<position with domains removed>"`

And probably also clear gtid_slave_pos in tools-db-1 (primary) as it's no longer replicating from other servers.

SET GLOBAL gtid_slave_pos = '';
fnegri edited parent tasks, added: T324965: Clean up old gtid_domain_id ; removed: T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4.Apr 18 2023, 3:53 PM
fnegri changed the task status from Open to Stalled.Apr 18 2023, 4:02 PM
fnegri added subscribers: • Marostegui, jcrespo.

After speaking with @Marostegui and @jcrespo, I'm postponing this for now as they experienced a few issues while trying to do the same thing in other servers. I'm linking this to T324965 that is tracking similar clean-up operations.

fnegri removed fnegri as the assignee of this task.Apr 18 2023, 4:02 PM
fnegri edited projects, added cloud-services-team; removed cloud-services-team (FY2022/2023-Q4).
JJMC89 moved this task from Backlog to ToolsDB on the Data-Services board.Apr 28 2023, 4:19 PM
Content licensed under Creative Commons Attribution-ShareAlike 4.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL