Every time we create a new toolsdb instance, a new gtid_domain_id is generated and written to /etc/my.cnf.
This should not be necessary as we're not using multi-source replication, and it only ends up messing up the GTID values if a replica is promoted to primary and both the old and new gtid_domain_id become part of gtid_binlog_pos:
gtid_binlog_pos | 2886731301-2886731301-4714442229,2886731673-2886731673-4887243158
gtid_domain_id was recently set to 0 in some production servers (T336228: Evaluate removing gtid_domain_id from the infra) but that task was Declined in favor of a different approach.
ToolsDB is slightly easier because we only have one primary and one replica. I think we could create a new replica host (e.g. tools-db-3) where we set gtid_domain_id = 0, then failover the existing primary to that new replica. New transactions after the failover will use the new domain id, and we can finally clean up the old domain ids, as we did for older domain ids in T334947: ToolsDB: discard obsolete GTID domains.
I think it would also be nice to set server_id to the last token of the hostname (e.g. 3 for tools-db-3) instead of deriving it from the hostname. This means we would get more readable GTIDs like 0-3-4714442229 instead of 2886731301-2886731301-4714442229.
Finally, we should set gtid_strict_mode to 1, as it can prevent some GTID-related issues.
Cc-ing @Marostegui for review.