Ok, so the simple "link a buster slave to the cluster" approach has the problem that the tools-redis classes try to install prometheus-redis-exporter, which isn't available for buster.

Mentioned in SAL (#wikimedia-cloud) [2020-04-03T18:23:05Z] <bstorm_> spin up tools-redis-1003 on stretch and connect to the cluster T248929

Stretch "just works" so we can stick with that until the upstream redis material is upgraded to buster.

It is working well. I think next step is to shut down the jessie slave

# Replication
role:slave
master_host:tools-redis-1002.tools.eqiad.wmflabs
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:2655171695832
slave_priority:100
slave_read_only:0
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

Mentioned in SAL (#wikimedia-cloud) [2020-04-03T18:53:11Z] <bstorm_> spin up tools-redis-1004 on stretch and connect to cluster T248929

Mentioned in SAL (#wikimedia-cloud) [2020-04-03T20:41:40Z] <bstorm_> deleting tools-redis-1003/4 to attach them to an anti-affinity group T248929

Change 585830 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] toolforge-redis: fail over tools-redis to tools-redis-1003

https://gerrit.wikimedia.org/r/585830

Change 585830 merged by Bstorm:
[operations/puppet@production] toolforge-redis: fail over tools-redis to tools-redis-1003

https://gerrit.wikimedia.org/r/585830

Mentioned in SAL (#wikimedia-cloud) [2020-04-03T22:32:51Z] <bstorm_> switch tools-redis-1003 to the active redis server T248929

Mentioned in SAL (#wikimedia-cloud) [2020-04-03T22:40:55Z] <bstorm_> shut down tools-redis-1001/2 T248929

I'll leave tools-redis-1001/2 shut down and clean them up on Monday.

Mentioned in SAL (#wikimedia-cloud) [2020-04-06T19:16:32Z] <bstorm_> deleted tools-redis-1001/2 T248929