Page MenuHomePhabricator

Investigate using orchestrator tags for different type of hosts.
Open, LowPublic

Description

Orchestrator provides a way to tag hosts.
Even though this has no effect on either the guy or the normal client operation, it can be a good way to use orchestrator as a way to also identify hosts.
On an ideal world, we could have a way to automatically tag hosts using puppet, or the source of truth, or the hosts identifying themselves.

Tags that could be useful would be:

  • Candidate masters
  • Backup sources

This is an example of a tag:

root@dborch1001:~# orchestrator-client -c tag -i db1077 --tag name=candidate
db1077:3306

Then listing the hosts with a given tag can be done from the CLI or even using the api:

root@dborch1001:~# orchestrator-client -c tagged -t candidate
db1077:3306

root@dborch1001:~# curl http://localhost:3000/api/tagged?tag=candidate
[{"Hostname":"db1077","Port":3306}]
root@dborch1001:~#

Seeing tagged hosts within a topology

root@dborch1001:~# orchestrator-client -c tag -i db1124 --tag name=master
db1124.eqiad.wmnet:3306
root@dborch1001:~# orchestrator-client -c tag -i db1125 --tag name=candidate
db1125.eqiad.wmnet:3306
root@dborch1001:~# /usr/bin/orchestrator-client -c which-cluster-master -a test-s4 < /usr/bin/orchestrator -c topology-tabulated -c topology-tags
db1124.eqiad.wmnet:3306   [0s,ok,10.4.20-MariaDB-log,rw,STATEMENT,>>,semi:master] [name=master]
+ db1125.eqiad.wmnet:3306 [0s,ok,10.4.21-MariaDB-log,ro,ROW,>>,GTID,semi:replica] [name=candidate]
Complete documentation is at https://github.com/openark/orchestrator/blob/master/docs/tags.md

Event Timeline

Marostegui renamed this task from Investigate using orchestrator tags for hosts different type of hosts. to Investigate using orchestrator for different type of hosts..
Marostegui triaged this task as Low priority.
Marostegui moved this task from Triage to Refine on the DBA board.
Marostegui renamed this task from Investigate using orchestrator for different type of hosts. to Investigate using orchestrator tags for different type of hosts..Nov 10 2020, 2:55 PM

I have tagged all candidates as candidates:

orchestrator-client -c tagged -tag name=candidate
db1100.eqiad.wmnet:3306
db1104.eqiad.wmnet:3306
db1123.eqiad.wmnet:3306
db1131.eqiad.wmnet:3306
db1160.eqiad.wmnet:3306
db1162.eqiad.wmnet:3306
db1163.eqiad.wmnet:3306
db1181.eqiad.wmnet:3306
db2080.codfw.wmnet:3306
db2090.codfw.wmnet:3306
db2107.codfw.wmnet:3306
db2112.codfw.wmnet:3306
db2113.codfw.wmnet:3306
db2114.codfw.wmnet:3306
db2118.codfw.wmnet:3306
db2127.codfw.wmnet:3306
es1021.eqiad.wmnet:3306
es1024.eqiad.wmnet:3306
es2020.codfw.wmnet:3306
es2024.codfw.wmnet:3306

Change 771257 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/software@master] switchover-tmpl.sh: Add orchestrator tag notes

https://gerrit.wikimedia.org/r/771257

Change 771257 merged by Marostegui:

[operations/software@master] switchover-tmpl.sh: Add orchestrator tag notes

https://gerrit.wikimedia.org/r/771257

Change 771329 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] db1149: Remove candidate master

https://gerrit.wikimedia.org/r/771329

Change 771329 merged by Marostegui:

[operations/puppet@production] db1149: Remove candidate master

https://gerrit.wikimedia.org/r/771329