Page MenuHomePhabricator

Set 'cluster' salt grain appropriately for all instances in beta cluster
Closed, DeclinedPublic

Description

To match closer to prod. As of May 4th 2017, they are almost set to misc, with a few exceptions:

root@deployment-salt02:~# salt --output=text -v '*' grains.get cluster|grep -v misc
deployment-cache-upload04.deployment-prep.eqiad.wmflabs: cache_upload
deployment-redis01.deployment-prep.eqiad.wmflabs: redis
deployment-redis02.deployment-prep.eqiad.wmflabs: redis
deployment-cache-text04.deployment-prep.eqiad.wmflabs: cache_text
root@deployment-salt02:~# salt --output=text -v '*' grains.get cluster
deployment-mediawiki05.deployment-prep.eqiad.wmflabs: misc
deployment-elastic06.deployment-prep.eqiad.wmflabs: misc
deployment-ms-fe02.deployment-prep.eqiad.wmflabs: misc
deployment-eventlogging03.deployment-prep.eqiad.wmflabs: misc
deployment-aqs01.deployment-prep.eqiad.wmflabs: misc
deployment-logstash2.deployment-prep.eqiad.wmflabs: misc
deployment-db03.deployment-prep.eqiad.wmflabs: misc
deployment-zotero01.deployment-prep.eqiad.wmflabs: misc
deployment-sca02.deployment-prep.eqiad.wmflabs: misc
deployment-redis01.deployment-prep.eqiad.wmflabs: redis
deployment-eventlogging04.deployment-prep.eqiad.wmflabs: misc
deployment-kafka01.deployment-prep.eqiad.wmflabs: misc
deployment-restbase02.deployment-prep.eqiad.wmflabs: misc
deployment-sca03.deployment-prep.eqiad.wmflabs: misc
deployment-sca01.deployment-prep.eqiad.wmflabs: misc
deployment-elastic07.deployment-prep.eqiad.wmflabs: misc
deployment-aqs03.deployment-prep.eqiad.wmflabs: misc
deployment-mathoid.deployment-prep.eqiad.wmflabs: misc
deployment-puppetmaster02.deployment-prep.eqiad.wmflabs: misc
deployment-kafka03.deployment-prep.eqiad.wmflabs: misc
deployment-ircd.deployment-prep.eqiad.wmflabs: misc
deployment-prometheus01.deployment-prep.eqiad.wmflabs: misc
deployment-mx.deployment-prep.eqiad.wmflabs: misc
deployment-mediawiki06.deployment-prep.eqiad.wmflabs: misc
deployment-mcs01.deployment-prep.eqiad.wmflabs: misc
deployment-etcd-01.deployment-prep.eqiad.wmflabs: misc
deployment-poolcounter04.deployment-prep.eqiad.wmflabs: misc
deployment-mediawiki04.deployment-prep.eqiad.wmflabs: misc
deployment-parsoid09.deployment-prep.eqiad.wmflabs: misc
deployment-pdf01.deployment-prep.eqiad.wmflabs: misc
deployment-jobrunner02.deployment-prep.eqiad.wmflabs: misc
deployment-apertium02.deployment-prep.eqiad.wmflabs: misc
deployment-tmh01.deployment-prep.eqiad.wmflabs: misc
deployment-memc04.deployment-prep.eqiad.wmflabs: misc
deployment-ms-be04.deployment-prep.eqiad.wmflabs: misc
deployment-memc05.deployment-prep.eqiad.wmflabs: misc
deployment-tin.deployment-prep.eqiad.wmflabs: misc
deployment-imagescaler01.deployment-prep.eqiad.wmflabs: misc
deployment-kafka05.deployment-prep.eqiad.wmflabs: misc
deployment-urldownloader.deployment-prep.eqiad.wmflabs: misc
deployment-puppetdb01.deployment-prep.eqiad.wmflabs: misc
deployment-ores-redis-01.deployment-prep.eqiad.wmflabs: misc
deployment-conf03.deployment-prep.eqiad.wmflabs: misc
deployment-restbase01.deployment-prep.eqiad.wmflabs: misc
deployment-trending01.deployment-prep.eqiad.wmflabs: misc
deployment-pdfrender02.deployment-prep.eqiad.wmflabs: misc
deployment-zookeeper01.deployment-prep.eqiad.wmflabs: misc
deployment-aqs02.deployment-prep.eqiad.wmflabs: misc
deployment-fluorine02.deployment-prep.eqiad.wmflabs: misc
deployment-changeprop.deployment-prep.eqiad.wmflabs: misc
deployment-cache-upload04.deployment-prep.eqiad.wmflabs: cache_upload
deployment-phab01.deployment-prep.eqiad.wmflabs: misc
deployment-sentry01.deployment-prep.eqiad.wmflabs: misc
deployment-mira.deployment-prep.eqiad.wmflabs: misc
deployment-sca04.deployment-prep.eqiad.wmflabs: misc
deployment-redis02.deployment-prep.eqiad.wmflabs: redis
deployment-kafka04.deployment-prep.eqiad.wmflabs: misc
deployment-elastic05.deployment-prep.eqiad.wmflabs: misc
deployment-stream.deployment-prep.eqiad.wmflabs: misc
deployment-salt02.deployment-prep.eqiad.wmflabs: misc
deployment-ms-be03.deployment-prep.eqiad.wmflabs: misc
deployment-db04.deployment-prep.eqiad.wmflabs: misc
deployment-secureredirexperiment.deployment-prep.eqiad.wmflabs: misc
deployment-cache-text04.deployment-prep.eqiad.wmflabs: cache_text

Event Timeline

yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added a subscriber: yuvipanda.
greg triaged this task as Medium priority.Jan 29 2015, 5:36 AM
greg moved this task from To Triage to Next: Maintenance on the Beta-Cluster-Infrastructure board.

@yuvipanda can you elaborate on what needs to be done for this task? What is a cluster salt grain? :D

AAAAAAAAAAA such a long time ago :D

So grains are what salt uses for targeting, and 'cluster' is supposed to set what kind of machines they are - ganglia, appserver, varnish, etc. This is set in the 'role' hiera setup for production (do a 'git grep cluster' under 'hieradata/role' in ops/puppet). Since there is no role backend for labs(T127771), this isn't accounted for here.

This, when done would afford the following:

  1. Salt targetting per-cluster in the same way as prod

That's the only thing - the other use for cluster in prod is for ganglia, which we don't have in labs anywhere...

Unfortunately I have no time to work on this atm :(

@yuvipanda: So we need to make a puppet change of some sort? Or we should do something like salt 'hostname.deployment-prep.eqiad.wmflabs' grains.set cluster <value> (based on a list of production hosts and their cluster value), and document this as being a step for new beta salt minions?

Krenair updated the task description. (Show Details)
<Krenair> so I could make a hiera page on wikitech for cache-text04
<Krenair> set cluster: cache_text
<chasemp> something like
<chasemp> https://wikitech.wikimedia.org/wiki/Hiera:Tools/host/tools-worker-1011
<chasemp> but for that key I imagine yes
<chasemp> I don't actually know what the right cluster values are
<Krenair> yeah I'm going to get someone to dump the result of that salt command on the ticket for prod
<YuviPanda> Krenair: yes, you can do that per-host now, but gets unweildy for large number of hosts
<Krenair> YuviPanda, so what should we do instead?
<YuviPanda> Krenair: make 'role' work for labs as well
<YuviPanda> Krenair: so you can define them once only
<Krenair> isn't that a separate ticket?
<YuviPanda> yeah but I think that should block this
<Krenair> So completing that ticket won't get us this for free?
<Krenair> ok
<YuviPanda> Krenair: oh, that depends on how that is completed. if we load things from ops/puppet's hieradata/role stuff in labs too, then yes it'll get us it for free.

I am tempted to just close this. Doesn't seem we rely on the cluster grain on beta?