Page MenuHomePhabricator

Set 'cluster' salt grain appropriately for all instances in beta cluster
Closed, DeclinedPublic

Description

To match closer to prod. As of May 4th 2017, they are almost set to misc, with a few exceptions:

root@deployment-salt02:~# salt --output=text -v '*' grains.get cluster|grep -v misc
deployment-cache-upload04.deployment-prep.eqiad.wmflabs: cache_upload
deployment-redis01.deployment-prep.eqiad.wmflabs: redis
deployment-redis02.deployment-prep.eqiad.wmflabs: redis
deployment-cache-text04.deployment-prep.eqiad.wmflabs: cache_text
root@deployment-salt02:~# salt --output=text -v '*' grains.get cluster
deployment-mediawiki05.deployment-prep.eqiad.wmflabs: misc
deployment-elastic06.deployment-prep.eqiad.wmflabs: misc
deployment-ms-fe02.deployment-prep.eqiad.wmflabs: misc
deployment-eventlogging03.deployment-prep.eqiad.wmflabs: misc
deployment-aqs01.deployment-prep.eqiad.wmflabs: misc
deployment-logstash2.deployment-prep.eqiad.wmflabs: misc
deployment-db03.deployment-prep.eqiad.wmflabs: misc
deployment-zotero01.deployment-prep.eqiad.wmflabs: misc
deployment-sca02.deployment-prep.eqiad.wmflabs: misc
deployment-redis01.deployment-prep.eqiad.wmflabs: redis
deployment-eventlogging04.deployment-prep.eqiad.wmflabs: misc
deployment-kafka01.deployment-prep.eqiad.wmflabs: misc
deployment-restbase02.deployment-prep.eqiad.wmflabs: misc
deployment-sca03.deployment-prep.eqiad.wmflabs: misc
deployment-sca01.deployment-prep.eqiad.wmflabs: misc
deployment-elastic07.deployment-prep.eqiad.wmflabs: misc
deployment-aqs03.deployment-prep.eqiad.wmflabs: misc
deployment-mathoid.deployment-prep.eqiad.wmflabs: misc
deployment-puppetmaster02.deployment-prep.eqiad.wmflabs: misc
deployment-kafka03.deployment-prep.eqiad.wmflabs: misc
deployment-ircd.deployment-prep.eqiad.wmflabs: misc
deployment-prometheus01.deployment-prep.eqiad.wmflabs: misc
deployment-mx.deployment-prep.eqiad.wmflabs: misc
deployment-mediawiki06.deployment-prep.eqiad.wmflabs: misc
deployment-mcs01.deployment-prep.eqiad.wmflabs: misc
deployment-etcd-01.deployment-prep.eqiad.wmflabs: misc
deployment-poolcounter04.deployment-prep.eqiad.wmflabs: misc
deployment-mediawiki04.deployment-prep.eqiad.wmflabs: misc
deployment-parsoid09.deployment-prep.eqiad.wmflabs: misc
deployment-pdf01.deployment-prep.eqiad.wmflabs: misc
deployment-jobrunner02.deployment-prep.eqiad.wmflabs: misc
deployment-apertium02.deployment-prep.eqiad.wmflabs: misc
deployment-tmh01.deployment-prep.eqiad.wmflabs: misc
deployment-memc04.deployment-prep.eqiad.wmflabs: misc
deployment-ms-be04.deployment-prep.eqiad.wmflabs: misc
deployment-memc05.deployment-prep.eqiad.wmflabs: misc
deployment-tin.deployment-prep.eqiad.wmflabs: misc
deployment-imagescaler01.deployment-prep.eqiad.wmflabs: misc
deployment-kafka05.deployment-prep.eqiad.wmflabs: misc
deployment-urldownloader.deployment-prep.eqiad.wmflabs: misc
deployment-puppetdb01.deployment-prep.eqiad.wmflabs: misc
deployment-ores-redis-01.deployment-prep.eqiad.wmflabs: misc
deployment-conf03.deployment-prep.eqiad.wmflabs: misc
deployment-restbase01.deployment-prep.eqiad.wmflabs: misc
deployment-trending01.deployment-prep.eqiad.wmflabs: misc
deployment-pdfrender02.deployment-prep.eqiad.wmflabs: misc
deployment-zookeeper01.deployment-prep.eqiad.wmflabs: misc
deployment-aqs02.deployment-prep.eqiad.wmflabs: misc
deployment-fluorine02.deployment-prep.eqiad.wmflabs: misc
deployment-changeprop.deployment-prep.eqiad.wmflabs: misc
deployment-cache-upload04.deployment-prep.eqiad.wmflabs: cache_upload
deployment-phab01.deployment-prep.eqiad.wmflabs: misc
deployment-sentry01.deployment-prep.eqiad.wmflabs: misc
deployment-mira.deployment-prep.eqiad.wmflabs: misc
deployment-sca04.deployment-prep.eqiad.wmflabs: misc
deployment-redis02.deployment-prep.eqiad.wmflabs: redis
deployment-kafka04.deployment-prep.eqiad.wmflabs: misc
deployment-elastic05.deployment-prep.eqiad.wmflabs: misc
deployment-stream.deployment-prep.eqiad.wmflabs: misc
deployment-salt02.deployment-prep.eqiad.wmflabs: misc
deployment-ms-be03.deployment-prep.eqiad.wmflabs: misc
deployment-db04.deployment-prep.eqiad.wmflabs: misc
deployment-secureredirexperiment.deployment-prep.eqiad.wmflabs: misc
deployment-cache-text04.deployment-prep.eqiad.wmflabs: cache_text

Event Timeline

yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added a subscriber: yuvipanda.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 19 2015, 8:32 PM
greg triaged this task as Normal priority.Jan 29 2015, 5:36 AM
greg moved this task from To Triage to Next: Maintenance on the Beta-Cluster-Infrastructure board.
hashar added a subscriber: hashar.Jun 15 2015, 7:06 PM

@yuvipanda can you elaborate on what needs to be done for this task? What is a cluster salt grain? :D

chasemp set Security to None.
hashar updated the task description. (Show Details)Oct 22 2015, 9:44 AM
hashar removed a subscriber: hashar.Oct 22 2015, 9:49 AM
Krenair added a subscriber: Krenair.

Needs information from @yuvipanda

AAAAAAAAAAA such a long time ago :D

So grains are what salt uses for targeting, and 'cluster' is supposed to set what kind of machines they are - ganglia, appserver, varnish, etc. This is set in the 'role' hiera setup for production (do a 'git grep cluster' under 'hieradata/role' in ops/puppet). Since there is no role backend for labs(T127771), this isn't accounted for here.

This, when done would afford the following:

  1. Salt targetting per-cluster in the same way as prod

That's the only thing - the other use for cluster in prod is for ganglia, which we don't have in labs anywhere...

yuvipanda removed yuvipanda as the assignee of this task.Apr 11 2016, 5:08 PM

Unfortunately I have no time to work on this atm :(

@yuvipanda: So we need to make a puppet change of some sort? Or we should do something like salt 'hostname.deployment-prep.eqiad.wmflabs' grains.set cluster <value> (based on a list of production hosts and their cluster value), and document this as being a step for new beta salt minions?

Krenair updated the task description. (Show Details)Apr 11 2016, 7:49 PM
Krenair updated the task description. (Show Details)
Krenair added a comment.EditedApr 11 2016, 7:54 PM
<Krenair> so I could make a hiera page on wikitech for cache-text04
<Krenair> set cluster: cache_text
<chasemp> something like
<chasemp> https://wikitech.wikimedia.org/wiki/Hiera:Tools/host/tools-worker-1011
<chasemp> but for that key I imagine yes
<chasemp> I don't actually know what the right cluster values are
<Krenair> yeah I'm going to get someone to dump the result of that salt command on the ticket for prod
<YuviPanda> Krenair: yes, you can do that per-host now, but gets unweildy for large number of hosts
<Krenair> YuviPanda, so what should we do instead?
<YuviPanda> Krenair: make 'role' work for labs as well
<YuviPanda> Krenair: so you can define them once only
<Krenair> isn't that a separate ticket?
<YuviPanda> yeah but I think that should block this
<Krenair> So completing that ticket won't get us this for free?
<Krenair> ok
<YuviPanda> Krenair: oh, that depends on how that is completed. if we load things from ops/puppet's hieradata/role stuff in labs too, then yes it'll get us it for free.
hashar updated the task description. (Show Details)Sep 12 2016, 12:29 PM
hashar updated the task description. (Show Details)May 4 2017, 7:44 AM

I am tempted to just close this. Doesn't seem we rely on the cluster grain on beta?

hashar closed this task as Declined.Jun 29 2017, 7:59 AM
Restricted Application removed a subscriber: Liuxinyu970226. · View Herald TranscriptJun 29 2017, 7:59 AM
bd808 changed the status of subtask T127771: Make Cloud VPS hiera role aware from Resolved to Declined.Mar 19 2018, 7:44 PM