Set 'cluster' salt grain appropriately for all instances in beta cluster
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	yuvipanda
	Jan 19 2015, 8:32 PM

Description

To match closer to prod. As of May 4th 2017, they are almost set to misc, with a few exceptions:

root@deployment-salt02:~# salt --output=text -v '*' grains.get cluster|grep -v misc
deployment-cache-upload04.deployment-prep.eqiad.wmflabs: cache_upload
deployment-redis01.deployment-prep.eqiad.wmflabs: redis
deployment-redis02.deployment-prep.eqiad.wmflabs: redis
deployment-cache-text04.deployment-prep.eqiad.wmflabs: cache_text

root@deployment-salt02:~# salt --output=text -v '*' grains.get cluster
deployment-mediawiki05.deployment-prep.eqiad.wmflabs: misc
deployment-elastic06.deployment-prep.eqiad.wmflabs: misc
deployment-ms-fe02.deployment-prep.eqiad.wmflabs: misc
deployment-eventlogging03.deployment-prep.eqiad.wmflabs: misc
deployment-aqs01.deployment-prep.eqiad.wmflabs: misc
deployment-logstash2.deployment-prep.eqiad.wmflabs: misc
deployment-db03.deployment-prep.eqiad.wmflabs: misc
deployment-zotero01.deployment-prep.eqiad.wmflabs: misc
deployment-sca02.deployment-prep.eqiad.wmflabs: misc
deployment-redis01.deployment-prep.eqiad.wmflabs: redis
deployment-eventlogging04.deployment-prep.eqiad.wmflabs: misc
deployment-kafka01.deployment-prep.eqiad.wmflabs: misc
deployment-restbase02.deployment-prep.eqiad.wmflabs: misc
deployment-sca03.deployment-prep.eqiad.wmflabs: misc
deployment-sca01.deployment-prep.eqiad.wmflabs: misc
deployment-elastic07.deployment-prep.eqiad.wmflabs: misc
deployment-aqs03.deployment-prep.eqiad.wmflabs: misc
deployment-mathoid.deployment-prep.eqiad.wmflabs: misc
deployment-puppetmaster02.deployment-prep.eqiad.wmflabs: misc
deployment-kafka03.deployment-prep.eqiad.wmflabs: misc
deployment-ircd.deployment-prep.eqiad.wmflabs: misc
deployment-prometheus01.deployment-prep.eqiad.wmflabs: misc
deployment-mx.deployment-prep.eqiad.wmflabs: misc
deployment-mediawiki06.deployment-prep.eqiad.wmflabs: misc
deployment-mcs01.deployment-prep.eqiad.wmflabs: misc
deployment-etcd-01.deployment-prep.eqiad.wmflabs: misc
deployment-poolcounter04.deployment-prep.eqiad.wmflabs: misc
deployment-mediawiki04.deployment-prep.eqiad.wmflabs: misc
deployment-parsoid09.deployment-prep.eqiad.wmflabs: misc
deployment-pdf01.deployment-prep.eqiad.wmflabs: misc
deployment-jobrunner02.deployment-prep.eqiad.wmflabs: misc
deployment-apertium02.deployment-prep.eqiad.wmflabs: misc
deployment-tmh01.deployment-prep.eqiad.wmflabs: misc
deployment-memc04.deployment-prep.eqiad.wmflabs: misc
deployment-ms-be04.deployment-prep.eqiad.wmflabs: misc
deployment-memc05.deployment-prep.eqiad.wmflabs: misc
deployment-tin.deployment-prep.eqiad.wmflabs: misc
deployment-imagescaler01.deployment-prep.eqiad.wmflabs: misc
deployment-kafka05.deployment-prep.eqiad.wmflabs: misc
deployment-urldownloader.deployment-prep.eqiad.wmflabs: misc
deployment-puppetdb01.deployment-prep.eqiad.wmflabs: misc
deployment-ores-redis-01.deployment-prep.eqiad.wmflabs: misc
deployment-conf03.deployment-prep.eqiad.wmflabs: misc
deployment-restbase01.deployment-prep.eqiad.wmflabs: misc
deployment-trending01.deployment-prep.eqiad.wmflabs: misc
deployment-pdfrender02.deployment-prep.eqiad.wmflabs: misc
deployment-zookeeper01.deployment-prep.eqiad.wmflabs: misc
deployment-aqs02.deployment-prep.eqiad.wmflabs: misc
deployment-fluorine02.deployment-prep.eqiad.wmflabs: misc
deployment-changeprop.deployment-prep.eqiad.wmflabs: misc
deployment-cache-upload04.deployment-prep.eqiad.wmflabs: cache_upload
deployment-phab01.deployment-prep.eqiad.wmflabs: misc
deployment-sentry01.deployment-prep.eqiad.wmflabs: misc
deployment-mira.deployment-prep.eqiad.wmflabs: misc
deployment-sca04.deployment-prep.eqiad.wmflabs: misc
deployment-redis02.deployment-prep.eqiad.wmflabs: redis
deployment-kafka04.deployment-prep.eqiad.wmflabs: misc
deployment-elastic05.deployment-prep.eqiad.wmflabs: misc
deployment-stream.deployment-prep.eqiad.wmflabs: misc
deployment-salt02.deployment-prep.eqiad.wmflabs: misc
deployment-ms-be03.deployment-prep.eqiad.wmflabs: misc
deployment-db04.deployment-prep.eqiad.wmflabs: misc
deployment-secureredirexperiment.deployment-prep.eqiad.wmflabs: misc
deployment-cache-text04.deployment-prep.eqiad.wmflabs: cache_text

Related Objects
Search...

Status	Assigned	Task
Open	None	T53494 Use Beta cluster as a true canary for code deployments (epic)
Open	None	T87220 Minimize infrastructure differences between Beta Cluster and production
Declined	None	T87199 Set 'cluster' salt grain appropriately for all instances in beta cluster
Declined	Andrew	T127771 Make Cloud VPS hiera role aware

Event Timeline

yuvipanda created this task.Jan 19 2015, 8:32 PM

yuvipanda raised the priority of this task from to Needs Triage.

yuvipanda updated the task description. (Show Details)

yuvipanda added projects: Beta-Cluster-Infrastructure, acl*sre-team.

yuvipanda subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 19 2015, 8:32 PM

yuvipanda added a parent task: T87220: Minimize infrastructure differences between Beta Cluster and production.Jan 20 2015, 8:23 AM

greg triaged this task as Medium priority.Jan 29 2015, 5:36 AM

greg moved this task from To Triage to Next: Maintenance on the Beta-Cluster-Infrastructure board.

Liuxinyu970226 subscribed.Apr 18 2015, 1:32 AM

@yuvipanda can you elaborate on what needs to be done for this task? What is a cluster salt grain? :D

• chasemp removed a project: SRE.Oct 21 2015, 11:12 PM

• chasemp set Security to None.

hashar updated the task description. (Show Details)Oct 22 2015, 9:44 AM

hashar unsubscribed.Oct 22 2015, 9:49 AM

Needs information from @yuvipanda

AAAAAAAAAAA such a long time ago :D

So grains are what salt uses for targeting, and 'cluster' is supposed to set what kind of machines they are - ganglia, appserver, varnish, etc. This is set in the 'role' hiera setup for production (do a 'git grep cluster' under 'hieradata/role' in ops/puppet). Since there is no role backend for labs(T127771), this isn't accounted for here.

This, when done would afford the following:

Salt targetting per-cluster in the same way as prod

That's the only thing - the other use for cluster in prod is for ganglia, which we don't have in labs anywhere...

Unfortunately I have no time to work on this atm :(

@yuvipanda: So we need to make a puppet change of some sort? Or we should do something like salt 'hostname.deployment-prep.eqiad.wmflabs' grains.set cluster <value> (based on a list of production hosts and their cluster value), and document this as being a step for new beta salt minions?

Krenair updated the task description. (Show Details)Apr 11 2016, 7:49 PM

Krenair updated the task description. (Show Details)

<Krenair> so I could make a hiera page on wikitech for cache-text04
<Krenair> set cluster: cache_text
<chasemp> something like
<chasemp> https://wikitech.wikimedia.org/wiki/Hiera:Tools/host/tools-worker-1011
<chasemp> but for that key I imagine yes
<chasemp> I don't actually know what the right cluster values are
<Krenair> yeah I'm going to get someone to dump the result of that salt command on the ticket for prod
<YuviPanda> Krenair: yes, you can do that per-host now, but gets unweildy for large number of hosts
<Krenair> YuviPanda, so what should we do instead?
<YuviPanda> Krenair: make 'role' work for labs as well
<YuviPanda> Krenair: so you can define them once only
<Krenair> isn't that a separate ticket?
<YuviPanda> yeah but I think that should block this
<Krenair> So completing that ticket won't get us this for free?
<Krenair> ok
<YuviPanda> Krenair: oh, that depends on how that is completed. if we load things from ops/puppet's hieradata/role stuff in labs too, then yes it'll get us it for free.

Krenair added a subtask: T127771: Make Cloud VPS hiera role aware.Apr 11 2016, 7:55 PM

greg moved this task from Next: Maintenance to Backlog on the Beta-Cluster-Infrastructure board.Aug 5 2016, 8:56 PM

hashar updated the task description. (Show Details)Sep 12 2016, 12:29 PM

I am tempted to just close this. Doesn't seem we rely on the cluster grain on beta?

• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:55 PM

hashar closed this task as Declined.Jun 29 2017, 7:59 AM

Restricted Application removed a subscriber: Liuxinyu970226. · View Herald TranscriptJun 29 2017, 7:59 AM

Andrew closed subtask T127771: Make Cloud VPS hiera role aware as Resolved.Mar 19 2018, 7:42 PM

bd808 changed the status of subtask T127771: Make Cloud VPS hiera role aware from Resolved to Declined.Mar 19 2018, 7:44 PM

Set 'cluster' salt grain appropriately for all instances in beta clusterClosed, DeclinedPublicActions

Description

Related ObjectsSearch...

Event Timeline

Set 'cluster' salt grain appropriately for all instances in beta cluster
Closed, DeclinedPublic
Actions

Related Objects
Search...