Page MenuHomePhabricator

Use a Puppet ENC to define which classes are included in which nodes (in Labs)
Closed, DeclinedPublic

Description

Currently wikitech directly updates LDAP, which puppet master then uses (via https://docs.puppetlabs.com/guides/ldap_nodes.html) to figure out which roles to apply. Instead, use https://docs.puppetlabs.com/guides/external_nodes.html + an API on wikitech/horizon/ponies + a UI to let people decide which roles to apply where.

Also get rid of all puppet vars, replace them with hiera.

Details

Related Gerrit Patches:

Event Timeline

yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added a subscriber: yuvipanda.
chasemp triaged this task as Medium priority.Jan 6 2015, 11:02 PM
chasemp removed a project: acl*sre-team.
chasemp set Security to None.
coren moved this task from Triage to Backlog on the Cloud-Services board.Feb 10 2015, 9:23 PM

So we could just have the ENC take in YAML as well, and spit out node defs from that. In the future we can have horizon / wikitech have a UI for specifying these. Something like...

.*:
    - role::puppet::self
staging-puppetmaster:
    - role::salt::master
    - role::foo::bar
staging-mw-app-.*:
    - role::mediawiki::appserver

etc. Basically match a regex to a bunch of roles. We could even mandate that these always be role classes, although not sure how useful that would be.

Perhaps in a glorious future this can replace site.pp too? :)

Note that we can't use an ENC with our current LDAP terminus setup. We can have one or the other..

scfc added a subscriber: scfc.Mar 5 2015, 4:27 PM

We could make the ENC script query LDAP additionally, but it's probably less error-prone to make a hard cut (freeze wikitech/LDAP, convert LDAP to ENC input, switch Puppet).

If we make it query LDAP, we can actually have this be an addition to the LDAP terminus for labs. Maybe even parameterize it, so it is turned on only for some projects.

Alright, so how about we make this a small webservice with pluggable backends? Early backend would be file based (yay for beta / staging), and eventually we can use mysql / something along those lines for horizon.

The backend would store an ordered list of regexes and the roles to be applied for each regex. Puppet global variables not supported. We can build them to be composable, so one can fallback to another (File+LDAP, for example)

@Andrew does this sound good?

Or we could go completely low-fi, and just have it be a simple script that's called by the ENC. This is the CGI model, and is far simpler. Perhaps I can start here, use it for staging / deployment-prep, and then see how it goes.

And no, it is only *like* CGI. Puppet ENC works by startying a script with a param (the name of the node) and expecting back YAML about the roles/variables. It doesn't care about http.

Change 196628 had a related patch set uploaded (by Yuvipanda):
ldap yaml file puppet ENC for self hosted puppetmasters

https://gerrit.wikimedia.org/r/196628

Change 196628 merged by Yuvipanda:
ldap yaml file puppet ENC for self hosted puppetmasters

https://gerrit.wikimedia.org/r/196628

scfc added a comment.Mar 17 2015, 2:37 PM

Looks cool; needs documentation, though, for the code flow in modules/puppetmaster/files/ldap-yaml-enc.py and how to use it as well.

scfc added a comment.Mar 18 2015, 1:03 PM

That's a bummer. If we could make the regular expression match of the host name a fact, we might use a "normal" hiera hierarchy. But I don't think the flexibility of ldap-yaml-enc.py is within reach with pure Puppet.

What might be possible is convention: If host names are always labelled $instanceproject-$typeofinstance-$something, we could make $typeofinstance a fact and then use that.

scfc added a comment.Mar 18 2015, 1:04 PM

(Or ${instanceproject}-${typeofinstance}${number}, seems to be another common pattern.)

@scfc Assuming you meant 'bummer' is the fact that we can't get rid of hiera_include classes? Anyway, when we migrate to horizon, I expect the ENC there to be much closer to the flexibility offered by the YAML+LDAP one than with how wikitech offers it now....

And it's possible that we can just make the labs puppetmaster itself use this ENC rather than ldap terminus as it does now. This wouldn't cause any loss in functionality at all (since it falls back to LDAP), and will allow using the yaml stuff even for projects that don't have a custom puppetmaster. To do that, we'll need to basically rewrite the ENC to be a persistant process that a small script can hit via http / some interface, and be able to re-load changes from the yaml files as they happen...

scfc added a comment.Mar 18 2015, 1:59 PM

With bummer I meant https://gerrit.wikimedia.org/r/#/c/197487/. The more places there are where configurations are set, the more confusing it gets :-). For example, I assume https://gerrit.wikimedia.org/r/#/c/197482/ had the potential to disrupt projects that used classes in hiera, but were not aware of nodes/. If this ENC could fully replace hiera's classes, that would be amendable, but with "doesn't work in all cases" there can't be a single source of truth.

Change 197712 had a related patch set uploaded (by Yuvipanda):
WIP: Make yaml ldap ENC into a http service

https://gerrit.wikimedia.org/r/197712

yuvipanda removed yuvipanda as the assignee of this task.Nov 11 2015, 4:31 PM

Change 197712 abandoned by Yuvipanda:
WIP: Make yaml ldap ENC into a http service

https://gerrit.wikimedia.org/r/197712

yuvipanda closed this task as Declined.Jul 5 2016, 1:33 PM

Superseded by T91990 and its dependent tasks