Change 436439 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[operations/puppet@production] deployment-prep: Remove override for scap::sources

https://gerrit.wikimedia.org/r/436439

Change 436440 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[operations/puppet@production] puppetmaster: Add role_hierarchy to labs.hiera

https://gerrit.wikimedia.org/r/436440

Change 436439 abandoned by Krinkle:
deployment-prep: Remove override for scap::sources

Reason:
Discussing at T161675

https://gerrit.wikimedia.org/r/436439

Change 436440 abandoned by Krinkle:
puppetmaster: Add role_hierarchy to labs.hiera

Reason:
Discussing at T161675

https://gerrit.wikimedia.org/r/436440

Change 436581 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[operations/puppet@production] Move scap::sources from role::deployment_server to common

https://gerrit.wikimedia.org/r/436581

Change 436581 abandoned by Krinkle:
Move scap::sources from role::deployment_server to common

https://gerrit.wikimedia.org/r/436581

Can I propose an alternative?

1. Get rid of that hiera thing
2. Write a small script that runs on a regular cron, which gets a list of all projects known to gerrit, checks whether a scap/ directory exists, and if it hasn't been cloned it does so

That can be pushed to every deployment server, and will remove a puppet dep.

Gets all scap-enabled projects -- looking for a checkout and cloning if not present wouldn't be complicated.

#!/usr/bin/python

import json
import requests

projects = json.loads(requests.get('https://gerrit.wikimedia.org/r/projects/?pp=0').text[4:])

for project in projects:
	if projects[project]['state'] != 'ACTIVE':
		print('{}: Project isn\'t active'.format(project))
		continue
	if 'web_links' not in projects[project] or len(projects[project]['web_links']) == 0:
		print('{}: No web links are available for this project'.format(project))
		continue

	url = None
	for web_link in projects[project]['web_links']:
		if web_link['name'] == 'gitiles':
			url = web_link['url']
	if not url:
		print('{}: No gitiles url found'.format(project))
		continue

	# Try getting the scap directory
	scap_request = requests.get('{}/+/HEAD/scap'.format(url))
	if scap_request.status_code == 404:
		print('{}: No scap dir found'.format(project))
	else:
		print('{}: SCAP-ish!'.format(project))

I think nthis request is ultimatly asking for $_role support in the wmcs puppet hiera config. This is something that has come up a few times before specifically

In T280324#7008427, @dcaro wrote:

Some previous conversations on trying to add roles to the cloud hiera lookup:

• T120165 -> two years ago, got denied in favor of horizon puppet prefixes
• T211029 -> three years ago, when it was decided to use version control for the cloud enc
• T127771 -> three years ago, a request to add role support, unblocked by horizon puppet prefixes

and T280324 which was resolved as the machine got decommissioned.

@dcaro had a patch out for this but it got abandoned when the above got closed. perhaps they can resurect that patch. in the mean time i think that scap::sources have been added to hieradata/cloud/eqiad1/deployment-prep/common.yaml

I believe the following 2 steps should result in the same thing we would get if the role keyword was working:

• copy/paste the content from hieradata/role/common/deployment_server/kubernetes.yaml into hieradata/cloud/eqiad1/deployment-prep/common.yaml

• edit a puppet project or prefix tab in Horizon and ensure all the profiles are applied that are also included in modules/role/manifests/deployment_server.pp