Page MenuHomePhabricator

WDQS puppet/hiera configs are too distributed
Closed, ResolvedPublic

Description

Right now, to change some value on WDQS cluster, we need to edit at least 5 hiera files:

hieradata/labs/wikidata-query/common.yaml
hieradata/role/common/wdqs.yaml
hieradata/role/common/wdqs/autodeploy.yaml
hieradata/role/common/wdqs/internal.yaml
hieradata/role/common/wdqs/test.yaml

Example: https://gerrit.wikimedia.org/r/c/operations/puppet/+/475455

This is both time-consuming and fragile - it is extremely easy to forget one of the files or mess up one of the changes, and it's hard to verify change on 14 hosts and ensure it's the same everywhere. It would be much better if we have one "master" config and other configs would contain only differences from the master, not whole set of variables no matter what, that need to be edited each time.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
jcrespo added a subscriber: jcrespo.

Your ticket seems reasonable, however I am unsure if you need something from SRE explicitly at the moment- AFAIK, nothing in the puppet style policy requires such a complex setup. Maybe common information should be moved to a profile or to a different hiera pattern. You are encouraged to do architectural improvements on puppet as it better suits your needs. Moving this to "radar" unless you have a concrete actionable for us at the time- but of course feel free to ask questions.

What we should probably do in this case is define default values to the hiera calls in profile::wdqs, and override only what needs to be different. At least for parameters where a default would make sense.

Change 479376 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/puppet@production] wdqs: reduce hiera configs via profile defaults

https://gerrit.wikimedia.org/r/479376

Change 479376 merged by BBlack:
[operations/puppet@production] wdqs: reduce hiera configs via profile defaults

https://gerrit.wikimedia.org/r/479376