Page MenuHomePhabricator

WDQS puppet/hiera configs are too distributed
Closed, ResolvedPublic

Description

Right now, to change some value on WDQS cluster, we need to edit at least 5 hiera files:

hieradata/labs/wikidata-query/common.yaml
hieradata/role/common/wdqs.yaml
hieradata/role/common/wdqs/autodeploy.yaml
hieradata/role/common/wdqs/internal.yaml
hieradata/role/common/wdqs/test.yaml

Example: https://gerrit.wikimedia.org/r/c/operations/puppet/+/475455

This is both time-consuming and fragile - it is extremely easy to forget one of the files or mess up one of the changes, and it's hard to verify change on 14 hosts and ensure it's the same everywhere. It would be much better if we have one "master" config and other configs would contain only differences from the master, not whole set of variables no matter what, that need to be edited each time.

Event Timeline

Restricted Application added a project: Wikidata. · View Herald TranscriptNov 26 2018, 6:25 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
jcrespo moved this task from Backlog to Radar on the SRE board.Nov 27 2018, 4:58 PM
jcrespo added a subscriber: jcrespo.

Your ticket seems reasonable, however I am unsure if you need something from SRE explicitly at the moment- AFAIK, nothing in the puppet style policy requires such a complex setup. Maybe common information should be moved to a profile or to a different hiera pattern. You are encouraged to do architectural improvements on puppet as it better suits your needs. Moving this to "radar" unless you have a concrete actionable for us at the time- but of course feel free to ask questions.

Gehel added a comment.Dec 3 2018, 9:57 PM

What we should probably do in this case is define default values to the hiera calls in profile::wdqs, and override only what needs to be different. At least for parameters where a default would make sense.

Change 479376 had a related patch set uploaded (by Mathew.onipe; owner: Mathew.onipe):
[operations/puppet@production] wdqs: reduce hiera configs via profile defaults

https://gerrit.wikimedia.org/r/479376

Mathew.onipe triaged this task as Medium priority.

Change 479376 merged by BBlack:
[operations/puppet@production] wdqs: reduce hiera configs via profile defaults

https://gerrit.wikimedia.org/r/479376

Addshore moved this task from incoming to monitoring on the Wikidata board.Dec 19 2018, 12:45 PM
debt closed this task as Resolved.Dec 21 2018, 1:35 PM