Page MenuHomePhabricator
Create Task
Maniphest T303272

Move VTRS db passwords to a different hiera location
Closed, ResolvedPublic
Actions

Description

The mariadb grants file for m2 (modules/profile/templates/mariadb/grants/production-m2.sql.erb) need to reference the VTRS db passwords. This is currently not possible, as those passwords live in a part of hiera that is not accessible to the mariadb profile.

@jbond suggested that if we move the VRTS passwords to hieradata/common/profile/vrts.yaml, then they can be referenced from both the VTRS and mariadb roles.

I have a CR that makes the required changes to the mariadb code: https://gerrit.wikimedia.org/r/c/operations/puppet/+/764744, but the VTRS side needs to change first.

Details

SubjectRepoBranchLines +/-
C:vtrs: dont unwrap this password as its not Sensitiveoperations/puppetproduction+1 -1
P:mail::mx: move passwords to hieraoperations/puppetproduction+8 -8
P:mail::mx: move passwords to hieraoperations/puppetproduction+5 -5
mariadb: Reference the actual VRTS passwords in the m2 grants file.operations/puppetproduction+8 -6
vtrs: move password to profile name spacelabs/privatemaster+0 -0
Customize query in gerrit

Related Objects

Event Timeline

Kormat created this task.Mar 8 2022, 2:35 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 8 2022, 2:35 PM
gerritbot added a comment.Mar 8 2022, 2:35 PM

Change 764744 had a related patch set uploaded (by Kormat; author: Kormat):

[operations/puppet@production] mariadb: Reference the actual OTRS passwords in the m2 grants file.

https://gerrit.wikimedia.org/r/764744

gerritbot added a project: Patch-For-Review.Mar 8 2022, 2:35 PM
jbond triaged this task as Medium priority.Mar 21 2022, 11:39 AM
jbond added a project: Znuny.
jbond added a comment.EditedMar 22 2022, 12:54 PM

The change has been made on the private repo

git show b9303238                                                                              [12:52:40]
commit b9303238c94fe4a839a30d38f1ee8df919678e01 (HEAD -> master)
Author: gitpuppet for private repo <git@puppetmaster1001.eqiad.wmnet>
Date:   Tue Mar 22 12:52:08 2022 +0000

    (jbond) vtrs: move password to profile name space
    
    Allows access to db hosts
    
    Bug: T303272

diff --git a/hieradata/role/common/vrts.yaml b/hieradata/common/profile/vrts.yaml
similarity index 100%
rename from hieradata/role/common/vrts.yaml
rename to hieradata/common/profile/vrts.yaml

and labs private

gerritbot added a comment.Mar 22 2022, 12:56 PM

Change 772821 had a related patch set uploaded (by Jbond; author: John Bond):

[labs/private@master] vtrs: move password to profile name space

https://gerrit.wikimedia.org/r/772821

gerritbot added a comment.Mar 22 2022, 12:56 PM

Change 772821 merged by Jbond:

[labs/private@master] vtrs: move password to profile name space

https://gerrit.wikimedia.org/r/772821

jbond mentioned this in rLPRIa74c1059fc55: vtrs: move password to profile name space.Mar 22 2022, 1:19 PM
Dzahn added a comment.Mar 24 2022, 11:46 PM

@Arnoldokoth Are you already aware of this change?

gerritbot added a comment.Apr 29 2022, 9:53 AM

Change 764744 merged by Kormat:

[operations/puppet@production] mariadb: Reference the actual VRTS passwords in the m2 grants file.

https://gerrit.wikimedia.org/r/764744

Maintenance_bot removed a project: Patch-For-Review.Apr 29 2022, 10:31 AM
Arnoldokoth added a comment.EditedAug 19 2022, 6:37 PM

Hey @Kormat @jbond I was wondering if this was resolved. I noticed the file now references some vrts passwords.

# less modules/profile/templates/mariadb/grants/production-m2.sql.erb  | grep "vrts"
IDENTIFIED BY '<%= @vrts_exim_database_pass %>'
IDENTIFIED BY '<%= @vrts_database_pw %>'
IDENTIFIED BY '<%= @vrts_exim_database_pass %>'
IDENTIFIED BY '<%= @vrts_database_pw %>'
IDENTIFIED BY '<%= @vrts_exim_database_pass %>'
IDENTIFIED BY '<%= @vrts_database_pw %>'
jijiki moved this task from Incoming 🐫 to 🙈🙉🙊Backlog on the serviceops board.Sep 28 2022, 2:19 PM
LSobanski edited projects, added collaboration-services; removed serviceops.Oct 20 2022, 12:31 PM
Dzahn added a comment.Oct 20 2022, 3:50 PM

@Arnoldokoth @jbond Looks to me like what has happened is that a new password has been added in hieradata/common/profile/vrts.yaml but there is still an old password both in modules/passwords/manifests/init.pp and in hieradata/role/common/mail/mx.yaml.

It's called $vrts_mysql_password and has the same value but it's not equal to the value of profile::vrts::database_pass.

I think what needs to happen is to double check if that old one is still valid and if code uses it and then to remove it from the old places.

Dzahn added a comment.Oct 21 2022, 9:16 PM

I did a mysql -h m2-master.eqiad.wmnet -u otrs -p otrs from otrs1001 and could confirm that the password at hieradata/common/profile/vrts.yaml:profile::vrts::database_pass works for that and only exists once.

The password at modules/passwords/manifests/init.pp: $vrts_mysql_password which has the same value as the hieradata/role/common/mail/mx.yaml:profile::mail::mx::vrts_mysql_password and hieradata/common/profile/vrts.yaml:profile::vrts::exim_database_pass is separate and for exim.

It is still duplicated in multiple places.

gerritbot added a comment.Oct 24 2022, 9:07 AM

Change 848214 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] P:mail::mx: move passwords to hiera

https://gerrit.wikimedia.org/r/848214

gerritbot added a project: Patch-For-Review.Oct 24 2022, 9:07 AM
gerritbot added a comment.Oct 24 2022, 9:32 AM

Change 848214 merged by Jbond:

[operations/puppet@production] P:mail::mx: move passwords to hiera

https://gerrit.wikimedia.org/r/848214

gerritbot added a comment.Oct 24 2022, 9:37 AM

Change 845761 had a related patch set uploaded (by Jbond; author: Jbond):

[operations/puppet@production] P:mail::mx: move passwords to hiera

https://gerrit.wikimedia.org/r/845761

gerritbot added a comment.Oct 24 2022, 9:48 AM

Change 845761 merged by Jbond:

[operations/puppet@production] P:mail::mx: move passwords to hiera

https://gerrit.wikimedia.org/r/845761

jbond closed this task as Resolved.Oct 24 2022, 10:02 AM
jbond claimed this task.
In T303272#8336257, @Dzahn wrote:

The password at modules/passwords/manifests/init.pp: $vrts_mysql_password which has the same value as the hieradata/role/common/mail/mx.yaml:profile::mail::mx::vrts_mysql_password and hieradata/common/profile/vrts.yaml:profile::vrts::exim_database_pass is separate and for exim.

It is still duplicated in multiple places.

AFAIK the password was only duplicated in theses two places. i have updated the config in hieradata/role/common/mail/mx.yaml to alias the value in hieradata/common/profile/vrts.yaml i have also removed passwords::exim as its not no longer used

Dzahn awarded a token.Oct 24 2022, 2:51 PM
Arnoldokoth added a comment.Oct 24 2022, 7:15 PM

Hey @jbond Seems like the merge broke Puppet on otrs1001.eqiad.wmnet. It fails with the following error:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, Failed to parse template vrts/exim4.conf.vrts.erb:
  Filepath: /etc/puppet/modules/vrts/templates/exim4.conf.vrts.erb
  Line: 22
  Detail: undefined method `unwrap' for "<redacted>":String
 (file: /etc/puppet/modules/vrts/manifests/mail.pp, line: 41, column: 20) on node otrs1001.eqiad.wmnet
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Arnoldokoth reopened this task as Open.Oct 24 2022, 7:15 PM
Dzahn added a comment.Oct 24 2022, 7:28 PM

Seems like "unwrap" is ok in .epp templates but not in .erb templates.

Should the exim .erb template be converted to an .epp template to fix this?

https://puppet.com/docs/puppet/7/lang_template_epp.html#lang_template_epp

vs

https://puppet.com/docs/puppet/7/lang_template_erb.html

gerritbot added a comment.Oct 24 2022, 7:34 PM

Change 848481 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] C:vtrs: dont unwrap this password as its not Sensitive

https://gerrit.wikimedia.org/r/848481

gerritbot added a comment.Oct 24 2022, 7:43 PM

Change 848481 merged by Jbond:

[operations/puppet@production] C:vtrs: dont unwrap this password as its not Sensitive

https://gerrit.wikimedia.org/r/848481

jbond closed this task as Resolved.Oct 24 2022, 7:47 PM
In T303272#8339600, @Arnoldokoth wrote:

Hey @jbond Seems like the merge broke Puppet on otrs1001.eqiad.wmnet. It fails with the following error:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, Failed to parse template vrts/exim4.conf.vrts.erb:
  Filepath: /etc/puppet/modules/vrts/templates/exim4.conf.vrts.erb
  Line: 22
  Detail: undefined method `unwrap' for "<redacted>":String
 (file: /etc/puppet/modules/vrts/manifests/mail.pp, line: 41, column: 20) on node otrs1001.eqiad.wmnet
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Sorry about that should be fixed now

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL