Page MenuHomePhabricator

deployment-imagescaler02 is not responding to salt
Closed, DeclinedPublic

Description

krenair@deployment-salt02:~$ sudo salt 'deployment-imagescaler02.deployment-prep.eqiad.wmflabs' cmd.run id
krenair@deployment-salt02:~$

all the others are fine, just not this one

Event Timeline

Aug 19 15:50:16 deployment-imagescaler02 systemd[1]: Starting The Salt Minion...
Aug 19 15:50:16 deployment-imagescaler02 systemd[1]: Started The Salt Minion.
Aug 19 15:50:18 deployment-imagescaler02 salt-minion[13497]: [CRITICAL] The specified fingerprint in the master configuration file:
Aug 19 15:50:18 deployment-imagescaler02 salt-minion[13497]: 49:5c:c2:d5:7a:fc:33:a9:c6:81:ea:21:9b:ca:62:21
Aug 19 15:50:18 deployment-imagescaler02 salt-minion[13497]: Does not match the authenticating master's key:
Aug 19 15:50:18 deployment-imagescaler02 salt-minion[13497]: ce:70:88:21:59:a6:3e:7d:90:a9:b4:8d:7d:63:7a:a8:2d:34:76:79:55:0a:9f:a9:e2:63:2e:05:63:0f:4d:19
Aug 19 15:50:18 deployment-imagescaler02 salt-minion[13497]: Verify that the configured fingerprint matches the fingerprint of the correct master and that this minion is not subject to a man-in-the-middle
Aug 19 15:50:18 deployment-imagescaler02 systemd[1]: salt-minion.service: Main process exited, code=exited, status=42/n/a
Aug 19 15:50:18 deployment-imagescaler02 systemd[1]: salt-minion.service: Unit entered failed state.
Aug 19 15:50:18 deployment-imagescaler02 systemd[1]: salt-minion.service: Failed with result 'exit-code'.

am guessing this is md5 vs. sha256. also noticed this box is running Debian Stretch

krenair@deployment-salt02:~$ sudo cat /etc/salt/pki/master/master.pub | grep -v BEGIN | grep -v END | md5sum
495cc2d57afc33a9c681ea219bca6221  -
krenair@deployment-salt02:~$ sudo cat /etc/salt/pki/master/master.pub | grep -v BEGIN | grep -v END | sha256sum
ce70882159a63e7d90a9b48d7d637aa82d347679550a9fa9e2632e05630f4d19  -

It works if I set hash_type: md5 in /etc/salt/minion, guess I need to puppetise this one way or the other

For now I've just edited the individual instance's puppet config in horizon to use the sha256 fingerprint. @Andrew, we should think about which way we want to go on this. Setting hash_type to md5 will work but should we instead consider taking both fingerprints in config and using sha256 where possible?

Interesting, I seem to remember seeing something like this in production too but it self healed once puppet was running on the box