17:54 < mutante> !log tin - keyholder disarm and arm again using new passphrase
16:51 < mutante> !log mira - arming keyholder after setting service/dumps/eventlogging/phabricator key passphrases to the same one 
                 (T154943)

16:56 < icinga-wm> PROBLEM - Keyholder SSH agent on tin is CRITICAL: CRITICAL: Keyholder is not armed. Run keyholder arm to arm it.


16:58 < mutante> oh come on, it's totally armed per "status"
17:01 < volans> mutante: the paths are missing
17:02 < volans> no sorry
17:02 < mutante> volans: you mean in the output of "keyholder status"? that's just the key comments
17:02 < mutante> which are not set
17:02 < volans> mutante: I thought was that but actually is the ssh-add -l that is empty

17:03 < volans> the command in /usr/lib/nagios/plugins/check_keyholder applied to the outout of ssh-add is returning just rsa
17:03 < volans> because that should be the path
17:04 < volans> and it doesnt' match with the sorted paths in configured_keys() that are taken from a find

17:08 < mutante> volans: hmm, i dont get why analytics would be the only one different
17:08 < mutante> thanks, so i guess i removed the key comment by just not providing it
17:09 < volans> mutante: yeah, seems so
17:09 < mutante> but weird how one key kept it

17:26 < mutante> volans: "Comments are only supported for RSA1 keys.
17:27 < mutante> so even with -C / -c not adding the comment silently
17:29 < volans> mutante: how did they have the comment before?
17:29 < volans> maybe rsa1 is the only one that allow to change it?
17:32 < mutante>  This operation is only supported for RSA1 keys. 

17:32 < volans> adding them manually in the public key is not enough I guess... :-P

...

18:31 < icinga-wm> RECOVERY - Keyholder SSH agent on mira is OK: OK: Keyholder is armed with all configured keys.
18:32 < volans> mutante: \o/
18:33 < mutante> yea, but it's only a revert :/
18:33 < mutante> i talked in #openssh for a while even
18:33 < mutante> they say old ssh-add version .. but .. eh
18:34 < mutante> and that's not the comment line, that's just the full path to the key
18:34 < mutante> https://paste.pound-python.org/show/UOsCwU9cRuEPVdLKcLxm/  etc
18:34 < volans> that disappear when changing the password?
18:34 < mutante> yes
18:34 < mutante> except for one of them
18:34 < volans> WAT?!?!

18:35 < mutante> 18:18 < BasketCase> I am not sure why manual ssh-add doesn't add a comment
18:35 < mutante> 18:19 < BasketCase> also, manual ssh-add actually added the comment from the pub file for my ed25519 key but not my rsa key
18:35 < mutante> 18:22 < BasketCase> I am on 7.3 btw
18:35 < mutante> 18:22 < BasketCase> not whatever obsolete junk Debian has

18:36 < mutante> yea, eh, i am reverting to get back to it later
18:36 < mutante> totally unexpected rabbit hole here
18:36 < volans> ok, otherwise we could just create new ones
18:36 < mutante> yes, true
18:37 < volans> probably easier and also safer, as an not-scheduled key rotation :)

18:39 < icinga-wm> RECOVERY - Keyholder SSH agent on tin is OK: OK: Keyholder is armed with all configured keys.

18:14 < mutante> man ssh-keygen -c Requests changing the comment in the private and public key files. "
18:14 < BasketCase> at least it is on my system
18:14 < BasketCase> yes, v1 keys had an embedded comment
18:14 < mutante> 4096 35:c7:36:72:0f:a9:9e:0c:a8:d7:bb:5e:bf:17:9b:94 /etc/keyholder.d/analytics_deploy (RSA)
18:14 < mutante> 2048 6d:54:92:8b:39:10:f5:9b:84:40:36:ef:3c:9a:6d:d8 rsa w/o comment (RSA)
18:15 < mutante> i have this stuff
18:15 < mutante> see the "w/o comment" part
18:15 < mutante> that is my problem
18:15 < BasketCase> your version is old
18:16 < mutante> so i have to create new keys?
18:16 < mutante> still doesnt get how he removed the existing comments 
18:16 < BasketCase> no, your ssh-add command is old
18:16 < mutante> while changing passphrase
18:16 < BasketCase> it is still using an md5 hash
18:16 < BasketCase> when I look deeper, if I manually add a key to my agent it says there is no comment like that
18:17 < BasketCase> my keys added via pam_ssh have the file name as the comment
18:17 < BasketCase> neither is the comment stored in the file

18:17 < mutante> hmm.. i need the file name in the comment for all of them, yes

18:18 < BasketCase> I am not sure why manual ssh-add doesn't add a comment
18:19 < BasketCase> also, manual ssh-add actually added the comment from the pub file for my ed25519 key but not my rsa key

18:20 < mutante> ugh, did _not_ expect these changes
18:22 < BasketCase> https://paste.pound-python.org/show/UOsCwU9cRuEPVdLKcLxm/
18:22 < BasketCase> I am on 7.3 btw
18:22 < BasketCase> not whatever obsolete junk Debian has
18:24 < BasketCase> I have no idea why ed25519 was treated differently there
18:24 < mutante> i see. thanks.. hmm
18:25 < BasketCase> but the agent I get from pam_ssh shows this: https://paste.pound-python.org/show/p7xcGzVOzLOzKlWoEVEG/
18:25 < BasketCase> none of those are the real comment but they are something specific at least
18:26  * BasketCase loves pam_ssh

18:28 < BasketCase> if your ssh key fingerprints are still md5 I am not sure if my suggestion of ssh-keygen -o will work
18:29 < BasketCase> I think the new format came before a secure hash but I don't feel like looking it up to make sure
18:29 < mutante> ok, for now i am taking the files before i touched them at all
18:29 < BasketCase> if you have ed25519 support you have ssh-keygen -o
18:31 < mutante> so after reverting to the unchanged keyfiles i have the file pathes back 
18:31 < mutante> for all of them. same ssh-keygen
18:33 < BasketCase> did you do the new format transition I was suggesting when you upgraded?  maybe the current behavior doesn't apply to the old format?
18:34 < BasketCase> the file command would identify the private key files as different types
18:36 < BasketCase> "PEM RSA private key" is the old format "OpenSSH private key" is the new format
18:36 < BasketCase> (assuming a modern file magic)
18:39 < mutante> i did not to that yet, no
18:39 < mutante> i think i'll create new keys tomorrow

18:39 < BasketCase> then I am not sure why behavior changed
18:40 < BasketCase> the new format is much more resistant to brute force attacks btw

18:40 < BasketCase> essentially, the old kdf was just md5
18:41 < BasketCase> so the encryption key was just an md5 hash

[tin:~] $ sudo keyholder status
keyholder-agent: active
..
- 2048 6d:54:92:8b:39:10:f5:9b:84:40:36:ef:3c:9a:6d:d8 /etc/keyholder.d/deploy_service (RSA)
- 2048 86:c9:17:ab:b7:00:79:b5:8a:c5:b5:ee:29:24:c9:2f /etc/keyholder.d/dumpsdeploy (RSA)

[tin:~] $ sudo keyholder status
keyholder-agent: active
..
- 2048 6d:54:92:8b:39:10:f5:9b:84:40:36:ef:3c:9a:6d:d8 rsa w/o comment (RSA)
- 2048 86:c9:17:ab:b7:00:79:b5:8a:c5:b5:ee:29:24:c9:2f rsa w/o comment (RSA)
..