Welcome @hnowlan!

This checklist is from a template for onboarding in SRE.

I started by adding you to the ops mailing lists and i can see you already have a Wikitech user (great!).

Feel free to start with something like the checkbox confirming you can login on Phabricator and setup 2FA.

Also if you want to generate an SSH key and paste it here on the ticket that would be a step towards getting your shell access setup.

I'll talk to you about the details and other check boxes tomorrow. (am in PST)

SSH key: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICIFfF8+3TrSBaPBKPwbmnBM7e0C9/TFHs9/2hHiq+3t nosmo@ocasey

Change 563557 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] admins: add Hugh Nowlan to ldap_only_admins (wmf)

https://gerrit.wikimedia.org/r/563557

Change 563557 merged by Dzahn:
[operations/puppet@production] admins: add Hugh Nowlan to ldap_only_admins (wmf)

https://gerrit.wikimedia.org/r/563557

Mentioned in SAL (#wikimedia-operations) [2020-01-10T19:47:00Z] <mutante> LDAP - add Hugh Nowlan to "wmf" group (T242309)

@hnowlan The LDAP group gave you access to a bunch of web-based logins now: See https://wikitech.wikimedia.org/wiki/LDAP/Groups#wmf_group

@hnowlan One more thing we'll need for the "pwstore" part will be a GPG key. If you already have one or want to create one you can go ahead uploading that to a keyserver and getting some signatures from other SREs.

Thanks Daniel! I've created a new key and uploaded it here http://keys.gnupg.net/pks/lookup?op=vindex&fingerprint=on&search=0x63514D67ADFD2615

Change 564171 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] admin: upgrade Hugh Nowlan to root shell user (ops)

https://gerrit.wikimedia.org/r/564171

To whomever needs the comment, as Hugh Nowlan's manager I approve his being approved for shell access, provided it is approved by the appropriate individuals within SRE.

Change 564171 merged by Giuseppe Lavagetto:
[operations/puppet@production] admin: upgrade Hugh Nowlan to root shell user (ops)

https://gerrit.wikimedia.org/r/564171

@Dzahn can we please ensure this procedure is finished before next week?

Change 566823 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] icinga: let Hugh Nowlan run commands on all hosts and services

https://gerrit.wikimedia.org/r/566823

Change 566823 merged by Dzahn:
[operations/puppet@production] icinga: let Hugh Nowlan run commands on all hosts and services

https://gerrit.wikimedia.org/r/566823

Regarding the GPG key i see it on the keyserver but it has no new signatures yet. Looks like we are waiting for upload of a new version with the new sigs on it.

Unfortunately it seems I don't have permissions to issue commands. I attempted to downtime a service on a host that's not yet in use (restbase2023) and received a "Not Authorized" error. And yeah, I don't think people have signed things since all-hands yet.

@hnowlan There's an error in the username configured in https://gerrit.wikimedia.org/r/566823, let me fix that.

Change 570611 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Fix username in Icinga authorization config for Hugh

https://gerrit.wikimedia.org/r/570611

Change 570611 merged by Muehlenhoff:
[operations/puppet@production] Fix username in Icinga authorization config for Hugh

https://gerrit.wikimedia.org/r/570611

Moritz clarified how case sensitive logins affect Icinga - I've since logged in as Hnowlan and I can confirm I can run commands successfully.

Aww, thanks for conforming and thanks Moritz for fixing it. This is exactly why i wanted to test it. The capitalization caught us a couple times before.

@hnowlan can you provide you gpg key id

My GPG key ID is 0x63514D67ADFD2615, fp is B858 7E40 78C9 95C4 AAF0 1CFC 6351 4D67 ADFD 2615 (on keyservers here). It doesn't have any signatures yet, don't think people have gotten to those post all-hands

Oh good I'm guessing from your response that you did attend the key signing party at all hands. Also should mention that the link you provided didn't work for me i hit this error this is likely because sks is now pretty broken. i have uploaded your key to keys.openpgp.org however I'm not sure what the current guidance is on what keyservers to use wikitech still lists pool.sks-keyservers.net which dose have you key.

@MoritzMuehlenhoff should we still be using pool.sks-keyservers.net or should we update our documentation to use keys.openpgp.org (which strips signatures so not great), or potential set up our own server?

@MoritzMuehlenhoff: Could you please answer the last comment? Thanks!

While the keyserver networks have some structural issues which are pending some changes and a number of keys have been DDoSed by malicious key updates, there's currently not actual impediment to simply continue to use the SKS network (GPG clients have also added countermeasures against malformed keys, which were shipped in security updates of distros). As such, we can simply continue to use it for now.

Long term I'd like to simply store GPG keys for @wikimedia.org staff in our DNS zone file.

@hnowlan Do you see a signature other than your own with gpg --list-sigs 63514D67ADFD2615?

Could you gpg --armor --output hnowlan.pub --export 0x63514D67ADFD2615 and upload hnowlan.pub to a production server. For example a bastion host or https://people.wikimedia.org/~hnowlan/ or even just paste it here on the ticket or a pastebin and let me know?

Thanks!

its getting harder and harder to use signitures with gpg. Anyway i found a signd copy available on keys.gnupg.net which is signed by myself [8B4182B1196B2D27] and Riccardo [ 7509CEA4650AE684]

Here's a copy of the signed key https://people.wikimedia.org/~hnowlan/hnowlan.asc which is the same as the one on keys.gnupg.net.

wget https://people.wikimedia.org/~hnowlan/hnowlan.asc
..
 gpg --import hnowlan.asc 
gpg: key 63514D67ADFD2615: "Hugh Nowlan <hnowlan@wikimedia.org>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1

Note how it stays "unchanged" no matter how i import it.

Then i remove it completely and reimport it again and still nothing:

gpg --delete-key 63514D67ADFD2615

 gpg --import hnowlan.asc

 gpg --list-sigs 63514D67ADFD2615
pub   rsa4096 2020-01-13 [SC] [expires: 2022-01-12]
      B8587E4078C995C4AAF01CFC63514D67ADFD2615
uid           [ unknown] Hugh Nowlan <hnowlan@wikimedia.org>
sig 3        63514D67ADFD2615 2020-01-13  Hugh Nowlan <hnowlan@wikimedia.org>
sub   rsa4096 2020-01-13 [E] [expires: 2022-01-12]
sig          63514D67ADFD2615 2020-01-13  Hugh Nowlan <hnowlan@wikimedia.org>

In T242309#6092868, @Dzahn wrote:

gpg --keyserver keys.gnupg.net --search-keys hnowlan@wikimedia.org

Yes i know its very annoying, i get similar, has my signature obviously but missing Riccardo's :(

git/cas-overlay-template [ gpg --keyserver keys.gnupg.net --receive-keys 63514D67ADFD2615         staging ] 12:32 PM
gpg: key 63514D67ADFD2615: "Hugh Nowlan <hnowlan@wikimedia.org>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
git/cas-overlay-template [ gpg --list-sigs B8587E4078C995C4AAF01CFC63514D67ADFD2615               staging ] 12:32 PM
pub   rsa4096 2020-01-13 [SC] [expires: 2022-01-12]
      B8587E4078C995C4AAF01CFC63514D67ADFD2615
uid           [  full  ] Hugh Nowlan <hnowlan@wikimedia.org>
sig 3        63514D67ADFD2615 2020-01-13  Hugh Nowlan <hnowlan@wikimedia.org>
sig          8B4182B1196B2D27 2020-04-21  John Bond (Wikimedia) <jbond@wikimedia.org>
sub   rsa4096 2020-01-13 [E] [expires: 2022-01-12]
sig          63514D67ADFD2615 2020-01-13  Hugh Nowlan <hnowlan@wikimedia.org>

It worked after i manually copy/pasted the key from http://keys.gnupg.net/pks/lookup?op=get&search=0x63514D67ADFD2615 and imported it. Then i see the signatures. But if i use --recv-keys from _the same server_ they are not there. It's strange. But i can move forward now.

added Hugh to the .users file in pwstore, reencrypted files and pushed them. It should work now.