Wed, Feb 13
Unlike 1004 which is working properly, Apache on on 2003 is returning "(52) Empty reply from server"
Wed, Feb 6
Working, thanks @ayounsi !
Tue, Feb 5
This is done, incidentally the dumps are now big enough to cause disk alerts about /srv
Mon, Feb 4
Just kicked this off, will update when it finishes, probably tomorrow.
Wed, Jan 30
@Cmjohnson yep, sorry forgot about this ticket! Thanks for your help.
Jan 17 2019
@Pcoombe yes indeed, I made a subtask for that for tracking and to remind myself.
@Pcoombe unfortunately the available versions on Jessie (current OS) and Stretch (upcoming OS) are both old (0.14 and 0.18 respectively). Buster (2 OS versions out) has 1.0 but that's a ways off. So we'd want to look at packaging it ourselves which is doable, unless someone around here (analytics?) has already done that which would be easier.
Jan 16 2019
Jan 9 2019
@ayounsi thanks for the help, this looks good
Jan 8 2019
@ayounsi the new rules are at 1546987554
Jan 7 2019
@CDanis all good here, go ahead and remove the old service at your convenience.
@ayounsi thanks! Fundraising grafana is now fixed. i pushed up 1546890827 which removes krypton.
@ayounsi the updated config for this is at 1546888529
Deployed iptables change:
@Pythoncoder thanks for the report, I shuffled the tags around so this gets seen by the right people.
Jan 3 2019
@Eileenmcnaughton sure, looking at the grants, would that be just select on dev_*?
Jan 2 2019
Dec 19 2018
Dec 18 2018
While currently tech can see the alerts in IRC, Jeff and I get SMS about it, which means dropping what you are doing, getting out of bed, etc. So it would be good to keep that alert stream to things that are actionable by ops. Another icinga channel might be the answer, but there is the open question of who pays for our phones.
There are a couple related tickets here: T202419
Does this mean fr-tech wants to get paged by icinga?
@Cstone has logged in to everything
c917cff add cstone mysql grants
Dec 17 2018
07a03cd add cstone secrets
36e3166 add cstone user
Thanks everyone, inspiring to see the commitment to donor privacy in action.
Dec 13 2018
Dec 12 2018
@Pcoombe can we get contact/employment info for @Jksamra updated on https://collab.wikimedia.org/wiki/Fundraising#Contact_List ?
Dec 11 2018
@Dzahn yep looks good now! Thanks :)
@Dzahn it goes back as far as current syslog, I'll dig back and see when it started. Fwiw I can telnet to 126.96.36.199 5667 and it works, but 188.8.131.52 says Connection Refused. iptables&pfw both look to be open.
Dec 10 2018
Dec 6 2018
She got into Civi, we are awaiting the Yubikey for shell stuff.
This is fine for now.
Dec 4 2018
From: Lisa Gruwell <firstname.lastname@example.org> Date: Mon, 3 Dec 2018 14:57:51 -0800 Subject: Re: New employee access request To: Casey Dentinger <email@example.com>
Dec 3 2018
@Jksamra cleaning house, re-open if you have trouble logging in
Nov 29 2018
@TSkaff cleaning house, re-open if you are still having trouble
@spatton cleaning house, re-open if you are still having trouble
@Jksamra there are now 2 files in your home directory on frdev1001, jsamra.p12 (the encrypted certificate) and jsamra.pw (the password to decrypt the cert). Let me know if you need a hand retrieving/installing
Nov 28 2018
Nasty. Can you try removing/re-adding the cert?
@CCogdill_WMF hrm, I think this is a keychain problem based on: https://www.nicksherlock.com/2017/09/fixing-err_ssl_client_auth_signature_failed-on-macos/
Nov 27 2018
@Jksamra you should, collab is actually meant for contractor access, but at the moment I'm not sure who admins access to that wiki...
@spatton my mistake, I forgot you have an account on frdev1001, so that is where your cert and password are located. Hit me up on IRC if you need help retrieving them.
@Jksamra sure, we can get you set up. Can you update your contact info on here? https://collab.wikimedia.org/wiki/Fundraising#Contact_List
@spatton you should have received a new one on or around 10/22 via email, and the password via SMS. Can you check?
@TSkaff ok, you should have received the cert and password, let me know if you can get logged in
@CCogdill_WMF cert and password sent, let me know if it works
this is done
Nov 21 2018
@MNoorWMF ok, pw is sent, instructions are here: https://collab.wikimedia.org/wiki/Fundraising/Engineering/SSL_Client_Authentication
@MNoorWMF I use what is listed on: https://collab.wikimedia.org/wiki/Fundraising#Contact_List
Nov 20 2018
@MNoorWMF I sent you a new cert on 11/13, can you check your mail from then?
@LeanneS thanks! Closing, but re-open if there are problems.
@jkim_wikimedia cleaning house, please re-open if you need help.
@CCogdill_WMF try now, new key is in place
Nov 19 2018
@CCogdill_WMF if you still have your old ssh config, use that one. There is pretty much infinite variation to what it could look like so stick to what you are used to if possible.
@CCogdill_WMF ok, we'll set you up with a new key.
@jgleeson this looks resolved based on how that chart renders now, please re-open if I'm mistaken
Nov 16 2018
He should have the cert and password now
Nov 15 2018
She has logged in now
Nov 14 2018
@Jgreen figured this out, the root cert (is that the right term here?) had expired and not been replaced due to some puppet cruft. Wildly opaque and unhelpful nginx behavior. Good times.
@CCogdill_WMF @KHaggard ok these accounts are all set up, the next step would be ssh config: https://wikitech.wikimedia.org/wiki/Fundraising/tech/ssh_config
c59e853 add khaggard ssh key, yubikey, and mysql grants
8500d98 add khaggard shell account
Nov 13 2018
@CCogdill_WMF thanks, looks good, setting up the accounts now.
@KHaggard I show it as delivered to your number from: https://collab.wikimedia.org/wiki/Fundraising#Contact_List
Civi cert and password have been sent