Tue, Mar 2
Mon, Mar 1
Something worth considering here, in addition to the naming scheme for SLIs themselves, is recording metrics that represent the SLOs values themselves (e.g. 0.1 percent)
Mon, Feb 22
Thanks @ayounsi it's been re-enabled and puppet has been run
FWIW here's a quick review of current gerrit alerting in case it helps when thinking about checks to include in gitlab monitoring.
Fri, Feb 12
Wed, Feb 10
Sorry, I should have clarified this initially, afaict a proxy won't work for this case because logstash configures this at the JVM level and would have unwanted effects on the other inputs and outputs. So I was curious what other approaches might be recommended for this type of outward connection?
Hey @ayounsi, what approach would you recommend for outward connectivity from logstash frontend hosts (logstash1023 for instance) to imap.gmail.com:993?
Tue, Feb 9
LGTM thanks @Papaul!
@Papaul sure, sounds good. This host is not yet in production so there will be no prep/depool needed before the re-rack.
Mon, Feb 8
Feb 5 2021
Hey @Cmjohnson, @elukey, sure this should be no problem. I've set a reminder in my calendar to stop services on this host ahead of the window, and yup as long as the host/network config stays the same ES should do the right thing when services are brought back up. Would like to monitor it as it comes up though, just shoot a ping when ready. Thanks!
Feb 4 2021
Feb 2 2021
That's really exciting! Yes I'd love do see this happen as well, and am on board with the plan that you outlined. Time will be the main constraint for me right now, but yes let's get it started on prep work and then and if necessary can plan out the more time consuming components for the next Q.
Feb 1 2021
Hey @Cmjohnson, when do you estimate this one will be racked and installed?
Jan 27 2021
Jan 15 2021
I hear you, it depends on the use case a bit, but in general a screen shot or similar (along with saving useful views as visualizations and dashboards) will be more durable in the long-term because, for example, logs will age off after 90d.
Yes /goto/ links will need to be re-created. We have updated the links within the operations/puppet repository, and for things like bookmarks simply log in to logstash.wikimedia.org and search for the dashboard then hit share to obtain an updated /goto/ url.
Jan 14 2021
Jan 13 2021
Was hoping for some feedback on the above patch, but since it's been a few days I've gone ahead and merged it. The listinfo page in this task description looks to have improved to me, in that copy/pasting a sampling of text into a translator gives back a meaningful result. How does it look to you @Mormegil?
Jan 12 2021
Jan 7 2021
Jan 6 2021
Dec 16 2020
An API key for klaxxon (discussed via IRC, that's what this is going to be used for, see linked task as well) has been created and added to the pwstore file 'victorops'.
Dec 14 2020
Dec 11 2020
I think we can go without it, we plan to replace these older hosts in the near future and also have some logstash refresh hardware that was just ordered. Thanks!
Dec 10 2020
Dec 9 2020
The missing cache pop metrics have been backfilled using the above method and the thanos bucket web viewer no longer shows a gap. I think we're good here!
Dec 8 2020
After some testing, I think this may be a viable approach for backfilling:
Dec 3 2020
Dec 2 2020
Copies of the missing blocks have been made into /root/gap_blocks on each of the prometheus pop instances
Nov 20 2020
Apache2 on deployment-logstash03 was erroring with [auth_cas:error] [pid 18928:tid 139767719112768] MOD_AUTH_CAS: CASLoginURL or CASValidateURL not defined.
Nov 19 2020
Nov 18 2020
Hi @IJethroBT-WMF, the requested access has been granted. I'll transition this to closed now, but please reopen if any follow-up is needed. Thanks!
Hi @STran, you have been added to the wmf LDAP group. I'll transition this to closed now, but please reopen if any follow-up is needed. Thanks!
Hi @Swagoel, the requested access has been granted and will be fully active within 30 minutes. I'll transition this to closed now, but please reopen if any follow-up is needed. Thanks!
Hi @Tobias_Schumann_WMDE-ext, the requested access has been granted. I'll transition this to closed now, but please re-open if any follow-up is needed. Thanks!
Nov 17 2020
Hi @STran, for our records could you please give a high level description of what the requested access will be used for? Thanks in advance!
The requested shell and LDAP access has been granted, and will be fully active within 30 minutes. I'll transition this to closed now, but please re-open if any follow-up is needed. Thanks!
Hi @KEchavarriqueen, the requested group access has been granted. I'll transition this to closed now, but please don't hesitate to re-open if any follow up is needed. Thanks!
Nov 16 2020
Hi @hnowlan gmodena has been added to LDAP group wmf, and the above patch has been merged. Thanks for that!
Hi @KFrancis, could you please confirm that we have an NDA on file for Tobias? Thanks in advance!
I'll transition this to closed for the time being due to inactivity. When ready to proceed please add a comment of manager approval and re-open the task. Thanks in advance!
Hi @DNdubane_WMF, could you please coordinate obtaining a comment from your manager approving this request?