Page MenuHomePhabricator
Create Task
Maniphest T316560

Update the videoscaler alert to point at the correct runbook
Closed, ResolvedPublic
Actions

Description

The videoscaler alert points at a non-existent anchor in the runbooks page and should be updated.

11:41 AM <•jinxer-wm> (ProbeDown) firing: Service videoscaler:443 has failed probes (http_videoscaler_ip4) #page - https://wikitech.wikimedia.org/wiki/Runbook#videoscaler:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/service&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown

https://wikitech.wikimedia.org/wiki/Application_servers/Runbook#Jobrunners may be the right documentation (needs verification)?

Related Objects

Event Timeline

LSobanski created this task.Aug 29 2022, 3:10 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 29 2022, 3:10 PM
RhinosF1 added a subscriber: RhinosF1.Aug 29 2022, 3:16 PM

The way these docs work. You have to add a link to it as the run book is the same for all probe down alerts.

RhinosF1 updated the task description. (Show Details)Aug 29 2022, 3:16 PM
Dzahn added a subscriber: Dzahn.Aug 29 2022, 5:09 PM

What RhinosF1 said. There is only one runbook URL per check and if everything uses the same "Probe Down" check then they will all have the same link.

And combined with the move from Icinga to Alertmanager/Prometheus it is a question for observability.

RLazarus closed this task as Resolved.Aug 29 2022, 5:24 PM
RLazarus claimed this task.
RLazarus added a subscriber: RLazarus.

T312947 already tracks the larger question of how to organize runbooks for ProbeDown effectively.

In the specific case, I agree with @LSobanski that for this particular alert, Application_servers/Runbook#Jobrunners is the right place to link to, so I added a section https://wikitech.wikimedia.org/wiki/Runbook#videoscaler:443 (meaning the link in the alert now works) and I'm resolving this. (I added jobrunner:443 too, while I was at it.)

Dzahn added a comment.Aug 29 2022, 6:42 PM

Oh, good point about using the link with an anchor and just creating that. Thanks.

Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL