Page MenuHomePhabricator

Update Runbook wikis for the application and LVS servers
Closed, ResolvedPublic

Description

During a recent incident a request as made for the following wikies to be updated with fix instrauctions for SRE's

@Legoktm are you able to provide more info on what is missing

Event Timeline

jbond triaged this task as Medium priority.Mar 31 2021, 12:56 PM
jbond created this task.

I'm not sure if we should put jobrunner stuff on the LVS page, LVS was working fine, it just happened to be what paged because the entire jobrunner cluster was failing.

I added some more notes to the appservers runbook: https://wikitech.wikimedia.org/w/index.php?title=Application_servers%2FRunbook&type=revision&diff=1906219&oldid=1905942 that basically sums up what we did yesterday I believe.

akosiaris claimed this task.
akosiaris subscribed.

I'm not sure if we should put jobrunner stuff on the LVS page, LVS was working fine, it just happened to be what paged because the entire jobrunner cluster was failing.

Agreed. LVS paging was the paging symptom, not the cause itself. And the page was rather telling about the fact being about the jobrunners specifically. I think we lost 0 time caring about LVS specifically.

I added some more notes to the appservers runbook: https://wikitech.wikimedia.org/w/index.php?title=Application_servers%2FRunbook&type=revision&diff=1906219&oldid=1905942 that basically sums up what we did yesterday I believe.

LGTM. Thanks! I think we can resolve this, but feel free to reopen

Aklapper renamed this task from Update Runboook wikis for the application and LVS servers to Update Runbook wikis for the application and LVS servers.Apr 2 2021, 9:57 AM