Page MenuHomePhabricator

Improve scap canary check messages
Closed, ResolvedPublic

Description

Proposed improvements:

  • 20:48:44 Waiting for canary traffic...
    • Indicate how long it is waiting, like: Waiting XX seconds for canary traffic...
    • Mostly hypothetical, but I suspect in an emergency I won't remember --force, would hit this message and be like oh crap. Would be nice if there was something like "type "skip" here to skip in case of emergencies".
  • 20:49:04 Executing check 'Logstash Error rate for mw1279.eqiad.wmnet'
    • Error should be lowercase probably.
    • Does it really need one message per host? These messages were basically half of all the sync-file output.

Details

TitleReferenceAuthorSource BranchDest Branch
canary_checks: Improve format of Waiting for canary traffic messagerepos/releng/scap!217dancymaster-I7240ab57c5461e5805562bfcf42a29b2d3680b83master
Mention how long we're waiting for canary trafficrepos/releng/scap!215dancymaster-I3745ed1b83962bc72c9e0874060cc9efbf61dea4master
Customize query in GitLab

Event Timeline

mmodell moved this task from Needs triage to Debt on the Scap board.
dancy lowered the priority of this task from Medium to Low.Feb 15 2024, 9:28 PM
dancy removed a subscriber: mmodell.

dancy opened https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/217

canary_checks: Improve format of Waiting for canary traffic message

dancy merged https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/217

canary_checks: Improve format of Waiting for canary traffic message

Latest canary phase output:

16:47:26 Started sync-check-canaries
16:47:46 sync-canaries: 100% (in-flight: 0; ok: 4; fail: 0; left: 0)
16:47:46 Per-host sync duration: average 17.4s, median 17.4s
16:47:46 rsync transfer: average 373,855 bytes/host, total 1,495,420 bytes
16:47:46 Finished Canaries Synced (duration: 00m 19s)
16:47:46 Executing check 'Check endpoints for mw2272.codfw.wmnet'
16:47:46 Executing check 'Check endpoints for mw2374.codfw.wmnet'
16:47:46 Executing check 'Check endpoints for mw2376.codfw.wmnet'
16:47:46 Executing check 'Check endpoints for mw2271.codfw.wmnet'
16:47:47 Finished Canary Endpoint Check Complete (duration: 00m 00s)
16:47:47 Waiting 20 seconds for canary traffic...
16:48:06 Executing check 'Logstash canary error rate'
16:48:06 Finished sync-check-canaries (duration: 00m 39s)
dancy claimed this task.

Scap's messages have been updated according to the suggestions made in this ticket (mostly as a side effect of the work on T357402).

This last bit:

  • Mostly hypothetical, but I suspect in an emergency I won't remember --force, would hit this message and be like oh crap. Would be nice if there was something like "type "skip" here to skip in case of emergencies".

is not merely changes message text, so it warrants its own ticket if this behavior is still desired.

Marking this ticket resolved.