Page MenuHomePhabricator

scap: hide helmfile operations behind a progress bar
Closed, ResolvedPublic

Description

Scap currently logs both the start and end of each helmfile operation.

10:22:46 Started Running helmfile -e eqiad --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-web
10:22:46 Started Running helmfile -e eqiad --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-api-int
10:22:46 Started Running helmfile -e codfw --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-web
10:22:46 Started Running helmfile -e eqiad --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-api-ext
10:22:46 Started Running helmfile -e codfw --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-api-int
10:22:46 Started Running helmfile -e codfw --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-api-ext
10:22:46 Started Running helmfile -e eqiad --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-jobrunner
10:22:46 Started Running helmfile -e codfw --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-jobrunner
10:23:06 Finished Running helmfile -e codfw --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-jobrunner (duration: 00m 20s)
10:23:08 Finished Running helmfile -e eqiad --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-api-ext (duration: 00m 21s)
10:23:10 Finished Running helmfile -e eqiad --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-jobrunner (duration: 00m 23s)
10:23:11 Finished Running helmfile -e codfw --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-api-ext (duration: 00m 24s)
10:23:23 Finished Running helmfile -e eqiad --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-web (duration: 00m 37s)
10:23:24 Finished Running helmfile -e eqiad --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-api-int (duration: 00m 37s)
10:23:25 Finished Running helmfile -e codfw --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-api-int (duration: 00m 38s)
10:23:27 Finished Running helmfile -e codfw --selector name=canary apply in /srv/deployment-charts/helmfile.d/services/mw-web (duration: 00m 40s)

Can we hide the details of individual deployments and use a progress bar instead?

Details

Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Simple k8s deployment progress reportingrepos/releng/scap!292dancymaster-I609482aa2fed02a864543b5c3987d0dad91de254master
Customize query in GitLab

Event Timeline

dancy changed the task status from Open to In Progress.Apr 26 2024, 4:54 PM
dancy claimed this task.
dancy triaged this task as Medium priority.

Mentioned in SAL (#wikimedia-operations) [2024-04-26T17:48:27Z] <dancy@deploy1002> Started scap: Testing T325530

Deployed in scap 4.80.0

Sample output:

17:51:21 Finished sync-canaries-k8s (duration: 01m 07s)
17:51:21 Started sync-prod-k8s
17:52:32 K8s deployment progress:  14% (ok: 280; fail: 0; left: 1680) /

Mentioned in SAL (#wikimedia-operations) [2024-04-26T17:57:42Z] <dancy@deploy1002> Finished scap: Testing T325530 (duration: 09m 14s)