Page MenuHomePhabricator

scap phatality deployment problem
Closed, ResolvedPublic

Description

18:05:47 releng/phatality: fetch stage(s): 100% (in-flight: 0; ok: 12; fail: 0; left: 0)
18:05:48 releng/phatality: config_deploy stage(s): 100% (in-flight: 0; ok: 12; fail: 0; left: 0)
18:05:51 ['/usr/bin/scap', 'deploy-local', '-v', '--repo', 'releng/phatality', '-g', 'default', 'promote', '--refresh-config'] (ran as deploy-service@logstash1032.eqiad.wmnet) returned [1]: Registering scripts in directory '/srv/deployment/releng/phatality-cache/revs/c2cb594a75503ba4c25e2e8653b56efa3950abba/scap/scripts'
Registering scripts in directory '/srv/deployment/releng/phatality-cache/revs/c2cb594a75503ba4c25e2e8653b56efa3950abba/scap/scripts'
Executing check 'install_zip'
Check 'install_zip' failed: Removing existing phatality plugin before installation
Removing phatality...
Unable to remove plugin because of error: "EACCES: permission denied, unlink '/usr/share/opensearch-dashboards/plugins/phatality/opensearch_dashboards.json'"

Executing check 'restart_dashboards'
Check 'restart_dashboards' completed, output:

18:05:52 releng/phatality: promote stage(s): 100% (in-flight: 0; ok: 11; fail: 1; left: 0)
18:05:52 1 targets had deploy errors
18:05:52 1 targets failed

A general problem here is that a failing check does not appear to stop the next check from running. That's not acceptable if the checks are used for installation steps as they are in the phatality repo.

Event Timeline

dancy triaged this task as Medium priority.Mon, Sep 16, 6:18 PM
dancy updated the task description. (Show Details)
dancy renamed this task from scap phataility deployment problem to scap phatality deployment problem.Mon, Sep 16, 6:22 PM

Change #1073286 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[releng/phatality@master] scap/checks.yaml: Combine two installation step checks into one

https://gerrit.wikimedia.org/r/1073286

Change #1073286 merged by jenkins-bot:

[releng/phatality@master] scap/checks.yaml: Combine two installation step checks into one

https://gerrit.wikimedia.org/r/1073286

Mentioned in SAL (#wikimedia-operations) [2024-09-17T14:46:56Z] <dancy@deploy1003> Started deploy [releng/phatality@84c7283]: T374880

Mentioned in SAL (#wikimedia-operations) [2024-09-17T14:47:05Z] <dancy@deploy1003> Finished deploy [releng/phatality@84c7283]: T374880 (duration: 00m 09s)

Mentioned in SAL (#wikimedia-operations) [2024-09-17T14:52:36Z] <dancy@deploy1003> Started deploy [releng/phatality@84c7283]: T374880

Mentioned in SAL (#wikimedia-operations) [2024-09-17T14:52:42Z] <dancy@deploy1003> Finished deploy [releng/phatality@84c7283]: T374880 (duration: 00m 06s)

Considering this resolved after https://gerrit.wikimedia.org/r/1073286. However I do have some remaining suspicions about the rollback behavior should scap deploy fail. I don't get the impression that the checks run on rollback, so a rollback isn't really a rollback for phatality.