Page MenuHomePhabricator

Add DEPLOY_DIR env var to scap command checks
Open, NormalPublic

Description

Just talking about this in IRC with @bd808 and @Smalyshev

Scap3 command checks could execute in an environment where $DEPLOY_DIR is set to the directory of the current deployed rev.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 4 2017, 8:59 PM

Should we just make it the cwd during check execution?

+1 on setting the env var, but should it really be the current rev dir? Why not point it to the symlink?

thcipriani triaged this task as High priority.Jan 30 2017, 4:51 PM
thcipriani lowered the priority of this task from High to Normal.
thcipriani moved this task from Needs triage to Experiments on the Scap board.
awight added a subscriber: awight.Jan 30 2018, 9:25 PM

Should we just make it the cwd during check execution?

For my purposes (T181071) that would be great, so I'm submitting a patch. I'm unsure what the right directory is for the rollback stage, looking forward to the review comments :D

Why not point it to the symlink?

Any stage before promote won't have a symlink available, because the "current" links will still point to the current (old) version of the target repo.

awight added a comment.Feb 7 2018, 2:28 PM

Just noting that we're blocked by this bug, and I have a patch D955: Set cwd as a courtesy to fetch-check commands ready for review. Please let us know when this might make it into your team's workflow, thanks!

awight added a comment.EditedFeb 26 2018, 7:19 PM

I'm trying this out on beta, using target code https://gerrit.wikimedia.org/r/#/c/392682/ (PS 9) and deployment-tin reports scap version 3.8.0-1~20180222003831.298

There must be a missing import, because scap/script.py is included in the deployed version of scap.

19:14:56 [deployment-ores01.deployment-prep.eqiad.wmflabs] Unhandled error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/scap/cli.py", line 330, in run
    exit_status = app.main(app.extra_arguments)
  File "/usr/lib/python2.7/dist-packages/scap/deploy.py", line 149, in main
    status = self._execute_checks(stage, group)
  File "/usr/lib/python2.7/dist-packages/scap/utils.py", line 402, in context_wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/scap/deploy.py", line 480, in _execute_checks
    chks = checks.load(self.config)
  File "/usr/lib/python2.7/dist-packages/scap/checks.py", line 165, in load
    raise CheckInvalid(msg)
CheckInvalid: unknown check type 'script'

I'm trying this out on beta, using target code https://gerrit.wikimedia.org/r/#/c/392682/ (PS 9) and deployment-tin reports scap version 3.8.0-1~20180222003831.298
There must be a missing import, because scap/script.py is included in the deployed version of scap.

19:14:56 [deployment-ores01.deployment-prep.eqiad.wmflabs] Unhandled error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/scap/cli.py", line 330, in run
    exit_status = app.main(app.extra_arguments)
  File "/usr/lib/python2.7/dist-packages/scap/deploy.py", line 149, in main
    status = self._execute_checks(stage, group)
  File "/usr/lib/python2.7/dist-packages/scap/utils.py", line 402, in context_wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/scap/deploy.py", line 480, in _execute_checks
    chks = checks.load(self.config)
  File "/usr/lib/python2.7/dist-packages/scap/checks.py", line 165, in load
    raise CheckInvalid(msg)
CheckInvalid: unknown check type 'script'

Looks like this machine is running an old version of scap because puppet can't run due to:

thcipriani@deployment-ores01:~$ sudo tail /var/log/puppet.log
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Sleeping 7 for random splay
Info: Using configured environment 'future'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Could not find class role::ores::worker for deployment-ores01.deployment-prep.eqiad.wmflabs on node deployment-ores01.deployment-prep.eqiad.wmflabs
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
thcipriani@deployment-ores01:~$

I manually updated scap on this machine so this should work now. Filed T188316 for the puppet problems.

@thcipriani Wow, thanks for digging that up!

It's working well now.

Just noting here that the lack of cwd means my script has to do the following to calculate the deployment root:

deploy_dir="$( realpath "$( dirname "${BASH_SOURCE[0]}" )/../.." )"

It's working well now.
Just noting here that the lack of cwd means my script has to do the following to calculate the deployment root:

deploy_dir="$( realpath "$( dirname "${BASH_SOURCE[0]}" )/../.." )"

This should be the same path that is available inside any scap check being run via the new environment variable: $SCAP_REV_PATH. That points to the current revision directory being deployed.

Confirmed, thanks for reminding me about the new variable!

FWIW, I now have a script-check which has been smoke-tested on the beta cluster and behaves like we want. I've set this task to block mine, so we'll just throw a small party when it can all be deployed. No pressure ;-)

It is probably just about time for another scap release, eh? </canadianAccent>

awight removed a subscriber: awight.Mar 21 2019, 4:01 PM