Change Details

= Overview of how it would work = == Assumptions == * There is a user named `scap` that exists on all scap targets. * Members of the release engineering team can `ssh scap@<target>` from deploy1002. * A relenger has tagged a commit in scap for a new release. == Expected operation == On deploy server: * cd /srv/deployment/scap (which is a checkout of the scap git repo) * git checkout <some tag> * bin/scap install-scap * `scap install-scap` then: * Updates `__pycache__` (however that works) * runs some query to collect a list of hosts that should have scap installed * rsync's/srv/deployment/scap/scap/ to scap@target:/home/scap/scap/ for each host. == Prereqs == For this to work there must be a stub scap deb package preinstalled on all targets which contains a /usr/bin/scap which is a symlink to /home/scap/scap/bin/scap. This package would exclude the stuff currently in /usr/lib/python3/dist-packages/scap. [] Find a way to query the list of scap targets. See https://phabricator.wikimedia.org/T302919#7748986 for a recent list. I assume that cumin is involved. Whatever we use needs to be accessible from deployment.eqiad.wmnet. [] Need a user (maybe named `scap`, and assumed hereafter in this text) that can be ssh'd into on each of those hosts (similar to how deployers can ssh as mwdeploy to mediawiki targets during scap sync operations.. how is that set up? keyholder is used). [] Change scap's deb to just include the /usr/bin/scap -> /home/scap/scap/bin/scap symlink. [] New dependency: rsync [] What's the version number of this package? It needs to be something that won't be confusing. [] Looks like /usr/share/bash-completion/completions/scap should a few other bits should be included as well. Those could be symlinks too. == Bootstrapping the deploy server == Use puppet to * ensure that the scap stub deb is installed. * ensure that deploy-servers:/srv/deployment/scap is a clone of the scap git repo. Once checked out, puppet should leave it alone. == Transition plan == [] Run `scap install-scap` to prime the targets. Any uses of scap on these targets will continue to use the code from the current scap deb. [] Merge a commit that makes the scap deb be the stubbed form (just symlinks, as described above). Choose a version number for this deb. The version number selected must make clear that it is unrelated to any particular version of scap. [] Follow the usual scap release procedure to tag and request deployment by serviceops. [] When serviceops has completed deployment of the stub, /usr/bin/scap on targets will now point to the code previously installed by `scap install-scap` == Outstanding notes/problems== [] What about beta ? [] Installing scap somewhere other than /usr/lib/python3/dist-packages will break scripts that expect to be able to `import scap` (such as `stage-train` in the releases repo). Hopefully allowing scap to deploy itself will reduce the need/desire to extend scap outside of scap. [] What about freshly-provisioned hosts? Running scap on them will fail until someone runs `scap install-scap` on the deploy server.

= Overview of how it would work = == Assumptions == * There is a user named `scap` that exists on all scap targets. * Members of the release engineering team can `ssh scap@<target>` from deploy1002. * A relenger has tagged a commit in scap for a new release. == Expected operation == On deploy server: * cd /srv/deployment/scap (which is a checkout of the scap git repo) * git checkout <some tag> * bin/scap install-scap * `scap install-scap` then: * runs some query to collect a list of hosts that should have scap installed * rsync's/srv/deployment/scap/scap/ to scap@target:/home/scap/scap/ for each host. == Prereqs == For this to work there must be a stub scap deb package preinstalled on all targets which contains a /usr/bin/scap which is a symlink to /home/scap/scap/bin/scap. This package would exclude the stuff currently in /usr/lib/python3/dist-packages/scap. [] Find a way to query the list of scap targets. See https://phabricator.wikimedia.org/T302919#7748986 for a recent list. I assume that cumin is involved. Whatever we use needs to be accessible from deployment.eqiad.wmnet. [] Need a user (maybe named `scap`, and assumed hereafter in this text) that can be ssh'd into on each of those hosts (similar to how deployers can ssh as mwdeploy to mediawiki targets during scap sync operations.. how is that set up? keyholder is used). [] Change scap's deb to just include the /usr/bin/scap -> /home/scap/scap/bin/scap symlink. [] New dependency: rsync [] What's the version number of this package? It needs to be something that won't be confusing. [] Looks like /usr/share/bash-completion/completions/scap should a few other bits should be included as well. Those could be symlinks too. == Bootstrapping the deploy server == Use puppet to * ensure that the scap stub deb is installed. * ensure that deploy-servers:/srv/deployment/scap is a clone of the scap git repo. Once checked out, puppet should leave it alone. == Transition plan == [] Run `scap install-scap` to prime the targets. Any uses of scap on these targets will continue to use the code from the current scap deb. [] Merge a commit that makes the scap deb be the stubbed form (just symlinks, as described above). Choose a version number for this deb. The version number selected must make clear that it is unrelated to any particular version of scap. [] Follow the usual scap release procedure to tag and request deployment by serviceops. [] When serviceops has completed deployment of the stub, /usr/bin/scap on targets will now point to the code previously installed by `scap install-scap` == Outstanding notes/problems== [] What about beta ? [] Installing scap somewhere other than /usr/lib/python3/dist-packages will break scripts that expect to be able to `import scap` (such as `stage-train` in the releases repo). Hopefully allowing scap to deploy itself will reduce the need/desire to extend scap outside of scap. [] What about freshly-provisioned hosts? Running scap on them will fail until someone runs `scap install-scap` on the deploy server.