The beta cluster (specifically host deployment-deploy01) has an aptly tree in /srv/packages which is owned by the jenkins-deploy user. In T250157 I made some puppet configuration changes to ensure that the aptly::server role runs its aptly commands under the jenkins-deploy user (on deployment-deploy01). This took care of failures that were occurring during beta-publish-deb jobs. While investigating that problem I discovered that running read-only aptly commands (like aptly repo list) as root will case some files in the tree to have their ownership changed to root, breaking the tree for the jenkins-deploy user.

There are a few approaches we can take to deal with this:

• Do nothing and accept the risk of this happening and clean up when it does.
• Modify aptly so that if it runs as root and finds that the aptly tree is owned by a non-root user, become that user before performing the operation. This means maintaining a fork of the aptly codebase until mod gets merged in upstream.
• Stick with the stock aptly package and use Puppet to deploy a binstub (such as the one below) to handle the change of user when running as root. In this case the Puppet scripts would have to rename /usr/bin/aptly to something else (but only once) and drop the binstub in to replace it.
• Other ideas?

#!/bin/bash

set -eu

### Config ###
aptly_config=/etc/aptly.conf
### End config ###

aptlyroot=$(jq .rootDir -r $aptly_config)

# This will be a number
aptlyroot_owner=$(stat -c '%u' $aptlyroot)

# If not running as the owner of the aptly root directory, sudo to the
# owner before doing work.  This avoids a situation where a read-only
# operation run by root causes some files in /srv/packages/db to be
# owned by root.  xref T250157

cmdprefix=
if [ $(id -u) != $aptlyroot_owner ]; then
    cmdprefix="sudo -u #$aptlyroot_owner --set-home"
fi

exec $cmdprefix /usr/bin/aptly.real "$@"