Page MenuHomePhabricator

"/usr/local/bin/tox: Permission denied" on integration-slave14xx instances
Closed, ResolvedPublic

Description

https://integration.wikimedia.org/ci/job/operations-puppet-tox-data_admin_lint/13444/console

17:29:25 Building remotely on integration-slave1404
..
17:29:51 [operations-puppet-tox-data_admin_lint] $ /bin/bash -xe /tmp/hudson1591695731787047168.sh
17:29:51 + rm -fR log
17:29:51 + mkdir -p log
17:29:51 + set -o pipefail
17:29:51 + PY_COLORS=1
17:29:51 + tox -v -e data_admin_lint
17:29:51 /tmp/hudson1591695731787047168.sh: line 5: /usr/local/bin/tox: Permission denied
17:29:51 + tee log/data_admin_lint.log
17:29:51 Build step 'Execute shell' marked build as failure

Event Timeline

Krinkle raised the priority of this task from to High.
Krinkle updated the task description. (Show Details)
Krinkle added subscribers: Krinkle, Aklapper.

Looks like this broke when the umask for puppet changed to be more restrictive (only readable by root).

# contint/packages/labs.pp
    package { 'tox':
        ensure   => present,
        provider => 'pip',
        require  => Package['python-pip'],
    }

The package resource, however, does not support a umask parameter.
https://docs.puppetlabs.com/references/latest/type.html#package

Looks like this is a known issue with the Gem and Pip package providers.

https://tickets.puppetlabs.com/browse/PUP-1328

Pending a workaround or a reevaluation of the restrictive umask, added the following patch to our setup script might work. The only reason I'd do this is because we already had this on the old instances and there's no point in blocking it

#
## PATCH: Fix tox permissions (T91525)
#
$ umask 0022; sudo pip uninstall --yes tox && sudo sudo pip install tox && ls -l /usr/local/bin/tox

EDIT: Actually, it then fails because of a lower dependency that was still there:

18:26:25 IOError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/virtualenv-12.0.7.dist-info/METADATA'

Could also be uninstalled I guess, but getting more and more hacky.

https://wikitech.wikimedia.org/wiki/Nova_Resource:Integration/Setup

akosiaris claimed this task.
akosiaris subscribed.

Fixed in https://gerrit.wikimedia.org/r/#/c/194557/ . You will probably need to remove the package/recreate the machine