Page MenuHomePhabricator

/vagrant/cache/apt permissions warnings with Stretch sandboxed apt settings
Open, LowPublic

Description

I see this with apt
W: chown to _apt:root of directory /vagrant/cache/apt/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
W: chmod 0700 of directory /vagrant/cache/apt/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
E: Could not open lock file /vagrant/cache/apt/lock - open (13: Permission denied)
E: Unable to lock directory /vagrant/cache/apt/
it's because in stretch it has a _apt user now.

Event Timeline

bd808 created this task.Dec 18 2017, 3:27 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 18 2017, 3:27 PM
bd808 added a comment.Dec 25 2017, 6:02 AM

I do not see this error when using VirtualBox file shares (vagrant config nfs_shares off), but I'll check with NFS to see if I can recreate it.

bd808 added a comment.Dec 25 2017, 6:38 AM

Confirmed that this warning occurs with NFS shares and not with VirtualBox shares. It does not seem to effect actual usage or Puppet's processing of package resources.

$ vagrant ssh
$ mount|grep /vagrant
10.11.12.1:/Users/bd808/projects/wmf/vagrant-disposable on /vagrant type nfs (rw,noatime,vers=3,rsize=16384,wsize=16384,namlen=255,hard,proto=udp,timeo=11,retrans=3,sec=sys,mountaddr=10.11.12.1,mountvers=3,mountport=875,mountproto=udp,local_lock=none,addr=10.11.12.1)
$ ls -ld /vagrant/cache/apt
drwxr-xr-x 448 vagrant_share dialout 15232 Dec 24 21:58 /vagrant/cache/apt
$ ls -ld /vagrant/cache/apt/partial
drwx------ 3 vagrant_share dialout 102 Dec 24 21:58 /vagrant/cache/apt/partial
$ sudo apt-get install less
Reading package lists... Done
Building dependency tree
Reading state information... Done
less is already the newest version (481-2.1).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
W: chown to _apt:root of directory /vagrant/cache/apt/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
$ vagrant config nfs_shares no
$ vagrant reload
$ vagrant ssh
$ mount|grep /vagrant
vagrant-root on /vagrant type vboxsf (rw,nodev,relatime)
vagrant-logs on /vagrant/logs type vboxsf (rw,nodev,relatime)
$ ls -ld /vagrant/cache/apt
drwxr-xr-x 1 vagrant www-data 15232 Dec 24 21:58 /vagrant/cache/apt
$ ls -ld /vagrant/cache/apt/partial
drwx------ 1 vagrant www-data 102 Dec 24 21:58 /vagrant/cache/apt/partial
$ sudo apt-get install less
Reading package lists... Done
Building dependency tree
Reading state information... Done
less is already the newest version (481-2.1).
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.

Even on VirtualBox shares warnings can be produced by apt commands:

$ sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  libssl1.0.2 linux-image-4.9.0-4-amd64 linux-libc-dev sensible-utils
4 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 41.4 MB of archives.
After this operation, 41.0 kB disk space will be freed.
Do you want to continue? [Y/n]
...
W: Download is performed unsandboxed as root as file '/vagrant/cache/apt/partial/sensible-utils_0.0.9+deb9u1_all.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)

Running chmod 0777 cache/apt/partial on my host computer seems to have made the warning go away for subsequent package downloads. I'm not quite sure how we can make this type of change automatically and portably. To my knowledge git does not have a method for managing directory permissions natively. Since the change needs to be made on the host computer we can't do it with Puppet. We might be able to add something to the mediawiki-vagrant plugin that manages permissions like this on the host side. This seems like a lot of work to fix a minor security warning though.

bd808 renamed this task from /vagrant/cache/apt permissions issues with Stretch to /vagrant/cache/apt permissions warnings with Stretch sandboxed apt settings.Dec 25 2017, 6:39 AM

Change 400198 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[mediawiki/vagrant@stretch-migration] vagrant: Add plugin to set host file permissions

https://gerrit.wikimedia.org/r/400198

Change 400198 merged by jenkins-bot:
[mediawiki/vagrant@stretch-migration] vagrant: Add plugin to set host file permissions

https://gerrit.wikimedia.org/r/400198

bd808 closed this task as Resolved.Jan 9 2018, 1:38 AM
bd808 claimed this task.

Hmm I still get these errors. Do I have to recreate the vagrant machine?

bd808 added a comment.Jan 9 2018, 2:42 AM

Hmm I still get these errors. Do I have to recreate the vagrant machine?

vagrant provision should run the plugin and change the permissions on the host system.

If you are seeing things like E: Could not open lock file /vagrant/cache/apt/lock - open (13: Permission denied) that sounds like a problem with your MediaWiki-Vagrant clone/permissions on the host computer beyond what I attempted to fix with the plugin.

bd808 reopened this task as Open.Jan 9 2018, 4:12 AM
bd808 removed bd808 as the assignee of this task.
bd808 triaged this task as Low priority.

Testing on a Cloud VPS server shows that warnings are still happening:

$ sudo apt-get install tmux
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libutempter0
The following NEW packages will be installed:
  libutempter0 tmux
0 upgraded, 2 newly installed, 0 to remove and 73 not upgraded.
Need to get 272 kB of archives.
After this operation, 677 kB of additional disk space will be used.
W: chown to _apt:root of directory /vagrant/cache/apt/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
Do you want to continue? [Y/n]
Get:1 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 libutempter0 amd64 1.1.6-3 [7,812 B]
Get:2 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 tmux amd64 2.3-4 [265 kB]
Fetched 272 kB in 0s (4,185 kB/s)
Selecting previously unselected package libutempter0:amd64.
(Reading database ... 38691 files and directories currently installed.)
Preparing to unpack .../libutempter0_1.1.6-3_amd64.deb ...
Unpacking libutempter0:amd64 (1.1.6-3) ...
Selecting previously unselected package tmux.
Preparing to unpack .../cache/apt/tmux_2.3-4_amd64.deb ...
Unpacking tmux (2.3-4) ...
Setting up libutempter0:amd64 (1.1.6-3) ...
Processing triggers for libc-bin (2.24-11+deb9u1) ...
Setting up tmux (2.3-4) ...
Processing triggers for man-db (2.7.6.1-2) ...
W: chown to root:root of file /vagrant/cache/apt/partial/libutempter0_1.1.6-3_amd64.deb failed - 201::URIDone (1: Operation not permitted)
W: chown to root:root of file /vagrant/cache/apt/partial/tmux_2.3-4_amd64.deb failed - 201::URIDone (1: Operation not permitted)

With VirtualBox and NFS shares it also has warnings still:

$ sudo apt-get install tmux
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  tmux
0 upgraded, 1 newly installed, 0 to remove and 22 not upgraded.
Need to get 0 B/265 kB of archives.
After this operation, 635 kB of additional disk space will be used.
Selecting previously unselected package tmux.
(Reading database ... 46880 files and directories currently installed.)
Preparing to unpack .../cache/apt/tmux_2.3-4_amd64.deb ...
Unpacking tmux (2.3-4) ...
Setting up tmux (2.3-4) ...
Processing triggers for man-db (2.7.6.1-2) ...
W: chown to _apt:root of directory /vagrant/cache/apt/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)

With VirtualBox and VBox shares, there are no warnings:

$ sudo apt-get install tmux
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libutempter0
The following NEW packages will be installed:
  libutempter0 tmux
0 upgraded, 2 newly installed, 0 to remove and 4 not upgraded.
Need to get 272 kB of archives.
After this operation, 677 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 libutempter0 amd64 1.1.6-3 [7,812 B]
Get:2 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 tmux amd64 2.3-4 [265 kB]
Fetched 272 kB in 0s (534 kB/s)
Selecting previously unselected package libutempter0:amd64.
(Reading database ... 46755 files and directories currently installed.)
Preparing to unpack .../libutempter0_1.1.6-3_amd64.deb ...
Unpacking libutempter0:amd64 (1.1.6-3) ...
Selecting previously unselected package tmux.
Preparing to unpack .../cache/apt/tmux_2.3-4_amd64.deb ...
Unpacking tmux (2.3-4) ...
Setting up libutempter0:amd64 (1.1.6-3) ...
Processing triggers for libc-bin (2.24-11+deb9u1) ...
Setting up tmux (2.3-4) ...
Processing triggers for man-db (2.7.6.1-2) ...
bd808 added a comment.Jan 9 2018, 5:02 AM
$ chmod a=rwx cache/apt/partial
$ ls -ld cache/apt/partial
drwxrwxrwx  3 bd808  staff   102B Jan  8 21:02 cache/apt/partial/
$ vagrant provision
==> default: Running provisioner: lsb_check...
==> default: Running provisioner: file_perms...
==> default: Running provisioner: shell...
    default: Running: /var/folders/g5/zhwkm1n11yndb6g0pqy66lhm0000gn/T/vagrant-shell20180108-3018-17a5bf4.sh
==> default: Running provisioner: puppet...
==> default: Running Puppet with environment vagrant...
==> default: Info: Loading facts
==> default: Notice: Compiled catalog for vagrantdisposable.mediawiki-vagrant.dev in environment vagrant in 2.47 seconds
==> default: Info: Applying configuration version '1515471478.372b1cbd'
==> default: Notice: Applied catalog in 6.10 seconds
$ ls -ld cache/apt/partial
drwx------  3 bd808  staff   102B Jan  8 21:02 cache/apt/partial/

...
After a lot more poking, it seems that something (apt?) inside the Puppet run is silently chmoding the directory back to 0700 permissions. I've added enough debugging locally to see that the file_perms provisiner is changing the permissions to 0777 as expected and they are seen inside the VM before Puppet runs. After Puppet runs they are reset back to 0700.

rafidaslam added a comment.EditedOct 25 2018, 5:16 PM

Yes, I think so.
I think the culprit is the apt, since when I did a chmod a=rwx cache/apt/partial, and then I did a sudo apt-get install git (or any package) in my vagrant through a ssh session, the apt got stuck (didn't prompt a yes/no question), and then when I check cache/apt/partial directory again, its permission changed to 0700 again..

(In my local computer)

$ chmod a=rwx cache/apt/partial
$ ls -la cache/apt/
total 16
drwxr-xr-x. 3 refeed wheel 4096 Oct 25 15:14 ./
drwxr-xr-x. 5 refeed wheel 4096 Oct 24 20:14 ../
-rw-r--r--. 1 refeed wheel   17 Oct 24 20:14 .gitignore
-rw-r-----. 1 refeed wheel    0 Oct 25 15:14 lock
drwxrwxrwx. 2 refeed wheel 4096 Oct 24 20:14 partial/

(Then I switched to my other terminal tab which contained a ssh session to my vagrant)

vagrant@mediawikivagrant:~$ sudo apt-get install git
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  git-man liberror-perl
Suggested packages:
  git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk gitweb git-arch git-cvs git-mediawiki git-svn
Recommended packages:
  rsync

(It got stuck here)

(Switch back to my local terminal)

$ ls -la cache/apt/
total 16
drwxr-xr-x. 3 refeed wheel 4096 Oct 25 15:14 ./
drwxr-xr-x. 5 refeed wheel 4096 Oct 24 20:14 ../
-rw-r--r--. 1 refeed wheel   17 Oct 24 20:14 .gitignore
-rw-r-----. 1 refeed wheel    0 Oct 25 15:14 lock
drwx------. 2 refeed wheel 4096 Oct 24 20:14 partial/