Page MenuHomePhabricator

Jessie branch fails to run apt, puppet after VM is reloaded
Closed, DuplicatePublic

Description

On https://gerrit.wikimedia.org/r/#/c/329209/ @Gilles wrote:

I got the same error as Gergo. After resuming my work VM (I work inside an Ubuntu VM on a Windows machine). Literally nothing happened between the last "vagrant provision" I did a few days ago and the one I tried after resuming the paused Ubundu VM.
The file vagrant is looking for exists:

gilles@ubuntu:~/Documents/Mediawiki/vagrant$ vagrant provision
==> default: Running provisioner: lsb_check...
==> default: Running provisioner: shell...
    default: Running: /tmp/vagrant-shell20170109-41388-12wl00c.sh
==> default: Running provisioner: puppet...
==> default: Running Puppet with site.pp...
==> default: Error: Could not run: Could not find file /vagrant/puppet/manifests/site.pp
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.
gilles@ubuntu:~/Documents/Mediawiki/vagrant$ ls -al puppet/manifests/site.pp
-rw-rw-r-- 1 gilles gilles 1739 Sep 24 18:11 puppet/manifests/site.pp

I've found the cause, it's when you enable a role that reloads the VM. You don't even need to actually provision the role, in fact, just marking the VM for needing to restart on provisioning is enough. Steps to reproduce:

  • vagrant up
  • vagrant roles enable varnish
  • vagrant roles reset
  • vagrant provision You'll get a first error, apt-related:
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!
apt-get update
Stdout from the command:
Stderr from the command:
E: Archives directory /vagrant/cache/apt/partial is missing. - Acquire (2: No such file or directory)

Then if you attempt to provision again, you get the missing site.pp error.

Event Timeline

bd808 created this task.Jan 9 2017, 3:34 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 9 2017, 3:34 PM
bd808 updated the task description. (Show Details)Jan 9 2017, 3:36 PM
bd808 added a comment.Jan 9 2017, 3:40 PM

I'll play with this locally and see if I can figure out what is happening. It sounds to like the the vagrant mounted shares, our weird forced reload plugin command, and maybe the shell provisioning step that bootstraps Puppet in the VM may be working against each other in strange ways. Part of me thinks that maybe we should abandon our homegrown apt cache persistence in favor of recommending that people install the https://github.com/fgrehm/vagrant-cachier plugin.

This seems to be fixed now? Not sure what fixed it, though.

bd808 added a comment.Jan 31 2017, 6:20 AM

This seems to be fixed now? Not sure what fixed it, though.

I can't recreate this either. Should we close it as invalid until someone can come up with a reproduction case?

Gilles closed this task as Invalid.Jan 31 2017, 10:16 AM
Gilles reopened this task as Open.Mar 16 2017, 1:56 PM

This seems to be back. Repro steps using the Varnish role still "work".

Fresh VM without roles, then:

gilles@ubuntu:~/Documents/Mediawiki/vagrant$ vagrant roles enable varnish
Ok. Run `vagrant provision` to apply your changes.

Note the following settings have changed and your environment will be reloaded.
forward_ports: { 6081: 6081 } -> { 6081: 6081, 6082: 6082 }
gilles@ubuntu:~/Documents/Mediawiki/vagrant$ vagrant provision
==> default: Running provisioner: lsb_check...
==> default: Running provisioner: mediawiki_reload...
==> default: Reloading vagrant...
==> default: Attempting graceful shutdown of VM...
==> default: Checking if box 'debian/contrib-jessie64' is up to date...
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
    default: Adapter 2: hostonly
==> default: Forwarding ports...
    default: 8080 (guest) => 8080 (host) (adapter 1)
    default: 443 (guest) => 4430 (host) (adapter 1)
    default: 6081 (guest) => 6081 (host) (adapter 1)
    default: 6082 (guest) => 6082 (host) (adapter 1)
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
==> default: Machine booted and ready!
[default] GuestAdditions versions on your host (5.0.24) and guest (4.3.36) do not match.
Reading package lists...
Building dependency tree...
Reading state information...
dkms is already the newest version.
The following packages were automatically installed and are no longer required:
  gyp libc-ares-dev libc-ares2 libjs-node-uuid libjs-underscore libv8-3.14-dev
  libv8-3.14.5 node-abbrev node-ansi node-ansi-color-table node-archy
  node-async node-block-stream node-combined-stream node-cookie-jar
  node-delayed-stream node-forever-agent node-form-data node-fstream
  node-fstream-ignore node-github-url-from-git node-glob node-graceful-fs
  node-gyp node-inherits node-ini node-json-stringify-safe node-lockfile
  node-lru-cache node-mime node-minimatch node-mkdirp node-mute-stream
  node-node-uuid node-nopt node-normalize-package-data node-npmlog node-once
  node-osenv node-qs node-read node-read-package-json node-request node-retry
  node-rimraf node-semver node-sha node-sigmund node-slide node-tar
  node-tunnel-agent node-underscore node-which
Use 'apt-get autoremove' to remove them.
The following packages will be upgraded:
  linux-headers-3.16.0-4-amd64 linux-headers-3.16.0-4-common
2 upgraded, 0 newly installed, 0 to remove and 25 not upgraded.
E: Archives directory /vagrant/cache/apt/partial is missing. - Acquire (2: No such file or directory)
E: Archives directory /vagrant/cache/apt/partial is missing. - Acquire (2: No such file or directory)
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default: 
    default: Guest Additions Version: 4.3.36
    default: VirtualBox Version: 5.0
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

apt-get update

Stdout from the command:



Stderr from the command:

E: Archives directory /vagrant/cache/apt/partial is missing. - Acquire (2: No such file or directory)
bd808 added a comment.Mar 16 2017, 3:15 PM

E: Archives directory /vagrant/cache/apt/partial is missing.

Do you still have the vbguest plugin installed in your local Vagrant environment? We have stopped installing it by default, but I did not try to rig up any method to uninstall it if it already existed. Adding this to your Vagrantfile-extra.rb or ~/.vagrant.d/Vagrantfile would disable the vbguest plugin which tries to use apt in the VM prior to the phase where shares are mounted:

Vagrant.configure("2") do |config|
  if Vagrant.has_plugin?("vagrant-vbguest")
    config.vbguest.auto_update = false
  end
end

We could add that to the main Vagrantfile pretty easily.

Added that snippet to ~/.vagrant.d/Vagrantfile and it worked, thanks!