While working on T71223, the biggest problem was (and is) sharing folders between the host and the guest. On the other hand, at least for me the benefit is negligible: I already can use established tools to mount guest file systems or configure Git or Emacs TRAMP to ssh into a guest VM.
So it would be very nice if there was a way to let MediaWiki-Vagrant create a totally separate, isolated VM that populates its file system where necessary by fresh Git checkouts of mediawiki/core, etc. only.
We do need to get the Puppet configuration files into the VM, but it might be possible to do that optionally via the rsync synced folder plugin in addition to the NFS and VM container native sync options currently supported.
A similar thing that I would like to enable is placing git clones and other files on the VM's filesystem via hiera configuration. To do that effectively we need to ensure that /vagrant/* only appears in the Puppet configuration as a hiera managed parameter and that those parameters use common base parameters (e.g. mwv::services_dir) which can be easily changed in a puppet/hieradata/local.yaml file. If we can get to that point then it's pretty easy to add nicer support in the plugin for configuring the settings.
I tried the rsync method last year because I thought that would solve it, but (and maybe my memory is faulty; bringing up MediaWiki-Vagrant boxes takes a long time on my machine and produces a lot of output, so I am not so disciplined in testing) every time I reran vagrant provision, the "shared" folder was overwritten again, so no changes on the VM would survive.
I just found https://www.vagrantup.com/docs/provisioning/file.html which might be interesting as well (in fact, it looks very promising).
(Totally different approach: IIRC I saw a Labs project of yours about generating ISOs for MediaWiki-Vagrant. If there was a generator that created a generic Vagrant box with no shared folders and uploaded it daily/weekly, the bootstrapping issue would be solved and the start-up time reduced to zero. This would not satisfy the needs of developers who need to hack on the initial provisioning, before roles are applied, but for 08/15 stuff it would probably be very useful.)
The downside of using file would be that updating the Puppet config in the VM would not be possible (or would have to be done manually). Using rsync to provision the files would probably require moving the Puppet provisioned content out of /vagrant so that vagrant provision would not delete Puppet managed content.
(Totally different approach: IIRC I saw a Labs project of yours about generating ISOs for MediaWiki-Vagrant. If there was a generator that created a generic Vagrant box with no shared folders and uploaded it daily/weekly, the bootstrapping issue would be solved and the start-up time reduced to zero. This would not satisfy the needs of developers who need to hack on the initial provisioning, before roles are applied, but for 08/15 stuff it would probably be very useful.)
The server at https://mediawiki-vagrant-image.wmflabs.org/ which builds a new ISO image suitable for making USB sicks for install parties. That particular server uses LXC rather than VirtualBox and thus would not be able to generate VBox images. There is a similar idea proposed in T123005: Spin custom MediaWiki-Vagrant box with base packages pre-installed.
No, I mean the Puppet configuration would be a "normal" Git repository of mediawiki/vagrant inside the guest; whether typing git pull on the host or the guest doesn't matter much IMHO. I just hacked together a proof-of-concept Vagrantfile:
Vagrant.configure('2') do |config|
config.vm.hostname = 'mediawiki-vagrantlet.dev'
config.package.name = 'mediawiki-vagrantlet.box'
config.vm.provider :libvirt do |libvirt, override|
libvirt.memory = 2048
override.vm.box = 'trusty-cloud'
# override.vm.network :private_network, ip: settings[:static_ip]
end
# Install git.
config.vm.provision 'shell',
inline: 'apt-get update -qq && apt-get install -y git'
# Set up /vagrant.
config.vm.provision 'shell',
inline: 'mkdir -p /vagrant'
config.vm.provision 'shell',
inline: 'chown vagrant:vagrant /vagrant'
# Clone mediawiki/vagrant to /vagrant.
config.vm.provision 'shell',
inline: 'test -d /vagrant/.git || git clone --recursive https://gerrit.wikimedia.org/r/mediawiki/vagrant.git /vagrant',
privileged: false
# Disable the default synced folder.
config.vm.synced_folder '.', '/vagrant', disabled: true
# Configure Puppet.
config.vm.provision :puppet do |puppet|
# Use empty module path to avoid an extra mount.
# See --modulepath below
puppet.module_path = []
# Tell Vagrant that the manifests are already on the guest
puppet.manifests_path = [:guest, '/vagrant/puppet/manifests']
puppet.manifest_file = 'site.pp'
puppet.options = [
'--modulepath', '/vagrant/puppet/modules',
'--hiera_config', '/vagrant/puppet/hiera.yaml',
'--verbose',
'--config_version', '/vagrant/puppet/extra/config-version',
'--logdest', "/vagrant/logs/puppet/puppet.#{nil || 'unknown'}.log",
'--logdest', 'console',
'--write-catalog-summary',
'--detailed-exitcodes',
]
# For more output, uncomment the following line:
puppet.options << '--debug' if ENV.include?('PUPPET_DEBUG')
# Windows's Command Prompt has poor support for ANSI escape sequences.
puppet.options << '--color=false' if Vagrant::Util::Platform.windows?
puppet.facter = {
'fqdn' => config.vm.hostname,
#'git_user' => settings[:git_user],
'forwarded_port' => 8080,
'forwarded_https_port' => 443,
# 'shared_apt_cache' => '/vagrant/cache/apt/',
'environment' => ENV['MWV_ENVIRONMENT'] || 'vagrant',
'vmhost' => Socket.gethostname,
# T86282: Force Puppet's LANG env var by exploiting a factor quirk.
# See https://stackoverflow.com/a/23502693/582542
'x=x LANG=en_US.UTF-8 x' => 'x',
}
puppet.facter['share_owner'] = 'vagrant'
puppet.facter['share_group'] = 'www-data'
# if settings[:http_port] != 80 && ENV['MWV_ENVIRONMENT'] != 'labs'
# puppet.facter['port_fragment'] = ":#{settings[:http_port]}"
# end
# Derive a host IP from the configured static IP by getting the first
# usable IP in the 8-bit network
# if settings[:static_ip]
# network = IPAddr.new("#{settings[:static_ip]}/24")
# puppet.facter['host_ip'] = network.to_range.take(2).last.to_s
# end
end
endFor libvirt, this brings up a VM that sets itself up and seems to work (network stuff missing currently). In the spirit of leanness, I probably also replace the vagrant role and .settings.yaml bits with direct Ruby code in Vagrantfile-extra.rb so that a user doesn't have to install the mediawiki-vagrant gem (IMHO simplest Ruby code is just as easy to write as typing vagrant role enable […]).
I am very motivated by the speed-up: The initial Puppet run with no NFS took 1812.60 seconds which is much faster than my earlier attempts.