Page MenuHomePhabricator

Spin custom MediaWiki-Vagrant box with base packages pre-installed
Open, MediumPublic

Description

Basically, install the Debian packages (but nothing else) that a no-role instance uses, then export that as a MediaWiki-Vagrant box. Then, use that as the base box going forward.

This should make destroy/up much faster.

This could be re-done manually periodically (monthly?) initially. We could file a separate task to automate it later.

Idea how to do this:

  1. In a new directory, make sure all roles are disabled.
  2. vagrant up --provision.
  3. dpkg --get-selections > packages.txt
  4. Start a fresh VM (mediawiki-vagrant-base-box) using stock trusty-server-cloudimg-amd64-vagrant-disk1.box.
  5. On that VM:

5a. sudo apt-get install dselect
5b. sudo dselect update
5c. sudo dpkg --set-selections < packages.txt
5d. sudo apt-get -y update
5e. sudo apt-get dselect-upgrade

  1. Make sure right version of Guest Additions is installed.
  2. vagrant package --base mediawiki-vagrant-base-box

See https://docs.vagrantup.com/v2/virtualbox/boxes.html .

Event Timeline

• Mattflaschen-WMF raised the priority of this task from to Needs Triage.
• Mattflaschen-WMF updated the task description. (Show Details)

I don't have any hard numbers timing a destroy && up cycle but my seat of the pants inclination would be that the initial apt-get update takes much more time than $IP/maintenance/install.php.

Vagrant 1.8's new vagrant snapshot command might really go a lot farther towards improving speed for a typical destroy && up testing cycle. Vagrant 1.8 is also now using linked clones for VM images which is intended to reduce both disk usage and time needed for creating new VMs.

Thanks.

Yeah, the suggestion is not motivated by avoiding install.php. It's rather mainly about APT package downloads, and apt-get update to a certain extent.

vagrant snapshot will definitely be helpful for various scenarios (once I upgrade to 1.8).

It can also solve the "avoid repeated base downloads" case if there is a named snapshot with no roles.

If we did do a base box, I'm think it shouldn't even have MediaWiki installed, just Debian packages. That would allow testing early phases of provisioning, e.g. DB creation.

dduvall triaged this task as Medium priority.EditedJan 12 2016, 7:07 AM
In T123005#1926832, @Mattflaschen wrote:

vagrant snapshot will definitely be helpful for various scenarios (once I upgrade to 1.8).

You can use the vagrant-vbox-snapshot plugin with earlier versions. I've been using snapshots on 1.7 to speed up test provisions and it has worked fairly well—once in a while the nfs mount gets messed up after a restore and I have to restart the VM but it's been rare.

If we did do a base box, I'm think it shouldn't even have MediaWiki installed, just Debian packages. That would allow testing early phases of provisioning, e.g. DB creation.

I like this idea. The clone of core is also a lengthy process but I'm not sure how to integrate that into the base box without having to move it to the nfs/vboxsf mount upon the first provision (which might be a little ugly).

I like this idea. The clone of core is also a lengthy process but I'm not sure how to integrate that into the base box without having to move it to the nfs/vboxsf mount upon the first provision (which might be a little ugly).

Also, while time-consuming, the clone of core and extensions is a one-time thing. It doesn't need to be redone every destroy/up cycle.

The goal is to make the Debian packages also a one-time thing.

Producing a custom base image like this could also help with things like T161707: Upgrade linux kernel used on Vagrant to 4+.