Page MenuHomePhabricator

Spin custom MediaWiki-Vagrant box with base packages pre-installed
Open, NormalPublic

Description

Basically, install the Debian packages (but nothing else) that a no-role instance uses, then export that as a MediaWiki-Vagrant box. Then, use that as the base box going forward.

This should make destroy/up much faster.

This could be re-done manually periodically (monthly?) initially. We could file a separate task to automate it later.

Idea how to do this:

  1. In a new directory, make sure all roles are disabled.
  2. vagrant up --provision.
  3. dpkg --get-selections > packages.txt
  4. Start a fresh VM (mediawiki-vagrant-base-box) using stock trusty-server-cloudimg-amd64-vagrant-disk1.box.
  5. On that VM:

5a. sudo apt-get install dselect
5b. sudo dselect update
5c. sudo dpkg --set-selections < packages.txt
5d. sudo apt-get -y update
5e. sudo apt-get dselect-upgrade

  1. Make sure right version of Guest Additions is installed.
  2. vagrant package --base mediawiki-vagrant-base-box

See https://docs.vagrantup.com/v2/virtualbox/boxes.html .

Event Timeline

Mattflaschen-WMF raised the priority of this task from to Needs Triage.
Mattflaschen-WMF updated the task description. (Show Details)
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptJan 6 2016, 10:41 PM
bd808 added a comment.Jan 8 2016, 5:51 PM

I don't have any hard numbers timing a destroy && up cycle but my seat of the pants inclination would be that the initial apt-get update takes much more time than $IP/maintenance/install.php.

Vagrant 1.8's new vagrant snapshot command might really go a lot farther towards improving speed for a typical destroy && up testing cycle. Vagrant 1.8 is also now using linked clones for VM images which is intended to reduce both disk usage and time needed for creating new VMs.

Mattflaschen-WMF added a comment.EditedJan 12 2016, 2:54 AM

Thanks.

Yeah, the suggestion is not motivated by avoiding install.php. It's rather mainly about APT package downloads, and apt-get update to a certain extent.

vagrant snapshot will definitely be helpful for various scenarios (once I upgrade to 1.8).

It can also solve the "avoid repeated base downloads" case if there is a named snapshot with no roles.

If we did do a base box, I'm think it shouldn't even have MediaWiki installed, just Debian packages. That would allow testing early phases of provisioning, e.g. DB creation.

Mattflaschen-WMF set Security to None.
dduvall triaged this task as Normal priority.EditedJan 12 2016, 7:07 AM
In T123005#1926832, @Mattflaschen wrote:

vagrant snapshot will definitely be helpful for various scenarios (once I upgrade to 1.8).

You can use the vagrant-vbox-snapshot plugin with earlier versions. I've been using snapshots on 1.7 to speed up test provisions and it has worked fairly well—once in a while the nfs mount gets messed up after a restore and I have to restart the VM but it's been rare.

If we did do a base box, I'm think it shouldn't even have MediaWiki installed, just Debian packages. That would allow testing early phases of provisioning, e.g. DB creation.

I like this idea. The clone of core is also a lengthy process but I'm not sure how to integrate that into the base box without having to move it to the nfs/vboxsf mount upon the first provision (which might be a little ugly).

I like this idea. The clone of core is also a lengthy process but I'm not sure how to integrate that into the base box without having to move it to the nfs/vboxsf mount upon the first provision (which might be a little ugly).

Also, while time-consuming, the clone of core and extensions is a one-time thing. It doesn't need to be redone every destroy/up cycle.

The goal is to make the Debian packages also a one-time thing.

Mattflaschen-WMF updated the task description. (Show Details)
scfc added a subscriber: scfc.Feb 29 2016, 8:10 PM
bd808 added a comment.Jul 13 2017, 3:27 AM

Producing a custom base image like this could also help with things like T161707: Upgrade linux kernel used on Vagrant to 4+.