Page MenuHomePhabricator

Write a diskimage-builder element to run puppet
Closed, ResolvedPublic

Description

Following discussions with @zeljkofilipin and @dduvall, we need a diskimage-builder element to execute puppet when creating the image.

@hashar wrote a thin wrapper with https://gerrit.wikimedia.org/r/#/c/234300/ . One can fetch that patch on integration-dev.integration.eqiad.wmflabs which has all the required packages.

Then install diskimage 1.47 (and potentially other requirements):

pip install --user -rdib/requirements.txt
export PATH=~/local/bin/

diskimage-builder uses debootstrap to creates a base Debian jessie system. That is rather long but cacheable using the DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE env variable. So to build an image:

cd dib
DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE=1 ./build_image.py

OpenStack has a puppet element already https://github.com/openstack-infra/project-config/tree/master/nodepool/elements/puppet . You can clone it and then point diskimage-builder to it using the ELEMENTS_PATH variable.

git clone https://github.com/openstack-infra/project-config

# hack build_image.py to insert the 'puppet' element
ELEMENTS_PATH=project-config/nodepool/elements DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE=1 ./build_image.py

It is all hardcoded for OpenStack usage though. I have poked upstream to have the element upstreamed to diskimage-builder though I haven't insisted.

We probably want our own element in integration/config.git /dib/ based on https://github.com/openstack-infra/project-config/blob/master/nodepool/elements/puppet/bin/prepare-node . Maybe name it wikimedia-puppet.

Details

Related Gerrit Patches:

Event Timeline

hashar raised the priority of this task from to Needs Triage.
hashar updated the task description. (Show Details)
hashar triaged this task as High priority.
hashar moved this task from Backlog to In-progress on the Continuous-Integration-Scaling board.
hashar set Security to None.

Change 234975 had a related patch set uploaded (by Hashar):
(WIP) dib: wikimedia-puppet element (WIP)

https://gerrit.wikimedia.org/r/234975

I have a working dib element to run puppet out of operations/puppet.git though it doesn't install much it at least brings us the apt configuration and basic python packages from contint::packages::python https://gerrit.wikimedia.org/r/234975

Change 234975 merged by jenkins-bot:
nodepool: wikimedia-puppet element for dib

https://gerrit.wikimedia.org/r/234975

hashar closed this task as Resolved.Sep 11 2015, 9:07 PM

Bulk of the job has been completed. It currently just apply contint::packages::python.