Page MenuHomePhabricator

Write a diskimage-builder element to run puppet
Closed, ResolvedPublic

Description

Following discussions with @zeljkofilipin and @dduvall, we need a diskimage-builder element to execute puppet when creating the image.

@hashar wrote a thin wrapper with https://gerrit.wikimedia.org/r/#/c/234300/ . One can fetch that patch on integration-dev.integration.eqiad.wmflabs which has all the required packages.

Then install diskimage 1.47 (and potentially other requirements):

pip install --user -rdib/requirements.txt
export PATH=~/local/bin/

diskimage-builder uses debootstrap to creates a base Debian jessie system. That is rather long but cacheable using the DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE env variable. So to build an image:

cd dib
DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE=1 ./build_image.py

OpenStack has a puppet element already https://github.com/openstack-infra/project-config/tree/master/nodepool/elements/puppet . You can clone it and then point diskimage-builder to it using the ELEMENTS_PATH variable.

git clone https://github.com/openstack-infra/project-config

# hack build_image.py to insert the 'puppet' element
ELEMENTS_PATH=project-config/nodepool/elements DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE=1 ./build_image.py

It is all hardcoded for OpenStack usage though. I have poked upstream to have the element upstreamed to diskimage-builder though I haven't insisted.

We probably want our own element in integration/config.git /dib/ based on https://github.com/openstack-infra/project-config/blob/master/nodepool/elements/puppet/bin/prepare-node . Maybe name it wikimedia-puppet.

Event Timeline

hashar raised the priority of this task from to Needs Triage.
hashar updated the task description. (Show Details)
hashar added subscribers: gerritbot, dduvall, zeljkofilipin and 2 others.
hashar triaged this task as High priority.
hashar moved this task from Backlog to In-progress on the Continuous-Integration-Scaling board.
hashar set Security to None.

Change 234975 had a related patch set uploaded (by Hashar):
(WIP) dib: wikimedia-puppet element (WIP)

https://gerrit.wikimedia.org/r/234975

I have a working dib element to run puppet out of operations/puppet.git though it doesn't install much it at least brings us the apt configuration and basic python packages from contint::packages::python https://gerrit.wikimedia.org/r/234975

Change 234975 merged by jenkins-bot:
nodepool: wikimedia-puppet element for dib

https://gerrit.wikimedia.org/r/234975

Bulk of the job has been completed. It currently just apply contint::packages::python.