Page MenuHomePhabricator

wikidata-jenkins* instances needs puppet update
Closed, ResolvedPublic

Description

Labs op Andrew Bogott merged a few changes in operations/puppet.git to point instances to a new LDAP server. See announcement on labs-l:

https://lists.wikimedia.org/pipermail/labs-l/2014-September/002955.html

Unfortunately a few Wikidata Jenkins slaves instances are using role::puppet::self and thus are not self updating. The whole list can be found at https://wikitech.wikimedia.org/wiki/Ldap_rename#Instances_in_Danger

Among others:

wikidata-builder3
wikidata-jenkins1
wikidata-jenkins2
wikidata-jenkins3

Addshore is probably the most qualified since he has set up the instances originally.

A rebase of the local puppet repo would be needed to have them updated. Might want to do them one after the other to make sure nothing breaks.


Version: wmf-deployment
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=71419

Details

Reference
bz71411

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:47 AM
bzimport set Reference to bz71411.

If I don't appear to do this within the week please give this ticket a poke! :)

What blocks these from being on the normal puppet master?

The builder instances use:
https://github.com/wmde/puppet-builder
The jenkins instances use:
https://github.com/wmde/puppet-composer
(also has class ci::slave::labs applied)

See docs at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikidata-build

Simply pulling everything else from master shouldn't be an issue,everything custom we use are in these pretty modules that don't really rely on other things!

Those puppet failures after trying with an updated operations/puppet.git ( see https://bugzilla.wikimedia.org/show_bug.cgi?id=71419#c3 for the error ) on wikidata-builder3.eqiad.wmflabs are now solved. The problem was probably that old package versions of puppet and puppetmaster were not working well with the current puppet code.

Solved this by removing manifests/role/mariadb.pp and manifests/role/phabricator.pp . Then after a half working run it failed after puppet changed /etc/puppet/auth.conf . Tweaking that file made the next run go through. After a few more runs there was no change anymore. Then did an apt-get upgrade and another two puppet runs.

Perhaps first doing an apt-get upgrade and only after that running puppet with the updated repo will work better?

(Warning!: During apt-get upgrade the ldap config will become incorrect which would mean not being able to log in anymore until it gets corrected by running puppet, so take care.)