Page MenuHomePhabricator

puppet compiler fails with modules using puppetdb
Closed, ResolvedPublic

Description

Tracking this issue here so it doesn't get lost, I've ran into puppet compiler failing with

Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1134:in `issue_deprecation_warning')
Warning: The use of 'import' is deprecated at /mnt/jenkins-workspace/puppet-compiler/4582/change/src/manifests/site.pp:9. See http://links.puppetlabs.com/puppet-import-deprecation
   (at /usr/lib/ruby/vendor_ruby/puppet/parser/parser_support.rb:110:in `import')
Warning: You cannot collect without storeconfigs being set on line 80 in file /mnt/jenkins-workspace/puppet-compiler/4582/change/src/modules/ssh/manifests/server.pp
Warning: You cannot collect without storeconfigs being set on line 25 in file /mnt/jenkins-workspace/puppet-compiler/4582/change/src/modules/ganglia/manifests/monitor.pp
Warning: You cannot collect without storeconfigs being set on line 80 in file /mnt/jenkins-workspace/puppet-compiler/4582/change/src/modules/bacula/manifests/client.pp
Warning: You cannot collect without storeconfigs being set on line 44 in file /mnt/jenkins-workspace/puppet-compiler/4582/change/src/modules/bacula/manifests/client/job.pp
Error: Could not run: cannot load such file -- puppet/util/puppetdb

when trying to compile changes that use puppetdb (case in point, role::prometheus::ops). IIRC @Joe took a look and it is related to different the different way puppet behaves when standalone vs master

Event Timeline

fgiunchedi triaged this task as Medium priority.Nov 30 2016, 1:56 AM

Change 370192 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] puppetdb: spin off module, support hsqldb

https://gerrit.wikimedia.org/r/370192

Change 370192 merged by Giuseppe Lavagetto:
[operations/puppet@production] puppetdb: spin off module, support hsqldb

https://gerrit.wikimedia.org/r/370192

Change 370199 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] puppetdb: move more things to the module

https://gerrit.wikimedia.org/r/370199

Change 370199 merged by Giuseppe Lavagetto:
[operations/puppet@production] puppetdb: move more things to the module

https://gerrit.wikimedia.org/r/370199

Change 370627 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] labs/puppet3-diffs: Bump version of the puppet compiler

https://gerrit.wikimedia.org/r/370627

Change 370627 merged by Giuseppe Lavagetto:
[operations/puppet@production] labs/puppet3-diffs: Bump version of the puppet compiler

https://gerrit.wikimedia.org/r/370627

Status update:

  • puppetdb now runs on every puppet-compiler node, using the CA/certs combination used by the puppet compiler
  • the compiler copies over the puppetdb.conf file, if present, and can use it to compile catalogs

So in fact what remains to be done is to refresh the contents of the database regularly. For now, I'll just write a script that will populate it with enough data, that we can run on all nodes. Later on, we can create a separate server with puppetdb and rsync the data file to all servers.

I wrote a first version of the script that can be used to populate puppetdb; I'll upload it via puppet to all the compiler machines for now so that we can populate the db easily.

Then we can think of making that (and updating facts) easier/streamlined.

So one additional complication: we need to refresh the facts timestamp for every pcc run, as we don't want to incur in a case of https://tickets.puppetlabs.com/browse/PUP-5441

I'll add a CR for that and other things related to this task to puppet-compiler shortly.

Change 370802 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/software/puppet-compiler@master] Better support puppetdb

https://gerrit.wikimedia.org/r/370802

Change 370803 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/software/puppet-compiler@master] Add script to populate PuppetDB

https://gerrit.wikimedia.org/r/370803

Change 370802 merged by Giuseppe Lavagetto:
[operations/software/puppet-compiler@master] Better support puppetdb

https://gerrit.wikimedia.org/r/370802

Change 370803 merged by jenkins-bot:
[operations/software/puppet-compiler@master] Add script to populate PuppetDB

https://gerrit.wikimedia.org/r/370803

Change 370995 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] puppet-compiler: bump version, run puppetdb-populate

https://gerrit.wikimedia.org/r/370995

Change 370995 merged by Giuseppe Lavagetto:
[operations/puppet@production] puppet-compiler: bump version, run puppetdb-populate

https://gerrit.wikimedia.org/r/370995