Page MenuHomePhabricator

Integrate the puppet compiler in the puppet CI pipeline
Open, MediumPublic


The puppet compiler is very useful, but ATM its utility is limited by the fact we cannot know automatically which servers to check when a change is introduced. Also, it cannot be triggered directly from gerrit for this exact reason.

Given the added value that PCC gives us, we should aim in making it easy to trigger a comprehensive run from gerrit.

Ideally, this could be automated in the following way:

  1. when a keyword is set as a comment in gerrit, zuul starts the job
  2. the compiler analyzes the change for changed classes and node stanzas in site.pp
  3. the compiler queries puppetdb to find which hosts have the changed classes applied
  4. the compiler finds which node stanzas to pick from site.pp
  5. the compiler runs and the results are linked by jenkins-bot on the gerrit task

In order to do that we need the following things:

  • Have the puppet compiler be able to run in multiple instances, with a proper proxy in front
  • Have the puppet compiler be able to query puppetdb (this might need a move to production given the sensitivity of the queries)
  • Implement the node selection via puppetdb.


Related Gerrit Patches:

Event Timeline

Joe created this task.May 22 2017, 3:44 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 22 2017, 3:44 PM
Paladox added a subscriber: Paladox.Jul 3 2017, 3:59 PM

Me and @hashar was talking about this a few months ago.

I've created this to get jjb to generate the job and also created so that we can write in a comment and it will check against the nodes.

fgiunchedi triaged this task as Medium priority.Jul 21 2017, 10:20 AM
EddieGP edited projects, added puppet-compiler; removed Puppet.Apr 7 2018, 12:27 AM
Joe added a project: Puppet.Apr 8 2018, 5:14 AM
Volans added a subscriber: Volans.Oct 24 2018, 4:19 PM
herron added a subscriber: herron.Oct 24 2018, 4:19 PM
Krenair added a subscriber: Krenair.Nov 2 2018, 1:06 PM

We have a Jenkins job T97513 which has been made to recognizes Hosts: in commit message to passe a list of nodes to the PPC. 91b70b4612abc6443a571dcc397622c5ddbe82af

The job can be triggered by commenting check experimental in Gerrit.

jbond added a subscriber: jbond.Mar 20 2019, 11:42 AM
ayounsi added a subscriber: ayounsi.EditedApr 5 2019, 12:29 AM

That's useful, thank you.

It didn't work for
I think it's because it added a coma behind the regex.
+ NODES='re:dns.*,'

If we compare it to this successful one:
+ NODES='re:cloudcontrol.*'

I tried it on as well, the extra comma adds a ghost failed host, see

In addition, Jenkins doesn't seem to like having more than Change-id and Bug in the footer:

jbond added a comment.Apr 5 2019, 9:47 AM

In addition, Jenkins doesn't seem to like having more than Change-id and Bug in the footer:

Seems to be in the code wonder if it just needs a release

Change 518974 had a related patch set uploaded (by Jbond; owner: John Bond):
[integration/config@master] ppc check experimental: removing trailing comma

I hit trailing comma issue reported by ayounsi above and have attempted a fix. CR which triggered the error is

Change 518974 merged by jenkins-bot:
[integration/config@master] ppc check experimental: removing trailing comma