| jbond | |
| Oct 24 2019, 12:08 PM |
| F37137894: Capture d’écran du 2023-07-13 12-08-08.png | |
| Jul 13 2023, 10:12 AM |
The puppet master command and therefore the puppet master --compile command used by our puppet catalouge compiler is to be removed in puppet 6 and a new compile tool will be created· This is not something we can fix in the short term but something we should keep an eye on
another differ https://github.com/camptocamp/puppet-catalog-diff
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T294906 Puppet Improvements | |||
| Duplicate | jbond | T265138 Work required to prepare for puppet 7 | |||
| Open | None | T330490 Next steps for Puppet 7 | |||
| Resolved | jbond | T236373 update pcc with puppet 7 support | |||
| Declined | None | T236481 Investigate using the rich_data option to support Binary and binary_file for binary data |
Need to check this further but it may be possible to switch to puppet catalogue compile I also wonder if we can use puppet facts upload to upload facts to the compilers?
Change 936321 had a related patch set uploaded (by Jbond; author: jbond):
[operations/software/puppet-compiler@master] puppet: switch to puppet7 command
@hashar is there a way we can control which worker a pcc job will run on based on the some param we pass to the operations-puppet-catalog-compiler jenkins job.
To expand what i would like to do is have some pcc workers running puppet5 and some running puppet7. then have some parameter for the jenkins job, e.g. PUPPET_VERSION = ENUM[5, 7] which then controls which worker Jenkins sends the job too?
Once we have it configured in the jenkins job we can then add some control to pcc and "check experimental"
m'ybe yes, m'ybe no an idiom coming from Normandy meaning: probably. That was my immediate first thought last night.
You most probably know about all of what is following, but I will summarize the context for historical purpose and ensure we are in the same reference frame. There are at least two ways to trigger the Puppet compiler each with their own possibilities and limitations, hence my maybe yes, maybe not tongue-in-cheek.
The whole chain is:
We can ask Zuul to override the label on which the job will be run, by altering the build parameters in Zuul before it get send to Jenkins. That is done by adding the ZUUL_NODE parameter to the build but Afaik nothing uses it. The devil is that when the build is being processed by Zuul, I don't see how we can determine whether the job should run on a Puppet 5 or Puppet 7 node.
We could add a new pipeline in Zuul but that adds overhead to the whole system. It is not ideal. Maybe it is easier to create a 2nd job which is made to run on a different label such as {puppet7-compiler-node) which would be applied on the new node. As a result check experimental would trigger TWO catalogue compilation (though we can ask Zuul to run the Puppet 5 job first and wait for it to be a success before triggering the Puppet 7 job at the expense of doubling the feedback time).
That lets human run a different Jenkins job: operations-puppet-catalog-compiler by using the Jenkins REST API. The job can be passed any parameters as long as they are defined in the job definition (in integration/config jjb/operations-puppet-catalog-compiler.yml):
node: puppet-compiler-node parameters: - string: name: NUM_THREADS default: 4 description: "Number of threads used on the labs instance to compile catalog. Don't use more than four!" - string: name: GERRIT_CHANGE_NUMBER description: "The Gerrit change number (without patchset number) that will be fetched from Gerrit to compile the catalogs" - string: name: GERRIT_PRIVATE_CHANGE_NUMBER description: "The Gerrit change number (without patchset number) of the private repo that will be fetched from Gerrit to compile the change catalog" - string: name: LIST_OF_NODES description: | List of nodes (comma separated) to compile catalogs against. Leave blank to have the software select nodes for you (by parsing site.pp and matches that with the node list it had). - string: name: FAIL_FAST default: "" description: "If set to anything, will stop the compilation when the first failure is found and abort."
The job is currently tied to the puppet-compiler-node. I think we can remove the explicitly label and replace that with a job parameter with a list of labels to pick from (puppet-compiler-node and puppet7-compiler-node) with one of them being the default. If another value is passed the build request will be rejected. I think I once did that, I would have to lookup whether it is doable in Jenkins. Assuming it is doable, a parameter such as NODE_LABEL would be added and the utils/pcc.py can optionally set it as NODE_LABEL=puppet7-compiler-node to trigger the build on the Puppet 7 nodes. Additionally the parameter will be made available as an environment variable in the build.
Conclusion
Once Puppet 7 workers nodes are created on the WMCS project we would attach them as Jenkins agents with the label puppet7-compiler-node. Then:
Create an alias of the existing operations-puppet-catalog-compiler-puppet-test to a new job operations-puppet-catalog-compiler-puppet7-test with the new label and add that new Job to the operations/puppet experimental pipeline. That would cause check experimental to trigger BOTH jobs in parallel.
Investigate how to make the manual job operations-puppet-catalog-compiler to allow users to select the label it will run on :)
Change 937618 had a related patch set uploaded (by Hashar; author: Hashar):
[integration/config@master] Add Puppet 7 catalogue compiler job
Change 937791 had a related patch set uploaded (by Hashar; author: Hashar):
[integration/config@master] jjb: allow Jenkins label selection to PCC job
Mentioned in SAL (#wikimedia-releng) [2023-07-13T08:00:30Z] <hashar> integration: on CI Jenkins installing Node and Label parameter and updating the Token macro plugin T236373
@hashar thanks for the write up and the CR's, this all looks quite promising. ill work on getting pcc-worker4 up and running.
Mentioned in SAL (#wikimedia-releng) [2023-07-13T09:32:40Z] <hashar> integration: on CI Jenkins, removed the Node and Label parameter plugin, it doesn't have the features I was looking for. T236373
Mentioned in SAL (#wikimedia-releng) [2023-07-13T10:04:51Z] <hashar> integration: updating Matrix Combination CI Jenkins plugin from 1.31 to 1.32 to fix: grey ball images are broken since Jenkins-2.333 (JENKINS-69777) # T236373
For the job being run manually, I have made a copy at https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler-bis/build which offers the label selection:
Which defaults to run on Jenkins agents having the puppet5-compiler-node label (it will have to be added to the existing Puppet 5 node. One can then tick puppet7-compiler-node to also run the build on the to be added Puppet 7 pcc workers. But that will results in two different PCC runs each with their own reports. Then given they run on different nodes, I don't see alternatives.
Change 936321 merged by Jbond:
[operations/software/puppet-compiler@master] puppet: switch to puppet7 command
Change 960035 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] pcc: move pcc1004 to pcc version 3
Change 960035 merged by Jbond:
[operations/puppet@production] pcc: move pcc1004 to pcc version 3
Change 937618 merged by jenkins-bot:
[integration/config@master] Add Puppet 7 catalogue compiler job
Mentioned in SAL (#wikimedia-releng) [2023-10-16T13:43:44Z] <hashar> Added label puppet5-compiler-node to pcc-worker1001, pcc-worker1002 and pcc-worker1003 on top of the existing puppet-compiler-node label # T236373
Mentioned in SAL (#wikimedia-operations) [2023-10-17T09:59:41Z] <hashar> Deleted operations-puppet-catalog-compiler Jenkins job to replace it with a new job letting one picks the Puppet version(s) to compile against | T236373
Change 937791 merged by jenkins-bot:
[integration/config@master] jjb: convert PCC job to a matrix job
Change 966500 had a related patch set uploaded (by Hashar; author: Hashar):
[integration/config@master] jjb: rename puppet-compiler-node to puppet5-compiler-node
Mentioned in SAL (#wikimedia-releng) [2023-10-17T10:10:13Z] <hashar> Removing puppet-compiler-node label from the PCC workers, superseded by puppet5-compiler-node label | T236373
Change 966500 merged by jenkins-bot:
[integration/config@master] jjb: rename puppet-compiler-node to puppet5-compiler-node
Change 967148 had a related patch set uploaded (by Jbond; author: jbond):
[operations/software/puppet-compiler@2.x] puppet: switch to puppet7 command
Change 967929 had a related patch set uploaded (by Hashar; author: Hashar):
[integration/config@master] zuul: make Puppet 7 compiler non voting
Change 967929 merged by jenkins-bot:
[integration/config@master] zuul: make Puppet 7 compiler non voting