Default to the Puppet 7 PCC CI test, make it voting and eventually remove the Puppet 5 one
Open, MediumPublic
Actions

Assigned To

None

Authored By

	MoritzMuehlenhoff
	Jun 13 2024, 11:48 AM

Description

Currently the Puppet compiler runs two jobs

operations-puppet-catalog-compiler-test (voting) compiles the catalogues on Puppet 5
operations-puppet-catalog-compiler-puppet7-test (non-voting) compiles the catalogues on Puppet 5

Currently we have less than 20 roles on Puppet 5 and they are pretty much all on Puppet 5 because the systems are still using Buster. There are no additional Puppet 7-specific surprises we expect, as such I think we should:

Make the Puppet 7 job voting now
Remove the Puppet 5 check when all roles are migrated (except the roles which won't be migrated (puppetmaster::backend and puppetmaster:.frontend))

Details

Related Changes in Gerrit:

Subject	Repo	Branch	Lines +/-
Zuul: [operations/puppet] Stop running PCC with Puppet 5	integration/config	master	+0 -1
jjb: Remove Puppet 5 PCC jobs	integration/config	master	+3 -13
Stop uploading puppet facts to PCC from puppetmaster1001	operations/puppet	production	+1 -1
jjb: Default to Puppet 7 for PCC jobs	integration/config	master	+1 -1
zuul: [operations/puppet] Make PCC voting on Puppet 7	integration/config	master	+0 -4

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T365798 Shutdown of Puppet 5 servers
		Open		None	T367399 Default to the Puppet 7 PCC CI test, make it voting and eventually remove the Puppet 5 one

Event Timeline

MoritzMuehlenhoff created this task.Jun 13 2024, 11:48 AM

Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptJun 13 2024, 11:48 AM

elukey subscribed.Jun 17 2024, 3:11 PM

• joanna_borun triaged this task as High priority.Jun 17 2024, 3:11 PM

Change #1047024 had a related patch set uploaded (by Majavah; author: Majavah):

[integration/config@master] zuul: [operations/puppet] Make PCC voting on Puppet 7

https://gerrit.wikimedia.org/r/1047024

Change #1047025 had a related patch set uploaded (by Majavah; author: Majavah):

[integration/config@master] jjb: Default to Puppet 7 for PCC jobs

https://gerrit.wikimedia.org/r/1047025

Change #1047024 merged by jenkins-bot:

[integration/config@master] zuul: [operations/puppet] Make PCC voting on Puppet 7

https://gerrit.wikimedia.org/r/1047024

Mentioned in SAL (#wikimedia-releng) [2024-06-18T14:40:44Z] <hashar> Reloaded Zuul for https://gerrit.wikimedia.org/r/1047024 "Make PCC voting on Puppet 7" # T367399

Change #1047025 merged by jenkins-bot:

[integration/config@master] jjb: Default to Puppet 7 for PCC jobs

https://gerrit.wikimedia.org/r/1047025

Did one of these changes possinbly break PCC here?
https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler-test/3739/console

https://gerrit.wikimedia.org/r/c/operations/puppet/+/1042898 compiled file in the past, but after a rebase and PCC re-run triggered the error above.

In T367399#9909402, @MoritzMuehlenhoff wrote:

Did one of these changes possinbly break PCC here?
https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler-test/3739/console

https://gerrit.wikimedia.org/r/c/operations/puppet/+/1042898 compiled file in the past, but after a rebase and PCC re-run triggered the error above.

I re-ran PCC on this change and wasn't able to reproduce the issue. Perhaps some other change fixed the issue?

I briefly chatted with @hashar about this task. At present PCC does not vote. We could make it voting by moving it to test-prio. However, doing so would cause PCC to run on every patchset, which would probably be too taxing on CI. Instead he would need to setup a new pipeline that is voting but not run automatically. At present both puppet 7 & 5 run when hitting the button or commenting check experimental. @hashar also opened a task T371407 to improve the UI. In the short term the current situation is okay, so lowering the priority to medium.

From T371407:

The red chipset now has the v5 mention (v7 passes and is collapsed in the blue / INFO one):

v5 and v7 are each different runs:

gerrit_split_pcc_runs.png (228×287 px, 17 KB)

I think that disambiguates what is failing.

What I am wondering is: if the host has a fact or hiera value stating it has migrated to Puppet v7, maybe we could have the Puppet v5 compiler to skip that host or report it with a new state "migrated"?

In T367399#10029018, @hashar wrote:

What I am wondering is: if the host has a fact or hiera value stating it has migrated to Puppet v7, maybe we could have the Puppet v5 compiler to skip that host or report it with a new state "migrated"?

I like the idea, but I think there may be too many corner cases to worry about, hopefully puppet v5 won't be with us too much longer!

Change #1075187 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Stop uploading puppet facts to PCC from puppetmaster1001

https://gerrit.wikimedia.org/r/1075187

taavi added a project: Puppet CI.Fri, Dec 5, 1:26 PM

Change #1215593 had a related patch set uploaded (by Majavah; author: Majavah):

[integration/config@master] Zuul: [operations/puppet] Stop running PCC with Puppet 5

https://gerrit.wikimedia.org/r/1215593

Change #1215594 had a related patch set uploaded (by Majavah; author: Majavah):

[integration/config@master] jjb: Remove Puppet 5 PCC jobs

https://gerrit.wikimedia.org/r/1215594

Change #1215593 merged by jenkins-bot:

[integration/config@master] Zuul: [operations/puppet] Stop running PCC with Puppet 5

https://gerrit.wikimedia.org/r/1215593

Mentioned in SAL (#wikimedia-releng) [2025-12-05T13:31:13Z] <James_F> Zuul: [operations/puppet] Stop running PCC with Puppet 5, for T367399

Change #1215594 merged by jenkins-bot: