Page MenuHomePhabricator
Create Task
Maniphest T351450

Migrate Cloud VPS puppet infrastructure to Puppet 7
Closed, ResolvedPublic
Actions

Description

Puppet server v7 only runs on Bookworm. So almost all existing puppetmasters will need to be rebuild as new VMs. Steps for each server will include:

  • build new server VM, apply "profile::puppet::agent::force_puppet7: true", allow puppet to stabilize, apply "role::puppetserver::cloud_vps_project", allow puppet to stabilize
  • Migrate local changes in operations/puppet and labs/private to the new server
  • clone certs from existing v5 puppet server to new puppet server
    • is this possible if the new server has a different name than the old one?
  • move clients from old server to new server

An alternative to cloning certs is to just treat the move from old puppet master to new puppet server as clean break requiring regeneration of all certs (the same dance that we do when moving from the central puppet master to a project-local one).

Details

Related Changes in Gerrit:
Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedAndrewT351450 Migrate Cloud VPS puppet infrastructure to Puppet 7
 ResolvedtaaviT351452 Migrate per-project Puppet servers to Puppet 7
 ResolvedtaaviT351453 Migrate Puppet servers in Cloud Services team managed projects to Puppet 7
 ResolvedAndrewT351494 Migrate Toolforge to Puppet 7
 ResolvedAndrewT359810 Are clouddb-wikireplicas-query-1 and the cloudb-services project still useful?
 ResolvedfnegriT365975 [cloud-vps] migrate DNS zones away from deprecated clouddb-services project
 InvalidtaaviT351454 Write script or cookbook to migrate data from a Puppet 5 puppetmaster to a Puppet 7 puppetserver
 ResolvedAndrewT351509 Update Wikitech documentation on per-project Puppet servers
 ResolvedJeltoT360459 Update gitlab-runners project puppetmaster
 ResolvedAndrewT360461 Update Integration project puppetmaster
 ResolvedbrennenT360470 Update devtools project puppetmaster
 ResolvedAndrewT360823 Increase instance and volume quota in devtools project for puppetmaster upgrade
 ResolvedAndrewT360940 Update traffic project puppetmaster
 ResolvedAndrewT361371 Update mailman project puppetmaster
 ResolvedAndrewT361591 Update pki project puppetmaster
 ResolvedAndrewT361592 Update monitoring project puppetmaster
 ResolvedAndrewT361593 Update puppet-dev project puppetmaster
 ResolvedAndrewT361594 Update mariadbtest project puppetmaster
 ResolvedDwisehauptT361595 Update puppet civicrm-prototype puppetmaster
 ResolvedAndrewT361596 Update puppet wikidata-query puppetmaster
 InvalidNoneT351468 Andrew tries to make a cloud-vps puppet7 server
 ResolvedtaaviT351499 Restrict creation of new Debian Buster VMs
 ResolvedAndrewT351510 Build new Bullseye and Bookworm base images with Puppet 7
 ResolvedAndrewT351451 Migrate Cloud VPS central puppet server to Puppet 7
 ResolvedAndrewT351455 Update designate-sink cert cleaning hook to work with Puppet 7 CA changes
 ResolvedAndrewT361388 Update maps-experiments project puppetmaster

Event Timeline

taavi created this task.Nov 16 2023, 6:47 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 16 2023, 6:47 PM
taavi closed subtask T351499: Restrict creation of new Debian Buster VMs as Resolved.Nov 17 2023, 9:39 AM
fnegri moved this task from Inbox to Soon! on the cloud-services-team board.Jan 10 2024, 4:15 PM
joanna_borun removed a project: Infrastructure-Foundations.Jan 29 2024, 4:03 PM
fnegri edited projects, added Goal, cloud-services-team (FY2023/2024-Q3-Q4); removed cloud-services-team.Feb 1 2024, 11:21 AM
Andrew closed subtask T351468: Andrew tries to make a cloud-vps puppet7 server as Invalid.Feb 28 2024, 10:35 PM
Andrew updated the task description. (Show Details)Feb 29 2024, 10:37 PM
Andrew claimed this task.Mar 4 2024, 10:25 PM
gerritbot added a comment.Mar 4 2024, 10:25 PM

Change 1008554 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] role::puppetserver::cloud_vps_project: remove firewall config

https://gerrit.wikimedia.org/r/1008554

gerritbot added a project: Patch-For-Review.Mar 4 2024, 10:25 PM
Andrew added a comment.Mar 5 2024, 3:13 PM

puppet7 servers need > 1 Gb of RAM or they swap

gerritbot added a comment.Mar 5 2024, 6:03 PM

Change 1008554 merged by Andrew Bogott:

[operations/puppet@production] role::puppetserver::cloud_vps_project: remove firewall config

https://gerrit.wikimedia.org/r/1008554

Maintenance_bot removed a project: Patch-For-Review.Mar 5 2024, 6:30 PM
gerritbot added a comment.Mar 8 2024, 8:36 PM

Change 1009798 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] git-sync-upstream: on puppet7, deploy code after update

https://gerrit.wikimedia.org/r/1009798

gerritbot added a project: Patch-For-Review.Mar 8 2024, 8:36 PM
gerritbot added a comment.Mar 11 2024, 11:49 AM

Change 1010168 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] hieradata: WMCS: try to evict Puppet cache after more operations

https://gerrit.wikimedia.org/r/1010168

gerritbot added a comment.Mar 11 2024, 12:06 PM

Change 1010168 merged by Majavah:

[operations/puppet@production] hieradata: WMCS: try to evict Puppet cache after more operations

https://gerrit.wikimedia.org/r/1010168

gerritbot added a comment.Mar 13 2024, 5:29 PM

Change 1009798 abandoned by Andrew Bogott:

[operations/puppet@production] git-sync-upstream: on puppet7, deploy code after update

Reason:

dropped, using https://gerrit.wikimedia.org/r/c/operations/puppet/+/1010168 instead

https://gerrit.wikimedia.org/r/1009798

Maintenance_bot removed a project: Patch-For-Review.Mar 13 2024, 5:31 PM
gerritbot added a comment.Mar 14 2024, 12:36 PM

Change 1009798 restored by Andrew Bogott:

[operations/puppet@production] git-sync-upstream: on puppet7, deploy code after update

https://gerrit.wikimedia.org/r/1009798

gerritbot added a project: Patch-For-Review.Mar 14 2024, 12:36 PM
gerritbot added a comment.Mar 15 2024, 2:25 PM

Change 1009798 merged by Andrew Bogott:

[operations/puppet@production] git-sync-upstream: on puppet7, deploy code after update

https://gerrit.wikimedia.org/r/1009798

Andrew added a comment.Mar 15 2024, 7:25 PM

I have built a very large server (cloudinfra-cloudvps-puppetserver-1.cloudinfra.eqiad1.wikimedia.cloud) which I hope will be able to handle all of cloud-vps on its own. I'll change the DNS entry on Monday.

gerritbot added a comment.Mar 18 2024, 2:23 PM

Change 1012384 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] eqiad pdns: move the 'puppet' alias to the new puppetserver

https://gerrit.wikimedia.org/r/1012384

gerritbot added a comment.Mar 18 2024, 2:37 PM

Change 1012389 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] puppetserver: allow specifying an autosign script

https://gerrit.wikimedia.org/r/1012389

gerritbot added a comment.Mar 18 2024, 2:58 PM

Change 1012389 merged by Andrew Bogott:

[operations/puppet@production] puppetserver: allow specifying an autosign script

https://gerrit.wikimedia.org/r/1012389

gerritbot added a comment.Mar 18 2024, 2:59 PM

Change 1012384 merged by Andrew Bogott:

[operations/puppet@production] eqiad pdns: move the 'puppet' alias to the new puppetserver

https://gerrit.wikimedia.org/r/1012384

gerritbot added a comment.Mar 18 2024, 3:23 PM

Change 1012396 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] profile::puppetserver::wmcs: include validatelabsfqdn.py script

https://gerrit.wikimedia.org/r/1012396

gerritbot added a comment.Mar 18 2024, 3:27 PM

Change 1012396 merged by Andrew Bogott:

[operations/puppet@production] profile::puppetserver::wmcs: include validatelabsfqdn.py script

https://gerrit.wikimedia.org/r/1012396

gerritbot added a comment.Mar 18 2024, 5:36 PM

Change 1012414 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] puppetserver: allow specifying certname in [server] conf

https://gerrit.wikimedia.org/r/1012414

gerritbot added a comment.Mar 18 2024, 10:37 PM

Change 1012414 abandoned by Andrew Bogott:

[operations/puppet@production] puppetserver: allow specifying certname in [server] conf

Reason:

dropping in favor of dns_alt_names

https://gerrit.wikimedia.org/r/1012414

gerritbot added a comment.Mar 19 2024, 10:51 PM

Change 1012764 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] puppetserver: add puppet7-facts-export-nodb.py

https://gerrit.wikimedia.org/r/1012764

gerritbot added a comment.Mar 20 2024, 1:01 PM

Change 1012764 merged by Andrew Bogott:

[operations/puppet@production] puppetserver: add puppet7-facts-export-nodb.py

https://gerrit.wikimedia.org/r/1012764

gerritbot added a comment.Mar 20 2024, 2:59 PM

Change 1013066 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] pcc-db1001.yaml: update a bunch of cloud-vps puppetserver keys

https://gerrit.wikimedia.org/r/1013066

gerritbot added a comment.Mar 20 2024, 3:02 PM

Change 1013066 merged by Andrew Bogott:

[operations/puppet@production] pcc-db1001.yaml: update a bunch of cloud-vps puppetserver keys

https://gerrit.wikimedia.org/r/1013066

gerritbot added a comment.Mar 20 2024, 3:25 PM

Change 1013079 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] codfw1dev puppet: change the 'puppet' alias to point to the new puppet7 server

https://gerrit.wikimedia.org/r/1013079

gerritbot added a comment.Mar 20 2024, 3:54 PM

Change 1013079 merged by Andrew Bogott:

[operations/puppet@production] codfw1dev puppet: change the 'puppet' alias to point to the new puppet7 server

https://gerrit.wikimedia.org/r/1013079

gerritbot added a comment.Mar 20 2024, 4:47 PM

Change 1013100 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] pcc-db1001.yaml: further attempt to upload cloud-vps puppetserver key

https://gerrit.wikimedia.org/r/1013100

gerritbot added a comment.Mar 20 2024, 4:49 PM

Change 1013100 merged by Andrew Bogott:

[operations/puppet@production] pcc-db1001.yaml: further attempt to upload cloud-vps puppetserver key

https://gerrit.wikimedia.org/r/1013100

gerritbot added a comment.Mar 20 2024, 5:18 PM

Change 1013108 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] pcc-db1001.yaml: another round of keys for uploading puppet facts

https://gerrit.wikimedia.org/r/1013108

gerritbot added a comment.Mar 20 2024, 5:20 PM

Change 1013108 merged by Andrew Bogott:

[operations/puppet@production] pcc-db1001.yaml: another round of keys for uploading puppet facts

https://gerrit.wikimedia.org/r/1013108

gerritbot added a comment.Mar 20 2024, 6:40 PM

Change 1013126 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] pcc-db1001.yaml: yet newer keys for uploading puppet facts

https://gerrit.wikimedia.org/r/1013126

gerritbot added a comment.Mar 20 2024, 6:42 PM

Change 1013126 merged by Andrew Bogott:

[operations/puppet@production] pcc-db1001.yaml: yet newer keys for uploading puppet facts

https://gerrit.wikimedia.org/r/1013126

Andrew closed subtask T361388: Update maps-experiments project puppetmaster as Resolved.Mar 29 2024, 6:35 PM
Andrew closed subtask T351451: Migrate Cloud VPS central puppet server to Puppet 7 as Resolved.Apr 3 2024, 3:03 AM
Andrew added a comment.Apr 3 2024, 3:09 AM

Hm... the puppet servers themselves are upgraded but I'm not sure when to actually switch all VMs to puppet7. Ideally we'd have all the project puppet servers upgraded first.

gerritbot added a comment.Apr 9 2024, 4:58 PM

Change #1018316 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] cloud: switch (almost) everything to puppet 7

https://gerrit.wikimedia.org/r/1018316

gerritbot added a comment.Apr 9 2024, 6:45 PM

Change #1018316 merged by Andrew Bogott:

[operations/puppet@production] cloud: switch (almost) everything to puppet 7

https://gerrit.wikimedia.org/r/1018316

Andrew closed subtask T351510: Build new Bullseye and Bookworm base images with Puppet 7 as Resolved.Apr 10 2024, 9:14 PM
Andrew closed this task as Resolved.Apr 23 2024, 7:28 PM
Dzahn mentioned this in T363415: add bullseye support to deployment server puppet role - upgrade deployment server in devtools.May 10 2024, 9:22 PM
fnegri moved this task from Backlog to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.Jun 10 2024, 2:27 PM
fnegri reopened this task as In Progress.Jul 1 2024, 1:34 PM
fnegri moved this task from Done to In progress on the cloud-services-team (FY2023/2024-Q3-Q4) board.
fnegri subscribed.

Ceph hosts are still missing, blocked by T309789: [ceph] Upgrade hosts to bullseye

taavi closed this task as Resolved.Jul 1 2024, 1:38 PM

which is tracked in T349619: Migrate roles to puppet7 and not here.

fnegri moved this task from In progress to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.Jul 1 2024, 1:49 PM
taavi closed subtask T351452: Migrate per-project Puppet servers to Puppet 7 as Resolved.Feb 9 2025, 8:40 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits