currently the volatile endpoint is only configured on the puppet front ends. if we move to using SRV records then we would need this synced to all puppetservers. 🤞
Description
Description
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T330490 Next steps for Puppet 7 | |||
| Resolved | jbond | T341056 volatile: We need to configure the volatile endpoint on puppetserveres |
Event Timeline
Comment Actions
Change 948567 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetserver: Add support for defining additional mount points
Comment Actions
Change 948607 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] P:puppetserver: add support for extra_mounts
Comment Actions
Change 948608 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetserver: add volatile file mount
Comment Actions
Change 948567 merged by Jbond:
[operations/puppet@production] puppetserver: Add support for defining additional mount points
Comment Actions
Change 948607 merged by Jbond:
[operations/puppet@production] P:puppetserver: add support for extra_mounts
Comment Actions
Change 948608 merged by Jbond:
[operations/puppet@production] puppetserver: add volatile file mount
Comment Actions
Change 948647 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetserver: add volatile config
Comment Actions
Change 949978 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetserver: switch to useing ca_server instead of enable_ca
Comment Actions
Change 949978 merged by Jbond:
[operations/puppet@production] puppetserver: switch to useing ca_server instead of enable_ca
Comment Actions
Change 951138 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetmaster::fetch_swift_rings: rename profile
Comment Actions
Change 951138 merged by Jbond:
[operations/puppet@production] puppetmaster::fetch_swift_rings: rename profile
Comment Actions
Change 948647 merged by Jbond:
[operations/puppet@production] puppetserver: add volatile config
Comment Actions
Change 951440 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] external_clouds_vendors: add way top specify the private repo
Comment Actions
Change 951441 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetserver::volatile: pass through correct private repo path
Comment Actions
Change 951440 merged by Jbond:
[operations/puppet@production] external_clouds_vendors: add way top specify the private repo
Comment Actions
Change 951441 merged by Jbond:
[operations/puppet@production] puppetserver::volatile: pass through correct private repo path
Comment Actions
Change 951451 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetserver::volatile: ony update conftool on puppetmasters
Comment Actions
Change 951451 merged by Jbond:
[operations/puppet@production] puppetserver::volatile: ony update conftool on puppetmasters
Comment Actions
Change 951457 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetserver::volatile: we need to ensure if *not* empty
Comment Actions
Change 951457 merged by Jbond:
[operations/puppet@production] puppetserver::volatile: we need to ensure if *not* empty
Comment Actions
Change 951462 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] swift/thanos: allow puppetservers to also pull swift rings
Comment Actions
Change 951462 merged by Jbond:
[operations/puppet@production] swift/thanos: allow puppetservers to also pull swift rings
Comment Actions
Change 951506 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] profile::swift:fetch_rings: ensure we create directories
Comment Actions
Change 951506 merged by Jbond:
[operations/puppet@production] profile::swift:fetch_rings: ensure we create directories
Comment Actions
Change 951523 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetserver::rsync: open firwall port
Comment Actions
Change 951523 merged by Jbond:
[operations/puppet@production] puppetserver::rsync: open firwall port
Comment Actions
Change 951529 had a related patch set uploaded (by Jbond; author: jbond):
[operations/puppet@production] puppetserver::rsync: fix dir and ferm rule
Comment Actions
Change 951529 merged by Jbond:
[operations/puppet@production] puppetserver::rsync: fix dir and ferm rule
Comment Actions
volatile is now synced to all puppetserveres and agents using puppet7 can fetch data correctly
Comment Actions
During the preparation of the apt server migration I noticed that Puppet clients migrated to Puppet 7 receive outdated data, specifically I noticed this for the tftpboot data stored in volatile.
The current main apt servers (apt1001 and apt2001) are on Buster and thus Puppet 5. The new WIP apt server running bookworm (apt1002) has been migrated to Puppet 7 to catch errors early.
On apt1001 we have the latest version of the Bookworm image:
jmm@apt1001:/srv/tftpboot/bookworm-installer$ ls -lha total 20K dr-xr-xr-x 4 root root 4.0K Feb 12 08:19 . dr-xr-xr-x 64 root root 4.0K Oct 9 08:48 .. dr-xr-xr-x 3 root root 4.0K Feb 28 2023 debian-installer lrwxrwxrwx 1 root root 47 Feb 28 2023 ldlinux.c32 -> debian-installer/amd64/boot-screens/ldlinux.c32 lrwxrwxrwx 1 root root 33 Feb 28 2023 pxelinux.0 -> debian-installer/amd64/pxelinux.0 dr-xr-xr-x 2 root root 4.0K Feb 27 2023 pxelinux.cfg lrwxrwxrwx 1 root root 47 May 30 2023 splash.png -> debian-installer/amd64/boot-screens//splash.png -r--r--r-- 1 root root 65 Feb 12 08:19 version.info jmm@apt1001:/srv/tftpboot/bookworm-installer$ cat version.info Debian version: 12 (bookworm) Installer build: 20230607+deb12u5
But on apt1002:
jmm@apt1002:/srv/tftpboot/bookworm-installer$ ls -lha total 244K dr-xr-xr-x 4 root root 4.0K Nov 20 06:51 . dr-xr-xr-x 17 root root 4.0K Nov 20 06:51 .. dr-xr-xr-x 3 root root 4.0K Oct 11 13:59 debian-installer -r--r--r-- 1 root root 117K Nov 20 06:51 ldlinux.c32 -r--r--r-- 1 root root 42K Nov 20 06:51 pxelinux.0 dr-xr-xr-x 2 root root 4.0K Nov 20 06:51 pxelinux.cfg -r--r--r-- 1 root root 59K Nov 20 06:51 splash.png -r--r--r-- 1 root root 65 Nov 20 06:51 version.info jmm@apt1002:/srv/tftpboot/bookworm-installer$ cat version.info Debian version: 12 (bookworm) Installer build: 20230607+deb12u1
Initially assumed that were some kind of freshness issue, but pruning /srv/tftpboot/bookworm-installer re-generates the 20230607+deb12u1 version.
I also noticed that the same applies to the install servers (which are also on Puppet 7):
jmm@install3003:/srv/tftpboot/bookworm-installer$ ls -lha total 244K dr-xr-xr-x 4 root root 4.0K Nov 2 14:17 . dr-xr-xr-x 23 root root 4.0K Oct 9 08:48 .. dr-xr-xr-x 3 root root 4.0K Aug 17 06:50 debian-installer -r--r--r-- 1 root root 117K Nov 2 14:17 ldlinux.c32 -r--r--r-- 1 root root 42K Nov 2 14:17 pxelinux.0 dr-xr-xr-x 2 root root 4.0K Nov 2 14:17 pxelinux.cfg -r--r--r-- 1 root root 59K Nov 2 14:17 splash.png -r--r--r-- 1 root root 65 Nov 2 14:17 version.info jmm@install3003:/srv/tftpboot/bookworm-installer$ cat version.info Debian version: 12 (bookworm) Installer build: 20230607+deb12u1
@jhathaway Since we don't currently sync the contents of Puppet 5 volatile with Puppet 7 volatile for any other data source, it's probably best if we keep it simple and just establish that all updates of d-i simply need to happen twice (once for P5 and P7) until all consumers of the tftpboo data are migrated to Puppet 7? We refresh d-i images not very often (basically only after Debian makes a point release or when we create custom images like the buster image with Linux 5.10). What do you think?
Comment Actions
Change 1003004 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):
[operations/puppet@production] puppetserver: Also install the tool to update netboot images on puppet servers
Comment Actions
@Muehlenhoff I think that makes sense, are the updates run manually when Debian issues a release?
Comment Actions
Change 1003004 merged by Muehlenhoff:
[operations/puppet@production] puppetserver: Also install the tool to update netboot images on puppet servers
Comment Actions
Ack, thanks. Let's do that then.
are the updates run manually when Debian issues a release?
Yes, the netboot images don't contain firmware and since we need firmware very early for some NICs to even initiate the PXE boot, we hacked this script together to add them. And this use case will be needed for longer; while Debian created the new firmware-non-free section in the archive to allow regular installation media to include firmware, fixing the netboot image process to include is much more involved.
I think it's safe to say that the last person who ran that script before me was Faidon :-)
Comment Actions
Change 1003375 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):
[operations/puppet@production] update-netboot-image: Update instructions for Puppet 7
Comment Actions
Change 1003375 merged by Muehlenhoff:
[operations/puppet@production] update-netboot-image: Update instructions for Puppet 7
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2024-02-16T10:58:16Z] <moritzm> update bullseye/bookworm netboot images on the Puppet 7 volatile environment to the latest point releases (to bring in sync with volatile for Puppet 5) T341056