Page MenuHomePhabricator
Create Task
Maniphest T264276

Migrate puppetboard to Bullseye
Closed, ResolvedPublic
Actions

Description

puppetboard* are still on Stretch and should be moved to Buster. New VMs will be created in parallel and the old ones eventually shut down.

Details

Other Assignee
jbond
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

MoritzMuehlenhoff created this task.Oct 1 2020, 8:30 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2020, 8:30 AM
Volans added a comment.Oct 1 2020, 8:44 AM

@MoritzMuehlenhoff ping me when this work will start as we might want to upgrade puppetboard too. At the time I had to apply a couple of internal patches because not yet merged upstream. We should re-check the status and evaluate.

jbond added a comment.Oct 1 2020, 9:40 AM
In T264276#6507984, @Volans wrote:

@MoritzMuehlenhoff ping me when this work will start as we might want to upgrade puppetboard too. At the time I had to apply a couple of internal patches because not yet merged upstream. We should re-check the status and evaluate.

I couldn't see a PR on GitHub, if you point me to one i may be able to move it along

Volans added a comment.Oct 1 2020, 11:10 AM

The PRs I had there got eventually merged, at that time puppetboard was barely maintained so there was no release, but I guess they are included in the latest releases.
Then there is https://github.com/voxpupuli/puppetboard/issues/461 but I have totally forgotten what I might have done in our setup for that, but I'm sure I can reconstruct it.

gerritbot added a comment.Jan 22 2021, 1:36 PM

Change 657812 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Add puppetboard[12]002

https://gerrit.wikimedia.org/r/657812

gerritbot added a project: Patch-For-Review.Jan 22 2021, 1:36 PM
gerritbot added a comment.Jan 22 2021, 2:00 PM

Change 657812 merged by Muehlenhoff:
[operations/puppet@production] Add puppetboard[12]002

https://gerrit.wikimedia.org/r/657812

Maintenance_bot removed a project: Patch-For-Review.Jan 22 2021, 2:10 PM
jbond renamed this task from Migrate puppetboard to Buster to Migrate puppetboard to Bullseye.Oct 5 2021, 2:39 PM
jbond changed the task status from Open to In Progress.
jbond triaged this task as Medium priority.
jbond added a parent task: T292522: Upgrade puppetboard to the latest version.
gerritbot added a comment.Oct 5 2021, 2:40 PM

Change 726637 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] linux-host-entries: update puppetboard[12]002 servers to bullseye

https://gerrit.wikimedia.org/r/726637

gerritbot added a project: Patch-For-Review.Oct 5 2021, 2:40 PM

Change 726637 merged by Jbond:

[operations/puppet@production] linux-host-entries: update puppetboard[12]002 servers to bullseye

https://gerrit.wikimedia.org/r/726637

Maintenance_bot removed a project: Patch-For-Review.Oct 5 2021, 3:10 PM
gerritbot added a comment.Oct 6 2021, 11:17 AM

Change 726872 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] stdlib: update stdlib from version 7.0.1 to 8.1.0

https://gerrit.wikimedia.org/r/726872

gerritbot added a project: Patch-For-Review.Oct 6 2021, 11:17 AM
gerritbot added a comment.Oct 6 2021, 11:47 AM

Change 726878 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] sslcert: switch to using ensure_packages

https://gerrit.wikimedia.org/r/726878

gerritbot added a comment.Oct 6 2021, 11:54 AM

Change 726878 merged by Jbond:

[operations/puppet@production] sslcert: switch to using ensure_packages

https://gerrit.wikimedia.org/r/726878

gerritbot added a comment.Oct 7 2021, 11:10 AM

Change 726872 merged by Jbond:

[operations/puppet@production] stdlib: update stdlib from version 7.0.1 to 8.1.0

https://gerrit.wikimedia.org/r/726872

gerritbot added a comment.Oct 22 2021, 3:53 PM

Change 733002 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] P:puppetdb: update puppetboard hosts

https://gerrit.wikimedia.org/r/733002

gerritbot added a comment.Oct 22 2021, 4:00 PM

Change 733002 merged by Jbond:

[operations/puppet@production] P:puppetdb: update puppetboard hosts

https://gerrit.wikimedia.org/r/733002

gerritbot added a comment.Oct 25 2021, 11:09 AM

Change 734232 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] P:puppetmaster::ng: Enable catalog and update inventory facts

https://gerrit.wikimedia.org/r/734232

gerritbot added a comment.Oct 25 2021, 11:38 AM

Change 734232 merged by Jbond:

[operations/puppet@production] P:puppetmaster::ng: Enable catalog and update inventory facts

https://gerrit.wikimedia.org/r/734232

gerritbot added a comment.Oct 25 2021, 12:18 PM

Change 734255 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] P:puppetboard::ng: add notify for service

https://gerrit.wikimedia.org/r/734255

gerritbot added a comment.Oct 25 2021, 12:21 PM

Change 734255 merged by Jbond:

[operations/puppet@production] P:puppetboard::ng: add notify for service

https://gerrit.wikimedia.org/r/734255

gerritbot added a comment.Oct 25 2021, 12:36 PM

Change 734257 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] P:puppetboard::ng: drop icinga check

https://gerrit.wikimedia.org/r/734257

gerritbot added a comment.Oct 25 2021, 12:40 PM

Change 734257 merged by Jbond:

[operations/puppet@production] P:puppetboard::ng: drop icinga check

https://gerrit.wikimedia.org/r/734257

gerritbot added a comment.Nov 2 2021, 1:01 PM

Change 736228 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] O:puppetboard: add puppetboard.w.o to the cert SAN

https://gerrit.wikimedia.org/r/736228

gerritbot added a comment.Nov 2 2021, 1:03 PM

Change 736228 merged by Jbond:

[operations/puppet@production] O:puppetboard: add puppetboard.w.o to the cert SAN

https://gerrit.wikimedia.org/r/736228

jbond added a comment.Nov 2 2021, 1:26 PM

I have now built puppetboard[12]002 with bullseye pypuppetdb 2.4 and puppetboard to 3.1. i will leave the old systems around for ~1week before starting the decommissioning process

gerritbot added a comment.Nov 30 2021, 12:08 PM

Change 742704 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] scap::dsh: update puppetboard hosts to new host

https://gerrit.wikimedia.org/r/742704

Volans added a comment.Nov 30 2021, 2:14 PM

Uncomplete list of things to cleanup in random order that I think are not needed anymore if we can ditch the old puppetboard puppet module:

gerritbot added a comment.Nov 30 2021, 2:17 PM

Change 742704 merged by Jbond:

[operations/puppet@production] scap::dsh: remove puppetboard; hosts now use apt

https://gerrit.wikimedia.org/r/742704

jbond added a comment.Nov 30 2021, 2:19 PM

Thanks riccardo although unless im missing something:

In T264276#7537473, @Volans wrote:
  • puppet's profile::puppetdb::puppetboard_hosts

This is still needed for the ferm ACL's

Volans added a comment.Nov 30 2021, 2:23 PM
In T264276#7537486, @jbond wrote:

Thanks riccardo although unless im missing something:

In T264276#7537473, @Volans wrote:
  • puppet's profile::puppetdb::puppetboard_hosts

This is still needed for the ferm ACL's

Yes but I think still refer to all 4 hosts. I meant cleanup the old hostnames from there, sorry.

Forgot to add: manual cleanup of the /srv/deployment/puppetboard directory in the deployment hosts

jbond added a comment.Nov 30 2021, 2:24 PM

Yes but I think still refer to all 4 hosts. I meant cleanup the old hostnames from there, sorry.

ack thanks

Forgot to add: manual cleanup of the /srv/deployment/puppetboard directory in the deployment hosts

ack

gerritbot added a comment.Nov 30 2021, 2:40 PM

Change 742731 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] P:puppetdb: drop old puppetboard hosts

https://gerrit.wikimedia.org/r/742731

gerritbot added a comment.Nov 30 2021, 2:41 PM

Change 742731 merged by Jbond:

[operations/puppet@production] P:puppetdb: drop old puppetboard hosts

https://gerrit.wikimedia.org/r/742731

jbond closed this task as Resolved.Nov 30 2021, 2:44 PM
jbond updated Other Assignee, added: jbond.

Ill close this and haneld the rest of the deom in https://phabricator.wikimedia.org/T296744

ops-monitoring-bot added a comment.Dec 1 2021, 2:28 PM

cookbooks.sre.hosts.decommission executed by jbond@cumin1001 for hosts: puppetboard2001.codfw.wmnet

  • puppetboard2001.codfw.wmnet (PASS)
    • Downtimed host on Icinga
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster ganeti01.svc.codfw.wmnet to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster ganeti01.svc.codfw.wmnet to Netbox
ops-monitoring-bot added a comment.Dec 1 2021, 2:38 PM

cookbooks.sre.hosts.decommission executed by jbond@cumin1001 for hosts: puppetboard1001.eqiad.wmnet

  • puppetboard1001.eqiad.wmnet (PASS)
    • Downtimed host on Icinga
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster ganeti01.svc.eqiad.wmnet to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster ganeti01.svc.eqiad.wmnet to Netbox
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL