Page MenuHomePhabricator
Create Task
Maniphest T327867

Migrate the install servers to Bullseye
Closed, ResolvedPublic
Actions

Description

These are still on Buster currently. Replace them with new VMs:

sitenew VM namein serviceold VM decommissioned
eqiadinstall1004yesyes
codfwinstall2004yesyes
esamsinstall3002yesyes
ulsfoinstall4002yesyes
eqsininstall5002yesyes
drmrsinstall6002yesyes

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Remove Puppet references to install[12]003operations/puppetproduction+0 -18
Remove obsolete stub keytabslabs/privatemaster+0 -0
Remove Puppet references for install[3456]001operations/puppetproduction+0 -32
Remove installserver role from install1003operations/puppetproduction+1 -5
Update cloud proxiesoperations/puppetproduction+2 -2
Update next-server DHCP settings towards install1004operations/puppetproduction+42 -42
Point DHCP server in eqiad to install1004operations/homer/publicmaster+1 -1
Move webproxy in eqiad to install1004operations/dnsmaster+1 -1
Assign installserver role to install1004operations/puppetproduction+2 -6
Drop installserver role from install6001operations/puppetproduction+1 -9
Add testvm6001operations/puppetproduction+9 -0
Update tftp server settings for drmrsoperations/puppetproduction+4 -4
Point webproxy in drmrs to install6002operations/dnsmaster+1 -1
Assign installserver role to install6002operations/puppetproduction+1 -2
Remove installserver role from install5001operations/puppetproduction+2 -2
Point DHCP server for eqsin to install5002operations/puppetproduction+2 -2
Point DHCP server in eqsin to install5002operations/homer/publicmaster+1 -1
Point proxy in eqsin to install5002operations/dnsmaster+1 -1
Assign installserver role to install5002operations/puppetproduction+2 -2
Remove installserver role from install4001operations/puppetproduction+2 -2
Update DHCP config in ulsfooperations/puppetproduction+2 -2
Point DHCP server in ulsfo to install4002operations/homer/publicmaster+1 -1
Move webproxy in ulsfo to install4002operations/dnsmaster+1 -1
Setup install4002 as install serveroperations/puppetproduction+2 -2
Remove installserver role from install3001operations/puppetproduction+6 -2
Update DHCP config for esamsoperations/puppetproduction+3 -3
Point DHCP server in esams to install3002operations/homer/publicmaster+1 -1
Point the webproxy in esams to install3002operations/dnsmaster+1 -1
Apply installserver role to install3002operations/puppetproduction+5 -1
Remove installserver role from install2003operations/puppetproduction+5 -1
Point to install2004 for DHCP in codfwoperations/homer/publicmaster+1 -1
Move next-server settings from install2003->2004operations/puppetproduction+16 -16
Update cloudbastion rules for install2004operations/puppetproduction+6 -6
Move webproxy.codfw.wmnet to install2004operations/dnsmaster+1 -1
Apply role::installserver to install2004operations/puppetproduction+5 -1
New stub keytabs for the new install serverslabs/privatemaster+0 -0
Remove service toggle for TFTPoperations/puppetproduction+4 -17
Add new install serversoperations/puppetproduction+30 -0
Show related patches Customize query in gerrit

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
gerritbot added a comment.Jan 31 2023, 12:54 PM

Change 885336 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Move next-server settings from install2003->2004

https://gerrit.wikimedia.org/r/885336

MoritzMuehlenhoff added a comment.Jan 31 2023, 12:57 PM

install2004 has had the installserver role assigned and it's now acting the web proxy for codfw. The DHCP server is currrently stopped, tomorrow morning I'll merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/885336 and https://gerrit.wikimedia.org/r/c/operations/homer/public/+/885326 and test a reimage and VM installation.

gerritbot added a comment.Jan 31 2023, 3:37 PM

Change 885304 merged by Muehlenhoff:

[operations/puppet@production] Update cloudbastion rules for install2004

https://gerrit.wikimedia.org/r/885304

gerritbot added a comment.Feb 1 2023, 8:44 AM

Change 885336 merged by Muehlenhoff:

[operations/puppet@production] Move next-server settings from install2003->2004

https://gerrit.wikimedia.org/r/885336

MoritzMuehlenhoff updated the task description. (Show Details)Feb 1 2023, 8:46 AM
gerritbot added a comment.Feb 1 2023, 8:54 AM

Change 885326 merged by Muehlenhoff:

[operations/homer/public@master] Point to install2004 for DHCP in codfw

https://gerrit.wikimedia.org/r/885326

Maintenance_bot removed a project: Patch-For-Review.Feb 1 2023, 9:30 AM
ops-monitoring-bot added a comment.Feb 1 2023, 11:27 AM

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: testvm2001.codfw.wmnet

  • testvm2001.codfw.wmnet (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster codfw_test to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster codfw_test to Netbox
ops-monitoring-bot added a comment.Feb 1 2023, 12:41 PM

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: testvm2003.codfw.wmnet

  • testvm2003.codfw.wmnet (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster codfw_test to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster codfw_test to Netbox
gerritbot added a comment.Feb 1 2023, 12:48 PM

Change 885799 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove installserver role from install2003

https://gerrit.wikimedia.org/r/885799

gerritbot added a project: Patch-For-Review.Feb 1 2023, 12:48 PM
gerritbot added a comment.Feb 1 2023, 12:51 PM

Change 885799 merged by Muehlenhoff:

[operations/puppet@production] Remove installserver role from install2003

https://gerrit.wikimedia.org/r/885799

MoritzMuehlenhoff added a comment.Feb 1 2023, 12:55 PM

install2004 is fully in service now, I have tested a reimage on baremetal and VM installation successfully. atftpd, dhcpd and nginx have been stopped on install2003, but I'm keeping the VM for a few more days just in case.

MoritzMuehlenhoff triaged this task as Medium priority.Feb 1 2023, 12:56 PM
MoritzMuehlenhoff updated the task description. (Show Details)
gerritbot added a comment.Feb 1 2023, 1:18 PM

Change 885805 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/homer/public@master] Point DHCP server in esams to install3002

https://gerrit.wikimedia.org/r/885805

gerritbot added a comment.Feb 1 2023, 1:18 PM

Change 885806 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/homer/public@master] Point DHCP server in ulsfo to install4002

https://gerrit.wikimedia.org/r/885806

gerritbot added a comment.Feb 2 2023, 7:55 AM

Change 885982 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Point the webproxy in esams to install3002

https://gerrit.wikimedia.org/r/885982

gerritbot added a comment.Feb 2 2023, 7:56 AM

Change 885983 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Apply installserver role to install3002

https://gerrit.wikimedia.org/r/885983

gerritbot added a comment.Feb 2 2023, 8:00 AM

Change 885983 merged by Muehlenhoff:

[operations/puppet@production] Apply installserver role to install3002

https://gerrit.wikimedia.org/r/885983

gerritbot added a comment.Feb 2 2023, 8:17 AM

Change 885984 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Update DHCP config for esams

https://gerrit.wikimedia.org/r/885984

gerritbot added a comment.Feb 2 2023, 8:43 AM

Change 885982 merged by Muehlenhoff:

[operations/dns@master] Point the webproxy in esams to install3002

https://gerrit.wikimedia.org/r/885982

gerritbot added a comment.Feb 2 2023, 8:44 AM

Change 885805 merged by jenkins-bot:

[operations/homer/public@master] Point DHCP server in esams to install3002

https://gerrit.wikimedia.org/r/885805

gerritbot added a comment.Feb 2 2023, 8:57 AM

Change 885984 merged by Muehlenhoff:

[operations/puppet@production] Update DHCP config for esams

https://gerrit.wikimedia.org/r/885984

gerritbot added a comment.Feb 2 2023, 9:04 AM

Change 885989 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove installserver role from install3001

https://gerrit.wikimedia.org/r/885989

gerritbot added a comment.Feb 2 2023, 9:53 AM

Change 885989 merged by Muehlenhoff:

[operations/puppet@production] Remove installserver role from install3001

https://gerrit.wikimedia.org/r/885989

MoritzMuehlenhoff updated the task description. (Show Details)Feb 2 2023, 10:02 AM
gerritbot added a comment.Feb 2 2023, 12:41 PM

Change 886031 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Move webproxy in ulsfo to install4002

https://gerrit.wikimedia.org/r/886031

gerritbot added a comment.Feb 2 2023, 12:53 PM

Change 886036 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Setup install4002 as install server

https://gerrit.wikimedia.org/r/886036

gerritbot added a comment.Feb 2 2023, 12:54 PM

Change 886037 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Update DHCP config in ulsfo

https://gerrit.wikimedia.org/r/886037

gerritbot added a comment.Feb 2 2023, 1:11 PM

Change 886036 merged by Muehlenhoff:

[operations/puppet@production] Setup install4002 as install server

https://gerrit.wikimedia.org/r/886036

gerritbot added a comment.Feb 2 2023, 1:38 PM

Change 886031 merged by Muehlenhoff:

[operations/dns@master] Move webproxy in ulsfo to install4002

https://gerrit.wikimedia.org/r/886031

gerritbot added a comment.Feb 2 2023, 1:42 PM

Change 885806 merged by Muehlenhoff:

[operations/homer/public@master] Point DHCP server in ulsfo to install4002

https://gerrit.wikimedia.org/r/885806

gerritbot added a comment.Feb 2 2023, 1:55 PM

Change 886037 merged by Muehlenhoff:

[operations/puppet@production] Update DHCP config in ulsfo

https://gerrit.wikimedia.org/r/886037

gerritbot added a comment.Feb 2 2023, 2:02 PM

Change 886049 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove installserver role from install4001

https://gerrit.wikimedia.org/r/886049

gerritbot added a comment.Feb 2 2023, 2:06 PM

Change 886049 merged by Muehlenhoff:

[operations/puppet@production] Remove installserver role from install4001

https://gerrit.wikimedia.org/r/886049

MoritzMuehlenhoff updated the task description. (Show Details)Feb 2 2023, 2:12 PM
gerritbot added a comment.Feb 2 2023, 2:15 PM

Change 886053 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/homer/public@master] Point DHCP server in eqsin to install5002

https://gerrit.wikimedia.org/r/886053

ayounsi mentioned this in T327812: eqsin hosts are not rebooting when running sre.hosts.reimage cookbook.Feb 3 2023, 7:45 AM
gerritbot added a comment.Feb 3 2023, 8:16 AM

Change 886313 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Point proxy in eqsin to install5002

https://gerrit.wikimedia.org/r/886313

gerritbot added a comment.Feb 3 2023, 8:17 AM

Change 886314 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Assign installserver role to install5002

https://gerrit.wikimedia.org/r/886314

gerritbot added a comment.Feb 3 2023, 8:24 AM

Change 886314 merged by Muehlenhoff:

[operations/puppet@production] Assign installserver role to install5002

https://gerrit.wikimedia.org/r/886314

gerritbot added a comment.Feb 3 2023, 9:31 AM

Change 886313 merged by Muehlenhoff:

[operations/dns@master] Point proxy in eqsin to install5002

https://gerrit.wikimedia.org/r/886313

gerritbot added a comment.Feb 3 2023, 9:36 AM

Change 886320 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Point DHCP server for eqsin to install5002

https://gerrit.wikimedia.org/r/886320

gerritbot added a comment.Feb 3 2023, 9:41 AM

Change 886053 merged by Muehlenhoff:

[operations/homer/public@master] Point DHCP server in eqsin to install5002

https://gerrit.wikimedia.org/r/886053

gerritbot added a comment.Feb 3 2023, 9:52 AM

Change 886320 merged by Muehlenhoff:

[operations/puppet@production] Point DHCP server for eqsin to install5002

https://gerrit.wikimedia.org/r/886320

gerritbot added a comment.Feb 3 2023, 10:10 AM

Change 886326 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove installserver role from install5001

https://gerrit.wikimedia.org/r/886326

gerritbot added a comment.Feb 3 2023, 10:13 AM

Change 886326 merged by Muehlenhoff:

[operations/puppet@production] Remove installserver role from install5001

https://gerrit.wikimedia.org/r/886326

MoritzMuehlenhoff updated the task description. (Show Details)Feb 3 2023, 10:20 AM
Maintenance_bot removed a project: Patch-For-Review.Feb 3 2023, 10:30 AM
gerritbot added a comment.Feb 6 2023, 12:57 PM

Change 886888 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/homer/public@master] Point DHCP server in eqiad to install1004

https://gerrit.wikimedia.org/r/886888

gerritbot added a project: Patch-For-Review.Feb 6 2023, 12:57 PM

Change 886889 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Move webproxy in eqiad to install1004

https://gerrit.wikimedia.org/r/886889

gerritbot added a comment.Feb 7 2023, 11:04 AM

Change 887302 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Assign installserver role to install6002

https://gerrit.wikimedia.org/r/887302

gerritbot added a comment.Feb 7 2023, 11:07 AM

Change 887302 merged by Muehlenhoff:

[operations/puppet@production] Assign installserver role to install6002

https://gerrit.wikimedia.org/r/887302

gerritbot added a comment.Feb 7 2023, 11:17 AM

Change 887304 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Point webproxy in drmrs to install6002

https://gerrit.wikimedia.org/r/887304

gerritbot added a comment.Feb 7 2023, 11:19 AM

Change 887305 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Update tftp server settings for drmrs

https://gerrit.wikimedia.org/r/887305

gerritbot added a comment.Feb 7 2023, 11:30 AM

Change 887304 merged by Muehlenhoff:

[operations/dns@master] Point webproxy in drmrs to install6002

https://gerrit.wikimedia.org/r/887304

gerritbot added a comment.Feb 7 2023, 12:04 PM

Change 887305 merged by Muehlenhoff:

[operations/puppet@production] Update tftp server settings for drmrs

https://gerrit.wikimedia.org/r/887305

gerritbot added a comment.Feb 7 2023, 12:43 PM

Change 887310 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Add testvm6001

https://gerrit.wikimedia.org/r/887310

gerritbot added a comment.Feb 7 2023, 12:58 PM

Change 887310 merged by Muehlenhoff:

[operations/puppet@production] Add testvm6001

https://gerrit.wikimedia.org/r/887310

gerritbot added a comment.Feb 7 2023, 1:02 PM

Change 887311 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Drop installserver role from install6001

https://gerrit.wikimedia.org/r/887311

gerritbot added a comment.Feb 7 2023, 1:10 PM

Change 887311 merged by Muehlenhoff:

[operations/puppet@production] Drop installserver role from install6001

https://gerrit.wikimedia.org/r/887311

MoritzMuehlenhoff updated the task description. (Show Details)Feb 7 2023, 3:14 PM
Stashbot added a comment.Feb 8 2023, 2:29 PM

Mentioned in SAL (#wikimedia-operations) [2023-02-08T14:29:37Z] <moritzm> test install of testvm6001 T327867

gerritbot added a comment.Feb 8 2023, 2:57 PM

Change 887790 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Assign installserver role to install1004

https://gerrit.wikimedia.org/r/887790

gerritbot added a comment.Feb 8 2023, 2:59 PM

Change 887790 merged by Muehlenhoff:

[operations/puppet@production] Assign installserver role to install1004

https://gerrit.wikimedia.org/r/887790

gerritbot added a comment.Feb 8 2023, 3:27 PM

Change 886889 merged by Muehlenhoff:

[operations/dns@master] Move webproxy in eqiad to install1004

https://gerrit.wikimedia.org/r/886889

gerritbot added a comment.Feb 8 2023, 3:42 PM

Change 886888 merged by Muehlenhoff:

[operations/homer/public@master] Point DHCP server in eqiad to install1004

https://gerrit.wikimedia.org/r/886888

gerritbot added a comment.Feb 8 2023, 3:59 PM

Change 887796 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Update next-server DHCP settings towards install1004

https://gerrit.wikimedia.org/r/887796

gerritbot added a comment.Feb 8 2023, 4:04 PM

Change 887796 merged by Muehlenhoff:

[operations/puppet@production] Update next-server DHCP settings towards install1004

https://gerrit.wikimedia.org/r/887796

gerritbot added a comment.Feb 8 2023, 4:10 PM

Change 887798 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Update cloud proxies

https://gerrit.wikimedia.org/r/887798

gerritbot added a comment.Feb 9 2023, 9:05 AM

Change 887798 merged by Muehlenhoff:

[operations/puppet@production] Update cloud proxies

https://gerrit.wikimedia.org/r/887798

gerritbot added a comment.Feb 9 2023, 10:12 AM

Change 887941 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove installserver role from install1003

https://gerrit.wikimedia.org/r/887941

gerritbot added a comment.Feb 9 2023, 10:21 AM

Change 887941 merged by Muehlenhoff:

[operations/puppet@production] Remove installserver role from install1003

https://gerrit.wikimedia.org/r/887941

Maintenance_bot removed a project: Patch-For-Review.Feb 9 2023, 10:30 AM
MoritzMuehlenhoff updated the task description. (Show Details)Feb 9 2023, 11:28 AM

All install servers are running Bullseye now, the only missing bit is to remove the old VMs.

ops-monitoring-bot added a comment.Feb 10 2023, 9:56 AM

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: install6001.wikimedia.org

  • install6001.wikimedia.org (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster drmrs01 to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster drmrs01 to Netbox
ops-monitoring-bot added a comment.Feb 10 2023, 10:07 AM

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: install5001.wikimedia.org

  • install5001.wikimedia.org (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster eqsin to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster eqsin to Netbox
ops-monitoring-bot added a comment.Feb 10 2023, 10:21 AM

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: install4001.wikimedia.org

  • install4001.wikimedia.org (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster ulsfo to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster ulsfo to Netbox
gerritbot added a comment.Feb 10 2023, 10:22 AM

Change 888192 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove Puppet references for install[3456]001

https://gerrit.wikimedia.org/r/888192

gerritbot added a project: Patch-For-Review.Feb 10 2023, 10:22 AM
ops-monitoring-bot added a comment.Feb 10 2023, 10:33 AM

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: install3001.wikimedia.org

  • install3001.wikimedia.org (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster esams to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster esams to Netbox
MoritzMuehlenhoff updated the task description. (Show Details)Feb 10 2023, 10:34 AM
MoritzMuehlenhoff updated the task description. (Show Details)
gerritbot added a comment.Feb 10 2023, 10:35 AM

Change 888192 merged by Muehlenhoff:

[operations/puppet@production] Remove Puppet references for install[3456]001

https://gerrit.wikimedia.org/r/888192

gerritbot added a comment.Feb 10 2023, 10:38 AM

Change 888196 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[labs/private@master] Remove obsolete stub keytabs

https://gerrit.wikimedia.org/r/888196

gerritbot added a comment.Feb 10 2023, 10:45 AM

Change 888196 merged by Muehlenhoff:

[labs/private@master] Remove obsolete stub keytabs

https://gerrit.wikimedia.org/r/888196

Muehlenhoff mentioned this in rLPRI40624495175e: Remove obsolete stub keytabs.Feb 10 2023, 10:46 AM
ops-monitoring-bot added a comment.Feb 13 2023, 11:26 AM

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: install2003.wikimedia.org

  • install2003.wikimedia.org (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster codfw to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster codfw to Netbox
gerritbot added a comment.Feb 13 2023, 11:29 AM

Change 888656 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove Puppet references to install[12]003

https://gerrit.wikimedia.org/r/888656

gerritbot added a comment.Feb 13 2023, 11:30 AM

Change 888656 merged by Muehlenhoff:

[operations/puppet@production] Remove Puppet references to install[12]003

https://gerrit.wikimedia.org/r/888656

Maintenance_bot removed a project: Patch-For-Review.Feb 13 2023, 11:32 AM
ops-monitoring-bot added a comment.Feb 13 2023, 11:41 AM

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: install1003.wikimedia.org

  • install1003.wikimedia.org (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster eqiad to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster eqiad to Netbox
MoritzMuehlenhoff closed this task as Resolved.Feb 13 2023, 11:41 AM
MoritzMuehlenhoff claimed this task.
MoritzMuehlenhoff updated the task description. (Show Details)

All install servers are now on Bullseye.

ops-monitoring-bot added a comment.Feb 14 2023, 8:50 AM

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: testvm6001.drmrs.wmnet

  • testvm6001.drmrs.wmnet (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster drmrs01 to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster drmrs01 to Netbox
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits