Page MenuHomePhabricator

Migrate the install servers to Bullseye
Open, MediumPublic

Description

These are still on Buster currently. Replace them with new VMs:

sitenew VM namein service
eqiadinstall1004
codfwinstall2004yes
esamsinstall3002yes
ulsfoinstall4002yes
eqsininstall5002yes
drmrsinstall6002

Details

ProjectBranchLines +/-Subject
operations/homer/publicmaster+1 -1
operations/dnsmaster+1 -1
operations/puppetproduction+2 -2
operations/puppetproduction+2 -2
operations/homer/publicmaster+1 -1
operations/dnsmaster+1 -1
operations/puppetproduction+2 -2
operations/puppetproduction+2 -2
operations/puppetproduction+2 -2
operations/homer/publicmaster+1 -1
operations/dnsmaster+1 -1
operations/puppetproduction+2 -2
operations/puppetproduction+6 -2
operations/puppetproduction+3 -3
operations/homer/publicmaster+1 -1
operations/dnsmaster+1 -1
operations/puppetproduction+5 -1
operations/puppetproduction+5 -1
operations/homer/publicmaster+1 -1
operations/puppetproduction+16 -16
operations/puppetproduction+6 -6
operations/dnsmaster+1 -1
operations/puppetproduction+5 -1
labs/privatemaster+0 -0
operations/puppetproduction+4 -17
operations/puppetproduction+30 -0
Show related patches Customize query in gerrit

Event Timeline

Peachey88 renamed this task from MIgrate the install servers to Bullseye to Migrate the install servers to Bullseye.Wed, Jan 25, 8:59 AM

Change 883581 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Add new install servers

https://gerrit.wikimedia.org/r/883581

Change 883581 merged by Muehlenhoff:

[operations/puppet@production] Add new install servers

https://gerrit.wikimedia.org/r/883581

Change 884310 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove service toggle for TFTP

https://gerrit.wikimedia.org/r/884310

Change 884310 merged by Dzahn:

[operations/puppet@production] Remove service toggle for TFTP

https://gerrit.wikimedia.org/r/884310

Change 885246 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Apply role::installserver to install2004

https://gerrit.wikimedia.org/r/885246

Change 885263 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[labs/private@master] New stub keytabs for the new install servers

https://gerrit.wikimedia.org/r/885263

Change 885263 merged by Muehlenhoff:

[labs/private@master] New stub keytabs for the new install servers

https://gerrit.wikimedia.org/r/885263

Change 885246 merged by Muehlenhoff:

[operations/puppet@production] Apply role::installserver to install2004

https://gerrit.wikimedia.org/r/885246

Change 885285 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Move webproxy.codfw.wmnet to install2004

https://gerrit.wikimedia.org/r/885285

Change 885285 merged by Muehlenhoff:

[operations/dns@master] Move webproxy.codfw.wmnet to install2004

https://gerrit.wikimedia.org/r/885285

Change 885304 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Update cloudbastion rules for install2004

https://gerrit.wikimedia.org/r/885304

Change 885326 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/homer/public@master] Point to install2004 for DHCP in codfw

https://gerrit.wikimedia.org/r/885326

Change 885336 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Move next-server settings from install2003->2004

https://gerrit.wikimedia.org/r/885336

install2004 has had the installserver role assigned and it's now acting the web proxy for codfw. The DHCP server is currrently stopped, tomorrow morning I'll merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/885336 and https://gerrit.wikimedia.org/r/c/operations/homer/public/+/885326 and test a reimage and VM installation.

Change 885304 merged by Muehlenhoff:

[operations/puppet@production] Update cloudbastion rules for install2004

https://gerrit.wikimedia.org/r/885304

Change 885336 merged by Muehlenhoff:

[operations/puppet@production] Move next-server settings from install2003->2004

https://gerrit.wikimedia.org/r/885336

Change 885326 merged by Muehlenhoff:

[operations/homer/public@master] Point to install2004 for DHCP in codfw

https://gerrit.wikimedia.org/r/885326

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: testvm2001.codfw.wmnet

  • testvm2001.codfw.wmnet (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster codfw_test to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster codfw_test to Netbox

cookbooks.sre.hosts.decommission executed by jmm@cumin2002 for hosts: testvm2003.codfw.wmnet

  • testvm2003.codfw.wmnet (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found Ganeti VM
    • VM shutdown
    • Started forced sync of VMs in Ganeti cluster codfw_test to Netbox
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
    • VM removed
    • Started forced sync of VMs in Ganeti cluster codfw_test to Netbox

Change 885799 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove installserver role from install2003

https://gerrit.wikimedia.org/r/885799

Change 885799 merged by Muehlenhoff:

[operations/puppet@production] Remove installserver role from install2003

https://gerrit.wikimedia.org/r/885799

install2004 is fully in service now, I have tested a reimage on baremetal and VM installation successfully. atftpd, dhcpd and nginx have been stopped on install2003, but I'm keeping the VM for a few more days just in case.

MoritzMuehlenhoff updated the task description. (Show Details)

Change 885805 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/homer/public@master] Point DHCP server in esams to install3002

https://gerrit.wikimedia.org/r/885805

Change 885806 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/homer/public@master] Point DHCP server in ulsfo to install4002

https://gerrit.wikimedia.org/r/885806

Change 885982 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Point the webproxy in esams to install3002

https://gerrit.wikimedia.org/r/885982

Change 885983 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Apply installserver role to install3002

https://gerrit.wikimedia.org/r/885983

Change 885983 merged by Muehlenhoff:

[operations/puppet@production] Apply installserver role to install3002

https://gerrit.wikimedia.org/r/885983

Change 885984 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Update DHCP config for esams

https://gerrit.wikimedia.org/r/885984

Change 885982 merged by Muehlenhoff:

[operations/dns@master] Point the webproxy in esams to install3002

https://gerrit.wikimedia.org/r/885982

Change 885805 merged by jenkins-bot:

[operations/homer/public@master] Point DHCP server in esams to install3002

https://gerrit.wikimedia.org/r/885805

Change 885984 merged by Muehlenhoff:

[operations/puppet@production] Update DHCP config for esams

https://gerrit.wikimedia.org/r/885984

Change 885989 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove installserver role from install3001

https://gerrit.wikimedia.org/r/885989

Change 885989 merged by Muehlenhoff:

[operations/puppet@production] Remove installserver role from install3001

https://gerrit.wikimedia.org/r/885989

Change 886031 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Move webproxy in ulsfo to install4002

https://gerrit.wikimedia.org/r/886031

Change 886036 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Setup install4002 as install server

https://gerrit.wikimedia.org/r/886036

Change 886037 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Update DHCP config in ulsfo

https://gerrit.wikimedia.org/r/886037

Change 886036 merged by Muehlenhoff:

[operations/puppet@production] Setup install4002 as install server

https://gerrit.wikimedia.org/r/886036

Change 886031 merged by Muehlenhoff:

[operations/dns@master] Move webproxy in ulsfo to install4002

https://gerrit.wikimedia.org/r/886031

Change 885806 merged by Muehlenhoff:

[operations/homer/public@master] Point DHCP server in ulsfo to install4002

https://gerrit.wikimedia.org/r/885806

Change 886037 merged by Muehlenhoff:

[operations/puppet@production] Update DHCP config in ulsfo

https://gerrit.wikimedia.org/r/886037

Change 886049 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove installserver role from install4001

https://gerrit.wikimedia.org/r/886049

Change 886049 merged by Muehlenhoff:

[operations/puppet@production] Remove installserver role from install4001

https://gerrit.wikimedia.org/r/886049

Change 886053 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/homer/public@master] Point DHCP server in eqsin to install5002

https://gerrit.wikimedia.org/r/886053

Change 886313 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Point proxy in eqsin to install5002

https://gerrit.wikimedia.org/r/886313

Change 886314 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Assign installserver role to install5002

https://gerrit.wikimedia.org/r/886314

Change 886314 merged by Muehlenhoff:

[operations/puppet@production] Assign installserver role to install5002

https://gerrit.wikimedia.org/r/886314

Change 886313 merged by Muehlenhoff:

[operations/dns@master] Point proxy in eqsin to install5002

https://gerrit.wikimedia.org/r/886313

Change 886320 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Point DHCP server for eqsin to install5002

https://gerrit.wikimedia.org/r/886320

Change 886053 merged by Muehlenhoff:

[operations/homer/public@master] Point DHCP server in eqsin to install5002

https://gerrit.wikimedia.org/r/886053

Change 886320 merged by Muehlenhoff:

[operations/puppet@production] Point DHCP server for eqsin to install5002

https://gerrit.wikimedia.org/r/886320

Change 886326 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Remove installserver role from install5001

https://gerrit.wikimedia.org/r/886326

Change 886326 merged by Muehlenhoff:

[operations/puppet@production] Remove installserver role from install5001

https://gerrit.wikimedia.org/r/886326

Change 886888 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/homer/public@master] Point DHCP server in eqiad to install1004

https://gerrit.wikimedia.org/r/886888

Change 886889 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/dns@master] Move webproxy in eqiad to install1004

https://gerrit.wikimedia.org/r/886889