Completed auto-reimage of hosts:

['mw2295.codfw.wmnet', 'mw2296.codfw.wmnet']

Script wmf-auto-reimage was launched by legoktm on cumin1001.eqiad.wmnet for hosts:

['mw2295.codfw.wmnet', 'mw2296.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202101270030_legoktm_3434.log.

Icinga downtime set by ryankemper@cumin1001 for 2:00:00 1 host(s) and their services with reason: Enabling envoy for wdqs-internal

wdqs2008.codfw.wmnet

Icinga downtime set by ryankemper@cumin1001 for 2:00:00 1 host(s) and their services with reason: Enabling envoy for wdqs-internal

wdqs2005.codfw.wmnet

Icinga downtime set by ryankemper@cumin1001 for 2:00:00 1 host(s) and their services with reason: Enabling envoy for wdqs-internal

wdqs2006.codfw.wmnet

Icinga downtime set by ryankemper@cumin1001 for 2:00:00 1 host(s) and their services with reason: Enabling envoy for wdqs-internal

wdqs2004.codfw.wmnet

Icinga downtime set by ryankemper@cumin1001 for 2:00:00 1 host(s) and their services with reason: Enabling envoy for wdqs-internal

wdqs1011.eqiad.wmnet

Icinga downtime set by ryankemper@cumin1001 for 2:00:00 1 host(s) and their services with reason: Enabling envoy for wdqs-internal

wdqs1008.eqiad.wmnet

Icinga downtime set by ryankemper@cumin1001 for 2:00:00 1 host(s) and their services with reason: Enabling envoy for wdqs-internal

wdqs1003.eqiad.wmnet

Completed auto-reimage of hosts:

['mw1264.eqiad.wmnet']

Completed auto-reimage of hosts:

['mw2297.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2299.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2298.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2300.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2302.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2297.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262202_dzahn_1083_mw2297_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1264.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262201_dzahn_813_mw1264_eqiad_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2298.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262157_dzahn_32427_mw2298_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2299.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262156_dzahn_32245_mw2299_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2304.codfw.wmnet']

Script wmf-auto-reimage was launched by legoktm on cumin1001.eqiad.wmnet for hosts:

mw2300.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262136_legoktm_30392_mw2300_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2306.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2302.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262129_dzahn_24775_mw2302_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw1337.eqiad.wmnet']

Completed auto-reimage of hosts:

['mw1338.eqiad.wmnet']

cookbooks.sre.hosts.decommission executed by ryankemper@cumin1001 for hosts: relforge1002.eqiad.wmnet

• relforge1002.eqiad.wmnet (PASS)
  • Downtimed host on Icinga
  • Found physical host
  • Downtimed management interface on Icinga
  • Wiped bootloaders
  • Powered off
  • Set Netbox status to Decommissioning and deleted all non-mgmt interfaces and related IPs
  • Removed from DebMonitor
  • Removed from Puppet master and PuppetDB

Completed auto-reimage of hosts:

['mw2308.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2304.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262056_dzahn_14507_mw2304_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2306.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262052_dzahn_14030_mw2306_codfw_wmnet.log.

cookbooks.sre.hosts.decommission executed by ryankemper@cumin1001 for hosts: relforge1001.eqiad.wmnet

• relforge1001.eqiad.wmnet (PASS)
  • Downtimed host on Icinga
  • Found physical host
  • Downtimed management interface on Icinga
  • Wiped bootloaders
  • Powered off
  • Set Netbox status to Decommissioning and deleted all non-mgmt interfaces and related IPs
  • Removed from DebMonitor
  • Removed from Puppet master and PuppetDB

Completed auto-reimage of hosts:

['mw1413.eqiad.wmnet']

Completed auto-reimage of hosts:

['mw1411.eqiad.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2308.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262031_dzahn_6640_mw2308_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1338.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262016_dzahn_4545_mw1338_eqiad_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1337.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262015_dzahn_4477_mw1337_eqiad_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1413.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262009_dzahn_3710_mw1413_eqiad_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1411.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101262005_dzahn_3085_mw1411_eqiad_wmnet.log.

Completed auto-reimage of hosts:

['mw2317.codfw.wmnet']

Script wmf-auto-reimage was launched by legoktm on cumin1001.eqiad.wmnet for hosts:

mw2317.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101261855_legoktm_14699_mw2317_codfw_wmnet.log.

Completed auto-reimage of hosts:

['maps1009.eqiad.wmnet']

Completed auto-reimage of hosts:

['mw1408.eqiad.wmnet']

Completed auto-reimage of hosts:

['mw1409.eqiad.wmnet']

Completed auto-reimage of hosts:

['mw1337.eqiad.wmnet']

Completed auto-reimage of hosts:

['mw2321.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1408.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101261727_dzahn_17856_mw1408_eqiad_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1409.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101261726_dzahn_17778_mw1409_eqiad_wmnet.log.

Completed auto-reimage of hosts:

['mw1408.eqiad.wmnet']

Completed auto-reimage of hosts:

['mw1409.eqiad.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2321.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101261645_dzahn_17234_mw2321_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2321.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2321.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101261645_dzahn_16624_mw2321_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1337.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101261643_dzahn_14848_mw1337_eqiad_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1408.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101261640_dzahn_12287_mw1408_eqiad_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1409.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101261638_dzahn_10409_mw1409_eqiad_wmnet.log.

Script wmf-auto-reimage was launched by hnowlan on cumin1001.eqiad.wmnet for hosts:

['maps1009.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202101261627_hnowlan_28505.log.

Completed auto-reimage of hosts:

['maps1009.eqiad.wmnet']

Script wmf-auto-reimage was launched by hnowlan on cumin1001.eqiad.wmnet for hosts:

['maps1009.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202101261443_hnowlan_19435.log.

cookbooks.sre.hosts.decommission executed by elukey@cumin1001 for hosts: analytics1053.eqiad.wmnet

• analytics1053.eqiad.wmnet (FAIL)
  • Failed downtime host on Icinga (likely already removed)
  • Host steps raised exception: Invalid management FQDN analytics1053.mgmt.eqiad.wmnet for analytics1053.eqiad.wmnet

Completed auto-reimage of hosts:

['an-worker1131.eqiad.wmnet', 'an-worker1119.eqiad.wmnet']

Completed auto-reimage of hosts:

['db1160.eqiad.wmnet']

Script wmf-auto-reimage was launched by elukey on cumin1001.eqiad.wmnet for hosts:

['an-worker1119.eqiad.wmnet', 'an-worker1131.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202101260812_elukey_31698.log.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1160.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202101260805_marostegui_24756.log.

Completed auto-reimage of hosts:

['mw2318.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2319.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2320.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2331.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2318.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252247_dzahn_20078_mw2318_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2319.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252245_dzahn_19861_mw2319_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2331.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252241_dzahn_19428_mw2331_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2321.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2321.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252241_dzahn_19369_mw2321_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2320.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252240_dzahn_19303_mw2320_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2320.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin2001.codfw.wmnet for hosts:

mw2320.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252238_dzahn_31834_mw2320_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2320.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2320.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252238_dzahn_19183_mw2320_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2320.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2320.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252235_dzahn_19041_mw2320_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2321.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2321.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252235_dzahn_18999_mw2321_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw1338.eqiad.wmnet']

Completed auto-reimage of hosts:

['cloudgw2002-dev.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2323.codfw.wmnet']

Script wmf-auto-reimage was launched by pt1979 on cumin2001.codfw.wmnet for hosts:

cloudgw2002-dev.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252052_pt1979_12785_cloudgw2002-dev_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2322.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2326.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1338.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252044_dzahn_23847_mw1338_eqiad_wmnet.log.

Completed auto-reimage of hosts:

['mw1410.eqiad.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2323.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252003_dzahn_12169_mw2323_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2322.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252002_dzahn_11968_mw2322_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2326.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2326.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101252001_dzahn_11846_mw2326_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2326.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101251957_dzahn_11387_mw2326_codfw_wmnet.log.

Completed auto-reimage of hosts:

['mw2326.codfw.wmnet']

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw2326.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101251957_dzahn_11290_mw2326_codfw_wmnet.log.

Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for hosts:

mw1410.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202101251956_dzahn_11179_mw1410_eqiad_wmnet.log.

Completed auto-reimage of hosts:

['mw2324.codfw.wmnet']

Completed auto-reimage of hosts:

['mw2326.codfw.wmnet']

Completed auto-reimage of hosts:

['mw1412.eqiad.wmnet']