Page MenuHomePhabricator
Create Task
Maniphest T426044

Migrate Mediawiki memcached to Debian Trixie
Open, MediumPublic
Actions

Description

Update all memcached servers across both datacentres to Debian Trixie.

How?
We will reimage on of the hosts we are retiring from eqiad (mc1054) to trixie, add it to the pool for a while (2 days, maybe more), and if there is nothing odd, we can proceed with the upgrade.

eqiad

  • mc1055.eqiad.wmnet
  • mc1056.eqiad.wmnet
  • mc1057.eqiad.wmnet
  • mc1058.eqiad.wmnet
  • mc1059.eqiad.wmnet
  • mc1060.eqiad.wmnet
  • mc1061.eqiad.wmnet
  • mc1062.eqiad.wmnet
  • mc1063.eqiad.wmnet
  • mc1064.eqiad.wmnet
  • mc1065.eqiad.wmnet
  • mc1066.eqiad.wmnet
  • mc1067.eqiad.wmnet
  • mc1068.eqiad.wmnet
  • mc1069.eqiad.wmnet
  • mc1070.eqiad.wmnet
  • mc1071.eqiad.wmnet
  • mc1072.eqiad.wmnet
  • mc-wf1001.eqiad.wmnet
  • mc-wf1002.eqiad.wmnet

codfw

  • mc2038.codfw.wmnet
  • mc2039.codfw.wmnet
  • mc2040.codfw.wmnet
  • mc2041.codfw.wmnet
  • mc2042.codfw.wmnet
  • mc2043.codfw.wmnet
  • mc2044.codfw.wmnet
  • mc2045.codfw.wmnet
  • mc2046.codfw.wmnet
  • mc2047.codfw.wmnet
  • mc2048.codfw.wmnet
  • mc2049.codfw.wmnet
  • mc2050.codfw.wmnet
  • mc2051.codfw.wmnet
  • mc2052.codfw.wmnet
  • mc2053.codfw.wmnet
  • mc2054.codfw.wmnet
  • mc2055.codfw.wmnet
  • mc-gp2004.codfw.wmnet
  • mc-gp2005.codfw.wmnet
  • mc-gp2006.codfw.wmnet
  • mc-wf2001.codfw.wmnet
  • mc-wf2002.codfw.wmnet

Notes

  • codfw can wait for Q2 when we will also refresh the memcached cluster
  • @RLazarus we have two mc-wf servers per DC, what should we do in order to not disrupt the application using them?
  • For mc-gp servers needing to be re-IPed, the process will be to
    1. Degrade the gutter pool to 1 server (ie remove the two hosts from profile::mediawiki::mcrouter_wancache::shards
    2. run puppet on deployment servers and re-deploy mw-mcrouter in both DCs
    3. Reimage/re-IP (--move-vlan)
    4. rinse

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
mcrouter_wancache: Bring mc-gp1004 back into use.Blakeoperations/puppetproduction+2 -3
mcrouter_wancache: Swap gutterpool servers under maintenance.Blakeoperations/puppetproduction+12 -11
mcrouter_wancache: Remove 2 gutterpool servers for maintenance.Blakeoperations/puppetproduction+6 -4
mcrouter_wancache: swap mc1054 for mc1055 to enable decomBlakeoperations/puppetproduction+1 -1
mcrouter_wancache: swap mc1055 for mc1054 for trixie testingBlakeoperations/puppetproduction+1 -1
site.pp: re-add mc1054 for trixie testingBlakeoperations/puppetproduction+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
jijiki created subtask T426049: Package prometheus-memcached-exporter for Debian Trixie.May 12 2026, 12:08 PM
jijiki created subtask T426053: Update Wikitech Memcached documentation.May 12 2026, 12:11 PM
Nemoralis renamed this task from MIgrate Mediawiki memcached to Debian Trixie to Migrate Mediawiki memcached to Debian Trixie.May 12 2026, 12:12 PM
jijiki updated the task description. (Show Details)May 12 2026, 1:19 PM
jijiki updated the task description. (Show Details)
JMeybohm triaged this task as Medium priority.May 21 2026, 9:12 AM
JMeybohm moved this task from Inbox to Scheduled (this Q) on the ServiceOps new board.
Blake closed subtask T426047: Package memkeys for Debian Trixie as Resolved.May 21 2026, 9:55 AM
gerritbot added a comment.May 26 2026, 10:41 AM

Change #1293656 had a related patch set uploaded (by Blake; author: Blake):

[operations/puppet@production] site.pp: re-add mc1054 for trixie testing

https://gerrit.wikimedia.org/r/1293656

gerritbot added a project: Patch-For-Review.May 26 2026, 10:41 AM
gerritbot added a comment.May 26 2026, 1:41 PM

Change #1293656 merged by Blake:

[operations/puppet@production] site.pp: re-add mc1054 for trixie testing

https://gerrit.wikimedia.org/r/1293656

ops-monitoring-bot added a comment.May 26 2026, 1:56 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1054.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.May 26 2026, 2:31 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1054.eqiad.wmnet with OS trixie completed:

  • mc1054 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202605261414_blake_759281_mc1054.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
Maintenance_bot removed a project: Patch-For-Review.May 26 2026, 2:31 PM
Blake closed subtask T426049: Package prometheus-memcached-exporter for Debian Trixie as Resolved.May 26 2026, 3:21 PM
Blake subscribed.
gerritbot added a comment.May 27 2026, 8:38 AM

Change #1294216 had a related patch set uploaded (by Blake; author: Blake):

[operations/puppet@production] mcrouter_wancache: swap mc1055 for mc1054 for trixie testing

https://gerrit.wikimedia.org/r/1294216

gerritbot added a project: Patch-For-Review.May 27 2026, 8:38 AM
Blake added a comment.May 27 2026, 8:40 AM

i've verified that mc1054 is running the versions of memkeys and prometheus-memcached-exporter we expect, so i believe we can now add it to the pool.

jijiki updated the task description. (Show Details)May 28 2026, 6:43 AM
gerritbot added a comment.May 28 2026, 10:43 AM

Change #1294216 merged by Blake:

[operations/puppet@production] mcrouter_wancache: swap mc1055 for mc1054 for trixie testing

https://gerrit.wikimedia.org/r/1294216

Blake added a comment.May 28 2026, 11:15 AM

mc1054 has been added to the pool, and things look good (also, memkeys was tested, and that worked!). i'll reimage mc1055 to trixie on monday, and then swap these servers back so we can proceed with the decom of 1054.

Maintenance_bot removed a project: Patch-For-Review.May 28 2026, 11:31 AM
Blake claimed this task.May 29 2026, 11:39 AM
ops-monitoring-bot added a comment.Jun 2 2026, 8:56 AM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1055.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 9:30 AM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1055.eqiad.wmnet with OS trixie completed:

  • mc1055 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606020914_blake_4006665_mc1055.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
gerritbot added a comment.Jun 2 2026, 9:33 AM

Change #1296513 had a related patch set uploaded (by Blake; author: Blake):

[operations/puppet@production] mcrouter_wancache: swap mc1054 for mc1055 to enable decom

https://gerrit.wikimedia.org/r/1296513

gerritbot added a project: Patch-For-Review.Jun 2 2026, 9:33 AM
gerritbot added a comment.Jun 2 2026, 10:45 AM

Change #1296513 merged by Blake:

[operations/puppet@production] mcrouter_wancache: swap mc1054 for mc1055 to enable decom

https://gerrit.wikimedia.org/r/1296513

jijiki updated the task description. (Show Details)Jun 2 2026, 10:52 AM
jijiki added a subscriber: RLazarus.
ops-monitoring-bot added a comment.Jun 2 2026, 11:09 AM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1056.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 11:09 AM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1057.eqiad.wmnet with OS trixie

Maintenance_bot removed a project: Patch-For-Review.Jun 2 2026, 11:31 AM
ops-monitoring-bot added a comment.Jun 2 2026, 11:45 AM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1056.eqiad.wmnet with OS trixie completed:

  • mc1056 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021129_blake_4105790_mc1056.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 11:47 AM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1058.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 11:49 AM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1057.eqiad.wmnet with OS trixie completed:

  • mc1057 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021133_blake_4106022_mc1057.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 11:50 AM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1059.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 12:20 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1058.eqiad.wmnet with OS trixie completed:

  • mc1058 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021204_blake_4137421_mc1058.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 12:20 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1060.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 12:24 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1059.eqiad.wmnet with OS trixie completed:

  • mc1059 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021207_blake_4139522_mc1059.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 12:29 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1061.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 12:43 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1060.eqiad.wmnet with OS trixie executed with errors:

  • mc1060 (FAIL)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Unable to downtime the new host on Icinga/Alertmanager, the sre.hosts.downtime cookbook returned 99
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021235_blake_4168021_mc1060.out
    • The reimage failed, see the cookbook logs for the details. You can also try typing "sudo install-console mc1060.eqiad.wmnet" to get a root shell, but depending on the failure this may not work.
ops-monitoring-bot added a comment.Jun 2 2026, 12:47 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1062.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 12:49 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1061.eqiad.wmnet with OS trixie executed with errors:

  • mc1061 (FAIL)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Unable to downtime the new host on Icinga/Alertmanager, the sre.hosts.downtime cookbook returned 99
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021242_blake_4175315_mc1061.out
    • The reimage failed, see the cookbook logs for the details. You can also try typing "sudo install-console mc1061.eqiad.wmnet" to get a root shell, but depending on the failure this may not work.
ops-monitoring-bot added a comment.Jun 2 2026, 1:06 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1063.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 1:19 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1062.eqiad.wmnet with OS trixie completed:

  • mc1062 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021303_blake_4191962_mc1062.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 1:20 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1064.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 1:40 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1063.eqiad.wmnet with OS trixie completed:

  • mc1063 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021323_blake_15586_mc1063.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 1:43 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1065.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 2:00 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1064.eqiad.wmnet with OS trixie completed:

  • mc1064 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021342_blake_27735_mc1064.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 2:01 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1066.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 2:16 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1065.eqiad.wmnet with OS trixie completed:

  • mc1065 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021400_blake_47007_mc1065.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 2:17 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1067.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 2:36 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1066.eqiad.wmnet with OS trixie completed:

  • mc1066 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021420_blake_69523_mc1066.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 2:37 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1068.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 2:51 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1067.eqiad.wmnet with OS trixie completed:

  • mc1067 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021434_blake_82109_mc1067.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 3:02 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1069.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 3:12 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1068.eqiad.wmnet with OS trixie completed:

  • mc1068 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021454_blake_99001_mc1068.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 3:18 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1070.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 3:36 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1069.eqiad.wmnet with OS trixie completed:

  • mc1069 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021520_blake_119509_mc1069.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 3:37 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1071.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 3:51 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1070.eqiad.wmnet with OS trixie completed:

  • mc1070 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021535_blake_132357_mc1070.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 3:53 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc1072.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 2 2026, 4:10 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1071.eqiad.wmnet with OS trixie completed:

  • mc1071 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021554_blake_148381_mc1071.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 2 2026, 4:30 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc1072.eqiad.wmnet with OS trixie completed:

  • mc1072 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced UEFI HTTP Boot for next reboot
    • Host rebooted via Redfish
    • Host up (Debian installer)
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606021614_blake_162361_mc1072.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
Blake updated the task description. (Show Details)Jun 2 2026, 4:32 PM

All of the memcached hosts in the main pool in eqiad have been reimaged to Trixie.

RLazarus added a comment.Jun 4 2026, 4:32 PM

Word from the Abstract Wikipedia folks is that you should go ahead and reimage the mc-wf hosts without any prework -- just, one at a time please.

ops-monitoring-bot added a comment.Jun 9 2026, 2:33 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc-wf2001.codfw.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 9 2026, 3:16 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc-wf2001.codfw.wmnet with OS trixie completed:

  • mc-wf2001 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606091458_blake_2455955_mc-wf2001.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 9 2026, 3:27 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc-wf2002.codfw.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 9 2026, 4:07 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc-wf2002.codfw.wmnet with OS trixie completed:

  • mc-wf2002 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606091549_blake_2475963_mc-wf2002.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 9 2026, 4:12 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc-wf1001.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Jun 9 2026, 4:47 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc-wf1001.eqiad.wmnet with OS trixie completed:

  • mc-wf1001 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606091630_blake_2508357_mc-wf1001.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Jun 9 2026, 4:50 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc-wf1002.eqiad.wmnet with OS trixie

Blake updated the task description. (Show Details)Jun 9 2026, 5:25 PM
ops-monitoring-bot added a comment.Jun 9 2026, 5:32 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc-wf1002.eqiad.wmnet with OS trixie completed:

  • mc-wf1002 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606091714_blake_2523467_mc-wf1002.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
jijiki updated the task description. (Show Details)Jun 11 2026, 12:00 PM
gerritbot added a comment.Wed, Jun 17, 8:59 AM

Change #1303350 had a related patch set uploaded (by Blake; author: Blake):

[operations/puppet@production] mcrouter_wancache: Remove 2 gutterpool servers for maintenance.

https://gerrit.wikimedia.org/r/1303350

gerritbot added a project: Patch-For-Review.Wed, Jun 17, 8:59 AM
gerritbot added a comment.Wed, Jun 17, 12:22 PM

Change #1303350 merged by Blake:

[operations/puppet@production] mcrouter_wancache: Remove 2 gutterpool servers for maintenance.

https://gerrit.wikimedia.org/r/1303350

Maintenance_bot removed a project: Patch-For-Review.Wed, Jun 17, 12:31 PM
ops-monitoring-bot added a comment.Wed, Jun 17, 12:40 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc-gp1005.eqiad.wmnet with OS trixie

ops-monitoring-bot added a comment.Wed, Jun 17, 12:41 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc-gp1006.eqiad.wmnet with OS trixie

gerritbot added a comment.Wed, Jun 17, 1:22 PM

Change #1303419 had a related patch set uploaded (by Blake; author: Blake):

[operations/puppet@production] mcrouter_wancache: Swap gutterpool servers under maintenance.

https://gerrit.wikimedia.org/r/1303419

gerritbot added a project: Patch-For-Review.Wed, Jun 17, 1:22 PM
ops-monitoring-bot added a comment.Wed, Jun 17, 1:28 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc-gp1005.eqiad.wmnet with OS trixie completed:

  • mc-gp1005 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Host successfully migrated to the new VLAN
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606171310_blake_463902_mc-gp1005.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
ops-monitoring-bot added a comment.Wed, Jun 17, 1:32 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc-gp1006.eqiad.wmnet with OS trixie completed:

  • mc-gp1006 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Host successfully migrated to the new VLAN
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606171314_blake_464088_mc-gp1006.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
gerritbot added a comment.Wed, Jun 17, 2:09 PM

Change #1303419 merged by Blake:

[operations/puppet@production] mcrouter_wancache: Swap gutterpool servers under maintenance.

https://gerrit.wikimedia.org/r/1303419

ops-monitoring-bot added a comment.Wed, Jun 17, 2:19 PM

Cookbook cookbooks.sre.hosts.reimage was started by blake@cumin1003 for host mc-gp1004.eqiad.wmnet with OS trixie

Maintenance_bot removed a project: Patch-For-Review.Wed, Jun 17, 2:32 PM
ops-monitoring-bot added a comment.Wed, Jun 17, 3:03 PM

Cookbook cookbooks.sre.hosts.reimage started by blake@cumin1003 for host mc-gp1004.eqiad.wmnet with OS trixie completed:

  • mc-gp1004 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present and deleted any certificates
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Checked BIOS boot parameters are back to normal
    • Host up (new fresh trixie OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202606171444_blake_482916_mc-gp1004.out
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
gerritbot added a comment.Wed, Jun 17, 3:22 PM

Change #1303469 had a related patch set uploaded (by Blake; author: Blake):

[operations/puppet@production] mcrouter_wancache: Bring mc-gp1004 back into use.

https://gerrit.wikimedia.org/r/1303469

gerritbot added a project: Patch-For-Review.Wed, Jun 17, 3:22 PM
gerritbot added a comment.Thu, Jun 18, 9:19 AM

Change #1303469 merged by Blake:

[operations/puppet@production] mcrouter_wancache: Bring mc-gp1004 back into use.

https://gerrit.wikimedia.org/r/1303469

Maintenance_bot removed a project: Patch-For-Review.Thu, Jun 18, 9:31 AM
Blake updated the task description. (Show Details)Thu, Jun 18, 9:33 AM
jijiki mentioned this in T421711: ServiceOps: Re-IP eqiad private baremetal hosts to new per-rack vlans/subnets.Thu, Jun 18, 11:59 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits