Currently scheduled for Tue Sept 10th 2024 16:00 UTC
As part of the scheduled refresh of switch equipment in codfw rows C and D we need to move the network connections for servers in racks C4 and C5 from the old to new switch.
Hosts in this rack are managed by the following teams:
Collaboration Services Core Platform Data Persistence Data Platform Infrastructure Foundations Machine Learning Observability Search Platform ServiceOps Traffic
A full list of the specific hosts can be found below. We will use the sheet to plan the moves and co-ordinate with other SRE teams on actions required to ensure things go smoothly:
https://docs.google.com/spreadsheets/d/16xoZuDeC_-o6s70uEMnvdgn4BlT1f8__WPYprRuduIA#gid=1455278591
Server links will be moved one-by-one from old to the new switch. So no two hosts will be offline at once.
Based on previous experience each host is likely to only lose comms for ~10 seconds. It is inevitable that a small number of the new cables do not work, however, or there is some minor glitch in the move. So it is possible in an edge case that a host will be offline for 2-3 minutes. On previous occasions this happened with about 1 out of 20 hosts.