In the investigation on T119631, @RobH opened a ticket to investigate temperature issues. During UnitedLayers investigation, it was discovered that one of the NICs in each cabinet's PDUs is malfunctioning. Email from UL Support quoted below:
> It has come to our attention that your network management card for your
> (b-side)PDU in cabinet 1.23 and (a-side) PDU in cabinet 1.22 is
> malfunctioning. We are not able to remotely access and monitor your PDU
> because of this issue. We would like to work with you to fix the issue sooner
> than later so we do not run into any problems.
>
> There may be some downtime with a few of your servers associated with this
> work so we would like to schedule a maintenance window to your convenience so
> that we can go ahead and swap out the bad PDU. If we determine that downtime
> is needed, we can schedule this maintenance window during off hours so the
> impact to your business continuity is at a minimum.
>
> Please let me know if you have any questions or concerns. If you would like to
> discuss this over the phone please let me know and I can accommodate this for
> you.
We will need to coordinate a downtime window. All the systems (except the mgmt switches and mr1-ulsfo) have redundant power supplies, and should remain online during this PDU replacement.