Change 1012680 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/software/netbox-extras@master] Add Netbox script to change a server's NIC

https://gerrit.wikimedia.org/r/1012680

For the ML hosts - our K8s clusters don't currently require 10G bandwidth, and at the time we didn't want to "waste" 10G ports if not really needed. But if now it is not a problem anymore, we'd be happy to switch (let us know what is the current best practice regarding 1G vs 10G) :)

Feel free to test it on Netbox next

The steps to follow once this script is deployed :

1. (Optional) Upgrade idrac and NIC firmware: cookbook sre.hardware.upgrade-firmware -n -c idrac -n nic <fqdn>
2. Run the import from the PuppetDB Netbox script
3. Run the netbox "upgrade nic" script (this one)
4. Update E/N/I with the new interface name
5. Run homer to update the switch port
6. Physical recable
7. Reboot the server through console (or just reload the networking service if you're brave)
8. Run the import from the PuppetDB Netbox script a last time

So straightforward to be run by DCops and the service owner together. If there is a huge need, we can look at writing a cookbook to automate those steps more.

For the ML hosts - our K8s clusters don't currently require 10G bandwidth, and at the time we didn't want to "waste" 10G ports if not really needed. But if now it is not a problem anymore, we'd be happy to switch (let us know what is the current best practice regarding 1G vs 10G) :)

All new switches are 1/10/25G compatible, the rule of thumbs is "please have 10G NICs on the servers if budget permits", so far that I'd redirect you to @wiki_willy.

On the technical side, if your server needs 10G in the short term we can look at racking it in a 10G rack. If it can wait for the switch refresh, then that even better, and we can "upgrade" it when possible.

Hi @elukey - do you want me to change the Lift Wing expansion requests for 16x servers in FY24-25 to 10g? Thanks, Willy

@ayounsi thanks for the patch! LGTM.

Unfortunately I think the approach might not suit in a lot of cases, due to the Trident 3 port-block restriction.

Most servers connected on SFP-based 5100s - i.e. a switch that can do 1G or 10G on any port - are already at 10G I think? We have a lot of servers connected to 5120's using 1G SFPs, which are probably prime candidates for upgrading, but the port-block restriction will probably bite us here. If a switch port is currently running at 1G we can't change it to 10G unless the rest of the ports in its block-of-four are unused, or also changed at the same time.

So we probably need to add another field for the port number, so a different (and compatible) port can be selected. If we add a 'switch' field that will auto-populate once the device is selected we could even make that a drop-down, and restrict the options shown to the user to ports which are valid for the selected speed?

I started implementing a fix for that but it quickly gets complex as it means shutting down a port, and fully setting up another one. Before going that way let's see if it's something we want/need to do.
Also as I think you mentioned somewhere else, it would mess with @Papaul 's rack U to switch port mapping.

The validator in https://gerrit.wikimedia.org/r/c/operations/software/netbox-extras/+/985113 might make it more complex as well, as if it were in effect it would block any migration. I'd need to check if the validator triggers if we migrate all 4 ports at once. It's something we will eventually face.

If we go with the "move all 4 ports at once" option, the script could have a checkbox to skip the switch side changes (or speed changes), to be done in bulk.