@Jhancock.wm if you have some time this week or next week can you please check in rack C8 all the servers that have only 1G network card and update the "comment" section of the table in the task description. Thanks

@Papaul there's two more not marked in the comments that do not have 10G cards, but they are being decommed. civi2001 and frpig2001

@Jhancock.wm thank you no worry on civi2001 and frpig2001. So we have a total of 8 servers that are running on 1G and we will have to relocation those 8 servers to be racked using the first 8U space on the bottom of the rack. U1-U8 . if a server is already within that U space range no need to relocate. We will configure the first 12 ports (first group of 3 ) to be 1G (ge-0/0[0-11]) and the other servers will be connected to the switches using 10G nic. Note: ge-0/0/0 is already been used to connect to fmasw-c8 so the first server on U1 will be connect to ge-0/0/1

I wll discuss with @Jgreen and @Dwisehaupt the process to switch the servers with 10G Nic from 1G to 10G connection.

@ayounsi @cmooney I have been working on the migration process and put together the proposal below. I also had a meeting with @Jgreen and @Dwisehaupt to discuss about this proposal so far no objection on the proposal. Please review and let me know if there is anything we need to remove or add. Thanks
We will have 3 phases for this migration.

• Phase 1 ( Tuesday October 8th at 1:00pm CT) COMPLETE

1- Set schedule maintenance window in Librenms and Disable notifications
1- Move link from cr1-codfw to pfw3a and connect it to pfw1a (xe-0/2/0)
2- Move link from cr2-codfw to pfw3b and connect it to pfw1b(xe-7/2/0)
3- Verify links are up and BGP sessions are up
4- Move link from fasw-c8a to pfw3a and connect it to pfw1a (xe-0/2/1)
5- Move link from fasw-c8b to pfw3b and connect it to pfw1b (xe-7/2/1)
6- Verify links are up
7 - Have @Jgreen or @Dwisehaupt check all good on server side

Diagram before

• Phase 2 ( Wednesday October 9th at 1:00pm CT)

1- Set schedule maintenance window in Librenms and Disable notifications
2- Connect the link between fasw2-c8a (xe-0/0/46) and fasw-c8a (xe-0/2/3)
3- Move the link from pfw1a (xe-0/2/1) to fasw-c8a to fasw2-c8a (xe-0/0/47)
4- Move the link from pfw1b(xe-7/2/1) to fasw-c8b to fasw2-c8b(xe-0/0/47)
5- Have @Jgreen or @Dwisehaupt check all good on server side
6- Move the first 1G server (frmx2001) from fasw-c8a/b to fasw2-c8a/b
7 - Have @Jgreen or @Dwisehaupt check all good on server side
8- Servers move: This can be done the same day or next day depending on availability, best start with all 1G servers frist.
For the 1G to 10G migration process, @Jgreen or @Dwisehaupt will change the interface name on each server one at a time, once this is done, dcops will move the the server connection from fasw-c8a/b to fasw2-c8a/b and check interface status
9- Once all the servers moved to the new switches, remove the link from fasw-c8a on xe-0/0/47 to fmasw-c8 and plug it to fasw2-c8a
10- Remove the link between fasw-c8a and fasw2-c8a (xe-0/0/46) and disable the interface

Diagram before

Diagram after

Final Diagram

Phase 3
Clean up: I will open another task for decommission and clean up.

About phase 1. I checked the pfw1 config and steps here. Gave some feedback over IRC. Overall lgtm.

I didn't check phase 2 yet, will do tomorrow.

@Jgreen @Dwisehaupt when do you think you will have time to relocate the 4 servers in the table that have "YES" on the the New U space column? Best do the relocation before we start moving the servers to the new switch.

Thanks

@Jhancock.wm we are going to put civi2001 on the new switch on port 7 since on U6 we have a 2U server so we will just be using port 6 and port 7 will not in use. We can connect it on that port for the time being and when we decom it we can disable back the port.

Phase 2 lgtm, one point though : you need to trunk the management vlan between the old and new switch for fasw to be reachable between steps 3 and 9.

@ayounsi thanks for the feedback

We did phase 2 today, all the 1G nodes are now connected to the new fasw2-c8a/b. We will me moving the 10G nodes next week. Thanks to @Jgreen and @Jhancock.wm

All the servers are now running on the new switches
We did 2 fail over tests today each did last about 5 minutes and had no issues
The first fail over test was to power down fasw2-c8a and and see if traffic was going to switch to fasw2-c8b the second switch
The second fail over test was to power down pfw1a(node0) and see if the traffic was going to switch to pfw1b(node1) the second firewall and if the second firewall was going to take the primary role in the cluster and if when the first firewall comes back online had the secondary role in the cluster.
After the test i rebooted pfw1b so pfw1a can take back the primary role in the cluster.
Thanks again to @Jgreen