Page MenuHomePhabricator
Create Task
Maniphest T334231

Cabling for Eqiad racks E5-8 and F5-8
Closed, ResolvedPublic
Actions

Description

Creating this task to track the progress for cabling of new Leaf switches in Eqiad to the spine layer using 100GBase-CWDM4 optical links.

I believe some of this may have been done but not yet documented in Netbox (see T292095#8715082). @Jclark-ctr can you clarify what the situation is there?

In terms of the work to add the links in Netbox I can probably take care of it under T333441 once I know what's been done.

Related Objects
Search...

Event Timeline

cmooney triaged this task as Low priority.Apr 6 2023, 3:08 PM
cmooney created this task.
cmooney mentioned this in T292095: Q2:(Need By: TBD) Rows E/F network racking task.
Maintenance_bot added a project: SRE.Apr 6 2023, 3:29 PM
cmooney added a project: ops-eqiad.Apr 6 2023, 3:41 PM
cmooney added a comment.EditedJul 19 2023, 8:38 PM

@Jclark-ctr my apologies for some reason I thought these links had been cabled but seems from T338789 I didn't update the optic type so we need got them ordered. I've updated that, we can use those Dell optics no problem for rack E8/F8.

When they arrive can you cable up the links as follows? We've a link to both spines from every link:

ssw1-e1-eqiad links:

ssw1-e1-eqiadet-0/0/4lsw1-e5-eqiadet-0/0/54
ssw1-e1-eqiadet-0/0/5lsw1-e6-eqiadet-0/0/54
ssw1-e1-eqiadet-0/0/6lsw1-e7-eqiadet-0/0/54
ssw1-e1-eqiadet-0/0/7lsw1-e8-eqiadEthernet72 (Port 54)
ssw1-e1-eqiadet-0/0/12lsw1-f5-eqiadet-0/0/54
ssw1-e1-eqiadet-0/0/13lsw1-f6-eqiadet-0/0/54
ssw1-e1-eqiadet-0/0/14lsw1-f7-eqiadet-0/0/54
ssw1-e1-eqiadet-0/0/15lsw1-f8-eqiadEthernet72 (Port 54)

ssw1-f1-eqiad links:

ssw1-f1-eqiadet-0/0/4lsw1-e5-eqiadet-0/0/55
ssw1-f1-eqiadet-0/0/5lsw1-e6-eqiadet-0/0/55
ssw1-f1-eqiadet-0/0/6lsw1-e7-eqiadet-0/0/55
ssw1-f1-eqiadet-0/0/7lsw1-e8-eqiadEthernet76 (Port 55)
ssw1-f1-eqiadet-0/0/12lsw1-f5-eqiadet-0/0/55
ssw1-f1-eqiadet-0/0/13lsw1-f6-eqiadet-0/0/55
ssw1-f1-eqiadet-0/0/14lsw1-f7-eqiadet-0/0/55
ssw1-f1-eqiadet-0/0/15lsw1-f8-eqiadEthernet76 (Port 55)
Papaul moved this task from Backlog to Blocked on the ops-eqiad board.Aug 28 2023, 10:46 PM
Papaul subscribed.

Blocked on https://phabricator.wikimedia.org/T338789

cmooney added a comment.Sep 6 2023, 8:51 AM

Indeed blocked on the optics arriving, but to clarify the cable runs have been done we just need the optics to slot in and connect.

@Jclark-ctr correct me if I'm wrong there? And while we're on the subject we need to get the cable IDs for those runs corrected in Netbox, currently just have temporary ones:

https://netbox.wikimedia.org/dcim/cables/?q=changeme&site_id=6&color=&length=&length_unit=

ayounsi added a subtask: Unknown Object (Task).Oct 3 2023, 12:03 PM
ayounsi moved this task from Backlog to Next quarter on the netops board.
Jclark-ctr closed subtask Unknown Object (Task) as Resolved.Oct 5 2023, 8:06 PM
ayounsi renamed this task from Cabling for Eqiad racke E5-7 and F5-7 to Cabling for Eqiad racks E5-8 and F5-8.Oct 6 2023, 8:51 AM
ayounsi added a parent task: T335028: Put Dell SONiC switches in production.
cmooney added a comment.Oct 9 2023, 2:17 PM

@Jclark-ctr let me know when you have time to look at this now that the optics have been received.

thanks :)

Jclark-ctr added a comment.Oct 10 2023, 3:35 PM

ssw1-e1-eqiad links:

				                                                                             Cableid

ssw1-e1-eqiad et-0/0/4 lsw1-e5-eqiad et-0/0/54 202308001
ssw1-e1-eqiad et-0/0/5 lsw1-e6-eqiad et-0/0/54 202308002
ssw1-e1-eqiad et-0/0/6 lsw1-e7-eqiad et-0/0/54 202308003
ssw1-e1-eqiad et-0/0/7 lsw1-e8-eqiad Ethernet72 (Port 55) 202308004
ssw1-e1-eqiad et-0/0/12 lsw1-f5-eqiad et-0/0/54 202308005
ssw1-e1-eqiad et-0/0/13 lsw1-f6-eqiad et-0/0/54 202308006
ssw1-e1-eqiad et-0/0/14 lsw1-f7-eqiad et-0/0/54 202308007
ssw1-e1-eqiad et-0/0/15 lsw1-f8-eqiad Ethernet72 (Port 56) 202308008




ssw1-f1-eqiad links:






ssw1-f1-eqiad	et-0/0/4	lsw1-e5-eqiad	et-0/0/55	202308009

ssw1-f1-eqiad	et-0/0/5	lsw1-e6-eqiad	et-0/0/55	202308010

ssw1-f1-eqiad	et-0/0/6	lsw1-e7-eqiad	et-0/0/55	202308011

ssw1-f1-eqiad	et-0/0/7	lsw1-e8-eqiad	Ethernet76 (Port 55)	202308012

ssw1-f1-eqiad	et-0/0/12	lsw1-f5-eqiad	et-0/0/55	202308013

ssw1-f1-eqiad	et-0/0/13	lsw1-f6-eqiad	et-0/0/55	202308014

ssw1-f1-eqiad	et-0/0/14	lsw1-f7-eqiad	et-0/0/55	202308015

ssw1-f1-eqiad	et-0/0/15	lsw1-f8-eqiad	Ethernet76 (Port 56)	202308016
Jclark-ctr added a project: DC-Ops.Oct 10 2023, 3:39 PM
cmooney added a comment.Oct 10 2023, 9:54 PM
Thanks @Jclark-ctr, I can confirm things look good (including light levels and pings I've not added here).



cmooney@ssw1-f1-eqiad> show interfaces descriptions | match lsw 
et-0/0/0        up    up   Core: lsw1-e1-eqiad:et-0/0/55 {#g2207282819001216}
et-0/0/1        up    up   Core: lsw1-e2-eqiad:et-0/0/55 {#G2108191173001616}
et-0/0/2        up    up   Core: lsw1-e3-eqiad:et-0/0/55 {#G2108180567000814}
et-0/0/4        up    up   Core: lsw1-e5-eqiad:et-0/0/55 {#changeme_30040}
et-0/0/5        up    up   Core: lsw1-e6-eqiad:et-0/0/55 {#changeme_30048}
et-0/0/6        up    up   Core: lsw1-e7-eqiad:et-0/0/55 {#changeme_30056}
et-0/0/8        up    up   Core: lsw1-f1-eqiad:et-0/0/55 {#2023020701}
et-0/0/9        up    up   Core: lsw1-f2-eqiad:et-0/0/55 {#G2104265521010789}
et-0/0/10       up    up   Core: lsw1-f3-eqiad:et-0/0/55 {#G2103083181019496}
et-0/0/12       up    up   Core: lsw1-f5-eqiad:et-0/0/55 {#changeme_30072}
et-0/0/13       up    up   Core: lsw1-f6-eqiad:et-0/0/55 {#changeme_30080}
et-0/0/14       up    up   Core: lsw1-f7-eqiad:et-0/0/55 {#changeme_30088}

{master:0}
cmooney@ssw1-f1-eqiad> show ospf interface 
Interface           State   Area            DR ID           BDR ID          Nbrs
et-0/0/0.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/1.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/10.0         PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/12.0         PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/13.0         PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/14.0         PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/2.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/4.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/5.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/6.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/8.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/9.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
lo0.0               DRother 0.0.0.0         0.0.0.0         0.0.0.0            0





cmooney@ssw1-e1-eqiad> show interfaces descriptions | match lsw                                                                     
et-0/0/0        up    up   Core: lsw1-e1-eqiad:et-0/0/54 {#2023020702}
et-0/0/1        up    up   Core: lsw1-e2-eqiad:et-0/0/54 {#G2104265521010750}
et-0/0/2        up    up   Core: lsw1-e3-eqiad:et-0/0/54 {#G2103083181019492}
et-0/0/4        up    up   Core: lsw1-e5-eqiad:et-0/0/54 {#changeme_30039}
et-0/0/5        up    up   Core: lsw1-e6-eqiad:et-0/0/54 {#changeme_30047}
et-0/0/6        up    up   Core: lsw1-e7-eqiad:et-0/0/54 {#changeme_30055}
et-0/0/8        up    up   Core: lsw1-f1-eqiad:et-0/0/54 {#G2207282819001217}
et-0/0/9        up    up   Core: lsw1-f2-eqiad:et-0/0/54 {#G2108191173001103}
et-0/0/10       up    up   Core: lsw1-f3-eqiad:et-0/0/54 {#G2108191173001100}
et-0/0/12       up    up   Core: lsw1-f5-eqiad:et-0/0/54 {#changeme_30071}
et-0/0/13       up    up   Core: lsw1-f6-eqiad:et-0/0/54 {#changeme_30079}
et-0/0/14       up    up   Core: lsw1-f7-eqiad:et-0/0/54 {#changeme_30087}

{master:0}
cmooney@ssw1-e1-eqiad> show ospf interface 
Interface           State   Area            DR ID           BDR ID          Nbrs
et-0/0/0.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/1.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/10.0         PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/12.0         PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/13.0         PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/14.0         PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/2.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/4.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/5.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/6.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/8.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
et-0/0/9.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1
lo0.0               DRother 0.0.0.0         0.0.0.0         0.0.0.0            0



I'll get the cables updated in netbox when I have a moment and close off the task.
cmooney closed this task as Resolved.Oct 11 2023, 11:43 AM
Cable IDs updated now.
ayounsi reopened this task as Open.Oct 18 2023, 8:12 AM
ayounsi reassigned this task from cmooney to Jclark-ctr.
ayounsi moved this task from Blocked to Hardware Failure / Troubleshoot on the ops-eqiad board.
ayounsi subscribed.
I can't get the links to the Dell switches up, only looking at lsw1-e8 for now it seems like there is either no optic or they're not recognized.



lsw1-e8-eqiad# show interface transceiver Ethernet 72
Ethernet72
---------------------------------------------------------------------
Attribute                :  Value/State
---------------------------------------------------------------------
present                  :  NOT-PRESENT
lsw1-e8-eqiad# show interface transceiver Ethernet 76
Ethernet76
---------------------------------------------------------------------
Attribute                :  Value/State
---------------------------------------------------------------------
present                  :  NOT-PRESENT
Jclark-ctr added a comment.Oct 18 2023, 3:45 PM
Unsure if port is turned off or if fs dell optics are not compatible.    I put loopback on optic in dell switch and link did not come up
ayounsi closed this task as Resolved.Oct 23 2023, 8:59 AM
Thanks! I'll open a different task if needed to debug any remaining issue, but looks like it's a Dell bug so far.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits