Page MenuHomePhabricator

Q4/Q1:knams racking elevations & planning
Closed, ResolvedPublic

Description

This task will start as the knams elevation planning task and eventually become the racking task for those devices.

Hostname / Racking / Installation Details

Follow racking and cable mappings listed on the KNAMS Racking Elevation GSheet.

Elevation Considerations

  • Papaul requests to place large MX480s at bottom of rack.
    • Assumes we leave asw and msw at top of rack in elevation doc.
  • Leave room in each rack for a 1:1 replace/refresh of servers in 5 years.
  • Cross-rack connections should add 5 Meters to the run per Interxion, as we have to exit top of the rack, route on the ladder rack, and back down one rack over for all cross-rack connections.
    • All cross-rack connections are noted with (xrack) on the elevation and cable mappings gsheet.

PDU Installation Photos

Please note PDUs arrived without brackets, Interxion support had brackets and used them to mount. We need (2) of the flat silver rj45 cables to link ps1 to ps2 in each rack, as that cable was also missing.

20230413_144226.jpg (4×3 px, 1001 KB)

20230413_144231.jpg (4×3 px, 2 MB)

Related Objects

StatusSubtypeAssignedTask
ResolvedRequestNone
ResolvedRobH

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Note that we will only have one mx480 (current cr3-esams), the other router will be the current cr3-knams (mx204).

Future refresh will replaced that mx480 with a device similar to a mx204. So it might be better to keep the mx204 at the top, as well as space for the mx480 replacement.

Ideally we should also have 1 patch panel per rack.

Note that we will only have one mx480 (current cr3-esams), the other router will be the current cr3-knams (mx204).

Future refresh will replaced that mx480 with a device similar to a mx204. So it might be better to keep the mx204 at the top, as well as space for the mx480 replacement.

Ideally we should also have 1 patch panel per rack.

In reviewing the contract, its "Precabling/Patc h Panels : Fiber – 6 Ports" so it's a bundle and likely has to terminate in the same panel (or its no longer a bundle really.) We can ask I suppose, but I'd err towards 1 patch panel with 6 ports landing in 1 rack.

How long will the expectation of 1 MX204 + 1 MX480 run?

One of my concerns is our other caching sites use matched routers for redundancy and we could always shift one circuit to another router, which is slightly more complicated (or impossible) with MX480/MX204 split.

I hate the idea of a ton of space in one rack bottom for MX480, but then space at the top since then both racks will not be mirrored. If it is 1 MX480 for a short run, I'd almost prefer just putting all the servers racked fromtop down so future refresh can keep things mirrored.

In reviewing the contract, its "Precabling/Patc h Panels : Fiber – 6 Ports" so it's a bundle and likely has to terminate in the same panel (or its no longer a bundle really.) We can ask I suppose, but I'd err towards 1 patch panel with 6 ports landing in 1 rack.

We need at least 12, see T315937#8215121 so ideally 6 in each.

How long will the expectation of 1 MX204 + 1 MX480 run?

Looking at the device with the most recent linecards: https://netbox.wikimedia.org/dcim/devices/1898/inventory/ I'd expect at least 2 years if not more before we have to procure a new one (and justify replacing the whole box).

One of my concerns is our other caching sites use matched routers for redundancy and we could always shift one circuit to another router, which is slightly more complicated (or impossible) with MX480/MX204 split.

Not sure I understand what you mean here

One of my concerns is our other caching sites use matched routers for redundancy and we could always shift one circuit to another router, which is slightly more complicated (or impossible) with MX480/MX204 split.

Not sure I understand what you mean here

I think the comment is related to the two CRs being different models, so port layout etc. being different. Tbh it's always a manual call moving a circuit from one device say, so I think this is not a massive amount of additional complexity. Makes dealing with remote hands a bit harder but I think we should manage?

I hate the idea of a ton of space in one rack bottom for MX480, but then space at the top since then both racks will not be mirrored. If it is 1 MX480 for a short run, I'd almost prefer just putting all the servers racked fromtop down so future refresh can keep things mirrored.

Ideally, longer-term, it would be nice to have both racks fairly symmetric. But I'm not sure what the plan here would be, the MX204 I don't think will need refresh in 2 years time, thus we will replace the MX480 with something else that doesn't match.

Unless we want to replace both at that stage?

Unless we want to replace both at that stage?

Probably not

Ideally, longer-term, it would be nice to have both racks fairly symmetric. But I'm not sure what the plan here would be, the MX204 I don't think will need refresh in 2 years time, thus we will replace the MX480 with something else that doesn't match.

Hard to tell what the landscape will look like, but unless special crisis and seeing how popular the 1U routers are, I'd bet there will be MX204s or MX205s by that time. Worse case, a MX304 (2Us).
cr3-knams is from 2019, so we could maybe refresh it one year sooner and the MX480 1 year later to do both at the same time, or the MX480 2 years later, etc.

@RobH @ayounsi in terms of the CR to ASW connectivity I think this makes sense?

CRCR PortASWASW Port
cr4-knamset-1/0/0asw1-bw27-knamset-0/0/48
cr4-knamset-1/0/1asw1-by27-knamset-0/0/48
cr3-knamset-0/0/0asw1-bw27-knamset-0/0/50
cr3-knamset-0/0/1asw1-by27-knamset-0/0/50

As per discussion on IRC we can re-use the 40GBase-LR4 QSFP+ optics in the MX480 we'll bring over (cr4-knams). We should order an additional 3 (2 for the l MX204 to ASW links, plus 1 spare).

We need 5 single-mode LC-LC patches (including 1 spare) in appropriate lengths to connect the 4 ports listed in my previous comment.

RobH updated the task description. (Show Details)
RobH mentioned this in Unknown Object (Task).May 17 2023, 2:05 PM
RobH mentioned this in Unknown Object (Task).May 17 2023, 2:19 PM

Ideally we should also have 1 patch panel per rack.

In reviewing the contract, its "Precabling/Patc h Panels : Fiber – 6 Ports" so it's a bundle and likely has to terminate in the same panel (or its no longer a bundle really.) We can ask I suppose, but I'd err towards 1 patch panel with 6 ports landing in 1 rack.

I'd missed this part of the discussion. I asked them today if they could split out the fibre panel across 2 racks. Let's see what they say.

I was assuming the positions they gave us were for duplex ports, in other words X1-12 represented 12 fibre pairs. If we've only ordered 6 we'll need to get more, as we've 8 circuits at the moment. Let's wait and see what the response to my mail is.

ayounsi assigned this task to RobH.

I believe this is all done.