Page MenuHomePhabricator

Agree how to document intra-DC patch panels in Netbox
Open, LowPublic

Description

Requirement

With the current expansion in eqiad to a second cage, and the facility-provided fiber runs between cages, there is an increased requirement for us to document the patch panel usage and connections in Netbox. The new scenario means having devices directly connected to patch panels either side, which is unlike previous requirements where all panels terminated outside circuits.

Netbox Model

Netbox has functionality to model patch panels, basically passive devices in a fiber path, which have "front" and "rear" ports which can be used to document the connections to equipment one side, and equivalent patch-panel connection on the other.

For instance I have created two example patch panels in netbox-next:

https://netbox-next.wikimedia.org/dcim/devices/?role=patch-panel

I've connected the 12 'rear' ports on these back-to-back (these represent the DC-provided runs between the panels in either cage).

We can then connect devices to the 'front' ports of each, to document the device->device connection, but also record where the fibers go and usage of the panels. For instance I've created a new dummy rack (E1) and device (SPINE E1 Dummy), and connected a port on it to cr2-eqiad via the panels:

https://netbox-next.wikimedia.org/dcim/devices/3431/interfaces/

You can then trace the cable and see the runs either side from device to panel:

https://netbox-next.wikimedia.org/dcim/interfaces/20703/trace/

Creating this task to facilitate discussion and gather thoughts on this approach. Seems fine from my point of view, was a major improvement in my last place when Netbox introduced this functionality. But I'd especially like to see what DC-Ops make of the proposal as it probably relates most to them.

Event Timeline

I agree that's the textbook way of doing it and where we need to be in the longer term.

We currently document patch panels information in both the "termination" box of Netbox circuits (best effort), and a X-connect spreadsheet.
They're prone to errors as they're free form and no reports/checks to enforce correctness.

However having the patch panels in Netbox introduces new complexity (all hypothetical bellow):

  • Requires serial/asset tags
  • Automation update
  • Additional steps when patching circuits
  • Reports for consistency/correctness

And would require back-porting all the existing patch panels to not have different ways of doing things.

I suggest we first check if we could replace the X-connect spreadsheet by introducing that new way of doing things, and possibly a report to expose directly useful information. That way this new complexity will be balanced with eliminating a pain point.

In the meantime I lean toward maintaining the status-quo for the eqiad expansion use-case by modeling those links using the well understood circuits feature.
That's me trying to keep Netbox lean, but as I'll not be the main user of the feature DCops opinion is important here as well.

I'd like to remove using the google sheet and support any alternative that moves it wholly to netbox. I think with the circuits patch panel field entry combined with the comments field on the circuits in netbox could be enough to eliminate the gsheet. Currently I'm the only one maintaining it, and it doesn't really scale well to multiple maintainers since it has no real error checking. Having this info in netbox would get more eyes on it, as netbox is interacted with quite often in comparison (to the gsheet of xconnects). We wouldn't have to delete the old google sheet, just depreciate and no longer update it.

Then we could move to fully utilizing the netbox patch panel model from there. On the point of asset tags, I am not exactly certain who owns the patch panels in each of our deployments. We can determine this pretty easily though by emailing our account reps with the question. If we own them, there shouldn't be a problem slapping an asset tag on each of them. If we don't own them, we'll want to perhaps put an asset tag exception in for patch panels and asset tags.

@ayounsi While I agree with your analysis of the difficulties, I wonder if a hybrid approach might be possible.

What we need to model for the eqiad expansion is something we have nowhere else in our infra, i.e. connections between our own equipment which are routed via patch panels. I think it's entirely reasonable to say for this new use-case we will use Netbox, and continue to use existing methods for cross connects going to MMR / WAN circuits etc.

Adding fake "circuits", with us as a provider, and no structured detail on how it corresponds to the actual patch panel, just seems like we are piling on more technical debt.

That said if doing it the "proper" Netbox way messes up automation or reports then maybe that is a reason to hold off. I'm not really aware what problems we might have there. We will have lots of scripting and automation changes to support the new racks either way, so it might not be a whole lot of extra work. But that may be a valid reason to hold off for sure.

I'm easy enough either way. I agree it's probably more something for DC-Ops to decide on, my main reason for creating this task was to demo how it can be done. But long as we've an agreed way to document device->device connections in the coming weeks I'm happy.

That said if doing it the "proper" Netbox way messes up automation or reports then maybe that is a reason to hold off. I'm not really aware what problems we might have there. We will have lots of scripting and automation changes to support the new racks either way, so it might not be a whole lot of extra work. But that may be a valid reason to hold off for sure.

The asset tag checks in the report are straightforward I don't see any problem to adjust them to whatever you decide here, should take few minutes. I'd say don't worry too much about that and pick the solution that best work in the longer term for you all.

My main worry is that we will end up with two ways of documenting and managing X-connects (and overall point to point links). And to me this seems more of an issue than technical debt as we already have all the tooling around the circuit feature.

That's why I'd prefer going fully one way (stick with circuits) or the other (add all patch panels).

Here is an example of having those cross connects as circuits: https://netbox-next.wikimedia.org/circuits/circuits/107/
It's set at type: cross-connect, but could be transport or ICCC as well.
Small side benefit is that It will also allow us to track unused links see https://netbox-next.wikimedia.org/circuits/circuits/?q=&type=cross-connect&provider=equinix&commit_rate=&cf_metric=&cf_state=