Page MenuHomePhabricator

Homer: Netbox driven switch interfaces
Open, MediumPublic

Description

Since rOSHObf962ab217cdad3b19b94f47511fd99fed76939c is done, I started to re-write Homer interfaces support.
Now that it doesn't have to be generic anymore, I split into several functions, the main benefits are code re-usability, easier to understand (and review) as well as more future proof (eg. we can change the way switches are managed without touching routers, etc...).

starting with POPs, as they don't change often. Target switches L2 config have been populated using the j2nb script, and interface links have been imported automanually.

Importing links fully automatically from real-life to Netbox could be possible once those are done:

  • Some devices don't report their LLDP data properly to the switches, see T250367
  • Device interface are added to Netbox, see T244153

The feature already surfaced some small miss-configurations and other various cleanups that are now done.

While it does keep some of the current infrastructures standards (such as regrouping access only interfaces under a single interface-range same for the disabled interfaces) for ease of transition and management.
It is not possible to do the same for all interfaces, especially the ones with complex configurations (eg. with more than one vlan).

The changes I'll need to do on the POP switches for now, which can be done without impacting production are:

  • Remove apply-groups access-port and apply its configuration either directly to the access interface-range (which is pretty much mtu and interface-mode access
  • Remove interface-range infrastructure that is only there for the apply-groups access-port
  • Breakout interface-range LVS-balancer into their own per interface configuration stanza
  • Breakout interface-range ganeti into their own per interface configuration stanza (already like that in ulsfo)
  • Move ulsfo's customer-1montgomery into its own interface-range, as it's an access port, to not break standardization
  • Same for the locations where the RIPE atlas proves are not already in their own interface-range vlan-sandbox1....

Pinging @faidon or @mark for the above as it's a change of networking convention.

Once done in the POPs, we could structure eqiad/codfw switches the same way, while not managing them by Homer yet.
We will in parallel need to figure out the workflow for SRE to do changes using Netbox and Homer, which mean:

  • An abstraction layer to edit Netbox interfaces and links on host provisioning/changes/etc.
  • A way for non-root users to deploy interfaces changes (see also T244840)

Last, some reports to make sure Netbox data doesn't diverge from real life data, eg.:

  • Comparing LLDP data to Netbox data
  • Comparing links A/Z configurations
  • etc

Event Timeline

ayounsi triaged this task as Medium priority.Apr 16 2020, 7:07 PM
ayounsi created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 16 2020, 7:07 PM

Change 547584 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/homer/public@master] Netbox driven switch interfaces configuration

https://gerrit.wikimedia.org/r/547584

Change 589406 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/software/homer/deploy@master] WMF sepcific Netbox plugin for interfaces config

https://gerrit.wikimedia.org/r/589406

ayounsi updated the task description. (Show Details)Apr 16 2020, 7:16 PM
ayounsi updated the task description. (Show Details)Apr 16 2020, 7:22 PM
ayounsi updated the task description. (Show Details)Apr 17 2020, 6:03 PM
ayounsi added a subscriber: mark.

Mentioned in SAL (#wikimedia-operations) [2020-06-10T07:36:04Z] <XioNoX> make asw2-ulsfo interfaces Homer like - T250429

Mentioned in SAL (#wikimedia-operations) [2020-06-10T08:50:46Z] <XioNoX> make asw1-eqsin interfaces Homer like - T250429

Mentioned in SAL (#wikimedia-operations) [2020-06-11T06:37:47Z] <XioNoX> make asw2-esams interfaces Homer like - T250429

Change 589406 merged by Volans:
[operations/software/homer/deploy@master] WMF specific Netbox plugin for interfaces config

https://gerrit.wikimedia.org/r/589406

Change 547584 merged by Ayounsi:
[operations/homer/public@master] Netbox driven switch interfaces configuration

https://gerrit.wikimedia.org/r/547584

Change 604746 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Homer: enable wmf-plugin

https://gerrit.wikimedia.org/r/604746

Change 604746 merged by Ayounsi:
[operations/puppet@production] Homer: enable wmf-plugin

https://gerrit.wikimedia.org/r/604746