Page MenuHomePhabricator

Make more extensive use of Netbox custom fields
Open, Stalled, LowPublic

Description

Placeholder/tracking task to move data out of homer's config/*.yaml and unstructured Netbox fields into custom fields once Netbox is upgraded to the latest version.

For example:

  • "no-mon" text in interface description
  • urpf_strict in Homer's config
  • filters in Homer's config
  • Interface damping in Homer's config
  • extend the purchase_date and ticket to the inventory items (currently as free description text)

Event Timeline

ayounsi changed the task status from Open to Stalled.Mar 31 2022, 8:29 AM
ayounsi triaged this task as Low priority.
ayounsi created this task.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

One other thing we might want here is a var for IPv6 RAs. I added this optional var definition for devices.yaml so the definition could be added for the fxp interfaces on drmrs CRs:

https://gerrit.wikimedia.org/r/plugins/gitiles/operations/homer/public/+/refs/heads/master/config/devices.yaml#349

Or otherwise get rid of this piece of config if we can work a better way, I went ahead and added for now to progress the task.

One other thing we might want here is a var for IPv6 RAs

After some analysis we concluded the existing logic for identifying which interfaces need to send RAs in our Jinja templates is sufficient, so we won't need this. The changes referenced in the above patch have been removed so we are not defining this in YAML any more.

Two other things which we might want to add custom fields on interfaces:

  • Flag to indicate a "dymaic neighbor" BGP peer should be configured for the interfaces IP subnet(s)
    • i.e. We want such a peer for private vlans but not analytics or public
  • VRF membership (like on cloudsw's)

EDIT: Removed VRF membership as it is a built-in field in newer netbox releases.

Probably a good idea to add an "ASN" custom field on devices so we can document the AS numbers used for eBGP based L3 switches like in drmrs.

extend the purchase_date and ticket to the inventory items (currently as free description text)

Custom fields created, now they need to be migrated over

extend the purchase_date and ticket to the inventory items (currently as free description text)

I ran this on netbox-dev:

for item in InventoryItem.objects.all():
    if not item.description:
        continue
    result = re.search(r"^Purchase:(.*),Task:(T\d{5,}|RT #\d{3,})(.*)$", item.description)
    if not result:
        print(item.id)
    print(item.description)
    item.cf["purchase_date"] = result.group(1)
    item.cf["ticket"] = result.group(2)
    if result.group(3):
        item.description = result.group(3).strip()
    else:
        item.description = ""
    item.save()

@cmooney @Volans, can on of you review the code and/or the result on https://netbox-next.wikimedia.org/dcim/inventory-items/ (add the matching table columns) before I run it on prod?
To me it worked as expected.

@ayounsi some nits in the code/results:

  • https://netbox-next.wikimedia.org/dcim/inventory-items/62/ has a description of ,RMA:2019-08-08,RMA Task:T226422
  • it's the only one, so could also be fixed manually.
  • as for the item.description = result.group(3) line, it could contain just spaces (maybe could be trimmed), not sure if Netbox does already trim it itself.

Apart those nits the results looks good to me.

Yep, that's expected (and the only reason why I put the if result.group(3): block).
There is no clear way on how to document RMA tasks so leaving it in the description field is good enough for now.

  • as for the item.description = result.group(3) line, it could contain just spaces (maybe could be trimmed), not sure if Netbox does already trim it itself.

Thanks, updated the code above

edit: ran a backup of the DB and then the above code in prod, everything went fine.

Change 814742 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/software/netbox-extras@master] Remove test_juniper_inventory_descs

https://gerrit.wikimedia.org/r/814742

Change 814742 merged by jenkins-bot:

[operations/software/netbox-extras@master] Remove test_juniper_inventory_descs

https://gerrit.wikimedia.org/r/814742