Page MenuHomePhabricator

Upgrade Netbox to 4.x
Open, MediumPublic

Description

Logical follow up from T314933: Upgrade Netbox to latest 3.2.

The 5 major releases come with their load of new features, from which I quickly selected the interesting ones:

3.3

  • L2VPN Modeling (#8157)
  • Toggle Custom Field Visibility (#9166)
  • #8511 - Enable custom fields and tags for circuit terminations
  • #8995 - Enable arbitrary ordering of REST API results
  • #6454 - Include contextual help when creating first objects in UI
  • #10039 - Add "assign FHRP group" option to actions dropdown in interfaces list
  • #10447 - Enable reassigning an inventory item from one device to another

3.4

  • New Global Search (#10560)
  • Virtual Device Contexts (#7854)
  • Saved Filters (#9623)
  • JSON/YAML Bulk Imports (#4347)
  • Scheduled Reports & Scripts (#8366)
  • API for Staged Changes (#10851)
  • #10600 - Allow custom object fields to reference a user or group

3.5

  • Customizable Dashboard (#9416)
  • Remote Data Sources (#11558)
  • ASN Ranges (#8550)
  • Provider Accounts (#9047)
  • #10759 - Support Markdown rendering for custom field descriptions

3.6

  • Custom Field Choice Sets (#12988)
  • #8137 - Add a field for designating the out-of-band (OOB) IP address for devices

3.7

  • Event Rules (#14132)
  • Object Protection Rules (#10244)
  • Improved Custom Field Visibility Controls (#13299)
  • VPN Tunnels (#9816)
  • #12135 - Avoid orphaned interfaces by preventing the deletion of interfaces which have children assigned

4

But also their load of breaking changes:

3.3

  • Device position, device type height, and rack unit values are now reported as decimals (e.g. 1.0 or 1.5) to support modeling half-height rack units.
    • Not impactful
  • The nat_outside relation on the IP address model now returns a list of zero or more related IP addresses, rather than a single instance (or None).
    • Not used
  • Several fields on the cable API serializers have been altered or removed to support multiple-object cable terminations: [see table on the link]
  • As with the cable model, several API fields on all objects to which cables can be connected (interfaces, circuit terminations, etc.) have been changed: [see table on the link]
  • The cable path serialization returned by the /paths/ endpoint for pass-through ports has been simplified, and the following fields removed: origin_type, origin, destination_type, destination. (Additionally, is_complete has been added.)
    • Removed fields not used

3.4

  • Device and virtual machine names are no longer case-sensitive. Attempting to create e.g. "device1" and "DEVICE1" within the same site will raise a validation error.
    • Not impactful
  • The asn, noc_contact, admin_contact, and portal_url fields have been removed from the provider model. Please replicate any data remaining in these fields to the ASN and contact models introduced in NetBox v3.1 prior to upgrading.
    • Updated manually
  • The content_type fields on the CustomLink and ExportTemplate models have been renamed to content_types and now support the assignment of multiple content types per object.
    • Not used
  • Within the Python API, the cf property on an object with custom fields now returns deserialized values. For example, a custom field referencing an object will return the object instance rather than its numeric ID. To access the raw serialized values, reference the object's custom_field_data attribute instead.
    • Not used
  • The NetBoxModelCSVForm class has been renamed to NetBoxModelImportForm. Backward compatability with the previous name has been retained for this release, but will be dropped in NetBox v3.5.
    • Not used

3.5

  • The account field has been removed from the provider model. This information is now tracked using the new provider account model. Multiple accounts can be assigned per provider.
    • Will be updated automatically, not used externally
  • A minimum length of 50 characters is now enforced for the SECRET_KEY configuration parameter.
  • The JobResult model has been moved from the extras app to core and renamed to Job. Accordingly, its REST API endpoint has been moved from /api/extras/job-results/ to /api/core/jobs/.
  • The obj_type field on the Job model (previously JobResult) has been renamed to object_type for consistency with other models.
  • The JOBRESULT_RETENTION configuration parameter has been renamed to JOB_RETENTION.
  • The obj context variable is no longer passed when rendering custom links: Use object instead.
    • Updated
  • The REST API schema is now generated using the OpenAPI 3.0 spec
    • Not impactful
  • The URLs for the REST API schema documentation have changed:
    • /api/docs/ is now /api/schema/swagger-ui/
    • /api/redoc/ is now /api/schema/redoc/
    • Not impactful

3.6

  • PostgreSQL 11 is no longer supported (dropped in Django 4.2). NetBox v3.6 requires PostgreSQL 12 or later.
    • We're on 13
  • The device_role field on the Device model has been renamed to role. The device_role field has been temporarily retained on the REST API serializer for devices for backward compatibility, but is read-only.
    • Will need patches
  • The choices array field has been removed from the CustomField model. Any defined choices are automatically migrated to CustomFieldChoiceSets, accessible via the new choice_set field on the CustomField model.
    • TBD if impactful
  • The napalm_driver and napalm_args fields (which were deprecated in v3.5) have been removed from the Platform model.
    • Feature not in use

3.7

  • The following fields have been removed from the Webhook model: content_types, type_create, type_update, type_delete, type_job_start, type_job_end, enabled, and conditions. Webhooks are now tied to events via event rules. Existing webhooks will have event rules created automatically upon upgrade.
  • The ui_visibility field on the custom field model has been replaced with two new fields: ui_visible and ui_editable. Existing values will be migrated automatically upon upgrade.
  • Not impacted
  • The FeatureQuery class for querying content types by model feature has been removed. Plugins should now use the new with_feature() manager method on NetBox's proxy model for ContentType.
  • The ConfigRevision model has been moved from extras to core. Configuration history will be retained throughout the upgrade process.
  • Not impacted
  • The L2VPN and L2VPNTermination models have been moved from the ipam app to the new vpn app. All object data will be retained, however please note that the relevant API endpoints have moved to /api/vpn/.
  • Not impacted
  • The CustomFieldsMixin, SavedFiltersMixin, and TagsMixin classes have moved from the extras.forms.mixins module to netbox.forms.mixins.
  • Not impacted

4.0

  • Support for Python 3.8 and 3.9 has been removed.
  • The format for GraphQL query filters has changed. Please see the GraphQL documentation for details and examples.
  • The deprecated device_role & device_role_id filters for devices have been removed. (Use role and role_id instead.)
  • The obsolete device_role field has been removed from the REST API serializer for devices. (Use role instead.)
  • The legacy reports functionality has been dropped. Reports will be automatically converted to custom scripts on upgrade.
  • The parent and parent_id filters for locations now return only immediate children of the specified location. (Use ancestor and ancestor_id to return all descendants.)
  • The object_type field on the CustomField model has been renamed to related_object_type.
  • The utilities.utils module has been removed and its resources reorganized into separate modules organized by function.
  • The obsolete NullableCharField class has been removed. (Use Django's stock CharField class with null=True instead.)

Next steps:

TBD

Once Netbox-next is upgraded to 3.6:

  • Update cookbooks if needed
  • Update Homer (& wmf-netbox.py)

Once Netbox is upgraded:

  • Upgrade pynetbox to > 6.6 (eg. in Homer)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ayounsi updated the task description. (Show Details)

Change 918353 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Netbox 3.5: JOBRESULT_RETENTION -> JOB_RETENTION

https://gerrit.wikimedia.org/r/918353

Change 918359 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/software/netbox-extras@master] Netbox 3.5: getstats.py

https://gerrit.wikimedia.org/r/918359

Change 918518 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/software/netbox-extras@master] Netbox 3.5: multiple cable terminations and endpoints

https://gerrit.wikimedia.org/r/918518

ayounsi renamed this task from Upgrade Netbox to 3.5.x to Upgrade Netbox to 3.6.x.Aug 7 2023, 12:14 PM
ayounsi updated the task description. (Show Details)
ayounsi updated the task description. (Show Details)
ayounsi renamed this task from Upgrade Netbox to 3.6.x to Upgrade Netbox to 3.7.x.Dec 5 2023, 1:46 PM
ayounsi updated the task description. (Show Details)
ayounsi updated the task description. (Show Details)

Change 980815 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Netbox: remove SECURE_PROXY_SSL_HEADER

https://gerrit.wikimedia.org/r/980815

Change 980815 merged by Ayounsi:

[operations/puppet@production] Netbox: remove SECURE_PROXY_SSL_HEADER

https://gerrit.wikimedia.org/r/980815

https://docs.netbox.dev/en/stable/configuration/miscellaneous/#enforce_global_unique
ENFORCE_GLOBAL_UNIQUE - The default value for this parameter was changed from False to True in NetBox v3.7.

Change 1006001 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Set ENFORCE_GLOBAL_UNIQUE to True

https://gerrit.wikimedia.org/r/1006001

Mentioned in SAL (#wikimedia-operations) [2024-02-27T06:42:04Z] <XioNoX> Netbox: set ENFORCE_GLOBAL_UNIQUE to True - T336275

Change 1006001 merged by Ayounsi:

[operations/puppet@production] Netbox: set ENFORCE_GLOBAL_UNIQUE to True

https://gerrit.wikimedia.org/r/1006001

ayounsi renamed this task from Upgrade Netbox to 3.7.x to Upgrade Netbox to 4.x.Wed, Apr 3, 2:37 PM
ayounsi updated the task description. (Show Details)