Page MenuHomePhabricator

Configuration management for network operations
Closed, ResolvedPublic0 Estimate Story Points

Description

  • Productionize existing configuration management software jnt (potentially renaming it => Homer)
  • Integrate with Netbox for device selection
  • Integrate with Netbox for topology data gathering
  • Add safe push method for the configuration: interactive and sequential
  • [stretch] Evaluate Netbox to store network secrets => NO

Details

Related Gerrit Patches:
operations/puppet : productionHomer: set SSH_AUTH_SOCK
operations/puppet : productionHomer: specify username to connect to devices
operations/software/homer : masterMake transport username configurable
operations/puppet : productionHomer: add Netbox config
operations/puppet : productionhomer: add netbox credentials to the configuration
operations/software/homer/deploy : masterRelease v0.1.0
operations/software/homer : masterUpdated changelog for first release.
operations/software/homer : masternetbox: split generic and device-specific data
operations/software/homer : masterNetbox: expose additional metadata
operations/software/homer : masterdevices: allow to expose arbitrary metadata
operations/software/homer : masternetbox: allow to select the devices from Netbox
operations/software/homer : masterInitial support for Netbox integration
operations/software/homer : masterAdd commit action to the Homer class
operations/puppet : productionhomer: cleanup absented resource
operations/puppet : productionHomer: setup private repo
operations/puppet : productionHomer, remove rsync
operations/puppet : productionHomer: add bash wrapper to make it easy to run it
operations/puppet : productionHomer, add make package for scap deploy
operations/puppet : productionhomer->deploy-homer on deploy server
operations/puppet : productionHomer, use deploy-homer user for deploy + fix files perms
operations/puppet : productionhomer: add missing dependency virtualenv
operations/puppet : productionscap: add dsh group for homer
operations/puppet : productionDeploy homer
labs/private : masterAdd fake SSH keypair for user homer
operations/software/homer : masterCLI: suppress ncclient noisy logger
operations/software/homer : masterconfig: inject role and site to the configuration
operations/software/homer : mastertransports: add JunOS transport
operations/software/homer : masteractions: add generate action
operations/software/homer : mastertemplates: add rendering of templates
operations/software/homer : masterdevices: add logging
operations/software/homer : mastercli: rename action compile to generate
operations/software/homer : masterdevices: add query capability
operations/software/homer : masterConfiguration: load and merge private config
operations/software/homer : masterdoc: add configuration example in documentation
operations/software/homer : mastersetup.py: add missing PyYAML dependency
integration/docroot : masterdoc: add link to Homer documentation
operations/software/homer : masterInitial draft of devices configuration parsing
operations/software/homer : masterInitial draft of the CLI
operations/software/homer : masterInitial structure of the project
integration/config : masterSetup CI for operations/software/homer

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 532225 merged by jenkins-bot:
[operations/software/homer@master] Configuration: load and merge private config

https://gerrit.wikimedia.org/r/532225

Change 532226 merged by jenkins-bot:
[operations/software/homer@master] devices: add query capability

https://gerrit.wikimedia.org/r/532226

Change 532227 merged by jenkins-bot:
[operations/software/homer@master] cli: rename action compile to generate

https://gerrit.wikimedia.org/r/532227

Change 532452 merged by jenkins-bot:
[operations/software/homer@master] devices: add logging

https://gerrit.wikimedia.org/r/532452

Change 532453 merged by jenkins-bot:
[operations/software/homer@master] templates: add rendering of templates

https://gerrit.wikimedia.org/r/532453

Change 532454 merged by jenkins-bot:
[operations/software/homer@master] actions: add generate action

https://gerrit.wikimedia.org/r/532454

Change 533558 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] transports: add JunOS transport

https://gerrit.wikimedia.org/r/533558

Change 533568 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] config: inject role and site to the configuration

https://gerrit.wikimedia.org/r/533568

Change 533570 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] CLI: suppress ncclient noisy logger

https://gerrit.wikimedia.org/r/533570

Change 533558 merged by jenkins-bot:
[operations/software/homer@master] transports: add JunOS transport

https://gerrit.wikimedia.org/r/533558

Change 533568 merged by jenkins-bot:
[operations/software/homer@master] config: inject role and site to the configuration

https://gerrit.wikimedia.org/r/533568

Change 533570 merged by jenkins-bot:
[operations/software/homer@master] CLI: suppress ncclient noisy logger

https://gerrit.wikimedia.org/r/533570

Change 534538 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Deploy homer

https://gerrit.wikimedia.org/r/534538

Change 539169 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[labs/private@master] Add fake SSH keypair for user homer

https://gerrit.wikimedia.org/r/539169

Change 539169 merged by Ayounsi:
[labs/private@master] Add fake SSH keypair for user homer

https://gerrit.wikimedia.org/r/539169

Change 534538 merged by Volans:
[operations/puppet@production] Deploy homer

https://gerrit.wikimedia.org/r/534538

Change 539271 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] scap: add dsh group for homer

https://gerrit.wikimedia.org/r/539271

Change 539271 merged by Volans:
[operations/puppet@production] scap: add dsh group for homer

https://gerrit.wikimedia.org/r/539271

Change 539284 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] homer: add missing dependency virtualenv

https://gerrit.wikimedia.org/r/539284

Change 539284 merged by Volans:
[operations/puppet@production] homer: add missing dependency virtualenv

https://gerrit.wikimedia.org/r/539284

Change 539367 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Homer, use deploy-homer user for deploy + fix files perms

https://gerrit.wikimedia.org/r/539367

Change 539367 merged by Ayounsi:
[operations/puppet@production] Homer, use deploy-homer user for deploy + fix files perms

https://gerrit.wikimedia.org/r/539367

Change 539377 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] homer->deploy-homer on deploy server

https://gerrit.wikimedia.org/r/539377

Change 539377 merged by Ayounsi:
[operations/puppet@production] homer->deploy-homer on deploy server

https://gerrit.wikimedia.org/r/539377

Change 539390 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Homer, add make package for scap deploy

https://gerrit.wikimedia.org/r/539390

Change 539390 merged by Ayounsi:
[operations/puppet@production] Homer, add make package for scap deploy

https://gerrit.wikimedia.org/r/539390

Change 539404 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] Homer: add bash wrapper to make it easy to run it

https://gerrit.wikimedia.org/r/539404

Change 539404 merged by Volans:
[operations/puppet@production] Homer: add bash wrapper to make it easy to run it

https://gerrit.wikimedia.org/r/539404

Change 539411 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Homer, add secondary server for sync

https://gerrit.wikimedia.org/r/539411

Change 539411 merged by Ayounsi:
[operations/puppet@production] Homer, remove rsync

https://gerrit.wikimedia.org/r/539411

Change 539453 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] Homer: setup private repo

https://gerrit.wikimedia.org/r/539453

Volans updated the task description. (Show Details)Sep 27 2019, 6:22 AM

Change 539453 merged by Volans:
[operations/puppet@production] Homer: setup private repo

https://gerrit.wikimedia.org/r/539453

Change 539837 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] homer: cleanup absented resource

https://gerrit.wikimedia.org/r/539837

Change 539837 merged by Volans:
[operations/puppet@production] homer: cleanup absented resource

https://gerrit.wikimedia.org/r/539837

Change 539551 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] Add commit action to the Homer class

https://gerrit.wikimedia.org/r/539551

Change 539551 merged by Ayounsi:
[operations/software/homer@master] Add commit action to the Homer class

https://gerrit.wikimedia.org/r/539551

Change 543114 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] Initial support for Netbox integration

https://gerrit.wikimedia.org/r/543114

Change 543114 merged by jenkins-bot:
[operations/software/homer@master] Initial support for Netbox integration

https://gerrit.wikimedia.org/r/543114

Change 543890 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] netbox: allow to select the devices from Netbox

https://gerrit.wikimedia.org/r/543890

Change 544881 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] homer: add netbox credentials to the configuration

https://gerrit.wikimedia.org/r/544881

Change 543890 merged by jenkins-bot:
[operations/software/homer@master] netbox: allow to select the devices from Netbox

https://gerrit.wikimedia.org/r/543890

Volans updated the task description. (Show Details)Oct 31 2019, 2:34 PM

Basic integration with Netbox has been developed and is now merged, pending the next release. Some improvements are already WIP and should be ready for CR later today.

Change 547638 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] devices: allow to expose arbitrary metadata

https://gerrit.wikimedia.org/r/547638

Change 547639 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] Netbox: expose additional metadata

https://gerrit.wikimedia.org/r/547639

Change 547638 merged by jenkins-bot:
[operations/software/homer@master] devices: allow to expose arbitrary metadata

https://gerrit.wikimedia.org/r/547638

Change 547639 merged by jenkins-bot:
[operations/software/homer@master] Netbox: expose additional metadata

https://gerrit.wikimedia.org/r/547639

Change 556703 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] netbox: split generic and device-specific data

https://gerrit.wikimedia.org/r/556703

Change 556703 merged by jenkins-bot:
[operations/software/homer@master] netbox: split generic and device-specific data

https://gerrit.wikimedia.org/r/556703

Change 558435 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer@master] Updated changelog for first release.

https://gerrit.wikimedia.org/r/558435

Change 558435 merged by jenkins-bot:
[operations/software/homer@master] Updated changelog for first release.

https://gerrit.wikimedia.org/r/558435

Change 558442 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/homer/deploy@master] Release v0.1.0

https://gerrit.wikimedia.org/r/558442

Change 558442 merged by Volans:
[operations/software/homer/deploy@master] Release v0.1.0

https://gerrit.wikimedia.org/r/558442

Mentioned in SAL (#wikimedia-operations) [2019-12-17T10:13:12Z] <volans@deploy1001> Started deploy [homer/deploy@996f7be]: Homer release v0.1.0 - T228388

Mentioned in SAL (#wikimedia-operations) [2019-12-17T10:13:45Z] <volans@deploy1001> Finished deploy [homer/deploy@996f7be]: Homer release v0.1.0 - T228388 (duration: 00m 32s)

Volans updated the task description. (Show Details)Dec 17 2019, 10:17 AM

Change 558456 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Homer: add Netbox config

https://gerrit.wikimedia.org/r/558456

Change 544881 abandoned by Volans:
homer: add netbox credentials to the configuration

Reason:
Superseeded by https://gerrit.wikimedia.org/r/c/operations/puppet/ /558456

https://gerrit.wikimedia.org/r/544881

Change 558456 merged by Ayounsi:
[operations/puppet@production] Homer: add Netbox config

https://gerrit.wikimedia.org/r/558456

Change 558479 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/software/homer@master] Make transport username configurable

https://gerrit.wikimedia.org/r/558479

Change 558479 merged by Ayounsi:
[operations/software/homer@master] Make transport username configurable

https://gerrit.wikimedia.org/r/558479

Change 558497 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Homer: specify username to connect to devices

https://gerrit.wikimedia.org/r/558497

Mentioned in SAL (#wikimedia-operations) [2019-12-17T13:01:20Z] <ayounsi@deploy1001> Started deploy [homer/deploy@359de04]: Homer release v0.1.1 - T228388

Mentioned in SAL (#wikimedia-operations) [2019-12-17T13:01:44Z] <ayounsi@deploy1001> Finished deploy [homer/deploy@359de04]: Homer release v0.1.1 - T228388 (duration: 00m 30s)

Change 558497 merged by Ayounsi:
[operations/puppet@production] Homer: specify username to connect to devices

https://gerrit.wikimedia.org/r/558497

Change 558504 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Homer: set SSH_AUTH_SOCK

https://gerrit.wikimedia.org/r/558504

Change 558504 merged by Ayounsi:
[operations/puppet@production] Homer: set SSH_AUTH_SOCK

https://gerrit.wikimedia.org/r/558504

ayounsi updated the task description. (Show Details)Tue, Feb 4, 10:32 PM

Everything here is done.
Doc is there https://wikitech.wikimedia.org/wiki/Homer and has been tested by other SREs than Riccardo or me.

Future developments will need their own task.

ayounsi closed this task as Resolved.Tue, Feb 4, 10:34 PM