Page MenuHomePhabricator

Users management on SONiC
Closed, ResolvedPublic

Description

Problems

Junos

Junos supports creating users with no passwords but with SSH key only. This is our current way of interacting with network devices, both for human/cli access and for automation access (Netconf operates over SSH).

This is challenged with the more or less future introduction of TLS based management and monitoring as it requires username and password.

SONiC

SSH keys

while SoNiC's configuration API accepts an SSH key, it doesn't do anything with it.
The only known way to setup ssh-key authentication is like on any Linux system: add the public key to ~/.ssh/authorized_keys this works for both the admin and operator roles.
Note that the user directory (/home/user/) is only created at the first SSH access.
As data point,ssh-copy-id only works for users with admin roles as operator are sent to the more restrictive sonic-cli shell. For those an admin user need to create the relevant files.

SSH access is mostly useful for human/cli access (manual configuration/troubleshooting). However, as the system is Debian based, automation access over SSH would be useful to performs actions not supported by the API (SSH keys management, TLS certs, troubleshooting, etc)

Passwords

Note that the sonic-cli is a wrapper to the API.
Password change need to be done through the API. Changing the user's password through the passwd Linux tool doesn't update the API password.
Setting an empty password when creating a user (using the API) disable password authentication for the user (ssh or API).
When querying a user (or all users)' config through the API, only the role is returned, it's thus not possible to compare other data (like password hash or ssh key) to our source of truth.

Roles

SoNiC only supports admin (full read/write) and operator (mostly read) roles.

Management side

To add to the above constraints, we have to consider how the management side will keep the password safe.
On SONiC, mTLS "must contain the username in the common name (CN) field". On Junos username/password is still required.
When supported, JWT usually have a short-ish expiration time and require a user/pass to be generated.
Spicerack's passwords/keys can currently be defined in 4 different ways:

  • keyholder, for ssh keys: safest option
  • configuration file (readable by root only)
  • Env variable: cumbersome to define at each new sessions, prevents automated actions, risk of user setting it in their .bashrc file
  • User prompt: cumbersome to type at each action, prevents automated actions, risk of user leaking it in their bash history

Options

On the key management side

  • Would it be possible to use something similar to ssh-agent's keyholder, but tailored for passwords?
  • Store the "homer" password in a configuration file

On the SONiC configuration side

  • Move the list of users from homer-public to homer-private (until they're in the IDM) and add the hashed passwords to the users that need it (until T335870: Store network users in Bitu/LDAP is done)
  • Have a cookbook that manage the users and their SSH keys
    • This is not optimal as the cookbook will need to iterate over all devices and all users' dir to ensure the configured ssh key is the correct one. Also difficult to integrate this behavior in Homer
    • Bootstraping could be done with the admin user/password, which will then disable SSH password auth once the users are initially created (by modifying the ssh config file)
    • As the API doesn't expose the user's password (or hash) the cookbook (or automation) will require to either submit the defined password regularly (to prevent discrepancies) or grep the /etc/shadow file.
    • All of the above makes configuring users a multi-tool and multi steps process, increasing complexity and reducing reliability
  • A totally different approach, would be to have a daemon running on the device, ensuring that configuration is up to date, either Puppet or a custom script
    • This would need to be fully re-installed at each OS upgrade and brings questions like how to monitor it
  • Ditch ssh-key auth for SONiC (not ideal)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ayounsi updated the task description. (Show Details)

Change 926491 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/cookbooks@master] WIP: Cookbook to manage network users over gNMI

https://gerrit.wikimedia.org/r/926491

It would appear to me that given that we have the full freedom of a Debian-based OS here (after all next to the FLOSS aspect the major perk of SONiC), adding a small management service running on SONiC seems like a nice solution? E.g. a systemd timer which starts a script every 30 minutes which pulls the user list and deploys SSH keys? Is there some documentation to which extent customisation is possible and how a SONiC installation is updated/deployed, so that we could assess whether that's technically possible?

Yes, that would be possible even though there is no documented way on how to do this and what is supported or not.
The two main options I see is either via a docker image, or the more traditional "linux/debian" way. So even though both ways are technically possible deploying such script would require some kind of deployment mechanism, either Puppet (which is a can of worm if its own) or a custom cookbook, or something else.

The previously stated:

Password change need to be done through the API. Changing the user's password through the passwd Linux tool doesn't update the API password.

Is not correct.

So it opens the possibility of fully managing users with a side tool. Like we manage regular linux users.

Change 938853 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/cookbooks@master] Add cookbook to manage users SSH keys on SONiC devices

https://gerrit.wikimedia.org/r/938853

Change 938853 merged by jenkins-bot:

[operations/cookbooks@master] Add cookbook to manage users SSH keys on SONiC devices

https://gerrit.wikimedia.org/r/938853

Change 926491 abandoned by Ayounsi:

[operations/cookbooks@master] WIP: Cookbook to manage network users over gNMI

Reason:

If23c01c34127fe3a14c6533a6c4e891e30646832

https://gerrit.wikimedia.org/r/926491

ayounsi claimed this task.

This is done for now, more improvements to come from Dell, tracked in T342673.