So, our current use case is a bot that regularly parses a page, checks for updates and then pushes the update to GitHub with a pull request, authorizing with a deploy key via SSH (git pull/push, etc.) The alternative would be downgrading the project and dependencies to Python 3.5.3 and we already downgraded to 3.7.3 as the part of our shift to Kubernetes containers (which has been mostly okay) but downgrading to 3.5.3 would require us to use out-of-date dependencies which might lose support the next time we have to upgrade or fix something.
I think as Kubernetes on Toolforge begins to be used for things apart from webservices this will become more useful since ssh is quite mainstream for most use-cases.
Description
Details
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | Dvorapa | T319981 Migrate pywikibot from Toolforge GridEngine to Toolforge Kubernetes | |||
Resolved | • bd808 | T258841 Add ssh-client support to Kubernetes containers |
Event Timeline
@Bstorm Any thoughts on this? Including openssh-client on the base images seems somewhat reasonable to me.
Change 715215 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):
[operations/docker-images/toollabs-images@master] stretch-sssd: Add openssh-client
The other option for GitHub is to use git+https with OAuth instead of git+ssh: https://docs.github.com/en/github/extending-github/git-automation-with-oauth-tokens. You can't limit an OAuth token to a single repository unless you use a machine user, but that's usually a good idea if you're writing automatically anyway.
For T289850 Gerrit also supports read/write git+https (docs). Both token and LDAP auth are enabled (T218750).
The other way to do it would be using git credentials store and storing the credentials in-memory or file. In any case, the ideal way would be to make deploys happen using read-only SSH keys since that has no possibility of leaks and access is always restricted to one repository, that is the primary reason I wanted to use it in the first place. Eventually though, I just wrote the script separately in Python 3.5.3, split off the project into two and then ran it as a Toolforge cronjob.
Thanks, but that's not what I am trying to do. I would like to subscribe to Gerrit stream events which requires an SSH connection. There's also an existing tool that subscribes to these events (Gerrit-newcomer-bot] that will presumably also need this capability in a kubernetes deployment, when it is migrated to use that. uses paramiko, a pure python implementation of SSH. I suppose I can attempt to do the same and use a library for that in PHP; I'm already using phpseclib in my tool which is supposed to do that.
If there is another option besides adding a package to the images (such as PHP libraries), that is preferred. The images are too big as is.
Change 715215 abandoned by Kosta Harlan:
[operations/docker-images/toollabs-images@master] bullseye-sssd: Add openssh-client
Reason:
Change 918610 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):
[operations/docker-images/toollabs-images@master] {bullseye,buster}-sssd: add openssh-client
My measurement of https://gerrit.wikimedia.org/r/918610 applied to the Bullseye base image showed openssh-client adding approximately 5MB to the at rest image size as reported by docker image ls.
Change 918610 merged by jenkins-bot:
[operations/docker-images/toollabs-images@master] {bullseye,buster}-sssd: add openssh-client
Mentioned in SAL (#wikimedia-cloud) [2023-05-15T17:17:44Z] <bd808> Rebuilding bullseye and buster docker containers to pick up openssh-client package addition (T258841)
$ webservice python3.9 shell $ cd /tmp $ git clone git@gitlab.wikimedia.org:toolforge-repos/python-toolforge.git Cloning into 'python-toolforge'... The authenticity of host 'gitlab.wikimedia.org (208.80.154.145)' can't be established. ECDSA key fingerprint is SHA256:V1Q7vVIt/XR2yuue3kUa27/+cSwrCm8wLDfBROh8HMs. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added 'gitlab.wikimedia.org,208.80.154.145' (ECDSA) to the list of known hosts. git@gitlab.wikimedia.org: Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
The ssh git transport will work now. I would not actively suggest that anyone use it, but y'all can evaluate opsec risks of placing ssh private keys in the Toolforge multi-tenant environment for yourselves. 🦶🔫🤕