I propose adding firewall rules to only allow connections from WMF bastion hosts. This would limit their exposure in the event of misconfiguration or OpenSSH bug.

Prior to doing so, everyone who logs in will need to add a ProxyJump line to ~/.ssh/config, e.g.

Host wpr-mobile
    User uibuntu
    Hostname wpr-mobile.wmftest.org
    IdentityFile ~/.ssh/sitespeedio.pem
    ProxyJump primary.bastion.wmflabs.org

(repeat for other hosts as well)

@Peter: Please assign this task back to me once you've done the above, and I'll make the change.