Page MenuHomePhabricator

Users can't connect to login.toolforge.org: 'too many logins'
Closed, ResolvedPublic

Description

From the client side:

Too many logins for 'andrew'.
Last login: Sun Oct  9 17:21:50 2022 from 184.100.227.30
Connection to login.toolforge.org closed.

From auth.log:

Oct  9 17:22:36 tools-sgebastion-10 sshd[12996]: pam_unix(sshd:session): session opened for user sd by (uid=0)
Oct  9 17:22:36 tools-sgebastion-10 systemd-logind[1047]: New session 135353 of user sd.
Oct  9 17:22:36 tools-sgebastion-10 systemd: pam_limits(systemd-user:session): Too many logins (max 100) for sd
Oct  9 17:22:36 tools-sgebastion-10 systemd: pam_limits(systemd-user:session): conversation failed
Oct  9 17:22:36 tools-sgebastion-10 systemd: pam_unix(systemd-user:session): session opened for user sd by (uid=0)
Oct  9 17:22:37 tools-sgebastion-10 sshd[12996]: pam_limits(sshd:session): Too many logins (max 100) for sd
Oct  9 17:22:37 tools-sgebastion-10 sshd[12996]: error: PAM: pam_open_session(): Permission denied

Event Timeline

Andrew triaged this task as Unbreak Now! priority.Oct 9 2022, 5:24 PM

Mentioned in SAL (#wikimedia-cloud) [2022-10-09T17:29:26Z] <taavi> kill 10 idle tmux sessions of user 'hoi' on tools-sgebastion-10 T320352

root@tools-sgebastion-10:~# cat /etc/security/limits.conf
@wikidev        hard    cpu             720
@prometheus     -       nproc           150
*               hard    maxsyslogins    100
*               hard    nice            0
*               -       priority        10
*               hard    nofile          1024
*               soft    nofile          512
*               -       nproc           250

Change 840673 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Toolforge bastions: double max concurrent logins to 200

https://gerrit.wikimedia.org/r/840673

Change 840673 merged by Andrew Bogott:

[operations/puppet@production] Toolforge bastions: double max concurrent logins to 200

https://gerrit.wikimedia.org/r/840673

Andrew claimed this task.