Can't "become TOOLNAME" on Toolforge
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Magnus
	Oct 14 2017, 4:57 PM

Description

I ssh in, I get:

-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: Resource temporarily unavailable
magnus@tools-bastion-03:~$ become quickstatements
-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: Resource temporarily unavailable
-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: retry: No child processes
-bash: fork: Resource temporarily unavailable
magnus@tools-bastion-03:~$

Event Timeline

Magnus created this task.Oct 14 2017, 4:57 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 14 2017, 4:57 PM

Magnus triaged this task as Unbreak Now! priority.Oct 14 2017, 4:57 PM

Restricted Application added subscribers: Liuxinyu970226, Jay8g, TerraCodes. · View Herald TranscriptOct 14 2017, 4:57 PM

Which bastion is this? become WORKSFORME on both 02 and 03.

Uh, nevermind I see 03

You exhausted your process limit:

05:15:56 0 ✓ zhuyifei1999@tools-bastion-03: ~$ ulimit -a | grep processes
max user processes              (-u) 30
05:16:33 0 ✓ zhuyifei1999@tools-bastion-03: ~$ ps u -u magnus
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
magnus    3512  0.0  0.0 121532  2416 ?        SN   12:16   0:00 sshd: magnus@notty  
magnus    3513  0.0  0.0  14756  1464 ?        SNs  12:16   0:00 /usr/lib/openssh/sftp-server
magnus    4017  0.1  0.0  80684 11480 ?        SN   Oct13   1:28 mosh-server new -c 256 -s -l LANG=en_GB.UTF-8
magnus    4018  0.0  0.0  44788  6048 pts/57   SNs+ Oct13   0:00 -bash
magnus    4640  0.0  0.0 121540  2000 ?        SN   Oct10   0:01 sshd: magnus@pts/36 
magnus    4641  0.0  0.0  44788  6048 pts/36   SNs  Oct10   0:00 -bash
magnus    4673  0.3  0.0  79084  9508 ?        SN   16:51   0:05 mosh-server new -c 256 -s -l LANG=en_GB.UTF-8
magnus    4674  0.0  0.0  41656  2828 pts/17   SNs+ 16:51   0:00 -bash
magnus   11315  0.3  0.0  84016 14828 ?        SN   11:50   1:07 mosh-server new -c 256 -s -l LANG=en_GB.UTF-8
magnus   11316  0.0  0.0  44788  6052 pts/49   SNs  11:50   0:00 -bash
magnus   12729  0.0  0.0 121980  2504 ?        SN   11:51   0:00 sshd: magnus        
magnus   13170  0.0  0.0 122336  2744 ?        SN   Oct11   0:04 sshd: magnus        
magnus   13684  0.0  0.0  41840  2368 pts/17   SN+  16:59   0:00 -bash
magnus   14716  0.0  0.0 121536  2008 ?        SN   12:33   0:00 sshd: magnus        
magnus   16102  0.0  0.0 121680  2016 ?        SN   Oct11   0:04 sshd: magnus        
magnus   20188  0.0  0.0 121532  2004 ?        SN   13:18   0:00 sshd: magnus        
magnus   22023  0.0  0.0 121812  2340 ?        SN   Oct06   0:18 sshd: magnus@notty  
magnus   22025  0.0  0.0  10696  1220 ?        SNs  Oct06   0:02 /usr/lib/openssh/sftp-server
magnus   22092  0.0  0.0 122008  2504 ?        SN   12:01   0:00 sshd: magnus        
magnus   22886  0.0  0.0 121536  2004 ?        SN   Oct13   0:00 sshd: magnus@pts/11 
magnus   22887  0.0  0.0  44788  6048 pts/11   SNs  Oct13   0:00 -bash
magnus   23708  0.0  0.0 121536  2000 ?        SN   Oct13   0:02 sshd: magnus        
magnus   23787  0.0  0.0 121532  2012 ?        SN   11:33   0:00 sshd: magnus        
magnus   28840  0.0  0.0 121676  2008 ?        SN   Oct13   0:01 sshd: magnus        
magnus   30799  0.0  0.0 121668  2196 ?        SN   Oct10   0:06 sshd: magnus@pts/50 
magnus   30800  0.0  0.0  44788  6052 pts/50   SNs  Oct10   0:00 -bash
magnus   30836  0.0  0.0 121980  2484 ?        SN   Oct09   0:04 sshd: magnus        
magnus   31282  0.0  0.0 122812  3292 ?        SN   Oct09   0:04 sshd: magnus        
magnus   31299  0.0  0.0 121536  2004 ?        SN   Oct09   0:04 sshd: magnus        
magnus   31311  0.0  0.0 121540  2004 ?        SN   Oct09   0:03 sshd: magnus        
05:16:45 0 ✓ zhuyifei1999@tools-bastion-03: ~$ ps u -u magnus | wc -l
31

For self-recovery without root, you may want to try kill some processes via exec kill -TERM <pid>. exec is used here as it won't create new processes, but replace your current process, AFAIK. You will disconnect but you will be should able to start commands normally.

I killed 5 of the oldest open ssh connections:

$ ps uww -u magnus | wc -l
25

Restricted Application added a project: User-bd808. · View Herald TranscriptOct 14 2017, 5:55 PM

Liuxinyu970226 unsubscribed.Oct 15 2017, 3:35 AM

bd808 moved this task from To Do to Done on the User-bd808 board.Jul 15 2020, 9:17 PM

Can't "become TOOLNAME" on ToolforgeClosed, ResolvedPublicActions

Description

Event Timeline

Can't "become TOOLNAME" on Toolforge
Closed, ResolvedPublic
Actions