While deploying an emergency backport for T326788 I hit a lot of depool failures in codfw.
112 hosts failed:
01:29:56 php-fpm-restart: 100% (in-flight: 0; ok: 184; fail: 112; left: 0)
Here was the output from the php-fpm-restart phase:
1:21:33 Finished sync_wikiversions (duration: 00m 04s)
01:21:33 Started php-fpm-restarts
01:21:33 Running '/usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807' on 296 host(s)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2397.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,790 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,865 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:39,871 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:44,880 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:49,890 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:54,897 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:59,905 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,911 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:09,919 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:14,928 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:19,936 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2397.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:24,939 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:24,939 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2380.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,821 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,896 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:39,900 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:44,908 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:49,916 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:54,925 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:59,932 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,941 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:09,949 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:14,958 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:19,965 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2380.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:24,969 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:24,969 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2003.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,817 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,894 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:39,903 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:44,911 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:49,920 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:54,929 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:59,938 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,947 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:09,956 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:14,965 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:19,974 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2003.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:24,980 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:24,980 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2418.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,832 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,913 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:39,918 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:44,926 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:49,935 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:54,942 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:59,950 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,958 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:09,967 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:14,977 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:19,982 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2418.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:24,986 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:24,987 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2014.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,836 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,913 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:39,924 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:44,932 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:49,942 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:54,951 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:59,960 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,968 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:09,977 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:14,986 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:19,995 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2014.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:25,000 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:25,000 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2384.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,838 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,913 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:39,924 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:44,931 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:49,939 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:54,945 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:59,954 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,961 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:09,969 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:14,979 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:19,986 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2384.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:24,990 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:24,990 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2306.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,871 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,945 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:39,953 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:44,963 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:49,970 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:54,979 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:59,987 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,995 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:10,005 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:15,011 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:20,019 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2306.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:25,024 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:25,024 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2408.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,869 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,946 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:39,954 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:44,964 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:49,970 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:54,978 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:59,986 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,994 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:10,004 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:15,010 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:20,018 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2408.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:25,022 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:25,022 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2285.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,916 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,997 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:40,004 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:45,012 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:50,020 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:55,028 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:00,035 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:05,044 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:10,053 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:15,060 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:20,069 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2285.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:25,072 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:25,072 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2366.codfw.wmnet) returned [2]: 2023-01-16 01:21:34,919 [INFO] Depooling currently pooled services
2023-01-16 01:21:34,999 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:40,008 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:45,014 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:50,022 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:55,029 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:00,037 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:05,046 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:10,053 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:15,063 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:20,070 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2366.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:25,074 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:25,074 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:29 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2277.codfw.wmnet) returned [2]: 2023-01-16 01:21:39,646 [INFO] Depooling currently pooled services
2023-01-16 01:21:39,728 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:44,737 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:49,745 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:54,824 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:21:59,833 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,841 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:09,849 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:14,858 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:19,867 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:24,878 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2277.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:29,882 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:29,882 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:48 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2295.codfw.wmnet) returned [2]: 2023-01-16 01:21:58,728 [INFO] Depooling currently pooled services
2023-01-16 01:21:58,803 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:03,812 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:08,819 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:13,827 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:18,835 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:23,843 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:28,850 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:33,855 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:38,863 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:43,871 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2295.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:48,874 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:48,874 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:49 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2388.codfw.wmnet) returned [2]: 2023-01-16 01:21:59,223 [INFO] Depooling currently pooled services
2023-01-16 01:21:59,300 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,309 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:09,318 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:14,327 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:19,335 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:24,341 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:29,349 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:34,357 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:39,365 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:44,371 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2388.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:49,376 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:49,377 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:22:49 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2377.codfw.wmnet) returned [2]: 2023-01-16 01:21:59,227 [INFO] Depooling currently pooled services
2023-01-16 01:21:59,301 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:04,311 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:09,320 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:14,329 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:19,337 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:24,345 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:29,354 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:34,362 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:39,371 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:44,378 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2377.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:49,383 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:22:49,383 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:15 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2403.codfw.wmnet) returned [2]: 2023-01-16 01:22:25,158 [INFO] Depooling currently pooled services
2023-01-16 01:22:25,237 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:30,246 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:35,254 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:40,261 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:45,270 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:50,278 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:55,286 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:00,293 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:05,298 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:10,307 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2403.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:15,312 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:15,312 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:15 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2336.codfw.wmnet) returned [2]: 2023-01-16 01:22:25,197 [INFO] Depooling currently pooled services
2023-01-16 01:22:25,278 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:30,284 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:35,293 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:40,299 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:45,307 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:50,316 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:55,323 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:00,331 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:05,340 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:10,347 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2336.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:15,353 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:15,353 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:15 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2286.codfw.wmnet) returned [2]: 2023-01-16 01:22:25,211 [INFO] Depooling currently pooled services
2023-01-16 01:22:25,288 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:30,295 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:35,304 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:40,311 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:45,318 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:50,327 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:55,334 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:00,343 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:05,353 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:10,358 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2286.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:15,363 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:15,363 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:15 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2015.codfw.wmnet) returned [2]: 2023-01-16 01:22:25,204 [INFO] Depooling currently pooled services
2023-01-16 01:22:25,280 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:30,289 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:35,299 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:40,309 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:45,314 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:50,324 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:55,333 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:00,343 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:05,352 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:10,361 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2015.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:15,366 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:15,367 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:15 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2412.codfw.wmnet) returned [2]: 2023-01-16 01:22:25,254 [INFO] Depooling currently pooled services
2023-01-16 01:22:25,336 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:30,341 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:35,349 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:40,357 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:45,365 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:50,373 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:55,381 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:00,389 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:05,397 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:10,405 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2412.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:15,410 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:15,410 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:15 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2354.codfw.wmnet) returned [2]: 2023-01-16 01:22:25,298 [INFO] Depooling currently pooled services
2023-01-16 01:22:25,379 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:30,387 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:35,395 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:40,402 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:45,410 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:50,418 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:55,426 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:00,435 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:05,442 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:10,451 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2354.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:15,455 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:15,455 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:20 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2303.codfw.wmnet) returned [2]: 2023-01-16 01:22:30,090 [INFO] Depooling currently pooled services
2023-01-16 01:22:30,167 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:35,174 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:40,183 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:45,190 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:50,199 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:55,206 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:00,214 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:05,222 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:10,231 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:15,238 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2303.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:20,243 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:20,244 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:39 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2020.codfw.wmnet) returned [2]: 2023-01-16 01:22:48,837 [INFO] Depooling currently pooled services
2023-01-16 01:22:48,912 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:53,921 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:58,931 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:03,941 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:08,951 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:13,960 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:18,969 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:23,976 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:28,985 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:33,994 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2020.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:38,999 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:38,999 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:39 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2376.codfw.wmnet) returned [2]: 2023-01-16 01:22:49,014 [INFO] Depooling currently pooled services
2023-01-16 01:22:49,091 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:54,099 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:59,108 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:04,115 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:09,122 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:14,131 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:19,138 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:24,147 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:29,155 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:34,169 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2376.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:39,171 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:39,171 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:39 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2360.codfw.wmnet) returned [2]: 2023-01-16 01:22:49,101 [INFO] Depooling currently pooled services
2023-01-16 01:22:49,184 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:54,192 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:59,198 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:04,206 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:09,215 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:14,222 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:19,231 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:24,240 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:29,249 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:34,259 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2360.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:39,262 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:39,262 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:39 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2392.codfw.wmnet) returned [2]: 2023-01-16 01:22:49,596 [INFO] Depooling currently pooled services
2023-01-16 01:22:49,674 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:54,812 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:59,819 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:04,826 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:09,834 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:14,842 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:19,850 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:24,859 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:29,866 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:34,875 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2392.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:39,880 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:39,881 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:39 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2338.codfw.wmnet) returned [2]: 2023-01-16 01:22:49,542 [INFO] Depooling currently pooled services
2023-01-16 01:22:49,624 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:54,812 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:59,818 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:04,827 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:09,835 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:14,841 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:19,850 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:24,859 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:29,868 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:34,874 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2338.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:39,879 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:39,879 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:23:44 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2298.codfw.wmnet) returned [2]: 2023-01-16 01:22:54,065 [INFO] Depooling currently pooled services
2023-01-16 01:22:54,140 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:22:59,148 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:04,156 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:09,164 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:14,168 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:19,176 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:24,184 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:29,193 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:34,202 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:39,208 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2298.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:44,213 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:23:44,213 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:04 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2378.codfw.wmnet) returned [2]: 2023-01-16 01:23:13,809 [INFO] Depooling currently pooled services
2023-01-16 01:23:13,888 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:18,893 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:23,900 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:28,909 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:33,917 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:38,925 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:43,932 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:48,940 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:53,949 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:58,958 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2378.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:03,961 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:03,961 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:05 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2358.codfw.wmnet) returned [2]: 2023-01-16 01:23:15,539 [INFO] Depooling currently pooled services
2023-01-16 01:23:15,618 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:20,625 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:25,629 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:30,637 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:35,648 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:40,656 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:45,664 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:50,673 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:55,681 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:00,690 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2358.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:05,693 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:05,693 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:05 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2369.codfw.wmnet) returned [2]: 2023-01-16 01:23:15,580 [INFO] Depooling currently pooled services
2023-01-16 01:23:15,660 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:20,666 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:25,675 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:30,681 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:35,688 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:40,693 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:45,702 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:50,711 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:55,719 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:00,724 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2369.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:05,730 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:05,730 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:05 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2389.codfw.wmnet) returned [2]: 2023-01-16 01:23:15,623 [INFO] Depooling currently pooled services
2023-01-16 01:23:15,696 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:20,704 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:25,714 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:30,720 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:35,728 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:40,736 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:45,744 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:50,752 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:55,761 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:00,769 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2389.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:05,774 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:05,774 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:05 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2283.codfw.wmnet) returned [2]: 2023-01-16 01:23:15,673 [INFO] Depooling currently pooled services
2023-01-16 01:23:15,751 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:20,758 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:25,767 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:30,776 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:35,784 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:40,790 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:45,798 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:50,806 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:55,815 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:00,822 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2283.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:05,827 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:05,828 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:10 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2385.codfw.wmnet) returned [2]: 2023-01-16 01:23:20,460 [INFO] Depooling currently pooled services
2023-01-16 01:23:20,536 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:25,545 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:30,553 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:35,561 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:40,569 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:45,577 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:50,585 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:55,592 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:00,601 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:05,609 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2385.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:10,614 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:10,614 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:29 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2012.codfw.wmnet) returned [2]: 2023-01-16 01:23:39,123 [INFO] Depooling currently pooled services
2023-01-16 01:23:39,200 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:44,209 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:49,219 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:54,228 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:59,238 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:04,247 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:09,257 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:14,266 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:19,275 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:24,285 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2012.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:29,290 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:29,290 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:29 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2011.codfw.wmnet) returned [2]: 2023-01-16 01:23:39,215 [INFO] Depooling currently pooled services
2023-01-16 01:23:39,292 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:44,301 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:49,308 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:54,318 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:59,327 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:04,336 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:09,344 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:14,353 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:19,361 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:24,369 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2011.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:29,375 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:29,375 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:29 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2288.codfw.wmnet) returned [2]: 2023-01-16 01:23:39,347 [INFO] Depooling currently pooled services
2023-01-16 01:23:39,425 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:44,434 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:49,443 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:54,451 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:59,461 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:04,470 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:09,479 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:14,489 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:19,499 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:24,508 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2288.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:29,511 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:29,511 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:29 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2401.codfw.wmnet) returned [2]: 2023-01-16 01:23:39,377 [INFO] Depooling currently pooled services
2023-01-16 01:23:39,452 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:44,458 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:49,466 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:54,474 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:59,482 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:04,490 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:09,498 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:14,502 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:19,509 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:24,518 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2401.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:29,523 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:29,523 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:30 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2291.codfw.wmnet) returned [2]: 2023-01-16 01:23:39,479 [INFO] Depooling currently pooled services
2023-01-16 01:23:39,557 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:44,566 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:49,575 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:54,897 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:59,905 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:04,918 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:09,928 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:14,938 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:19,948 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:24,956 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2291.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:29,961 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:29,961 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:30 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2276.codfw.wmnet) returned [2]: 2023-01-16 01:23:40,100 [INFO] Depooling currently pooled services
2023-01-16 01:23:40,178 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:45,186 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:50,194 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:55,203 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:00,210 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:05,219 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:10,228 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:15,234 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:20,242 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:25,249 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2276.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:30,254 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:30,254 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:30 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2305.codfw.wmnet) returned [2]: 2023-01-16 01:23:40,095 [INFO] Depooling currently pooled services
2023-01-16 01:23:40,175 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:45,181 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:50,189 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:55,198 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:00,209 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:05,217 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:10,225 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:15,233 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:20,241 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:25,249 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2305.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:30,254 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:30,254 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:35 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2372.codfw.wmnet) returned [2]: 2023-01-16 01:23:44,430 [INFO] Depooling currently pooled services
2023-01-16 01:23:44,506 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:49,512 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:54,897 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:23:59,905 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:04,918 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:09,927 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:14,937 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:19,945 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:24,954 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:29,961 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2372.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:34,966 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:34,966 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:56 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2374.codfw.wmnet) returned [2]: 2023-01-16 01:24:05,942 [INFO] Depooling currently pooled services
2023-01-16 01:24:06,021 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:11,030 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:16,044 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:21,051 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:26,058 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:31,066 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:36,075 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:41,084 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:46,093 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:51,099 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2374.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:56,104 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:56,104 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:24:56 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2339.codfw.wmnet) returned [2]: 2023-01-16 01:24:05,956 [INFO] Depooling currently pooled services
2023-01-16 01:24:06,037 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:11,045 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:16,052 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:21,060 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:26,068 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:31,076 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:36,083 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:41,092 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:46,101 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:51,109 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2339.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:56,112 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:24:56,113 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:19 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2001.codfw.wmnet) returned [2]: 2023-01-16 01:24:29,504 [INFO] Depooling currently pooled services
2023-01-16 01:24:29,582 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:34,588 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:39,597 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:44,606 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:49,615 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:54,809 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:59,819 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:04,829 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:09,839 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:14,849 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2001.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:19,854 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:19,855 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:19 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2004.codfw.wmnet) returned [2]: 2023-01-16 01:24:29,582 [INFO] Depooling currently pooled services
2023-01-16 01:24:29,655 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:34,664 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:39,743 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:44,752 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:49,761 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:54,812 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:59,821 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:04,832 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:09,842 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:14,852 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2004.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:19,857 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:19,857 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:19 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2304.codfw.wmnet) returned [2]: 2023-01-16 01:24:29,737 [INFO] Depooling currently pooled services
2023-01-16 01:24:29,812 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:34,819 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:39,826 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:44,835 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:49,845 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:54,851 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:59,861 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:04,869 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:09,878 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:14,887 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2304.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:19,890 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:19,890 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:20 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2294.codfw.wmnet) returned [2]: 2023-01-16 01:24:29,823 [INFO] Depooling currently pooled services
2023-01-16 01:24:29,903 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:34,909 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:39,915 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:44,925 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:49,933 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:54,941 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:59,949 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:04,957 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:09,966 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:14,975 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2294.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:19,978 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:19,978 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:20 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2356.codfw.wmnet) returned [2]: 2023-01-16 01:24:30,190 [INFO] Depooling currently pooled services
2023-01-16 01:24:30,269 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:35,277 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:40,286 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:45,295 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:50,304 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:55,310 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:00,321 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:05,330 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:10,339 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:15,347 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2356.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:20,350 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:20,350 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:20 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2363.codfw.wmnet) returned [2]: 2023-01-16 01:24:30,354 [INFO] Depooling currently pooled services
2023-01-16 01:24:30,433 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:35,439 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:40,448 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:45,455 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:50,463 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:55,472 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:00,479 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:05,488 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:10,495 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:15,503 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2363.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:20,508 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:20,508 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:20 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2309.codfw.wmnet) returned [2]: 2023-01-16 01:24:30,462 [INFO] Depooling currently pooled services
2023-01-16 01:24:30,537 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:35,545 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:40,552 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:45,560 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:50,569 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:55,576 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:00,584 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:05,592 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:10,601 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:15,610 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2309.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:20,613 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:20,613 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:20 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2361.codfw.wmnet) returned [2]: 2023-01-16 01:24:30,480 [INFO] Depooling currently pooled services
2023-01-16 01:24:30,561 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:35,569 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:40,575 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:45,583 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:50,591 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:55,598 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:00,608 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:05,615 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:10,623 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:15,631 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2361.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:20,636 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:20,636 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2302.codfw.wmnet) returned [2]: 2023-01-16 01:24:35,184 [INFO] Depooling currently pooled services
2023-01-16 01:24:35,262 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:40,271 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:45,278 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:50,285 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:55,293 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:00,301 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:05,310 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:10,318 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:15,326 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:20,334 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2302.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:25,338 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:25,338 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2379.codfw.wmnet) returned [2]: 2023-01-16 01:24:35,174 [INFO] Depooling currently pooled services
2023-01-16 01:24:35,248 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:40,254 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:45,263 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:50,270 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:55,278 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:00,286 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:05,294 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:10,303 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:15,311 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:20,319 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2379.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:25,325 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:25,325 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:43 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2364.codfw.wmnet) returned [2]: 2023-01-16 01:24:53,494 [INFO] Depooling currently pooled services
2023-01-16 01:24:53,576 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:24:58,585 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:03,592 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:08,602 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:13,611 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:18,617 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:23,625 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:28,634 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:33,641 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:38,648 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2364.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:43,653 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:43,653 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:25:46 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2271.codfw.wmnet) returned [2]: 2023-01-16 01:24:56,339 [INFO] Depooling currently pooled services
2023-01-16 01:24:56,420 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:01,429 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:06,437 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:11,448 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:16,457 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:21,465 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:26,474 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:31,483 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:36,489 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:41,497 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2271.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:46,501 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:25:46,501 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:10 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2399.codfw.wmnet) returned [2]: 2023-01-16 01:25:20,098 [INFO] Depooling currently pooled services
2023-01-16 01:25:20,173 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:25,182 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:30,191 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:35,201 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:40,209 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:45,217 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:50,222 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:55,231 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:00,239 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:05,248 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2399.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:10,253 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:10,253 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:10 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2292.codfw.wmnet) returned [2]: 2023-01-16 01:25:20,189 [INFO] Depooling currently pooled services
2023-01-16 01:25:20,262 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:25,267 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:30,275 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:35,283 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:40,291 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:45,298 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:50,306 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:55,313 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:00,322 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:05,330 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2292.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:10,334 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:10,335 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:10 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2300.codfw.wmnet) returned [2]: 2023-01-16 01:25:20,252 [INFO] Depooling currently pooled services
2023-01-16 01:25:20,327 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:25,336 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:30,343 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:35,349 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:40,357 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:45,365 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:50,373 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:55,382 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:00,391 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:05,397 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2300.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:10,402 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:10,402 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:10 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2368.codfw.wmnet) returned [2]: 2023-01-16 01:25:20,576 [INFO] Depooling currently pooled services
2023-01-16 01:25:20,657 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:25,662 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:30,671 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:35,678 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:40,688 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:45,694 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:50,703 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:55,708 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:00,718 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:05,727 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2368.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:10,731 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:10,731 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:11 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2307.codfw.wmnet) returned [2]: 2023-01-16 01:25:20,828 [INFO] Depooling currently pooled services
2023-01-16 01:25:20,903 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:25,911 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:30,919 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:35,926 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:40,933 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:45,943 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:50,950 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:55,958 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:00,966 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:05,976 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2307.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:10,978 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:10,978 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:11 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2406.codfw.wmnet) returned [2]: 2023-01-16 01:25:20,844 [INFO] Depooling currently pooled services
2023-01-16 01:25:20,918 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:25,925 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:30,934 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:35,941 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:40,945 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:45,952 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:50,960 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:55,968 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:00,977 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:05,987 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2406.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:10,992 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:10,993 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:15 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2370.codfw.wmnet) returned [2]: 2023-01-16 01:25:25,557 [INFO] Depooling currently pooled services
2023-01-16 01:25:25,636 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:30,642 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:35,651 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:40,659 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:45,666 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:50,675 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:55,678 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:00,686 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:05,692 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:10,699 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2370.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:15,705 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:15,705 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:15 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2391.codfw.wmnet) returned [2]: 2023-01-16 01:25:25,543 [INFO] Depooling currently pooled services
2023-01-16 01:25:25,623 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:30,630 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:35,640 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:40,651 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:45,659 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:50,667 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:55,675 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:00,683 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:05,692 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:10,700 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2391.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:15,706 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:15,706 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:33 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2005.codfw.wmnet) returned [2]: 2023-01-16 01:25:43,601 [INFO] Depooling currently pooled services
2023-01-16 01:25:43,676 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:48,685 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:53,694 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:58,704 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:03,712 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:08,722 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:13,731 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:18,740 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:23,749 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:28,759 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2005.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:33,764 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:33,764 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:33 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2017.codfw.wmnet) returned [2]: 2023-01-16 01:25:43,610 [INFO] Depooling currently pooled services
2023-01-16 01:25:43,687 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:48,697 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:53,707 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:58,716 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:03,726 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:08,735 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:13,744 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:18,754 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:23,765 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:28,775 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2017.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:33,777 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:33,777 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:35 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2272.codfw.wmnet) returned [2]: 2023-01-16 01:25:45,101 [INFO] Depooling currently pooled services
2023-01-16 01:25:45,185 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:50,194 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:55,200 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:00,209 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:05,216 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:10,227 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:15,236 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:20,245 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:25,253 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:30,262 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2272.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:35,264 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:35,265 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:36 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2413.codfw.wmnet) returned [2]: 2023-01-16 01:25:46,723 [INFO] Depooling currently pooled services
2023-01-16 01:25:46,800 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:51,808 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:56,816 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:01,825 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:06,833 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:11,840 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:16,848 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:21,856 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:26,861 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:31,867 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2413.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:36,869 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:36,869 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:37 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2337.codfw.wmnet) returned [2]: 2023-01-16 01:25:47,008 [INFO] Depooling currently pooled services
2023-01-16 01:25:47,086 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:52,093 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:25:57,102 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:02,111 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:07,117 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:12,125 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:17,133 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:22,141 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:27,150 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:32,159 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2337.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:37,162 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:37,162 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:26:57 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2416.codfw.wmnet) returned [2]: 2023-01-16 01:26:07,735 [INFO] Depooling currently pooled services
2023-01-16 01:26:07,817 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:12,824 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:17,832 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:22,841 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:27,848 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:32,857 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:37,863 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:42,872 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:47,880 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:52,888 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2416.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:57,892 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:26:57,892 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:00 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2297.codfw.wmnet) returned [2]: 2023-01-16 01:26:10,475 [INFO] Depooling currently pooled services
2023-01-16 01:26:10,553 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:15,562 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:20,570 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:25,579 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:30,587 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:35,595 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:40,603 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:45,612 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:50,619 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:55,628 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2297.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:00,631 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:00,631 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:00 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2405.codfw.wmnet) returned [2]: 2023-01-16 01:26:10,552 [INFO] Depooling currently pooled services
2023-01-16 01:26:10,633 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:15,641 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:20,647 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:25,656 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:30,663 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:35,671 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:40,680 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:45,690 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:50,698 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:55,707 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2405.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:00,712 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:00,713 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:00 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2296.codfw.wmnet) returned [2]: 2023-01-16 01:26:10,612 [INFO] Depooling currently pooled services
2023-01-16 01:26:10,687 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:15,695 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:20,704 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:25,712 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:30,721 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:35,730 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:40,739 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:45,744 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:50,754 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:55,759 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2296.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:00,764 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:00,764 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:01 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2299.codfw.wmnet) returned [2]: 2023-01-16 01:26:10,939 [INFO] Depooling currently pooled services
2023-01-16 01:26:11,018 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:16,027 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:21,037 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:26,042 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:31,052 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:36,062 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:41,072 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:46,078 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:51,086 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:56,095 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2299.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:01,100 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:01,100 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:01 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2386.codfw.wmnet) returned [2]: 2023-01-16 01:26:11,195 [INFO] Depooling currently pooled services
2023-01-16 01:26:11,269 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:16,276 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:21,282 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:26,290 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:31,298 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:36,306 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:41,315 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:46,324 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:51,330 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:56,338 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2386.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:01,343 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:01,343 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:01 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2301.codfw.wmnet) returned [2]: 2023-01-16 01:26:11,197 [INFO] Depooling currently pooled services
2023-01-16 01:26:11,271 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:16,277 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:21,286 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:26,293 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:31,302 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:36,312 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:41,318 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:46,326 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:51,334 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:56,341 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2301.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:01,346 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:01,347 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:06 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2350.codfw.wmnet) returned [2]: 2023-01-16 01:26:15,922 [INFO] Depooling currently pooled services
2023-01-16 01:26:15,998 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:21,008 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:26,016 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:31,021 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:36,029 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:41,038 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:46,044 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:51,051 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:56,060 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:01,069 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2350.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:06,074 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:06,074 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:06 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2393.codfw.wmnet) returned [2]: 2023-01-16 01:26:15,924 [INFO] Depooling currently pooled services
2023-01-16 01:26:16,001 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:21,007 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:26,014 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:31,022 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:36,031 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:41,038 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:46,047 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:51,055 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:56,065 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:01,070 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2393.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:06,075 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:06,075 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:24 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2018.codfw.wmnet) returned [2]: 2023-01-16 01:26:34,000 [INFO] Depooling currently pooled services
2023-01-16 01:26:34,082 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:39,092 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:44,101 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:49,111 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:54,120 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:59,129 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:04,139 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:09,148 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:14,158 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:19,168 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2018.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:24,173 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:24,173 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:25 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2390.codfw.wmnet) returned [2]: 2023-01-16 01:26:35,472 [INFO] Depooling currently pooled services
2023-01-16 01:26:35,548 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:40,555 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:45,562 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:50,572 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:55,579 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:00,586 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:05,594 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:10,603 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:15,610 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:20,618 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2390.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:25,624 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:25,624 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:27 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2274.codfw.wmnet) returned [2]: 2023-01-16 01:26:37,086 [INFO] Depooling currently pooled services
2023-01-16 01:26:37,165 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:42,173 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:47,182 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:52,189 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:57,198 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:02,205 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:07,215 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:12,223 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:17,230 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:22,238 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2274.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:27,242 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:27,243 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:27 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2367.codfw.wmnet) returned [2]: 2023-01-16 01:26:37,385 [INFO] Depooling currently pooled services
2023-01-16 01:26:37,462 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:42,470 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:47,479 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:52,488 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:26:57,495 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:02,503 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:07,512 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:12,519 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:17,528 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:22,534 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2367.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:27,540 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:27,540 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:48 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2013.codfw.wmnet) returned [2]: 2023-01-16 01:26:57,886 [INFO] Depooling currently pooled services
2023-01-16 01:26:57,962 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:02,971 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:07,980 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:12,989 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:17,998 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:23,004 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:28,013 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:33,023 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:38,031 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:43,041 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2013.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:48,047 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:48,047 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:48 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2293.codfw.wmnet) returned [2]: 2023-01-16 01:26:58,101 [INFO] Depooling currently pooled services
2023-01-16 01:26:58,175 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:03,181 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:08,189 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:13,198 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:18,205 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:23,215 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:28,224 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:33,233 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:38,241 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:43,248 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2293.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:48,254 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:48,254 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:51 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2400.codfw.wmnet) returned [2]: 2023-01-16 01:27:00,923 [INFO] Depooling currently pooled services
2023-01-16 01:27:00,999 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:06,007 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:11,017 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:16,021 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:21,030 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:26,041 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:31,050 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:36,059 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:41,068 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:46,076 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2400.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:51,078 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:51,078 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:51 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2396.codfw.wmnet) returned [2]: 2023-01-16 01:27:00,971 [INFO] Depooling currently pooled services
2023-01-16 01:27:01,045 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:06,055 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:11,062 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:16,070 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:21,078 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:26,087 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:31,096 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:36,103 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:41,112 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:46,119 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2396.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:51,124 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:51,125 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:51 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2273.codfw.wmnet) returned [2]: 2023-01-16 01:27:01,564 [INFO] Depooling currently pooled services
2023-01-16 01:27:01,643 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:06,650 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:11,658 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:16,668 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:21,674 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:26,682 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:31,691 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:36,701 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:41,710 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:46,720 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2273.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:51,722 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:51,722 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:51 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2375.codfw.wmnet) returned [2]: 2023-01-16 01:27:01,573 [INFO] Depooling currently pooled services
2023-01-16 01:27:01,649 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:06,662 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:11,670 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:16,675 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:21,684 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:26,692 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:31,700 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:36,708 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:41,715 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:46,721 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2375.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:51,727 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:51,727 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:56 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2415.codfw.wmnet) returned [2]: 2023-01-16 01:27:06,295 [INFO] Depooling currently pooled services
2023-01-16 01:27:06,377 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:11,384 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:16,393 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:21,400 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:26,408 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:31,416 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:36,423 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:41,432 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:46,440 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:51,448 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2415.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:56,452 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:56,453 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:27:56 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2287.codfw.wmnet) returned [2]: 2023-01-16 01:27:06,301 [INFO] Depooling currently pooled services
2023-01-16 01:27:06,384 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:11,390 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:16,399 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:21,408 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:26,417 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:31,432 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:36,437 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:41,446 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:46,455 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:51,464 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2287.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:56,469 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:27:56,470 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:14 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2019.codfw.wmnet) returned [2]: 2023-01-16 01:27:24,398 [INFO] Depooling currently pooled services
2023-01-16 01:27:24,478 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:29,487 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:34,496 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:39,506 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:44,514 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:49,524 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:54,794 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:59,803 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:04,813 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:09,823 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2019.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:14,828 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:14,828 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:15 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2308.codfw.wmnet) returned [2]: 2023-01-16 01:27:25,081 [INFO] Depooling currently pooled services
2023-01-16 01:27:25,155 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:30,163 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:35,170 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:40,178 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:45,187 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:50,196 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:55,205 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:00,211 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:05,221 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:10,230 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2308.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:15,236 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:15,236 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:16 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2373.codfw.wmnet) returned [2]: 2023-01-16 01:27:25,842 [INFO] Depooling currently pooled services
2023-01-16 01:27:25,919 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:30,925 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:35,933 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:40,941 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:45,949 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:50,957 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:55,965 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:00,974 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:05,983 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:10,989 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2373.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:15,993 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:15,994 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:17 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2359.codfw.wmnet) returned [2]: 2023-01-16 01:27:27,460 [INFO] Depooling currently pooled services
2023-01-16 01:27:27,537 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:32,542 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:37,550 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:42,559 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:47,571 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:52,579 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:57,588 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:02,594 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:07,602 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:12,611 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2359.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:17,617 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:17,617 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:17 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2335.codfw.wmnet) returned [2]: 2023-01-16 01:27:27,758 [INFO] Depooling currently pooled services
2023-01-16 01:27:27,836 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:32,842 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:37,851 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:42,860 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:47,866 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:52,874 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:57,882 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:02,890 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:07,898 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:12,905 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2335.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:17,911 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:17,911 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:38 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@parse2016.codfw.wmnet) returned [2]: 2023-01-16 01:27:48,257 [INFO] Depooling currently pooled services
2023-01-16 01:27:48,335 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:53,344 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:58,353 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:03,363 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:08,373 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:13,382 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:18,391 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:23,400 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:28,410 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:33,419 [WARNING] LB lvs2009:9090 reports pool parsoid-php_443/parse2016.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:38,424 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:38,424 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:38 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2284.codfw.wmnet) returned [2]: 2023-01-16 01:27:48,475 [INFO] Depooling currently pooled services
2023-01-16 01:27:48,552 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:53,559 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:58,567 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:03,575 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:08,584 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:13,590 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:18,598 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:23,606 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:28,616 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:33,622 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2284.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:38,627 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:38,627 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:41 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2289.codfw.wmnet) returned [2]: 2023-01-16 01:27:51,345 [INFO] Depooling currently pooled services
2023-01-16 01:27:51,424 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:27:56,434 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:01,442 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:06,450 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:11,458 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:16,468 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:21,474 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:26,482 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:31,490 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:36,497 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2289.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:41,503 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:41,503 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:46 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2409.codfw.wmnet) returned [2]: 2023-01-16 01:27:56,660 [INFO] Depooling currently pooled services
2023-01-16 01:27:56,736 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:01,744 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:06,752 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:11,760 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:16,768 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:21,777 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:26,784 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:31,792 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:36,800 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:41,809 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2409.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:46,814 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:46,815 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:28:46 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2402.codfw.wmnet) returned [2]: 2023-01-16 01:27:56,677 [INFO] Depooling currently pooled services
2023-01-16 01:27:56,750 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:01,759 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:06,765 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:11,773 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:16,782 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:21,791 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:26,800 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:31,806 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:36,814 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:41,822 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2402.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:46,826 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:28:46,826 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:02 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2352.codfw.wmnet) returned [2]: 2023-01-16 01:28:12,111 [INFO] Depooling currently pooled services
2023-01-16 01:28:12,190 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:17,196 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:22,205 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:27,213 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:32,222 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:37,231 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:42,240 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:47,248 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:52,256 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:57,264 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2352.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:02,269 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:02,269 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:05 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2404.codfw.wmnet) returned [2]: 2023-01-16 01:28:15,443 [INFO] Depooling currently pooled services
2023-01-16 01:28:15,519 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:20,526 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:25,534 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:30,541 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:35,550 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:40,558 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:45,567 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:50,574 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:55,583 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:00,592 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2404.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:05,597 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:05,597 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:06 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2387.codfw.wmnet) returned [2]: 2023-01-16 01:28:16,203 [INFO] Depooling currently pooled services
2023-01-16 01:28:16,280 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:21,287 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:26,295 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:31,304 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:36,313 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:41,324 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:46,333 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:51,340 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:56,347 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:01,355 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2387.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:06,360 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:06,361 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:06 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2414.codfw.wmnet) returned [2]: 2023-01-16 01:28:16,562 [INFO] Depooling currently pooled services
2023-01-16 01:28:16,640 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:21,648 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:26,655 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:31,663 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:36,671 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:41,679 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:46,687 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:51,695 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:56,703 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:01,712 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2414.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:06,716 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:06,716 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:06 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2275.codfw.wmnet) returned [2]: 2023-01-16 01:28:16,680 [INFO] Depooling currently pooled services
2023-01-16 01:28:16,757 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:21,766 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:26,775 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:31,785 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:36,791 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:41,800 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:46,807 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:51,815 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:56,823 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:01,831 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2275.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:06,836 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:06,837 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:08 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2365.codfw.wmnet) returned [2]: 2023-01-16 01:28:18,128 [INFO] Depooling currently pooled services
2023-01-16 01:28:18,205 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:23,214 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:28,222 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:33,231 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:38,239 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:43,245 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:48,254 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:53,262 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:58,271 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:03,278 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2365.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:08,282 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:08,283 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:29 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2398.codfw.wmnet) returned [2]: 2023-01-16 01:28:38,767 [INFO] Depooling currently pooled services
2023-01-16 01:28:38,880 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:43,887 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:48,895 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:53,904 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:58,914 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:03,919 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:08,927 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:13,934 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:18,943 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:23,951 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2398.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:28,955 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:28,956 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:29 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2290.codfw.wmnet) returned [2]: 2023-01-16 01:28:38,848 [INFO] Depooling currently pooled services
2023-01-16 01:28:39,003 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:44,012 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:49,020 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:54,029 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:59,035 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:04,045 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:09,052 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:14,062 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:19,068 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:24,075 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2290.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:29,081 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:29,081 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:31 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2417.codfw.wmnet) returned [2]: 2023-01-16 01:28:41,721 [INFO] Depooling currently pooled services
2023-01-16 01:28:41,842 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:46,850 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:51,858 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:56,867 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:01,874 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:06,883 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:11,890 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:16,898 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:21,907 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:26,914 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2417.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:31,915 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:31,915 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:37 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2383.codfw.wmnet) returned [2]: 2023-01-16 01:28:47,023 [INFO] Depooling currently pooled services
2023-01-16 01:28:47,101 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:52,109 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:57,117 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:02,125 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:07,134 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:12,143 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:17,148 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:22,157 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:27,165 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:32,173 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2383.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:37,177 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:37,177 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:37 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2362.codfw.wmnet) returned [2]: 2023-01-16 01:28:47,043 [INFO] Depooling currently pooled services
2023-01-16 01:28:47,128 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:52,137 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:28:57,144 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:02,152 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:07,159 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:12,169 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:17,177 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:22,186 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:27,192 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:32,201 [WARNING] LB lvs2009:9090 reports pool api-https_443/mw2362.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:37,204 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:37,204 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:56 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2371.codfw.wmnet) returned [2]: 2023-01-16 01:29:06,579 [INFO] Depooling currently pooled services
2023-01-16 01:29:06,657 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:11,665 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:16,673 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:21,682 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:26,689 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:31,697 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:36,706 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:41,713 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:46,722 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:51,729 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2371.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:56,734 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:56,734 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:56 /usr/bin/sudo -u root -- /usr/local/sbin/restart-php-fpm-all php7.4-fpm 9223372036854775807 (ran as mwdeploy@mw2407.codfw.wmnet) returned [2]: 2023-01-16 01:29:06,704 [INFO] Depooling currently pooled services
2023-01-16 01:29:06,779 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:11,785 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:16,792 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:21,801 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:26,808 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:31,815 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:36,824 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:41,832 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:46,840 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:51,849 [WARNING] LB lvs2009:9090 reports pool appservers-https_443/mw2407.codfw.wmnet as enabled/up/pooled, should be disabled/*/not pooled
2023-01-16 01:29:56,852 [ERROR] Error depooling the servers: enabled/up/pooled
2023-01-16 01:29:56,852 [ERROR] Error running command with poolcounter: ('Failed executing ServiceRunner.run, return code %d', 127)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/poolcounter/client.py", line 377, in run
callback(*args)
File "/usr/local/bin/safe-service-restart", line 151, in run_and_raise
raise RuntimeError("Failed executing ServiceRunner.run, return code %d", rc)
RuntimeError: ('Failed executing ServiceRunner.run, return code %d', 127)
01:29:56 php-fpm-restart: 100% (in-flight: 0; ok: 184; fail: 112; left: 0)
01:29:56 49 hosts had failures restarting php-fpm
01:29:56 49 hosts had failures restarting php-fpm
01:29:56 14 hosts had failures restarting php-fpm
01:29:56 Finished php-fpm-restarts (duration: 08m 23s)
01:29:56 Running /usr/local/bin/mwscript purgeMessageBlobStore.php
01:29:57 Finished scap: Backport for [[gerrit:879798|LanguageDropdown: Check if the page is in talk namespaces instead (T316559 T326788)]] (duration: 24m 47s)Here are the hosts that had problems.
mwdeploy@mw2271.codfw.wmnet mwdeploy@mw2272.codfw.wmnet mwdeploy@mw2273.codfw.wmnet mwdeploy@mw2274.codfw.wmnet mwdeploy@mw2275.codfw.wmnet mwdeploy@mw2276.codfw.wmnet mwdeploy@mw2277.codfw.wmnet mwdeploy@mw2283.codfw.wmnet mwdeploy@mw2284.codfw.wmnet mwdeploy@mw2285.codfw.wmnet mwdeploy@mw2286.codfw.wmnet mwdeploy@mw2287.codfw.wmnet mwdeploy@mw2288.codfw.wmnet mwdeploy@mw2289.codfw.wmnet mwdeploy@mw2290.codfw.wmnet mwdeploy@mw2291.codfw.wmnet mwdeploy@mw2292.codfw.wmnet mwdeploy@mw2293.codfw.wmnet mwdeploy@mw2294.codfw.wmnet mwdeploy@mw2295.codfw.wmnet mwdeploy@mw2296.codfw.wmnet mwdeploy@mw2297.codfw.wmnet mwdeploy@mw2298.codfw.wmnet mwdeploy@mw2299.codfw.wmnet mwdeploy@mw2300.codfw.wmnet mwdeploy@mw2301.codfw.wmnet mwdeploy@mw2302.codfw.wmnet mwdeploy@mw2303.codfw.wmnet mwdeploy@mw2304.codfw.wmnet mwdeploy@mw2305.codfw.wmnet mwdeploy@mw2306.codfw.wmnet mwdeploy@mw2307.codfw.wmnet mwdeploy@mw2308.codfw.wmnet mwdeploy@mw2309.codfw.wmnet mwdeploy@mw2335.codfw.wmnet mwdeploy@mw2336.codfw.wmnet mwdeploy@mw2337.codfw.wmnet mwdeploy@mw2338.codfw.wmnet mwdeploy@mw2339.codfw.wmnet mwdeploy@mw2350.codfw.wmnet mwdeploy@mw2352.codfw.wmnet mwdeploy@mw2354.codfw.wmnet mwdeploy@mw2356.codfw.wmnet mwdeploy@mw2358.codfw.wmnet mwdeploy@mw2359.codfw.wmnet mwdeploy@mw2360.codfw.wmnet mwdeploy@mw2361.codfw.wmnet mwdeploy@mw2362.codfw.wmnet mwdeploy@mw2363.codfw.wmnet mwdeploy@mw2364.codfw.wmnet mwdeploy@mw2365.codfw.wmnet mwdeploy@mw2366.codfw.wmnet mwdeploy@mw2367.codfw.wmnet mwdeploy@mw2368.codfw.wmnet mwdeploy@mw2369.codfw.wmnet mwdeploy@mw2370.codfw.wmnet mwdeploy@mw2371.codfw.wmnet mwdeploy@mw2372.codfw.wmnet mwdeploy@mw2373.codfw.wmnet mwdeploy@mw2374.codfw.wmnet mwdeploy@mw2375.codfw.wmnet mwdeploy@mw2376.codfw.wmnet mwdeploy@mw2377.codfw.wmnet mwdeploy@mw2378.codfw.wmnet mwdeploy@mw2379.codfw.wmnet mwdeploy@mw2380.codfw.wmnet mwdeploy@mw2383.codfw.wmnet mwdeploy@mw2384.codfw.wmnet mwdeploy@mw2385.codfw.wmnet mwdeploy@mw2386.codfw.wmnet mwdeploy@mw2387.codfw.wmnet mwdeploy@mw2388.codfw.wmnet mwdeploy@mw2389.codfw.wmnet mwdeploy@mw2390.codfw.wmnet mwdeploy@mw2391.codfw.wmnet mwdeploy@mw2392.codfw.wmnet mwdeploy@mw2393.codfw.wmnet mwdeploy@mw2396.codfw.wmnet mwdeploy@mw2397.codfw.wmnet mwdeploy@mw2398.codfw.wmnet mwdeploy@mw2399.codfw.wmnet mwdeploy@mw2400.codfw.wmnet mwdeploy@mw2401.codfw.wmnet mwdeploy@mw2402.codfw.wmnet mwdeploy@mw2403.codfw.wmnet mwdeploy@mw2404.codfw.wmnet mwdeploy@mw2405.codfw.wmnet mwdeploy@mw2406.codfw.wmnet mwdeploy@mw2407.codfw.wmnet mwdeploy@mw2408.codfw.wmnet mwdeploy@mw2409.codfw.wmnet mwdeploy@mw2412.codfw.wmnet mwdeploy@mw2413.codfw.wmnet mwdeploy@mw2414.codfw.wmnet mwdeploy@mw2415.codfw.wmnet mwdeploy@mw2416.codfw.wmnet mwdeploy@mw2417.codfw.wmnet mwdeploy@mw2418.codfw.wmnet mwdeploy@parse2001.codfw.wmnet mwdeploy@parse2003.codfw.wmnet mwdeploy@parse2004.codfw.wmnet mwdeploy@parse2005.codfw.wmnet mwdeploy@parse2011.codfw.wmnet mwdeploy@parse2012.codfw.wmnet mwdeploy@parse2013.codfw.wmnet mwdeploy@parse2014.codfw.wmnet mwdeploy@parse2015.codfw.wmnet mwdeploy@parse2016.codfw.wmnet mwdeploy@parse2017.codfw.wmnet mwdeploy@parse2018.codfw.wmnet mwdeploy@parse2019.codfw.wmnet mwdeploy@parse2020.codfw.wmnet
This sounds similar to the codfw appserver degradation incident from Sept. Fortunately, @Ladsgroup was available to repool depooled machines.
But we should determine root cause before we resume deployments.