Page MenuHomePhabricator
Paste P8196

php segfault counts on integration-slave-docker hosts
ActivePublic

Authored by hashar on Mar 13 2019, 8:25 PM.
Tags
None
Referenced Files
F28383728: raw.txt
Mar 13 2019, 8:25 PM
Subscribers
None
$ sudo cumin --force 'name:docker' 'zgrep -c php.*segfault /var/log/kern.log*'
14 hosts will be targeted:
integration-slave-docker-[1021,1034,1037,1040-1041,1043,1045-1046,1048-1053].integration.eqiad.wmflabs
FORCE mode enabled, continuing without confirmation
===== NODE GROUP =====
(1) integration-slave-docker-1034.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:3
/var/log/kern.log.1:2
/var/log/kern.log.2.gz:1
/var/log/kern.log.3.gz:7
/var/log/kern.log.4.gz:10
===== NODE GROUP =====
(1) integration-slave-docker-1052.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:3
/var/log/kern.log.1:9
/var/log/kern.log.2.gz:9
/var/log/kern.log.3.gz:1
/var/log/kern.log.4.gz:0
===== NODE GROUP =====
(1) integration-slave-docker-1041.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:3
/var/log/kern.log.1:6
/var/log/kern.log.2.gz:0
/var/log/kern.log.3.gz:1
/var/log/kern.log.4.gz:1
===== NODE GROUP =====
(1) integration-slave-docker-1037.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:3
/var/log/kern.log.1:18
/var/log/kern.log.2.gz:15
/var/log/kern.log.3.gz:11
/var/log/kern.log.4.gz:2
===== NODE GROUP =====
(1) integration-slave-docker-1051.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:10
/var/log/kern.log.1:11
/var/log/kern.log.2.gz:3
/var/log/kern.log.3.gz:11
/var/log/kern.log.4.gz:0
===== NODE GROUP =====
(1) integration-slave-docker-1050.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:14
/var/log/kern.log.1:10
/var/log/kern.log.2.gz:5
/var/log/kern.log.3.gz:5
/var/log/kern.log.4.gz:0
===== NODE GROUP =====
(1) integration-slave-docker-1053.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:0
/var/log/kern.log.1:0
/var/log/kern.log.2.gz:0
/var/log/kern.log.3.gz:1
/var/log/kern.log.4.gz:0
===== NODE GROUP =====
(3) integration-slave-docker-[1021,1045,1049].integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:0
/var/log/kern.log.1:0
/var/log/kern.log.2.gz:0
/var/log/kern.log.3.gz:0
/var/log/kern.log.4.gz:0
===== NODE GROUP =====
(1) integration-slave-docker-1048.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:4
/var/log/kern.log.1:0
/var/log/kern.log.2.gz:5
/var/log/kern.log.3.gz:5
/var/log/kern.log.4.gz:0
===== NODE GROUP =====
(1) integration-slave-docker-1040.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:4
/var/log/kern.log.1:4
/var/log/kern.log.2.gz:5
/var/log/kern.log.3.gz:2
/var/log/kern.log.4.gz:4
===== NODE GROUP =====
(1) integration-slave-docker-1043.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:4
/var/log/kern.log.1:5
/var/log/kern.log.2.gz:2
/var/log/kern.log.3.gz:12
/var/log/kern.log.4.gz:2
===== NODE GROUP =====
(1) integration-slave-docker-1046.integration.eqiad.wmflabs
----- OUTPUT of 'zgrep -c php.*se...ar/log/kern.log*' -----
/var/log/kern.log:2
/var/log/kern.log.1:0
/var/log/kern.log.2.gz:0
/var/log/kern.log.3.gz:0
/var/log/kern.log.4.gz:1
================
PASS: |████████████████████████████████████████████▊ | 29% (4/14) [00:12<00:50, 5.01s/hosts]
FAIL: |███████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 71% (10/14) [00:12<00:08, 2.10s/hosts]
71.4% (10/14) of nodes failed to execute command 'zgrep -c php.*se...ar/log/kern.log*': integration-slave-docker-[1021,1041,1045-1046,1048-1053].integration.eqiad.wmflabs
28.6% (4/14) success ratio (< 100.0% threshold) for command: 'zgrep -c php.*se...ar/log/kern.log*'. Aborting.: integration-slave-docker-[1034,1037,1040,1043].integration.eqiad.wmflabs
28.6% (4/14) success ratio (< 100.0% threshold) of nodes successfully executed all commands. Aborting.: integration-slave-docker-[1034,1037,1040,1043].integration.eqiad.wmflabs