Unclean stop of jobrunner service via puppet
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	fgiunchedi
	Feb 16 2017, 9:31 AM

Description

Today I tried a roll-restart of jobrunner for T123728 with systemctl restart jobrunner via salt and when puppet tried to subsequently stop jobrunner it resulted in failure and triggering of icinga alert for 'systemd degraded'

root@mw2247:~# systemctl status jobrunner
● jobrunner.service - "Mediawiki job queue runner loop"
   Loaded: loaded (/lib/systemd/system/jobrunner.service; disabled)
   Active: failed (Result: exit-code) since Thu 2017-02-16 09:13:43 UTC; 4min 4s ago
  Process: 112358 ExecStart=/usr/bin/php /srv/deployment/jobrunner/jobrunner/redisJobRunnerService --config-file=${JOBRUNNER_CONFIG} ${DAEMON_OPTS} (code=exited, status=143)
 Main PID: 112358 (code=exited, status=143)

Feb 16 09:13:43 mw2247 jobrunner[112358]: [Thu Feb 16 09:13:43 2017] [hphp] [112358:7f008fff16c0:0:000026] [] LightProc...n pipe
Feb 16 09:13:43 mw2247 jobrunner[112358]: Sending SIGTERM to 6862.
Feb 16 09:13:43 mw2247 jobrunner[112358]: [Thu Feb 16 09:13:43 2017] [hphp] [112358:7f008fff16c0:0:000027] [] LightProc...n pipe
Feb 16 09:13:43 mw2247 jobrunner[112358]: [Thu Feb 16 09:13:43 2017] [hphp] [112358:7f008fff16c0:0:000028] [] LightProc...n pipe
Feb 16 09:13:43 mw2247 jobrunner[112358]: [Thu Feb 16 09:13:43 2017] [hphp] [112358:7f008fff16c0:0:000029] [] LightProc...n pipe
Feb 16 09:13:43 mw2247 jobrunner[112358]: [Thu Feb 16 09:13:43 2017] [hphp] [112358:7f008fff16c0:0:000030] [] LightProc...n pipe
Feb 16 09:13:43 mw2247 jobrunner[112358]: [Thu Feb 16 09:13:43 2017] [hphp] [112358:7f008fff16c0:0:000031] [] LightProc...n pipe
Feb 16 09:13:43 mw2247 systemd[1]: jobrunner.service: main process exited, code=exited, status=143/n/a
Feb 16 09:13:43 mw2247 systemd[1]: Stopped "Mediawiki job queue runner loop".
Feb 16 09:13:43 mw2247 systemd[1]: Unit jobrunner.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.