Page MenuHomePhabricator

prometheus-vhtcpd-stats cronspamming if vhtcpd is not running yet
Closed, ResolvedPublic

Description

The following cronspam is occasionally produced by prometheus-vhtcpd-stats upon server reboot:

Traceback (most recent call last):
  File "/usr/local/bin/prometheus-vhtcpd-stats", line 105, in <module>
    sys.exit(main())
  File "/usr/local/bin/prometheus-vhtcpd-stats", line 93, in main
    infile = open(args.statsfile, 'r')
IOError: [Errno 2] No such file or directory: '/tmp/vhtcpd.stats'

That seems to be due to the fact that the program is executed by cron every minute, possibly before vhtcpd's startup.

Event Timeline

ema created this task.Feb 6 2017, 4:42 PM
Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptFeb 6 2017, 4:42 PM
ema moved this task from Triage to Caching on the Traffic board.Feb 7 2017, 11:44 AM

Change 336773 had a related patch set uploaded (by Ema):
prometheus: run -vhtcpd-stats iff vhtcpd is running

https://gerrit.wikimedia.org/r/336773

Change 336773 merged by Ema:
prometheus: run -vhtcpd-stats iff vhtcpd is running

https://gerrit.wikimedia.org/r/336773

elukey closed this task as Resolved.Feb 10 2017, 7:22 AM
Volans reopened this task as Open.Apr 13 2017, 10:57 AM
Volans added a subscriber: Volans.

Re-opening because this is happening when rebooting hosts, see last days root@ mails

Change 366251 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: check for vhtcp.stats file

https://gerrit.wikimedia.org/r/366251

Change 366251 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: check for vhtcp.stats file

https://gerrit.wikimedia.org/r/366251

fgiunchedi closed this task as Resolved.Jul 19 2017, 2:37 PM
fgiunchedi claimed this task.

We're checking for file presence too now