Page MenuHomePhabricator

prometheus-puppet-agent-stats cronspam on missing puppet stats
Closed, ResolvedPublic

Description

Noted on a just-reinstalled machine

Traceback (most recent call last):
  File "/usr/local/bin/prometheus-puppet-agent-stats", line 106, in <module>
    sys.exit(main())
  File "/usr/local/bin/prometheus-puppet-agent-stats", line 97, in main
    collect_puppet_stats(args.puppet_state_dir, registry)
  File "/usr/local/bin/prometheus-puppet-agent-stats", line 74, in collect_puppet_stats
    _summary_stats(puppet_state_dir, registry)
  File "/usr/local/bin/prometheus-puppet-agent-stats", line 45, in _summary_stats
    with open(summary_file) as f:
IOError: [Errno 2] No such file or directory: '/var/lib/puppet/state/last_run_summary.yaml'

Sometimes for invalid status

Traceback (most recent call last):
  File "/usr/local/bin/prometheus-puppet-agent-stats", line 106, in <module>
    sys.exit(main())
  File "/usr/local/bin/prometheus-puppet-agent-stats", line 97, in main
    collect_puppet_stats(args.puppet_state_dir, registry)
  File "/usr/local/bin/prometheus-puppet-agent-stats", line 74, in collect_puppet_stats
    _summary_stats(puppet_state_dir, registry)
  File "/usr/local/bin/prometheus-puppet-agent-stats", line 53, in _summary_stats
    if 'time' in summary_yaml:
TypeError: argument of type 'NoneType' is not iterable

Event Timeline

Change 368160 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: handle non-existant or empty puppet last run summary

https://gerrit.wikimedia.org/r/368160

fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.Jul 27 2017, 10:31 AM

Change 368160 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: handle non-existant or empty puppet last run summary

https://gerrit.wikimedia.org/r/368160

fgiunchedi closed this task as Resolved.Jul 27 2017, 2:21 PM

Resolving, will reopen if cronspam shows up again

Change 373882 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: log puppet-agent-stats exceptions at DEBUG

https://gerrit.wikimedia.org/r/373882

Change 373882 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: log puppet-agent-stats exceptions at DEBUG

https://gerrit.wikimedia.org/r/373882