Since we use prometheus for monitoring puppet staleness, we are unable to tell whether puppet is stale on tools-k8s-master-01.tools.eqiad.wmflabs without logging into it, which is not really good. The cron that generates prometheus data fails for some reason with the following:
:~$ sudo /usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom Traceback (most recent call last): File "/usr/local/bin/prometheus-puppet-agent-stats", line 23, in <module> from prometheus_client import CollectorRegistry, Gauge, write_to_textfile File "/usr/lib/python3/dist-packages/prometheus_client/__init__.py", line 4, in <module> from . import exposition File "/usr/lib/python3/dist-packages/prometheus_client/exposition.py", line 10, in <module> from wsgiref.simple_server import make_server File "/usr/lib/python3.4/wsgiref/simple_server.py", line 13, in <module> from http.server import BaseHTTPRequestHandler, HTTPServer File "/usr/lib/python3.4/http/server.py", line 88, in <module> import http.client File "/usr/lib/python3.4/http/client.py", line 1014 raise InvalidURL(f"URL can't contain control characters. {url!r} " ^ SyntaxError: invalid syntax
That looks like an f-string, so I think this is just a python version error (which we saw recently).