swift-account-stats sometimes fails when running via cron with the following error:
Traceback (most recent call last): File "/usr/local/bin/swift-account-stats", line 76, in <module> sys.exit(main()) File "/usr/local/bin/swift-account-stats", line 49, in main headers = connection.head_account() File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1229, in head_account return self._retry(None, head_account) File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1181, in _retry self.url, self.token = self.get_auth() File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1155, in get_auth insecure=self.insecure) File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 318, in get_auth insecure=insecure) File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 228, in get_auth_1_0 {'X-Auth-User': user, 'X-Auth-Key': key}) File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 188, in request files=files, **self.requests_args) File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 177, in _request return requests.request(*arg, **kwarg) File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send r = adapter.send(request, **kwargs) File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send raise ConnectionError(e) requests.exceptions.ConnectionError: HTTPConnectionPool(host='ms-fe.svc.codfw.wmnet', port=80): Max retries exceeded with url: /auth/v1.0 (Caused by <class 'socket.error'>: [Errno 111] Connection refused)
Sometimes this only triggers a few cron emails, in other situations such as on January 6th we got > 100 messages.