On gallium, I have upgraded python-stated from 2.0.1 to 3.0.1 and restarted Zuul. The web app service at https://integration.wikimedia.org/zuul/status.json now throw an internal server error.
Using strace I found the following python stacktrace:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/paste/httpserver.py", line 1068, in process_request_in_thread
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 638, in __init__
self.handle()
File "/usr/lib/python2.7/dist-packages/paste/httpserver.py", line 442, in handle
BaseHTTPRequestHandler.handle(self)
File "/usr/lib/python2.7/BaseHTTPServer.py", line 340, in handle
self.handle_one_request()
File "/usr/lib/python2.7/dist-packages/paste/httpserver.py", line 437, in handle_one_request
self.wsgi_execute()
File "/usr/lib/python2.7/dist-packages/paste/httpserver.py", line 287, in wsgi_execute
self.wsgi_start_response)
File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
resp = self.call_func(req, *args, **self.kwargs)
File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
return self.func(req, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/zuul/webapp.py", line 130, in app
'must-revalidate',
NameError: global name 'reponse' is not definedSeems unrelated to the python-statsd upgrade, maybe some other module has been upgraded.