Page MenuHomePhabricator

prometheus-openldap-exporter: Request.write called on a request after Request.finish was called
Open, NormalPublic

Description

This has been going on in serpens and seaborgium for a while now:

ar  6 12:40:01 serpens CRON[3495]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]:  [twisted.python.log#info] 10.192.16.189 - - [06/Mar/2019:12:40:11 +0000] "GET /metrics HTTP/1.1" 200 1290 "-" "Prometheus/1.8.1+ds+k8s"
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]:  [twisted.internet.defer#critical] Unhandled error in Deferred:
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]:  [twisted.internet.defer#critical]
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011Traceback (most recent call last):
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011  File "/usr/lib/python2.7/dist-packages/ldaptor/protocols/ldap/ldapclient.py", line 171, in handle
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011    if handler(msg.value, *args, **kwargs):
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011  File "/usr/lib/python2.7/dist-packages/ldaptor/protocols/ldap/ldapsyntax.py", line 628, in _cbSearchMsg
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011    d.callback(None)
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 457, in callback
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011    self._startRunCallbacks(result)
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 565, in _startRunCallbacks
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011    self._runCallbacks()
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011--- <exception caught here> ---
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 651, in _runCallbacks
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011    current.result = callback(current.result, *args, **kw)
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011  File "/usr/bin/prometheus-openldap-exporter", line 115, in gotResultsOperations
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011    self.request.write('openldap_monitored_op{{{}}} {}\n'.format(labels, value).encode('utf-8'))
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011  File "/usr/lib/python2.7/dist-packages/twisted/web/server.py", line 220, in write
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011    http.Request.write(self, data)
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011  File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 896, in write
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011    raise RuntimeError('Request.write called on a request after '
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011exceptions.RuntimeError: Request.write called on a request after Request.finish was called.
Mar  6 12:40:11 serpens prometheus-openldap-exporter[429]: #011

Event Timeline

GTirloni created this task.Mar 6 2019, 12:44 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 6 2019, 12:44 PM
jbond triaged this task as Normal priority.Mar 7 2019, 1:21 PM
GTirloni removed a subscriber: GTirloni.Mar 21 2019, 9:06 PM
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptMar 21 2019, 9:06 PM