On kafka metadata change (e.g. broker restart), The EventLogging kafka consumer seems to die and not come back up:
2015-11-10 20:15:36,010 (Thread-13 ) Autocommitting consumer offset for consumer group mysql-m4-master and topic eventlogging-valid-mixed Exception in thread Thread-12: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 763, in run self.__target(*self.__args, **self.__kwargs) File "/usr/lib/python2.7/dist-packages/pykafka/simpleconsumer.py", line 269, in fetcher self.fetch() File "/usr/lib/python2.7/dist-packages/pykafka/simpleconsumer.py", line 500, in fetch min_bytes=self._fetch_min_bytes File "/usr/lib/python2.7/dist-packages/pykafka/broker.py", line 222, in fetch_messages return future.get(FetchResponse) File "/usr/lib/python2.7/dist-packages/pykafka/handlers.py", line 55, in get raise self.error SocketDisconnectedError
After this, the offset commit thread continues to run, but no new data is consumed from Kafka.