The ircecho bot (icinga-wm) quit the #wikimedia-operations channel with:
icinga-wm quit (Reason: Ping timeout: 256 seconds)
Checking it's status on einsteinium I found that it had thrown an exception few minutes before, so I restarted it. Here's the stack trace:
Dec 21 16:04:42 einsteinium ircecho[16665]: *** Connecting to IRC server chat.freenode.net... Dec 21 16:04:42 einsteinium ircecho[16665]: Using infile Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc-analytics.log Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc-fundraising.log Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc-interactive.log Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc-labs.log Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc-ores.log Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc-performance.log Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc-reading-web.log Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc-releng.log Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc-wikidata.log Dec 21 16:04:42 einsteinium ircecho[16665]: Opening: /var/log/icinga/irc.log Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: Starting notifier loop Dec 21 16:04:42 einsteinium ircecho[16665]: *** Connected Dec 21 16:04:42 einsteinium ircecho[16665]: Traceback (most recent call last): Dec 21 16:04:42 einsteinium ircecho[16665]: File "/usr/local/bin/ircecho", line 190, in <module> Dec 21 16:04:42 einsteinium ircecho[16665]: bot.start() Dec 21 16:04:42 einsteinium ircecho[16665]: File "/usr/lib/python2.7/dist-packages/irc/bot.py", line 266, in start Dec 21 16:04:42 einsteinium ircecho[16665]: super(SingleServerIRCBot, self).start() Dec 21 16:04:42 einsteinium ircecho[16665]: File "/usr/lib/python2.7/dist-packages/irc/client.py", line 1221, in start Dec 21 16:04:42 einsteinium ircecho[16665]: self.ircobj.process_forever() Dec 21 16:04:42 einsteinium ircecho[16665]: File "/usr/lib/python2.7/dist-packages/irc/client.py", line 267, in process_forever Dec 21 16:04:42 einsteinium ircecho[16665]: self.process_once(timeout) Dec 21 16:04:42 einsteinium ircecho[16665]: File "/usr/lib/python2.7/dist-packages/irc/client.py", line 248, in process_once Dec 21 16:04:42 einsteinium ircecho[16665]: self.process_data(i) Dec 21 16:04:42 einsteinium ircecho[16665]: File "/usr/lib/python2.7/dist-packages/irc/client.py", line 213, in process_data Dec 21 16:04:42 einsteinium ircecho[16665]: c.process_data() Dec 21 16:04:42 einsteinium ircecho[16665]: File "/usr/lib/python2.7/dist-packages/irc/client.py", line 557, in process_data Dec 21 16:04:42 einsteinium ircecho[16665]: for line in self.buffer: Dec 21 16:04:42 einsteinium ircecho[16665]: File "/usr/lib/python2.7/dist-packages/irc/buffer.py", line 84, in <genexpr> Dec 21 16:04:42 einsteinium ircecho[16665]: for line in super(DecodingLineBuffer, self).lines()) Dec 21 16:04:42 einsteinium ircecho[16665]: File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode Dec 21 16:04:42 einsteinium ircecho[16665]: return codecs.utf_8_decode(input, errors, True) Dec 21 16:04:42 einsteinium ircecho[16665]: UnicodeDecodeError: 'utf8' codec can't decode byte 0x93 in position 100: invalid start byte