While deploying we found by accident warnings in the mysql consumer log that say "Invalid string" followed by what look like unicode characters (eg. \xF0)
Steps taken to look for the problem:
- looked through past 7 days of logs for "Invalid string" and found nothing
- looked through git history around the time this bug was filed, checked the mysql handler and found no differences that would explain lack of logging
- made an input file with äê âain it and fed it to the consumer, it barfed with:
Traceback (most recent call last): File "eventlogging-consumer", line 53, in <module> eventlogging.drive(args.input, args.output) File "/home/otto/EventLogging/server/eventlogging/factory.py", line 123, in drive for event in reader: File "/home/otto/EventLogging/server/eventlogging/streams.py", line 100, in <genexpr> return (json.loads(dgram) for dgram in iter_unicode(stream)) File "/usr/lib/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode obj, end = self.scan_once(s, idx) ValueError: Invalid control character at: line 1 column 615 (char 614)
- The above error makes sense, because literal tabs are not allowed in JSON. I removed the tab and left the unicode and weirdo characters in there, and everything was fine.
Conclusion: EL consumer is fine and this bug does not appear to be happening any more.