Page MenuHomePhabricator

EventLogging mysql consumer cannot insert events that have a nested json schema that includes a plain "array" {oryx} [5 pts]
Closed, ResolvedPublic

Description

We ran into this problem while backfilling, we can't continue backfilling until this is fixed.

The error was (sanitized for posting):

/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:459: SAWarning: Unicode type received non-unicode bind param value
  param.append(processors[key](compiled_params[key]))
Traceback (most recent call last):
  File "eventlogging-consumer", line 50, in <module>
    eventlogging.drive(args.input, args.output)
  File "/usr/local/lib/python2.7/dist-packages/eventlogging-0.9_20151021-py2.7.egg/eventlogging/factory.py", line 124, in drive
    writer.send(event)
  File "/usr/lib/python2.7/contextlib.py", line 154, in __exit__
    self.thing.close()
  File "/usr/local/lib/python2.7/dist-packages/eventlogging-0.9_20151021-py2.7.egg/eventlogging/handlers.py", line 315, in sql_writer
    on_insert_callback=partial(insert_stats, stats))
  File "/usr/local/lib/python2.7/dist-packages/eventlogging-0.9_20151021-py2.7.egg/eventlogging/jrm.py", line 250, in store_sql_events
    insert(table, events, replace)
  File "/usr/local/lib/python2.7/dist-packages/eventlogging-0.9_20151021-py2.7.egg/eventlogging/jrm.py", line 217, in _insert_multi
    insert.execute()
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/expression.py", line 3147, in execute
    return e._execute_clauseelement(self, multiparams, params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1621, in _execute_clauseelement
    return connection._execute_clauseelement(elem, multiparams, params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 874, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1024, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 196, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 867, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 324, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1064, 'You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near \': "\'5.3\'", u\'b\': "\'<<stripped - was a coded string of alpha numeric info with underscores>>\'", u\'t\': \'1444812798\'},), 1, \' at line 1') 'INSERT IGNORE INTO `CentralNoticeBannerHistory_13447710` (uuid, `clientIp`, timestamp, `userAgent`, `webHost`, wiki, event_i, event_l, event_n, event_r) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)' (u'e5e06ed2a2735e4ca75da3612df15528', '<<client Ip hash stripped by Dan when making the task>>', '20151014085337', '"<<user agent stripped by Dan when making the task>>"', 'en.m.wikipedia.org', 'enwiki', '<<event_i stripped by Dan when making the task (it was an alpha-numeric string)>>', [{u's': u'5.3', u'b': u'<<stripped - was a coded string of alpha numeric info with underscores>>', u't': 1444812798}], 1, 0.01)

Event Timeline

Milimetric claimed this task.
Milimetric raised the priority of this task from to Needs Triage.
Milimetric updated the task description. (Show Details)
Milimetric added a project: Analytics-Kanban.
Milimetric added a subscriber: Milimetric.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 21 2015, 10:51 PM

Change 248045 had a related patch set uploaded (by Milimetric):
Exclude CentralNoticeBannerHistory from mysql

https://gerrit.wikimedia.org/r/248045

Nuria renamed this task from EventLogging mysql consumer can be killed by a bad event to EventLogging mysql consumer can be killed by a bad event with a nested array.Oct 22 2015, 3:21 PM
Nuria renamed this task from EventLogging mysql consumer can be killed by a bad event with a nested array to EventLogging mysql consumer cannot insert events that have a nested json schema that includes a plain "array".
Nuria set Security to None.

Change 248045 merged by Ottomata:
Exclude CentralNoticeBannerHistory from mysql

https://gerrit.wikimedia.org/r/248045

Milimetric moved this task from In Progress to Done on the Analytics-Kanban board.Oct 22 2015, 6:30 PM
Milimetric renamed this task from EventLogging mysql consumer cannot insert events that have a nested json schema that includes a plain "array" to EventLogging mysql consumer cannot insert events that have a nested json schema that includes a plain "array" {oryx} [5 pts].Oct 23 2015, 4:04 PM
Nuria closed this task as Resolved.Oct 23 2015, 4:12 PM