Page MenuHomePhabricator

"500 : Internal Server Error ordinal not in range(256)" when logging in with OAuth
Closed, ResolvedPublic

Description

I tried to log into https://paws.wmflabs.org/paws/hub/login with my username ("Vojtěch Dostál") using the classical OAuth procedure.

I get this error:

Event Timeline

Hi @Vojtech.dostal, sorry this eluded my radar. Are you still having trouble?

Yes, the problem persists :)

Chicocvenancio triaged this task as Normal priority.Dec 20 2018, 7:41 PM

Checking.

Thanks for looking into it .

Chicocvenancio raised the priority of this task from Normal to Unbreak Now!.EditedDec 20 2018, 7:47 PM

Seems to be encoded for Latin-1 for some reason. This is very weird and probably means it is affecting all users with non-latin characters, not sure when it broke as I have not looked into this for a few months.

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tornado/web.py", line 1543, in _execute
    result = yield result
  File "/usr/local/lib/python3.6/dist-packages/oauthenticator/oauth2.py", line 182, in get
    user = yield self.login_user()
  File "/usr/local/lib/python3.6/dist-packages/jupyterhub/handlers/base.py", line 480, in login_user
    user = self.user_from_username(username)
  File "/usr/local/lib/python3.6/dist-packages/jupyterhub/handlers/base.py", line 291, in user_from_username
    user = self.find_user(username)
  File "/usr/local/lib/python3.6/dist-packages/jupyterhub/handlers/base.py", line 286, in find_user
    orm_user = orm.User.find(db=self.db, name=name)
  File "/usr/local/lib/python3.6/dist-packages/jupyterhub/orm.py", line 194, in find
    return db.query(cls).filter(cls.name == name).first()
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 2835, in first
    ret = list(self[0:1])
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 2627, in __getitem__
    return list(res)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 2935, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 2958, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1416, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 508, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.6/dist-packages/pymysql/cursors.py", line 166, in execute
    result = self._query(query)
  File "/usr/local/lib/python3.6/dist-packages/pymysql/cursors.py", line 322, in _query
    conn.query(q)
  File "/usr/local/lib/python3.6/dist-packages/pymysql/connections.py", line 854, in query
    sql = sql.encode(self.encoding, 'surrogateescape')
UnicodeEncodeError: 'latin-1' codec can't encode character '\u011b' in position 319: ordinal not in range(256)
Restricted Application added subscribers: Liuxinyu970226, TerraCodes. · View Herald TranscriptDec 20 2018, 7:47 PM

All tables have the correct collation, problem is not at DB level.

MariaDB [s52771__paws]> show table status;
+---------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+--------------------+---------+
| Name                | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time | Check_time | Collation          | Checksum | Create_options     | Comment |
+---------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+--------------------+---------+
| alembic_version     | InnoDB |      10 | Compact    |    1 |          16384 |       16384 |               0 |            0 |         0 |           NULL | 2018-05-19 19:04:51 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL |                    |         |
| api_tokens          | InnoDB |      10 | Dynamic    |   50 |            327 |       16384 |               0 |        65536 |         0 |           3823 | 2018-06-21 21:15:31 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
| groups              | InnoDB |      10 | Dynamic    |    0 |              0 |       16384 |               0 |        16384 |         0 |              1 | 2018-05-19 19:04:51 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
| oauth_access_tokens | InnoDB |      10 | Dynamic    |  360 |            455 |      163840 |               0 |       147456 |         0 |           3888 | 2018-06-21 21:15:30 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
| oauth_clients       | InnoDB |      10 | Dynamic    |  254 |            516 |      131072 |               0 |        16384 |         0 |           3800 | 2018-06-21 21:15:31 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
| oauth_codes         | InnoDB |      10 | Dynamic    |   60 |            273 |       16384 |               0 |        32768 |         0 |           5372 | 2018-06-21 21:15:30 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
| servers             | InnoDB |      10 | Dynamic    |    5 |           3276 |       16384 |               0 |            0 |         0 |           3800 | 2018-05-19 19:04:51 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
| services            | InnoDB |      10 | Dynamic    |    0 |              0 |       16384 |               0 |        32768 |         0 |              2 | 2018-05-19 19:04:52 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
| spawners            | InnoDB |      10 | Dynamic    |  573 |            142 |       81920 |               0 |        32768 |         0 |            575 | 2018-06-21 21:15:31 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
| user_group_map      | InnoDB |      10 | Dynamic    |    0 |              0 |       16384 |               0 |        16384 |         0 |           NULL | 2018-05-19 19:04:51 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
| users               | InnoDB |      10 | Dynamic    |  500 |           3178 |     1589248 |               0 |        65536 |   4194304 |            575 | 2018-06-21 21:15:31 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL | row_format=DYNAMIC |         |
+---------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+--------------------+---------+

Pushed a probable fix just now. I'll create a user with non-Latin characters and test it.

Chicocvenancio closed this task as Resolved.Dec 20 2018, 8:27 PM
Chicocvenancio claimed this task.

Seems all is good now. Please test @Vojtech.dostal and reopen if necessary. For the record, this was due to SQLAlchemy and the solution was to add ?charset=utf8mb4 to the connection URL (it is encrypted in the repo but this was the commit).

Sorry for not catching this back in May when we moved to MariaDB (from sqlite).

It seems to be working all right! Sorry for not replying earlier.