Page MenuHomePhabricator

Usernames containing unicode characters unable to authenticate to Wikitech and Horizon
Closed, ResolvedPublic

Description

Creating new accounts with Unicode characters in the wiki username succeeds on Wikitech and leaves the user logged in and able to edit. If the user ever becomes logged out (because of session expiration or deliberate logout), they will be unable to log back in.

Verified by creating and using User:Tést őf Unícødë įn Usērnǎmė. Account creation worked as expected. Subsequent login attempts fail. The failures are caused by Unicode string handling failures in our OpenStack Keystone service.

Partial workaround: password reset will let you log into Wikitech (since it completely circumvents LDAP authentication). Does not help with Horizon.


When I try to log into Wikitech as Gergő Tisza, I get Incorrect username or password entered. (it doesn't even ask for the 2FA token, so it's not an issue with that). On Horizon it's An error occurred authenticating. Please try again later. On other LDAP-login-based services like Logstash the same password works, though. I haven't changed the password lately; I don't think somebody else changed it either, since that would log me out of Wikitech, which did not happen.

Event Timeline

aborrero moved this task from Inbox to Soon! on the cloud-services-team (Kanban) board.
aborrero added subscribers: Andrew, bd808, aborrero.

@Tgr do you have any idea when things last worked as expected for your logins to Wikitech and/or Horizon? The last major change that I know of to the Wikitech LDAPAuthentication configuration was in March when we started using cn:caseExactMatch: in lookups for T165795: Ldap auth extension vs. ldap vs. username Case.

A "cn:caseExactMatch:=Gergő Tisza" lookup seems to work as expected from the cli inside a Cloud VPS project:

$ ldapsearch -xLLL -P 3 -b"dc=wikimedia,dc=org" "cn:caseExactMatch:=Gergő Tisza" dn
dn: uid=tgr,ou=people,dc=wikimedia,dc=org

All I can find in the ELK cluster for Wikitech login failures related to this ticket is 2 occurrences of "Incorrect username or password entered.". One was recorded at 2019-07-29T13:22:18 and the other at 2019-07-29T13:24:06.

We don't aggrigate Horizon's logs to the ELK cluster, so I had to go poking around on the labweb* servers themselves to find:

2019-07-29 12:56:06.068920 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.0.130.
2019-07-29 12:57:40.793150 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.16.22.
2019-07-29 13:01:49.076918 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.48.101.
2019-07-29 13:02:30.613885 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.0.132.
2019-07-29 13:13:13.381522 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.32.69.
2019-07-29 13:13:43.783730 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.32.69.
$ python2 -c 'print "Gerg\xc5\x91 Tisza".decode("utf8")'
Gergő Tisza

Its looking like we may have to coordinate setting louder than normal logging for one or both of Wikitech and Horizon to get a useful error message here.

@Tgr do you have any idea when things last worked as expected for your logins to Wikitech and/or Horizon?

I don't. It's entirely possible that I haven't used it since March: Wikitech sessions are long-lived and I haven't used Cloud VPS recently. I did use Striker, but that one seems unaffected by this problem even now.

drive-by: If striker works but wikitech/Horizon don't then my first guess would be differing levels of case-sensitivity

I'm getting the same error as well. Wikitech automatically logs me out annoyingly often (like after 1 or 2 days), even though I always check the "remember me" option. Every time I try to log in, it says my password is invalid, and I have to reset it.

My username is "Jon Harald Søby", so it may be a Unicode issue somehow? (Seeing as Tgr's username also uses non-ASCII characters.)

bd808 renamed this task from Can't log into Wikitech/Horizon (as Gergő Tisza) to Usernames containing unicode characters unable to authenticate to Wikitech.Oct 21 2019, 9:07 PM
bd808 updated the task description. (Show Details)

The Tést őf Unícødë įn Usērnǎmė gives us something to play with. At least using command line tools it appears that cn:caseExactMatch: works fine with the username:

$ ldap 'cn:caseExactMatch:=Tést őf Unícødë įn Usērnǎmė'
dn: uid=unicodetest,ou=people,dc=wikimedia,dc=org
uid: unicodetest
sn:: VMOpc3QgxZFmIFVuw61jw7hkw6sgxK9uIFVzxJNybseObcSX
cn:: VMOpc3QgxZFmIFVuw61jw7hkw6sgxK9uIFVzxJNybseObcSX
objectClass: inetOrgPerson
objectClass: person
objectClass: ldapPublicKey
objectClass: posixAccount
objectClass: shadowAccount
uidNumber: 22360
gidNumber: 500
homeDirectory: /home/unicodetest
loginShell: /bin/bash
mail: bdavis+T229227@wikimedia.org

# pagedresults: cookie=

The funny sn:: and cn:: in the output above is how ldapsearch returns non-ascii results. These are base64 encoded values and the :: is the indicator that base64 is in use:

$ echo VMOpc3QgxZFmIFVuw61jw7hkw6sgxK9uIFVzxJNybseObcSX | base64 -d; echo
Tést őf Unícødë įn Usērnǎmė

I have some aliases in my shell that make this easier to work with:

$ ldap 'cn:caseExactMatch:=Tést őf Unícødë įn Usērnǎmė' cn sn | un64
dn: uid=unicodetest,ou=people,dc=wikimedia,dc=org
sn:: "Tést őf Unícødë įn Usērnǎmė"
cn:: "Tést őf Unícødë įn Usērnǎmė"

# pagedresults: cookie=
$ type -a ldap
ldap is aliased to `ldapsearch -xLLL -P 3 -E pr=40000/noprompt -o ldif-wrap=no -b"dc=wikimedia,dc=org"'
$ type -a un64
un64 is aliased to `awk 'BEGIN{FS=":: ";c="base64 -d"}{if(/\w+:: /) {print $2 |& c; close(c,"to"); c |& getline $2; close(c); printf("%s:: \"%s\"\n", $1, $2); next} print $0 }''

Some debugging using mwscript eval.php --wiki=labswiki -d 2 from labweb1001 seems to indicate that the auth attempt fails because of something in the OpenStack integration that is added to Wikitech by Extension:OpenStackManager.

$ mwscript eval.php --wiki=labswiki -d 2
[debug] [memcached] MemcachedPeclBagOStuff::initializeClient: initializing new c
lient instance.
[debug] [memcached] MainObjectStash using store MemcachedPeclBagOStuff
[debug] [memcached] MemcachedPeclBagOStuff::initializeClient: initializing new c
lient instance.
[debug] [objectcache] MainWANObjectCache using store MemcachedPeclBagOStuff
[debug] [DBReplication] Wikimedia\Rdbms\LBFactory::getChronologyProtector: reque
st info {
    "IPAddress": "",
    "UserAgent": "",
    "ChronologyProtection": null,
    "ChronologyClientId": null,
    "ChronologyPositionIndex": null
}
[debug] [DBConnection] Wikimedia\Rdbms\LoadBalancer::lazyLoadReplicationPosition
s: executed chronology callback.
[debug] [DBConnection] Wikimedia\Rdbms\LoadBalancer::getLocalConnection: opened
new connection for 0
> $wgLDAPDebug = 5;

> $ldap = LdapAuthenticationPlugin::getInstance();

> $username = 'Tést őf Unícødë įn Usērnǎmė';

> $password = 'the correct password was used outside of this paste -- bd808';

> $authed = $ldap->authenticate( $username, $password );
[info] [ldap] 2.1.0 Entering authenticate for username Tést őf Unícødë įn Usērnǎmė
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Entering Connect
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Using TLS or not using encryption.
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Using servers: ldap://ldap-labs.eqiad.wikimedia.org:389
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Using TLS
[info] [ldap] 2.1.0 PHP's LDAP connect method returned true (note, this does not imply it connected to the server).
[info] [ldap] 2.1.0 Entering getSearchString
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Entering getUserDN
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Doing a proxy bind
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Created a regular filter: (cn:caseExactMatch:=Tést őf Unícød
ë įn Usērnǎmė)
[info] [ldap] 2.1.0 Entering getBaseDN
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 basedn is ou=people,dc=wikimedia,dc=org
[info] [ldap] 2.1.0 Using base: ou=people,dc=wikimedia,dc=org
[info] [ldap] 2.1.0 userdn is: uid=unicodetest,ou=people,dc=wikimedia,dc=org
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Binding as the user
[info] [ldap] 2.1.0 Entering Connect
[info] [ldap] 2.1.0 Using TLS or not using encryption.
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Using servers: ldap://ldap-labs.eqiad.wikimedia.org:389
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Using TLS
[info] [ldap] 2.1.0 PHP's LDAP connect method returned true (note, this does not imply it connected to the server).
[info] [ldap] 2.1.0 Entering getUserDN
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 Created a regular filter: (cn:caseExactMatch:=Tést őf Unícødë įn Usērnǎmė)
[info] [ldap] 2.1.0 Entering getBaseDN
[info] [ldap] 2.1.0 Entering getDomain
[info] [ldap] 2.1.0 basedn is ou=people,dc=wikimedia,dc=org
[info] [ldap] 2.1.0 Using base: ou=people,dc=wikimedia,dc=org
[info] [ldap] 2.1.0 Fetching userdn using username: uid=unicodetest,ou=people,dc=wikimedia,dc=org
[info] [ldap] 2.1.0 Entering OpenStackNovaController::authenticate
[info] [ldap] 2.1.0 OpenStackNovaController::restCall fullurl: http://cloudcontrol1003.wikimedia.org:35357/v3/auth/tokens
[info] [ldap] 2.1.0 OpenStackNovaController::authenticate return code: 500

> var_dump( $authed );
bool(false)

The important part of the trace logging above is:

[info] [ldap] 2.1.0 Entering OpenStackNovaController::authenticate
[info] [ldap] 2.1.0 OpenStackNovaController::restCall fullurl: http://cloudcontrol1003.wikimedia.org:35357/v3/auth/tokens
[info] [ldap] 2.1.0 OpenStackNovaController::authenticate return code: 500

This is Hooks::run( 'ChainAuth', [ $username, $password, &$result ] ); running in LdapAuthenticationPlugin::authenticate() after the LDAP auth bind has succeeded.

So what is going on inside the hook? That is the work of OpenStackNovaUser::ChainAuth( $username, $password, &$result ).

⠀⠀⠀static function ChainAuth( $username, $password, &$result ) {
⠀⠀⠀ ⠀⠀⠀$user = new OpenStackNovaUser( $username );
⠀⠀⠀ ⠀⠀⠀$userNova = OpenStackNovaController::newFromUser( $user );
⠀⠀⠀ ⠀⠀⠀$token = $userNova->authenticate( $username, $password );
⠀⠀⠀ ⠀⠀⠀if ( $token ) {
⠀⠀⠀ ⠀⠀⠀ ⠀⠀⠀$result = true;
⠀⠀⠀ ⠀⠀⠀ ⠀⠀⠀# Add token to session, so that it can be referenced later
⠀⠀⠀ ⠀⠀⠀ ⠀⠀⠀$_SESSION['wsOpenStackToken'] = $token;
⠀⠀⠀ ⠀⠀⠀} else {
⠀⠀⠀ ⠀⠀⠀ ⠀⠀⠀$result = false;
⠀⠀⠀ ⠀⠀⠀}

⠀⠀⠀ ⠀⠀⠀return $result;
⠀⠀⠀}

Recreating the 500 error on the OpenStack side is pretty easy:

$ mwscript eval.php --wiki=labswiki
> $ldap = LdapAuthenticationPlugin::getInstance();

> $username = 'Tést őf Unícødë įn Usērnǎmė';

> $user = new OpenStackNovaUser( $username );

> $userNova = OpenStackNovaController::newFromUser( $user );

> $password = 'the correct password was used outside of this paste -- bd808';

> $wgLDAPDebug = 5;

> $token = $userNova->authenticate( $username, $password );
[info] [ldap] 2.1.0 Entering OpenStackNovaController::authenticate
[info] [ldap] 2.1.0 OpenStackNovaController::restCall fullurl: http://cloudcontrol1003.wikimedia.org:35357/v3/auth/tokens
[info] [ldap] 2.1.0 OpenStackNovaController::authenticate return code: 500

Hopping over to cloudcontrol1003.wikimedia.org and checking in /var/log/keystone/keystone.log shows us the problem with the authentication on the Keystone server:

(keystone.common.wsgi): 2019-10-21 22:11:24,895 ERROR 'ascii' codec can't encode character u'\xe9' in position 7: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 225, in __call__
    result = method(req, **params)
  File "/usr/lib/python2.7/dist-packages/keystone/auth/controllers.py", line 401, in authenticate_for_token
    self.authenticate(request, auth_info, auth_context)
  File "/usr/lib/python2.7/dist-packages/keystone/auth/controllers.py", line 528, in authenticate
    auth_context)
  File "/usr/lib/python2.7/dist-packages/wmfkeystoneauth/password_whitelist.py", line 66, in authenticate
    user_info = auth_plugins.UserAuthInfo.create(auth_payload, METHOD_NAME)
  File "/usr/lib/python2.7/dist-packages/keystone/auth/plugins/core.py", line 107, in create
    user_auth_info._validate_and_normalize_auth_data(auth_payload)
  File "/usr/lib/python2.7/dist-packages/keystone/auth/plugins/core.py", line 196, in _validate_and_normalize_auth_data
    auth_payload)
  File "/usr/lib/python2.7/dist-packages/keystone/auth/plugins/core.py", line 173, in _validate_and_normalize_auth_data
    user_name, domain_ref['id'])
  File "/usr/lib/python2.7/dist-packages/keystone/common/manager.py", line 124, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/core.py", line 416, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/core.py", line 426, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 1220, in decorate
    should_cache_fn)
  File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 825, in get_or_create
    async_creator) as value:
  File "/usr/lib/python2.7/dist-packages/dogpile/lock.py", line 154, in __enter__
    return self._enter()
  File "/usr/lib/python2.7/dist-packages/dogpile/lock.py", line 94, in _enter
    generated = self._enter_create(createdtime)
  File "/usr/lib/python2.7/dist-packages/dogpile/lock.py", line 145, in _enter_create
    created = self.creator()
  File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 792, in gen_value
    created_value = creator()
  File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 1216, in creator
    return fn(*arg, **kw)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/core.py", line 908, in get_user_by_name
    ref = driver.get_user_by_name(user_name, domain_id)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/backends/ldap/core.py", line 89, in get_user_by_name
    return self.user.filter_attributes(self.user.get_by_name(user_name))
  File "/usr/lib/python2.7/dist-packages/keystone/identity/backends/ldap/common.py", line 1528, in get_by_name
    res = self.get_all(query)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/backends/ldap/common.py", line 1930, in get_all
    return super(EnabledEmuMixIn, self).get_all(ldap_filter, hints)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/backends/ldap/common.py", line 1537, in get_all
    for x in self._ldap_get_all(hints, ldap_filter)]
  File "/usr/lib/python2.7/dist-packages/keystone/common/driver_hints.py", line 42, in wrapper
    return f(self, hints, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/backends/ldap/common.py", line 1498, in _ldap_get_all
    attrs)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/backends/ldap/common.py", line 947, in search_s
    attrlist_utf8, attrsonly)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/backends/ldap/common.py", line 640, in wrapper
    return func(self, conn, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/identity/backends/ldap/common.py", line 769, in search_s
    attrsonly)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 768, in search_s
    return self.search_ext_s(base,scope,filterstr,attrlist,attrsonly,None,None,timeout=self.timeout)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 1134, in search_ext_s
    return self._apply_method_s(SimpleLDAPObject.search_ext_s,*args,**kwargs)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 1071, in _apply_method_s
    return func(self,*args,**kwargs)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 761, in search_ext_s
    msgid = self.search_ext(base,scope,filterstr,attrlist,attrsonly,serverctrls,clientctrls,timeout,sizelimit)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 757, in search_ext
    timeout,sizelimit,
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 263, in _ldap_call
    result = func(*args,**kwargs)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 7: ordinal not in range(128)

This Python Unicode handling error points to this being another case of T234834: Various user visible errors in Cloud VPS projects following OpenStack upgrade on 2019-10-07 where the regression was caused by our upgrade from OpenStack "M" to "N".

bd808 renamed this task from Usernames containing unicode characters unable to authenticate to Wikitech to Usernames containing unicode characters unable to authenticate to Wikitech and Horizon.Oct 21 2019, 11:09 PM

This Python Unicode handling error points to this being another case of T234834: Various user visible errors in Cloud VPS projects following OpenStack upgrade on 2019-10-07 where the regression was caused by our upgrade from OpenStack "M" to "N".

The original bug report here was made before we upgraded to N, so the regression happened before then.

We're running python-pyldap 2.4.25.1-2. The issue is presumed fixed in 2.4.36 (and later). It's also probably fixed in python-ldap which provides the same bindings. So some options are:

@arturo, how would you feel about backporting and repo-ing python-pyldap2.4.45 for Stretch? (or 2.4.36 or 2.4.37, whichever is least disruptive to the dependency chain)

There is some weirdness going on in this package in Debian. A couple of src packages involved (python-ldap and python-pyldap) and apparently src:python-pyldap no longer exists. But src:python-ldap apparently dropped support for py2.
I need more time to focus and investigate.

You're probably lost in the same weeds I'm lost in. python-pyldap was a fork of python-ldap, and then the two merged back again. And in theory they've always provided the same bindings. So it would be fine to use python-ldap instead of python-pyldap, except then we'll be stuck fighting the dependencies in the python-keystone package.

This problem should be fixed for wikitech the next time the deployment train runs there (probably 2019-11-20). This was accomplished by removing the remaining integration between MediaWiki-extensions-OpenStackManager and our OpenStack Keystone service.

The problem will remain for Horizon until we can get Keystone in the eqiad1 deployment using a newer version of python-pyldap/python-ldap.

This is worth retesting -- lots of upgrades recently.

Keystone patch 'eca0829c4c65e6b64f08023ce2d5a55dc329248f' at least claims to address this issue (it's mostly about python3 but includes a bit for python2 as well).

A single line extracted from the above patch, plus version 3 of python-ldap, gets us working:

self.conn = ldap.initialize(url, bytes_mode=False)

Maybe we can monkey-patch that in until Rocky when it's fixed upstream

This is worth retesting -- lots of upgrades recently.

Still not working in its current unpatched state.

I've also just encountered this issue trying to log in to Horizon (for the first time), with the user name "Bartosz Dziewoński" :(

Change 560360 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Openstack keystone (ocata/stretch): pull in python-ldap from the buster repo

https://gerrit.wikimedia.org/r/560360

Change 560366 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Keystone hooks: monkeypatch ldap.initialize to set bytes_mode=false

https://gerrit.wikimedia.org/r/560366

Change 560360 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] Openstack keystone (ocata/stretch): pull in python-ldap v3

https://gerrit.wikimedia.org/r/560360

Mentioned in SAL (#wikimedia-operations) [2019-12-23T10:47:50Z] <arturo> import python-ldap 3.1.0-2~bpo9+1~wmf1 into stretch-wikimedia/component/python-ldap-bpo (T229227)

Mentioned in SAL (#wikimedia-operations) [2019-12-23T11:05:17Z] <arturo> import python-pyasn1 0.4.2-3~bpo9+1~wmf1 into stretch-wikimedia/component/python-ldap-bpo (T229227)

Packages are in the repo and are ready to install:

aborrero@cloudcontrol2003-dev:~ 4s $ sudo apt-get install python-ldap
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  kpartx libxmlsec1 libxmlsec1-openssl python-alabaster python-defusedxml python-imagesize python-libvirt python-pam python-passlib
  python-pysaml2 python-repoze.who python-sphinx python-zope.interface sphinx-common vlan websockify xmlsec1
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  python-pyasn1 python-pyasn1-modules
The following NEW packages will be installed:
  python-pyasn1-modules
The following packages will be upgraded:
  python-ldap python-pyasn1
2 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 191 kB of archives.
After this operation, 288 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://apt.wikimedia.org/wikimedia stretch-wikimedia/component/python-ldap-bpo amd64 python-pyasn1 all 0.4.2-3~bpo9+1~wmf1 [57.7 kB]
Get:2 http://mirrors.wikimedia.org/debian stretch/main amd64 python-pyasn1-modules all 0.0.7-0.1 [21.5 kB]
Get:3 http://apt.wikimedia.org/wikimedia stretch-wikimedia/component/python-ldap-bpo amd64 python-ldap amd64 3.1.0-2~bpo9+1~wmf1 [112 kB]
Fetched 191 kB in 0s (844 kB/s)      
No directory, logging in with HOME=/
INFO:debmonitor:Got 3 updates from dpkg hook version 3
INFO:debmonitor:Successfully sent the dpkg_hook update to the DebMonitor server
(Reading database ... 67698 files and directories currently installed.)
Preparing to unpack .../python-pyasn1_0.4.2-3~bpo9+1~wmf1_all.deb ...
Unpacking python-pyasn1 (0.4.2-3~bpo9+1~wmf1) over (0.1.9-2) ...
Selecting previously unselected package python-pyasn1-modules.
Preparing to unpack .../python-pyasn1-modules_0.0.7-0.1_all.deb ...
Unpacking python-pyasn1-modules (0.0.7-0.1) ...
Preparing to unpack .../python-ldap_3.1.0-2~bpo9+1~wmf1_amd64.deb ...
Unpacking python-ldap:amd64 (3.1.0-2~bpo9+1~wmf1) over (2.4.28-0.1) ...
Setting up python-pyasn1 (0.4.2-3~bpo9+1~wmf1) ...
Setting up python-pyasn1-modules (0.0.7-0.1) ...
Setting up python-ldap:amd64 (3.1.0-2~bpo9+1~wmf1) ...

Change 560366 abandoned by Andrew Bogott:
Keystone hooks: monkeypatch ldap.initialize to set bytes_mode=false

Reason:
This step turns out to be unneeded; upgrading python-ldap got things working.

https://gerrit.wikimedia.org/r/560366

Our unicode test account can now log in via both Horizon and Wikitech (once I configured 2fa for Horizon.) @Tgr, please re-open if you find otherwise.

I can indeed log in now. Thanks!