Page MenuHomePhabricator

Can't log into Wikitech/Horizon (as Gergő Tisza)
Open, HighPublic

Description

When I try to log into Wikitech as Gergő Tisza, I get Incorrect username or password entered. (it doesn't even ask for the 2FA token, so it's not an issue with that). On Horizon it's An error occurred authenticating. Please try again later. On other LDAP-login-based services like Logstash the same password works, though. I haven't changed the password lately; I don't think somebody else changed it either, since that would log me out of Wikitech, which did not happen.

Event Timeline

Tgr created this task.Jul 29 2019, 1:31 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 29 2019, 1:31 PM
aborrero triaged this task as High priority.Jul 29 2019, 1:32 PM
aborrero moved this task from Inbox to Important on the cloud-services-team (Kanban) board.
aborrero added subscribers: Andrew, bd808, aborrero.
bd808 added a comment.Jul 29 2019, 3:34 PM

@Tgr do you have any idea when things last worked as expected for your logins to Wikitech and/or Horizon? The last major change that I know of to the Wikitech LDAPAuthentication configuration was in March when we started using cn:caseExactMatch: in lookups for T165795: Ldap auth extension vs. ldap vs. username Case.

A "cn:caseExactMatch:=Gergő Tisza" lookup seems to work as expected from the cli inside a Cloud VPS project:

$ ldapsearch -xLLL -P 3 -b"dc=wikimedia,dc=org" "cn:caseExactMatch:=Gergő Tisza" dn
dn: uid=tgr,ou=people,dc=wikimedia,dc=org
bd808 added a comment.Jul 29 2019, 3:51 PM

All I can find in the ELK cluster for Wikitech login failures related to this ticket is 2 occurrences of "Incorrect username or password entered.". One was recorded at 2019-07-29T13:22:18 and the other at 2019-07-29T13:24:06.

We don't aggrigate Horizon's logs to the ELK cluster, so I had to go poking around on the labweb* servers themselves to find:

2019-07-29 12:56:06.068920 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.0.130.
2019-07-29 12:57:40.793150 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.16.22.
2019-07-29 13:01:49.076918 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.48.101.
2019-07-29 13:02:30.613885 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.0.132.
2019-07-29 13:13:13.381522 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.32.69.
2019-07-29 13:13:43.783730 Login failed for user "Gerg\xc5\x91 Tisza", remote address 10.64.32.69.
$ python2 -c 'print "Gerg\xc5\x91 Tisza".decode("utf8")'
Gergő Tisza

Its looking like we may have to coordinate setting louder than normal logging for one or both of Wikitech and Horizon to get a useful error message here.

Tgr added a comment.Jul 29 2019, 5:59 PM

@Tgr do you have any idea when things last worked as expected for your logins to Wikitech and/or Horizon?

I don't. It's entirely possible that I haven't used it since March: Wikitech sessions are long-lived and I haven't used Cloud VPS recently. I did use Striker, but that one seems unaffected by this problem even now.

drive-by: If striker works but wikitech/Horizon don't then my first guess would be differing levels of case-sensitivity