Page MenuHomePhabricator

Unable to log in to SUL account
Closed, InvalidPublic

Description

I am currently at a WMCZ outreach event and no participant is able to successfully login, including me using my password from the password manager. Resetting the password and logging in via the password that came via the email does not work either.

It seems our auth stack is completely and 100% broken.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Urbanecm_WMF triaged this task as Unbreak Now! priority.Nov 30 2023, 3:18 PM

This is an urgent problem.

Volans lowered the priority of this task from Unbreak Now! to High.Nov 30 2023, 3:32 PM
Volans subscribed.

This doesn't seem to be a widespread login problem at this time. (lowering the priority)
All indications so far points to a rate-limiting issue with multiple people sharing the same public IP.

Volans renamed this task from No one is able to log in to SUL account to Unable to log in to SUL account.Nov 30 2023, 3:33 PM

I cannot reproduce the issue anymore. Looks like someone was failing logins for too long and blocked everyone else. We might want to alter the message to say something else than "Invalid password", to avoid similar surprises in the future.

I cannot reproduce the issue anymore. Looks like someone was failing logins for too long and blocked everyone else. We might want to alter the message to say something else than "Invalid password", to avoid similar surprises in the future.

Using a very generic error message for blocked login attempts is a recommended best practice industry wide. Generally it is considered beneficial to provide as few signals as possible to attackers when rate limiting failed authentication attempts. Similar treatment is generally recommended for invalid usernames as well. I think MediaWiki's default is actually "Incorrect username or password entered. Please try again." in both cases. Ideally actual failure reasons would be logged or otherwise recorded related to the attempt for technical support and system troubleshooting.

I think trying to hide whether the user exists is bad practice, and we try to avoid it in MediaWiki. It might have some limited value in applications where users don't interact with each other and are invisible to the outside (SaaS apps for example), although usually you can probe usernames by trying to register them, anyway. But MediaWiki users are public so no sense in making login harder just to obscure information that's already public.

Not differentiating between password errors and throttling does seem like a meaningful hardening against password brute-forcing, though. It's not very useful because our code is public so an attacker can just check how many bad tries it takes to trigger throttling, and sophisticated attacks will use a botnet anyway, but it's probably at least a little useful? I don't have a strong opinion but if you want to change it, you should check with the Security team.

That said...

All indications so far points to a rate-limiting issue with multiple people sharing the same public IP.

the log events for cswiki (I assume it was a cswiki event?) don't really line up with that hypothesis. This was reported, presumably in real time, at 15:18 UTC, and we have no logs in that time slot. There are also no throttle hits for Martin Urbanec or Martin Urbanec (WMF). @Urbanecm_WMF maybe you can check the logs against the IP used at the event, if you know it?

More importantly, I don't think login throttling (if this is throttling via $wgPasswordAttemptThrottle and not some other mechanism) applies to multiple users sharing an IP at all? It should only throttle login attempts for a given username/IP pair (for 5 minutes after 5 failed, and 2 days after 150 failed attempts).

If it did apply to all users from a given IP, that would be a problem, as we have whole countries behind a single IP and we probably don't want to block them just because someone in that country is trying to brute-force passwords.

More importantly, I don't think login throttling (if this is throttling via $wgPasswordAttemptThrottle and not some other mechanism) applies to multiple users sharing an IP at all? It should only throttle login attempts for a given username/IP pair (for 5 minutes after 5 failed, and 2 days after 150 failed attempts).

As a test, I made 30 or so failed login attempts with one username, and had no problem logging in with a different username (I had to submit a captcha but wasn't throttled).

I'll close this as invalid because it's unclear what happened and I don't think we can figure it out now, but I'm pretty sure it wasn't throttling.