One question I have is which login page are we supposed to show to the user in the SUL3 case? I think it's going to be the login form on the shared domain right? Not the local one I assume?

I was able to reproduce the /wiki/Special:UserLogin?usesul3=1 part (which doesn't present the login form but just re-triggers a login chain) but couldn't reproduce /wiki/Special:UserLogin?usesul3=0. The later will present to me a login form that I can use locally to authenticate as a different user.

Just checked this today and we're having over 18K entries on logstash and over 1M entries in the last 30 days. Seems to be very frequent recently.

I think this started off as part of T380500: CentralAuthUser returning outdated data after user creation when we were investigating the frequent logout issue. @Tgr already worked on fixing one leg of it and there are other things to fix but I doubt that it's related to the logging here.

Thanks for filing @thcipriani.

Let's reopen this is things go up. This is a warning and we've got 1 instance in the last 1 month.

In T390784#10728004, @Tgr wrote:

Microstash is cross-DC memcached, right? So in theory not the most reliable - could be evicting data because the slab is full. It does seem unlikely though.

In T375796#10704803, @Tgr wrote:

We might also want to swap the central autologin lookup order of loginwiki and auth.wikimedia.org at some point.

But we few more days is a better cadence for conclusion so let's wait till end of week and if we no longer see them, we can resolve the task. I'll check again tomorrow evening :)

Looking at https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralAuth/+/1078400/89/includes/Hooks/Handlers/PageDisplayHookHandler.php, we were already using /start here for non-JS clients.

@Tgr wrote:
Maybe the same as T380500: CentralAuthUser returning outdated data after user creation?

In T388177#10706509, @matmarex wrote:

We should wait a few days to see if that really resolves the errors: https://logstash.wikimedia.org/goto/69ae850bf878fc526f85273c8547494f

On enwiki (in prod), I could reproduce this by checking the dev tools and saw an anon session created but can't reproduce this locally.

Anything left to do here? Or can this be marked as resolved?

Reading the class level docblock of the ChronologyProtector, it states the following (extract):

See (logs snippet) of the Special:UserLogin/return&wpLoginToken... request which is what triggers continue authentication on the local wiki - from debug log

In T388177#10694196, @Krinkle wrote:

In theory that can't happen because of ChronologyProtector, which ensures that the replicas we use have caught up to our own previous writes, given that it will be the same browser that created the account and the same browser getting redirected to read the data.

If this turns out incorrect, we need to file an Rdbms bug and confirm whether there are any missing CP-cookies or CP-query params in this journey.

In T388177#10639731, @Tgr wrote:

This always seems to be happening on the local continueAuthentication() call, presumably right after account creation.

Hm! What should be the expected behavior here, I have some 3 ideas that come to mind:

This seems really weird, vaguely following the Special:ConfirmEmail code, I see that it sometimes call Special:UserLogin as a GET request. I think maybe this is when we some weird behavior when cookies on shared domain are set but not on local domain (when continueAuthentication is called) - maybe some timeout(?). Then that throws the logic error but I wonder why/when that should ever happen.

Thanks @Reedy, we've added some logging, let's see if we can get more info about this as the patch goes out.

In T388252#10613965, @Reedy wrote:

rECAU9f8ff1156fed: Special: Add support for central autologin in SUL3 mode made assertLocalWikiIsValid accept a nullable string, but WikiMap::getWiki isn't defined to take a null(able string)...

We had some issues today, filed T388222: Remove(?) Office wiki from DAL as it's not an SUL wiki

In T388172#10612015, @Tgr wrote:

Seems like a SessionManager bug - those sessions clearly don't have the same priority:

[b2fa3a34-b31a-4584-9900-06adcb3bde76] /w/rest.php/wikibase/v0/entities/lexemes/L1041834/senses   MediaWiki\Session\SessionOverflowException: Multiple sessions for this request tied for top priority: [100]MediaWiki\Extension\OAuth\SessionProvider<anon>h5cshtuck442udqpv2lsupnhp2u8buao, [50]CentralAuthSessionProvider<anon>rspba02td80u2kaju3teghvqm0tf11qi

But also, the priority of CentralAuthSessionProvider should be 11, not 50 (it's hardcoded). What's going on?

In T388177#10612820, @Tgr wrote:

Maybe one of the ids is zero and this is some version of T380500: CentralAuthUser returning outdated data after user creation.

In T388177#10611985, @Tgr wrote:

Only a single instance (logstash). Do we seriously not log the user name/ID when this happens? Ugh.
It's very much not supposed to happen, but without any specifics, can't really investigate.

@Tgr, do you want to sign this off or is there something else you want me to have a look?

In T387789#10600098, @Stashbot wrote:

Mentioned in SAL (#wikimedia-operations) [2025-03-04T10:35:39Z] <xSavitar> T387789 Ran mwscript-k8s --comment="T387789" -f -- extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=enwiki --logwiki=metawiki 'JamesVilla44' 'DartsF4' --ignorestatus

In T384007#10585015, @Tgr wrote:

In one of my tests I ended up logged out after a successful registration (but then top-level autologin worked). Not sure if that's a SUL3 bug that happens infrequently, or just generic authentication fragility. I thought I saw a log message about session metadata conflict, but now I can't find it.

In T384232#10573645, @Tgr wrote:

Login is only prevented by blocks when $wgBlockDisablesLogin is true, which is usually the case for private wikis (which usually don't use SUL, although I think the stewardwiki is an exception).

In T384232#10573645, @Tgr wrote:

Login is only prevented by blocks when $wgBlockDisablesLogin is true, which is usually the case for private wikis (which usually don't use SUL, although I think the stewardwiki is an exception). CentralAuth global locks (which aren't quite blocks) do prevent login. But in neither case do we care much about the user experience, these are named blocks that are used when the user has no legitimate business on that wiki. IP blocks never prevent login AFAIK.

In T384232#10567548, @Tgr wrote:

Yeah, an IP block (https://test.wikipedia.org/wiki/Special:Block and you enter an IP in the target field).

@Tgr, there is a bullet that says blocked user cannot sign up, gets reasonable error message, not sure I understand what that means. Is there a way to block a user that doesn't yet exist in the system?

In T384232#10563856, @Tgr wrote:

NewUserMessage is apparently not enabled on any test wiki. We should probably change that.