Forked from T376267: ☂ Wikitech account linking and SUL error reporting:
@roti_WMDE, @Eevans, and @CDobbins have all reported being prompted for 2FA credentials when attempting to login to wikitech.wikimedia.org. When prompted, neither generated TOTP tokens or recovery tokens have been accepted.
OATHAuth user {user} failed OTP token/recovery code from {clientip} events are being recorded for these failed attempts as can be seen in log events from Wikitech.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | BUG REPORT | Reedy | T379398 Wikitech users being unexpectedly prompted for 2FA tokens | ||
| Resolved | taavi | T379442 Don't use a central user id of 0 for storing (or loading?) of 2FA rows |
I just tested login from an incognito window with my wikitech:User:BryanDavis account and it is reproducing the issue seen by others. This account did have local 2FA on Wikitech prior to the 2024-10-01 SUL migration. The account does not have global 2FA following unification with meta:User:BryanDavis. Wikitech is rejecting TOTP tokens generated with the seed found in labswiki.oathauth_devices for the BryanDavis account (local user.user_id = 1604).
I wonder if the easiest fix would be to rename (or archive and drop) the legacy oathauth_devices, oathauth_types, oathauth_users, and oathauth_users_restore tables in the labswiki schema?
The first report of this issue was T376267#10292062 on 2024-11-05. The first log event for that user is timestamped Nov 5, 2024 @ 10:43:17.723. There are however a number of OATHAuth user {user} failed OTP token/recovery code from {clientip} failures going back to the 2024-10-01 conversion date involving other users: https://logstash.wikimedia.org/goto/c13bd28c7ecdbe83c7e7b42fbd8846cd
oathauth_users and oathauth_users_restore are both ancient, and should be dropped.
oathauth_devices and oathauth_types would've been being used before SUL happened. They should be redundant now.
I wonder if there's some weird edge case of DatabaseVirtualDomains using local tables if they exist? Or just some edge case in OATHAuth generally...
if ( $wmgUseOATHAuth ) { wfLoadExtension( 'OATHAuth' ); if ( $wmgOATHAuthDisableRight ) { $wgGroupPermissions['user']['oathauth-enable'] = false; foreach ( $wmgPrivilegedGroups as $group ) { if ( isset( $wgGroupPermissions[$group] ) ) { $wgGroupPermissions[$group]['oathauth-enable'] = true; } } } $wgGroupPermissions['sysop']['oathauth-disable-for-user'] = false; $wgGroupPermissions['sysop']['oathauth-view-log'] = false; $wgGroupPermissions['sysop']['oathauth-verify-user'] = false; // T209749 if ( $wmgUseCentralAuth ) { $wgOATHAuthAccountPrefix = 'Wikimedia'; $wgVirtualDomainsMapping['virtual-oathauth'] = [ 'db' => 'centralauth' ]; // TODO: remove once relevant patches have been merged and shipped $wgOATHAuthDatabase = 'centralauth'; } if ( $wmgUseWebAuthn ) { wfLoadExtension( 'WebAuthn' ); } }
$ mw-debug-repl labswiki Psy Shell v0.12.3 (PHP 7.4.33 — cli) by Justin Hileman > $wmgUseOATHAuth = true > $wmgUseCentralAuth = true > $wgVirtualDomainsMapping['virtual-oathauth'] = [ "db" => "centralauth", ]
That all looks like I would expect. OATHAuth is enabled, wikitech is acting as a CentralAuth client, and database lookups for oauth data should use tables in the "centralauth" database.
Let's recreate the interesting part of the authn flow that seems to be going wrong and see what things look like:
> use MediaWiki\MediaWikiServices; > $svc = MediaWikiServices::getInstance(); > $userFactory = $svc->getUserFactory(); > $bd808 = $userFactory->newFromName( "BryanDavis" ); = MediaWiki\User\User {#8330 +mId: null, +mName: "BryanDavis", +mActorId: null, +mRealName: null, +mEmail: null, +mTouched: null, +mEmailAuthenticated: null, +mFrom: "name", mId: null, mName: "BryanDavis", mActorId: null, mRealName: null, mEmail: null, mTouched: null, mEmailAuthenticated: null, mFrom: "name", } > $userRepository = $svc->getService( "OATHUserRepository" ); > $authUser = $userRepository->findByUser( $bd808 ); = MediaWiki\Extension\OATHAuth\OATHUser {#8364} > $totp = $authUser->getModule(); > $totp->isEnabled( $authUser ); = true
This result is consistent the behavior of the login flow, but that behavior is still unexpected.
If those accounts aren't attached...
public function centralIdFromLocalUser( UserIdentity $user, $audience = self::AUDIENCE_PUBLIC, $flags = IDBAccessObject::READ_NORMAL ): int { return $this->isAttached( $user ) ? $this->centralIdFromName( $user->getName(), $audience, $flags ) : 0; }
There's a row in centralauth.oathauthdevices with oad_user of 0:
MariaDB [centralauth]> select oad_id, oad_user, oad_type, oad_name, oad_created from oathauth_devices where oad_user=0; +--------+----------+----------+----------+----------------+ | oad_id | oad_user | oad_type | oad_name | oad_created | +--------+----------+----------+----------+----------------+ | 4506 | 0 | 1 | NULL | 20241105020146 | +--------+----------+----------+----------+----------------+ 1 row in set (0.001 sec)
And we'll just do a lookup of that oad_user = 0:
$uid = $this->centralIdLookupFactory->getLookup() ->centralIdFromLocalUser( $user->getUser() ); $res = $this->dbProvider ->getReplicaDatabase( 'virtual-oathauth' ) ->newSelectQueryBuilder() ->select( [ 'oad_id', 'oad_data', 'oat_name', ] ) ->from( 'oathauth_devices' ) ->join( 'oathauth_types', null, [ 'oat_id = oad_type' ] ) ->where( [ 'oad_user' => $uid ] ) ->caller( __METHOD__ ) ->fetchResultSet();
https://meta.wikimedia.org/wiki/Special:CentralAuth?target=Robert%20Timm%20(WMDE) isn't attached
https://meta.wikimedia.org/wiki/Special:CentralAuth?target=EEvans%20(WMF) and https://meta.wikimedia.org/wiki/Special:CentralAuth?target=CDobbins-WMF don't have accounts listed on wikitech (so no username match)...
@roti_WMDE, @Eevans, @CDobbins can you advise which username you're trying to log in to wikitech with?
Mentioned in SAL (#wikimedia-operations) [2024-11-08T23:16:05Z] <Reedy> ran delete from oathauth_devices where oad_id=4506; on centralauth for T379398 because oad_user=0
I didn't test it before I deleted that row, but...
> $roti = $userFactory->newFromName( "Robert Timm (WMDE)" );
= MediaWiki\User\User {#5917
+mId: null,
+mName: "Robert Timm (WMDE)",
+mActorId: null,
+mRealName: null,
+mEmail: null,
+mTouched: null,
+mEmailAuthenticated: null,
+mFrom: "name",
mId: null,
mName: "Robert Timm (WMDE)",
mActorId: null,
mRealName: null,
mEmail: null,
mTouched: null,
mEmailAuthenticated: null,
mFrom: "name",
}
> $authUser = $userRepository->findByUser( $roti );
= MediaWiki\Extension\OATHAuth\OATHUser {#5904}
> $totp->isEnabled( $authUser );
= falseAnd when I reinserted it for testing...
> $totp = $authUser->getModule();
= MediaWiki\Extension\OATHAuth\Module\TOTP {#8352}
> $totp->isEnabled( $authUser );
= trueDeleted it again...
This statement was factually incorrect. I could have more correctly said that I do not remember setting up 2FA on my BryaDavis account. It does turn out that I have the secret for it in my TOTP generator of choice though and can authenticate using it.
Thankfully @Reedy found the real problem while I was chasing shadows on the cave wall.