Some more context…

1. I created two keys using Edge. (1) was a local key and (2) was the iPhone key (using QR code)

Platform Info

iPhone 14
iPhone OS 17.3.1

Wonder if this is some variant of T244088: Logging in at another wiki than WebAuth was set up fails, due to the different mobile domain...

Wonder if this is some variant of T244088: Logging in at another wiki than WebAuth was set up fails, due to the different mobile domain...

I believe they are separate because I also observed T244088 ( i can't log into wikivoyage with the wikipedia passkey) . but for T244088 I have a workaround : (1) log in on wikipedia (2) open up wikivoyage (on desktop)

video working experience using "show desktop site" on mobile safari

if we have measurements of "Authentication process was interrupted " we could segment by user -agent or device to measure incidence of this issue.

I'm blocked by another variant of this issue: login from a separate windows machine. I'm being prompted to "insert usb security key" but i have two passkeys registered : (1) from iphone and (1) from another windows machine. I would expect the option to pop a QR-CODE to proceed using iphone passkey

STEPS TO REPEAT (2 windows machine variant)

1. create (2) passkeys on windows machine-A
2. try to login on windows machine-B

ACTUAL RESULTS

I'm prompted "insert your security key into usb port" -- with no alternative credential.

EXPECTED RESULTS

I should be prompted for qr-code passkey authentication to use my iphone.
{F42403350}

this bug is pretty serious. I'd like to disable 2-FA but i also want to help get it fixed. I'll be locked out of my account if something happens to my first login session

That USB popup looks very Windows/Edge specific. I don't think the message is in our code, or anything we bring in via vendor.

In T358771#9603441, @Reedy wrote:

That USB popup looks very Windows/Edge specific. I don't think the message is in our code, or anything we bring in via vendor.

check out the "actual" vs "expected" (both are Edge experiences , 1 wikipedia and 1 github). I believe the site provides a list of eligible devices to the browser.

(i'm a bit new to webauthn) it seems that the site (wikipedia) sends a list of token public keys / token IDs to the browser to initiate token-based authentication.

I believe wikipedia may be truncating the eligible token list , hence the "insert USB token" prompt

More visual review of the protocol here:

https://webauthn.me/

https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blame/37f30cd/src/Key/WebAuthnKey.php#L112-L117

	protected $credentialTransports = [
		PublicKeyCredentialDescriptor::AUTHENTICATOR_TRANSPORT_USB,
		PublicKeyCredentialDescriptor::AUTHENTICATOR_TRANSPORT_NFC,
		PublicKeyCredentialDescriptor::AUTHENTICATOR_TRANSPORT_BLE,
		PublicKeyCredentialDescriptor::AUTHENTICATOR_TRANSPORT_INTERNAL
	];

It is the first one in the list, but...

I believe "HYBRID" is the one that supports the iPhone /passkey based login : https://web.dev/articles/passkey-registration

Is it possible that L112-L117 are for the credential setup phase, and that the authentication phase presents the list stored in the database?

Just brainstorming based on what I'm seeing

Here's my list of tokens on wikimedia

Had a look in the DB for your entries. You indeed have two devices enrolled.

Your windows one has been used 3 times, your iphone 0 (apparently, based on the counters).

They both list the same transports:

"transports":["usb","nfc","ble","internal"],

interesting.

"windows-pc" -- this one is internal (windows hello / TPM)
"iphone" -- this one is iPhone Passkey (added via QR-code) . I think it's supposed to be "hybrid"

I can break into the login phase (using chrome devtools) at https://en.wikipedia.org/w/extensions/WebAuthn/resources/login.js L3 and reproduce the issue.

I'm going to see if i can inject "hybrid" into the request to see if that fixes the issue. It may take me a while b/c i don't know the code

https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blob/979220702ab45fb4755ed45bd38cbbb05a411c22/resources/login.js#L3 in the repo

I will note upfront, we're using an old version of the webauthn libraries; specifically 3.3.12, which is 2 years old - https://packagist.org/packages/web-auth/webauthn-lib#v3.3.12

We can't upgrade to the 4.x (or newer versions) until T319432: Migrate WMF production from PHP 7.4 to PHP 8.1 has happend, basically.

https://packagist.org/packages/web-auth/webauthn-lib#v4.0.0
https://packagist.org/packages/web-auth/webauthn-lib#4.8.2

wow it worked!

tl;dr adding hybrid to list of transports fixes the issue

STEPS TO DEBUG

1. login until authn prompt & cancel the token prompt
2. set breakpoint at WebAuthn/resources/login.js L8 (after authenticator is assigned)
3. hit "reload" to break
4. open console and modify the allowCredentials object like this (inspect allowCredentials to find the index of your iphone cred-- the example below is my two creds) ...

authenticator.authInfo.allowCredentials[1].transports.push('hybrid')

5. "use another device" & proceed to login on iphone

screenshot evidence. video inbound
{F42406516}

debug session showing how to fix

https://github.com/wikimedia/mediawiki-vendor/blob/4e138440c4dfeb4896816143c1964da09fea12b6/web-auth/webauthn-lib/src/PublicKeyCredentialDescriptor.php#L26-L29

https://github.com/web-auth/webauthn-lib/blob/v4.0/src/PublicKeyCredentialDescriptor.php#L18-L26

https://github.com/web-auth/webauthn-lib/blob/4.8.2/src/PublicKeyCredentialDescriptor.php#L18-L34

The only mention I can find of hybrid is in two bug reports - https://github.com/search?q=repo%3Aweb-auth%2Fwebauthn-framework+hybrid&type=issues

We can certainly add it with https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blame/37f30cd/src/Key/WebAuthnKey.php#L112-L117 but that feels odd that the vendor library seems to know nothing about hybrid...

Aha

https://forums.developer.apple.com/forums/thread/710784

The QR code flow will be publicly formalized in an upcoming version of the CTAP2 spec. It uses the "hybrid" transport (formerly known as "caBLE").

final public const AUTHENTICATOR_TRANSPORT_CABLE = 'cable';

Which was added in https://github.com/web-auth/webauthn-lib/commit/6634a25e60c3a791f156673b82bb27c21ba7a562 (4.0.5+)

Can you run your test again for me, and see if cable indeed works instead of hybrid?

I'd be happier hard coding cable, with a TODO left to replace it with the proper upstream usage later...

yep cable works like hybrid

Great, I appreciate the debugging. This makes a patch fairly easy. Though, I will admit upfront, I don't know how well this will behave with stuff saved in the DB...

Change 1008987 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/WebAuthn@master] WebAuthnKey: Add "cable" to credentialTransports

https://gerrit.wikimedia.org/r/1008987

happy to help -- great partnership on this

with stuff saved in the DB...

I'll let you know if the transports reflects the new commit or if I need to re-install the credential (it's from the db)

I've also asked Upstream if we've got a chance of getting https://github.com/web-auth/webauthn-lib/commit/6634a25e60c3a791f156673b82bb27c21ba7a562 backported into the 3.3.x branch, and a followup release...

https://github.com/web-auth/webauthn-framework/issues/574

can I help testing out the change on the test env?

In T358771#9612543, @Tonymetz wrote:

can I help testing out the change on the test env?

Not really, yet. I'm pretty sure your testing confirms it's the right fix, at least for new DB entries. It's obviously currently unclear how that works for existing rows, and whether we need to do anything about it (yet).

It needs merging (ie C+2) in our version control, then it'll be live on the Beta-Cluster-Infrastructure...

what's a good way to track the launch status for this fix? i'm sorry I don't know too much about the deployment process

In T358771#9603948, @gerritbot wrote:

Change 1008987 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/WebAuthn@master] WebAuthnKey: Add "cable" to credentialTransports

https://gerrit.wikimedia.org/r/1008987

There'll be a comment like ^ saying this has been merged. Then depending on when it is merged, it should be rolled out on Wikimedia wikis within a week, or two at the most, depending on whether it is merged before the branch cut.

There may be another variant of the issue when logging in on meta.wikipedia.org. It seems that the viable webauthn credential list is being filtered either by site or by login device before being presented to the browser. I get a different experience on different browsers.

STEPS

1. use iPAD (safari) to log into "wikipedia library" via oauth e.g. https://meta.wikimedia.org/w/index.php?returnto=Special:OAuth/authenticate&returntoquery=%26oauth_consumer_key%3xxxxxbaf%26oauth_token%3Dxxxexxxxxxxx&title=Special:UserLogin&uselang=en&centralAuthAutologinTried=1&centralAuthError=Not+centrally+logged+in
2. enter username + password
3. try to use usb-c token or iphone passkey

ACTUAL RESULTS
iphone passkey does not work despite it being connected to the account. error message "Passkey was not found" (not exact)

EXPECTED RESULTS
both usb-c token and iphone passkey should work

any ETA on this one?

No, if there were any updates they would be here.

Can I ask what the blocker is?

There is no "blocker", only lack of unlimited time and lack of unlimited developer resources and lack of empty to-do lists - see also https://www.mediawiki.org/wiki/Bug_management/Development_prioritization

Reedy and I already got this fixed and he submitted a patch. So we're just waiting for it to get merged. What is blocking that?

It seems the commit was made Mar 5 so it's been 6 weeks. Can you guess how many more weeks it is going to be?

@Aklapper I don't appreciate the dismissive tone being used. I invested a lot of effort working together with Reedy to reproduce, debug & help formulate a fix. At the very least you could help clarify exactly what the next steps are here.

As mentioned before, the next step is that some other maintainer must review and approve ("+2") the change. For WebAuthn, that "someone" would most likely be me. However I am not planning to review any WebAuthn patches before I get some patches of my own merged there, and the constant pings in this task are very effective at bringing this patch to the very back of my mental to-do list.

1. it is a one-line change
2. it is a back-port of code from the webauthn repo
3. reedy & I recorded our testing procedure above with video and code samples

What is there to do?

I'm a diligent volunteer and you are burning a bridge here.

@taavi any updates on the webauthn tasks merge progress? Are we looking at another 6 weeks?

Webauthn security tokens are not being passed to wikifunctions.org

I have 2 passkeys (1 windows 11 and 1 iPhone) , and neither are recognized on wikifunctions.org

STEPS

1. Visit wikifunctions.org —> login
2. Enter username + cred
3. Receive 2fa prompt

EXPECTED RESULTS
Existing passkeys should work

ACTUAL RESULTS
No passkeys work

@taavi are we targeting 2024 or 2025 on this one?

@Tonymetz: Such comments are pretty unhelpful. "We" are not targeting any years.

what would be helpful would be an estimate.

a monthly reminder that ...

1. it is a one-line change
2. it is a back-port of code from the webauthn repo
3. reedy & I recorded our testing procedure above with video and code samples

In T358771#9830254, @Tonymetz wrote:

a monthly reminder that ...

Hi Tony,

there's no one that needs a reminder about this. if you want to do something productive, do Gerrit reviews of other changes on other tasks. in any case, you should not be commenting here.

the people who want to use webauthn do. What's the point of the feature if it's broken?

i did another round of testing and the feature still appears to be broken. The issue is likely locking users out of their accounts. There doesn't seem to be telemetry on the issue, so I worry that wikimedia staff is not paying attention to the number of users who are getting locked out.

In my recent case, I was locked out of mediawiki access. Luckily I had a en-wiki session open that allowed me to disable the webauthn .

I recommend either merging this patch to address the issue or disabling webauthn

2662 (SUL) users using 2FA, 84 of them using WebAuthn (but 196 total devices). Some having as many as 5 devices enrolled. 56 of those 84 have 2 or more devices. Meaning 28 only have 1.

I imagine the number of people affected is probably pretty low, but not zero. Otherwise we'd probably hear about it from more users.

See also: T376021: Migrate or disable WebAuthn on Wikimedia wikis

In T358771#10200249, @Reedy wrote:

I imagine the number of people affected is probably pretty low, but not zero. Otherwise we'd probably hear about it from more users.

See also: T376021: Migrate or disable WebAuthn on Wikimedia wikis

Thanks these are helpful insights. There still may be a blindspot. We expect the absolute numbers to be small. But what percentage of 2fa / webauthn users are having login issues? That part is unclear.

Users may be enrolling multiple devices to workaround the issue. webauthn is meant to allow enrollment on a single device and login on multiple devices. The current fault condition has potential to lockout users.

We have debugging examples and a patch that shows that multi-enrolled-device login is broken , and a lockout state is possible.

This feature will never reach broader adoption in this state. I guess the question is, does anyone care?

T244088: Logging in at another wiki than WebAuth was set up fails is a probably bigger blocker to uptake of WebAuthn than this. But work is in progress to hopefully fix that underlying issue

T376021: Migrate or disable WebAuthn on Wikimedia wikis documents many other issues with it.

Does anyone care? That's a harder question. WebAuthn was developed by outside contractors for the WMF, without any sort of long (or even short, frankly) term maintenance plan. Which helps explain the situation we're in with all the issues.

thanks for helping provide the context that is helpful. I'm happy to help provide some support on this if there's interest. I've worked on 2FA efforts before, and users require a bit of education and notification to help move adoption forward.

T244088 -- I believe this ticket is related to T244088 . the fault condition is preventing the complete list of enrolled devices from being presented on alternate wikis. I've personally experienced that state on wikitravel & wikimedia.org

We've talked about this task on the Platform team, and we're happy with the change you're suggesting in principle. @Tgr has already reviewed the patch, and he's happy with it security-wise, so the next step is for someone to test it locally. I volunteered to do that.

So far, I managed to set up OATHAuth and WebAuthn on my local test wiki, and made it available via HTTPS (with a self-signed certificate) under a named .local domain (not just an IP address) – these are both requirements for the Web Authentication API to work.

(By the way, enabling HTTPS on Apache on Ubuntu – what I happen to use – is apparently as easy as sudo a2enmod ssl; sudo a2ensite default-ssl.conf; sudo systemctl restart apache2, but I've read several long articles trying to tell me how to set up everything from scratch, each one suggesting subtly different solutions, before I figured that out.)

I was able to enable Web Authentication as the 2nd factor on a Windows PC (using a Yubikey USB token) and also on an iPhone SE 1st gen (also using the same Yubikey USB token), and then log in on both devices.

However, I am not even getting the option to log in using a passkey on either device, so I haven't been able to reproduce the bug or test the fix yet. I found this article: https://corbado.com/blog/passkeys-ios-15 which says I need to enable Touch ID first on the iPhone. I don't know what I'd need to do on Windows. I'll keep digging into this tomorrow.

One non-obvious thing (at least on Chrome) is that you need a TLS connection that has been established without errors, otherwise the browser rejects the call to initiate passkey check. That means it's not enough to click through the certificate security warning, you need to add it it to browser's CA store. (Some errors are logged to the JS console so that might help figuring out what's wrong.)

In T358771#10218878, @Tgr wrote:

One non-obvious thing (at least on Chrome) is that you need a TLS connection that has been established without errors, otherwise the browser rejects the call to initiate passkey check. That means it's not enough to click through the certificate security warning, you need to add it it to browser's CA store. (Some errors are logged to the JS console so that might help figuring out what's wrong.)

Yep, I ran into that when testing on Edge – I had to create a new cert with a SAN field, and then add it to the Windows CA store, before it let me attempt to use passkeys.

It was actually quite difficult to find a device/OS/browser combination that would let me use passkeys. I tested the following:

• Firefox 131 on Ubuntu 22.04 in a virtual machine – only offers USB security key as an option (I didn't test if it works)
• Chromium 129 on Ubuntu 22.04 in a virtual machine – only offers USB security key as an option (I didn't test if it works)
• Firefox 132 on Windows 10 – only offers USB security key as an option (it works!)
• Edge 129 on Windows 10 – finally, there's an option to use passkeys. It says it can store them locally if I enable Windows Hello (I didn't try), or using a separate authenticator device connected by QR code and Bluetooth (I tried it).
  • When the authenticator device is an iPhone SE 1st gen running iOS 15: does not work, the process gets stuck on "connecting" (with or without the patch)
  • When the authenticator device is some Android phone I borrowed: potentially works! It went further than the iPhone. But I didn't finish the process since I didn't want to store random credentials in another person's phone.
• Safari on iPhone SE 1st gen running iOS 15 (by itself, not as an auth device for desktop): after enabling Touch ID (for password autofill only) and iCloud keychain, it works! However, when I applied your patch, it stopped working – I was no longer able to log in using the stored passkey or remove it. I un-applied the patch and it worked again.

Now, this is not a representative set of devices, but it seems that applying the patch could also lock some users out of their accounts. Unless we can explain why this happened and resolve it, we probably shouldn't merge it.

I think the main thing I learned on this adventure is that passkeys are a technical support nightmare, and we should make sure we disable support for them as thoroughly as we can, until at least a few more years pass and browser/OS vendors figure them out better. USB security keys work well though.

@Tonymetz Also, I am having trouble understanding what is the actual workflow that doesn't work for you. Maybe I'm just confused, but you seem to be talking about several different ways to log in. Can you test things again and write down step-by-step 1) how do you create the second factor and 2) how do you use it to log in?

In T358771#9602229, @Tonymetz wrote:

I'm blocked by another variant of this issue: login from a separate windows machine. I'm being prompted to "insert usb security key" but i have two passkeys registered : (1) from iphone and (1) from another windows machine. I would expect the option to pop a QR-CODE to proceed using iphone passkey

STEPS TO REPEAT (2 windows machine variant)

1. create (2) passkeys on windows machine-A
2. try to login on windows machine-B

ACTUAL RESULTS

I'm prompted "insert your security key into usb port" -- with no alternative credential.

EXPECTED RESULTS

I should be prompted for qr-code passkey authentication to use my iphone.
{F42403350}

For what it's worth, I saw a similar interface, and clicking "Cancel" in that popup opens a second popup that lets you choose the QR code method. This seems like Windows's fault and not ours.

Adding the new transport also causes an extra prompt to appear when logging in on Edge on Windows, so that's not great either.

This could be fixed by using getTransports() when adding the second factor, and then providing that value in allowCredentials when requesting the second factor during login, as @taavi and @Tgr discussed here on the patch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WebAuthn/+/1008987/comment/e210ee68_dfd9b3bd/

If I understand correctly, this would also let us fix the iOS 15 issue. What I'm seeing is that iOS 15 Safari can create the second factor when cable is present, but can't authenticate using it. iOS 15 Safari actually doesn't have the getTransports() method (see MDN, I verified that locally), and it doesn't seem to support cable per my testing above, so we could simply hard-code it to ["usb", "nfc", "ble", "internal"] when the saved second factor doesn't specify it (either because getTransports() was unsupported, or because it was created before this change to the extension).

So, I think doing that would let us proceed with this change. I don't really understand how that would be done, though. web-auth/webauthn-lib doesn't seem to do anything related to that, is it something we'd have to implement ourselves?

In T358771#10219152, @matmarex wrote:

@Tonymetz Also, I am having trouble understanding what is the actual workflow that doesn't work for you. Maybe I'm just confused, but you seem to be talking about several different ways to log in.
Can you test things again and write down step-by-step

I'm happy to help test and support this. thanks for giving it some attention. it's been a few months so i'll try to reproduce a better test case.

1. how do you create the second factor and 2) how do you use it to log in?

IIRC from the earlier test case, it was something like this.

1. successfully add iphone token on windows-device-desktop
2. attempt login on a 2nd windows device, "windows-device-laptop", where I expected a QR code prompt (presentation of iphone token)

Step 2 failed. Instead of the presentation of iphone token qr code, the browser tried generic webauthn credentails (e.g. the windows-device-laptop's hello login, which didn't contain any tokens for wikipedia).

This is what led to a possible "locked out "case . Other variations have also failed, e.g. trying to log in on wikimedia. They are not recognizing the iphone token added on en.wikipedia.org.

In T358771#10223171, @matmarex wrote:

I don't really understand how that would be done, though. web-auth/webauthn-lib doesn't seem to do anything related to that, is it something we'd have to implement ourselves?

Yeah, AIUI the library is basically just a heap of value objects that serialize to and from the JSON structures used by the navigator.credentials browser APIs, so you can interact with those APIs with easy-to-read code. I think what would have to be done is:

• add transport setters to WebAuthnKey
• read the transports in mw.ext.webauthn.Registrator.prototype.formatCredential() and put it into the JSON string that gets submitted from the server
• read them from the JSON data and plug them into the newly created WebAuthnKey in Authenticator::continueRegistration()
• be able to load the data back in WebAuthnKey::setDataFromEncodedDBData()

Transports might not be the only thing for which this needs to be done, WebAuthnKey::setDataFromEncodedDBData() is suspiciously thin.