I did some debugging and found out that verification will fail when php extension gmp is not enabled. However, without it you would not be able to register the key in the first place.
Verfication failed occurs when authentication ceremony fails, most likely in the library. ( MediaWiki\Extension\WebAuthn\Key\WebAuthnKey::authenticationCeremony).
Any error coming from the library should be logged into the logger, on channel 'authentication'. Can you check the log?
Also, please check the DB if the key has been saved correctly

Thanks

In T237554#5643242, @ItSpiderman wrote:

I did some debugging and found out that verification will fail when php extension gmp is not enabled. However, without it you would not be able to register the key in the first place.
Verfication failed occurs when authentication ceremony fails, most likely in the library. ( MediaWiki\Extension\WebAuthn\Key\WebAuthnKey::authenticationCeremony).
Any error coming from the library should be logged into the logger, on channel 'authentication'. Can you check the log?
Also, please check the DB if the key has been saved correctly
Thanks

Yeah, GMP is indeed installed/enabled.

The DB row looks sane enough (comparing it to the one on the Hallo Welt dev wiki for OATHAuth).

I'm not sure the state of logging on beta, so will followup on that one and see if anything stands out in the logs

I can give you access the project on beta (shell access etc) if it helps?

Also, have you updated the Hallo Welt dev wiki to master (or thereabouts) and picked up any minor changes to the libraries? Just so we're able to compare something a bit more similar, incase it's related to PHP versions or other server type config/package versions

Change 549487 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Add 'authentication' => 'info' logging to beta temporarily

https://gerrit.wikimedia.org/r/549487

Change 549487 merged by jenkins-bot:
[operations/mediawiki-config@master] Add 'authentication' => 'info' logging to beta temporarily

https://gerrit.wikimedia.org/r/549487

I have just updated our dev system to master and pulled latest versions of both extensions, and was able to login normally using my phone's fingerprint sensor (left the FIDO key in the office).

It would be helpful to get the shell access, but i think all we can see would be in the logs (as this is most likely an exception in the library and that would be logged)

I tought it might be that the library has changed, and since you just installed the WebAuthn extension, you would possibly get the later version that i had on the dev system, but now i updated the library, and it still works. We had an issue of breaking changes in the library that required change in the extension before, but that does not seem to be the case now.

Thanks

I guess from these two (all that appeared on an login attempt)... "WebAuthn authentication failed due to: The credential ID is invalid."

{
  "_index": "logstash-mediawiki-2019.11.07",
  "_type": "mediawiki",
  "_id": "AW5GnqYweTItmKeUzHF9",
  "_version": 1,
  "_score": null,
  "_source": {
    "server": "en.wikipedia.beta.wmflabs.org",
    "phpversion": "7.2.24-1+0~20191026.31+debian9~1.gbpbbacde+wmf1",
    "wiki": "enwiki",
    "channel": "authentication",
    "program": "mediawiki",
    "type": "mediawiki",
    "http_method": "POST",
    "host": "deployment-mediawiki-07",
    "@version": 1,
    "shard": "s3",
    "timestamp": "2019-11-07T16:07:12+00:00",
    "severity": "warning",
    "unique_id": "XcRBMKwQBHcAAGB1VqgAAAAQ",
    "level": "WARNING",
    "ip": "172.16.4.21",
    "mwversion": "1.35.0-alpha",
    "logsource": "deployment-mediawiki-07",
    "message": "Webauthn login failed for user Reedy",
    "normalized_message": "Webauthn login failed for user Reedy",
    "url": "/w/index.php?title=Special:UserLogin&returnto=Special:Manage+Two-factor+authentication",
    "reqId": "XcRBMKwQBHcAAGB1VqgAAAAQ",
    "tags": [
      "input-kafka-rsyslog-udp-localhost",
      "rsyslog-udp-localhost",
      "kafka",
      "es"
    ],
    "referrer": "https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Special:UserLogin&returnto=Special:Manage+Two-factor+authentication",
    "@timestamp": "2019-11-07T16:07:12.556Z",
    "facility": "user"
  },
  "fields": {
    "@timestamp": [
      1573142832556
    ]
  },
  "highlight": {
    "channel": [
      "@kibana-highlighted-field@authentication@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1573142832556
  ]
}

{
  "_index": "logstash-mediawiki-2019.11.07",
  "_type": "mediawiki",
  "_id": "AW5GnqYweTItmKeUzHF8",
  "_version": 1,
  "_score": null,
  "_source": {
    "server": "en.wikipedia.beta.wmflabs.org",
    "phpversion": "7.2.24-1+0~20191026.31+debian9~1.gbpbbacde+wmf1",
    "wiki": "enwiki",
    "channel": "authentication",
    "program": "mediawiki",
    "type": "mediawiki",
    "http_method": "POST",
    "host": "deployment-mediawiki-07",
    "@version": 1,
    "shard": "s3",
    "timestamp": "2019-11-07T16:07:12+00:00",
    "severity": "warning",
    "unique_id": "XcRBMKwQBHcAAGB1VqgAAAAQ",
    "level": "WARNING",
    "ip": "172.16.4.21",
    "mwversion": "1.35.0-alpha",
    "logsource": "deployment-mediawiki-07",
    "message": "WebAuthn authentication failed due to: The credential ID is invalid.",
    "normalized_message": "WebAuthn authentication failed due to: The credential ID is invalid.",
    "url": "/w/index.php?title=Special:UserLogin&returnto=Special:Manage+Two-factor+authentication",
    "reqId": "XcRBMKwQBHcAAGB1VqgAAAAQ",
    "tags": [
      "input-kafka-rsyslog-udp-localhost",
      "rsyslog-udp-localhost",
      "kafka",
      "es"
    ],
    "referrer": "https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Special:UserLogin&returnto=Special:Manage+Two-factor+authentication",
    "@timestamp": "2019-11-07T16:07:12.555Z",
    "facility": "user"
  },
  "fields": {
    "@timestamp": [
      1573142832555
    ]
  },
  "highlight": {
    "channel": [
      "@kibana-highlighted-field@authentication@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1573142832555
  ]
}

In T237554#5644486, @gerritbot wrote:

Change 549487 merged by jenkins-bot:
[operations/mediawiki-config@master] Add 'authentication' => 'info' logging to beta temporarily
https://gerrit.wikimedia.org/r/549487

I don't know if bumping this upto debug would be useful

How can i set up WebAuthn for myself on https://en.wikipedia.beta.wmflabs.org? (i dont see the entry for 2FA)

Seems that relevant messages are written to log at this level, this is the error coming from the library, just unfortunatelly, not very helpful

In T237554#5644841, @ItSpiderman wrote:

How can i set up WebAuthn for myself on https://en.wikipedia.beta.wmflabs.org? (i dont see the entry for 2FA)

I've just made Dsavuljesku an admin on the wiki, you should be able to enable it now

As per https://en.wikipedia.beta.wmflabs.org/wiki/Special:ListGroupRights just "users" cannot enable 2FA, in the same way as for prod

This error occus after calling https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blob/master/src/WebAuthnCredentialRepository.php#L66-L71
It appears this does not return a valid credential.
From how PublicKeyCredentialSource::createFromArray works, it would either return a valid credential or throw another error, so my guess given credentialId is not found in the DB

In T237554#5644854, @ItSpiderman wrote:

This error occus after calling https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blob/master/src/WebAuthnCredentialRepository.php#L66-L71
It appears this does not return a valid credential.
From how PublicKeyCredentialSource::createFromArray works, it would either return a valid credential or throw another error, so my guess given credentialId is not found in the DB

publicKeyCredentialId is definitely in there.

I don't know offhand if/what of the db row is actually "sensitive", ie whether I should be posting it public on phab etc

But a slightly redacted version is below, which mostly looks like what was on your dev wiki too

{"keys":[{"userHandle":"<REDACTED>==","publicKeyCredentialId":"<REDACTED>==","credentialPublicKey":"<REDACTED>=","aaguid":"00000000-0000-0000-0000-000000000000","friendlyName":"yubikey5nfc","counter":21,"type":"public-key","transports":["usb","nfc","ble","internal"],"attestationType":"","trustPath":{"type":"Webauthn\\TrustPath\\EmptyTrustPath"}}]}

Thank you for admin access, now i do have the option to enable 2FA.

is it possible to do some debugging there?
On https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blob/master/src/WebAuthnCredentialRepository.php#L66
This should be true:
base64_encode( $publicKeyCredentialId ) == {value in db for publicKeyCredentialId}

In T237554#5644899, @ItSpiderman wrote:

Thank you for admin access, now i do have the option to enable 2FA.
is it possible to do some debugging there?
On https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blob/master/src/WebAuthnCredentialRepository.php#L66
This should be true:
base64_encode( $publicKeyCredentialId ) == {value in db for publicKeyCredentialId}

Yup, they do...

I debugged locally the whole path of authentication and this seems like an impossible bug...
if the statement from the previous post is true, that means that key is found that would mean that this condition is true

if ( isset( $this->credentials[$publicKeyCredentialId] ) ) {
	return PublicKeyCredentialSource::createFromArray( $this->credentials[$publicKeyCredentialId] );
}

so it attempts to create the credential source, which, if failed, would throw a different kind of exception, and would never return NULL that would cause the exception we see in the log.
Only way for the error we see to occur is if credentialId cannot be found so it never goes into this if statement branch

Can you please provide me with the shell access?

In T237554#5647511, @ItSpiderman wrote:

Can you please provide me with the shell access?

I did actually do it earlier in the week when I said I would ;).

https://wikitech.wikimedia.org/wiki/Help:Access Should explain how to get going with it

Thanks

I managed to log into bastion-eqiad1-01, is that the right server? How do I get to WebAuthn from there?

In T237554#5648208, @ItSpiderman wrote:

I managed to log into bastion-eqiad1-01, is that the right server? How do I get to WebAuthn from there?

You want deployment-deploy01.deployment-prep.eqiad.wmflabs, which you should be able to SSH to "directly" from your machine after doing https://wikitech.wikimedia.org/wiki/Help:Access#Accessing_instances_with_ProxyJump_ssh_option_(recommended)

When onto that host, you can do sql centrlauth to get the SQL database with the oathauth_users table

Code is at /srv/mediawiki-staging. I don't know offhand if you'll need to deploy any debugging or if you just want eval.php (mwscript eval.php enwiki)

Thank you, i got in.
I would like to know if this line is executed or not
https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blob/master/src/WebAuthnCredentialRepository.php#L69
I guess i would need to put some debugging statement in there

In T237554#5645032, @Reedy wrote:

In T237554#5644899, @ItSpiderman wrote:

Thank you for admin access, now i do have the option to enable 2FA.
is it possible to do some debugging there?
On https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blob/master/src/WebAuthnCredentialRepository.php#L66
This should be true:
base64_encode( $publicKeyCredentialId ) == {value in db for publicKeyCredentialId}

Yup, they do...

So this was indeed the case... But...

In T237554#5648349, @ItSpiderman wrote:

Thank you, i got in.
I would like to know if this line is executed or not
https://github.com/wikimedia/mediawiki-extensions-WebAuthn/blob/master/src/WebAuthnCredentialRepository.php#L69
I guess i would need to put some debugging statement in there

$this->credentials is an empty array for some reason (I didn't check that before)... Which I guess is the issue that's here. Something not loading correctly from the database?

I've just noticed the issue in WebAuthn

		/** @var OATHAuth $oath */
		$oath = MediaWikiServices::getInstance()->getService( 'OATHAuth' );
		$this->module = $oath->getModuleByKey( 'webauthn' );
		$this->db = MediaWikiServices::getInstance()->getDBLoadBalancer()->getConnection(
			DB_MASTER
		);
		$res = $this->db->select(
			'oathauth_users',
			[ 'id', 'data' ],
			[ 'module' => 'webauthn' ]
		);

It's just reading it from the local DB... And because beta is beta, it does have an oathauth_users table (the db updaters just create the tables every time even if they're removed). But it's empty, of course. So it doesn't error...

This is in comparison to for example the maintenance scripts in OATHAuth

		global $wgOATHAuthDatabase;
		$lb = MediaWikiServices::getInstance()->getDBLoadBalancerFactory()
			->getMainLB( $wgOATHAuthDatabase );
		$dbw = $lb->getConnectionRef( DB_MASTER, [], $wgOATHAuthDatabase );

MariaDB [enwiki]> select * from oathauth_users;
Empty set (0.00 sec)
MariaDB [centralauth]> select count(*) from oathauth_users;
+----------+
| count(*) |
+----------+
|       12 |
+----------+
1 row in set (0.00 sec)

For some routes through the code, I guess it uses the OATHAuth code which is fine, but others it uses the separate code in WebAuthn?

Maybe something we should address in some way, so that the read is done inside OATHAuth and passed through?

Change 549958 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WebAuthn@master] Use correct database ($wgOATHAuthDatabase) in WebAuthnCredentialRepository

https://gerrit.wikimedia.org/r/549958

In T237554#5649280, @gerritbot wrote:

Change 549958 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WebAuthn@master] Use correct database ($wgOATHAuthDatabase) in WebAuthnCredentialRepository
https://gerrit.wikimedia.org/r/549958

Need to amend the patch to probably not use global $wgOATHAuthDatabase; but get it from a Config object from one of the services somewhere along the lines

But that will fix the issue we're having here :)

Great! Yes, probably all DB access should go through OATHAuth, especially since this is just reading all keys. I will commit a patch for that in the following days

In T237554#5649351, @ItSpiderman wrote:

Great! Yes, probably all DB access should go through OATHAuth, especially since this is just reading all keys. I will commit a patch for that in the following days

Do you have C+2 access on the repo? If not, want to at least leave a C+1 and I'll self merge ;)

Change 549958 merged by jenkins-bot:
[mediawiki/extensions/WebAuthn@master] Use correct database ($wgOATHAuthDatabase) in WebAuthnCredentialRepository

https://gerrit.wikimedia.org/r/549958

Change 549975 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WebAuthn@REL1_34] Use correct database ($wgOATHAuthDatabase) in WebAuthnCredentialRepository

https://gerrit.wikimedia.org/r/549975

Change 549975 merged by jenkins-bot:
[mediawiki/extensions/WebAuthn@REL1_34] Use correct database ($wgOATHAuthDatabase) in WebAuthnCredentialRepository

https://gerrit.wikimedia.org/r/549975

Change 550424 had a related patch set uploaded (by ItSpiderman; owner: ItSpiderman):
[mediawiki/extensions/OATHAuth@master] Define fallback for request IP when persisting user

https://gerrit.wikimedia.org/r/550424

Change 550424 merged by jenkins-bot:
[mediawiki/extensions/OATHAuth@master] Define fallback for request IP when persisting user

https://gerrit.wikimedia.org/r/550424