For now, we might also want to check for a mw session token instead.

Please correct me if I’m wrong, but in this case, validation is just a matter of checking whether the token is present or not.

For JWT tokens:

• Key distribution: JWT public key(s) need to be distributed to CDN nodes. This could be managed via Puppet, confd, or a custom script that periodically fetches the key(s) from an internal MediaWiki endpoint.
• Validation: HAProxy provides a built-in jwt_verify() converter to perform signature verification. The example below demonstrates extracting the token from the Authorization header, checking the algorithm (alg) claim, the expiration date and validating the token

# Extract JWT from Authorization header and verify using a public certificate
http-request set-var(req.bearer) http_auth_bearer
http-request set-var(req.jwt_alg) var(req.bearer),jwt_header_query('$.alg')
http-request set-var(req.exp) http_auth_bearer,jwt_payload_query('$.exp','int')
http-request set-var(req.now) date()
# validate the algorithm
acl is_valid_session_token var(req.jwt_alg) -m str "RS256"
# validate the exp date
acl is_valid_session_token var(req.exp),sub(req.now) -m int lt 0
# validate the signature
acl is_valid_session_token var(req.bearer),jwt_verify(req.jwt_alg,"/path/to/crt.pem") 1
# acl is_human is out of scope for this example
acl is_human always_true
http-request set-var(req.trusted_request) str(C) if is_valid_session_token is_human
http-request set-var(req.trusted_request) str(B) if is_valid_session_token !is_human
http-request set-header X-Verified-User %[var(req.bearer),jwt_payload_query('$.sub')] if is_valid_session_token

• Signaling: For sucessfully verified tokens:
  • Bump the X-Trusted-Request grade to C (for humans) or B (for bots), following the trust model outlined in T399057
  • Populate an X-Verified-User header with the sub claim from the JWT. This header will be removed by Varnish before the request reaches any backend, to prevent leakage of identity information. (Note: We could append sub= or user= to X-Provenance but I'd only do that if we intend to let the data reach the backend servers, otherwise a separate header that gets wiped in Varnish makes everything easier/cleaner)

In T400238#11027254, @Vgutierrez wrote:

For now, we might also want to check for a mw session token instead.

Please correct me if I’m wrong, but in this case, validation is just a matter of checking whether the token is present or not.

Yes, that is correct right now, but that will likely change during this quarter, see T398815 and subtasks.

One complication regarding JWTs might be - I'm not sure we use a single key for all cases; we should probably make it a requirement for perf purposes? But that would potentially interact with T397924.

Generally I agree with your take, we can define the details of X-Trusted-Request but that's matter of deciding which grade corresponds to what.

it's not uncommon to have several keys in place at any given point in time, it should be fine in terms of performance as long as we keep it under control

@Tgr would it be possible to perform some lightweight validation of current MediaWiki session tokens? For example, checking whether the token has a specific length, or whether it's valid base64 / base64url encoded?

See T392633#10776362 for a full list of session tokens. We plan to treat everything other than OAuth 2 and session cookies as anonymous for rate limiting purposes, so I imagine you don't care about validating those. That leaves:

• OAuth 2 bearer tokens: already a JWT, although the contents will probably change somewhat (compare JWT fields in the task description of T399198 with the ones in T399198#11006680)
• <wikiId>Session cookie: 32-character random base-32 string
• <wikiId>Token, centralauth_Session, centralauth_Token cookies: 32-character random hex string
• <wikiId>UserID cookie: integer (assigned sequentially, in practice up to 8 digits right now)
• <wikiId>UserName, centralauth_User cookies: string, between 1-255 characters, more or less arbitrary

I don't think there is much point in validating the session cookies though, unless you just want to test the validation feature - they are easy to fake correctly.

@Tgr thanks, what's the source of truth for JWT keys at the moment? are the public keys being exposed somehow? (assuming some asymmetric encryption algorithm is being used)

@Tgr we're seeing requests at least targeting /w/api.php shipping a JWT token on a Authorization header missing the Bearer schema.. so something like Authorization: $TOKEN rather than Authorization: Bearer $TOKEN. I've been checking MediaWiki OAuth extension and it seems that it enforces the presence of Bearer in the header value, from https://github.com/wikimedia/mediawiki-extensions-OAuth/blob/4ba0262eaba44506ec0179bf18673d1c684c6a4d/src/ResourceServer.php#L69:

	public static function isOAuth2Request( $request ) {
		$authHeader = $request->getHeader( 'authorization' );

		// Normalize to array
		if ( is_string( $authHeader ) ) {
			$authHeader = [ $authHeader ];
		}
		if ( $authHeader && strpos( $authHeader[0], 'Bearer' ) === 0 ) {
			return true;
		}
		return false;

those requests seem to be processed as expected and not rejected with a 4XX status code. what I'm missing here?

In T400238#11043336, @Vgutierrez wrote:

what's the source of truth for JWT keys at the moment?

There are two, private MediaWiki config and private puppet. See T392647#10802093 for details.

are the public keys being exposed somehow?

Not that I'm aware.

(assuming some asymmetric encryption algorithm is being used)

Yes, RS256.

Earlier I claimed we use HS256; that was incorrect. We use that in OAuth 1, but for ID tokens (a certain kind of API response for our semi-OIDC protocol), not for session tokens.

In T400238#11044178, @Vgutierrez wrote:

@Tgr we're seeing requests at least targeting /w/api.php shipping a JWT token on a Authorization header missing the Bearer schema.. so something like Authorization: $TOKEN rather than Authorization: Bearer $TOKEN.

The syntax of the header is Authorization: <auth-scheme> <authorization-parameters>. We use four auth schemes in MediaWiki: Bearer for OAuth 2, OAuth for OAuth 1 (note that the Authorization header is one of several possible ways to authenticate an OAuth 1 request), CentralAuthToken for CentralAuth tokens in the REST API, NetworkSession for the NetworkSession extension (but that goes through the service mesh, not the API gateway).

If you don't see either of those four keywords, it's probably just a confused client.

those requests seem to be processed as expected and not rejected with a 4XX status code. what I'm missing here?

isOAuth2Request() returning false just means that the OAuth 2 session handler won't be the one to handle the request. So Authorization: Bearer <invalid> will result in a 404 (the OAuth 2 session handler will handle the request and will fail to validate it), Authorization: NetworkSession <invalid> similarly so, but Authorization: SomeRandomStuff ... will just be ignored by all the auth-header-related handlers, and handled by the default session handler (ie. treated as an anonymous request).

Change #1174420 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] varnish: Expand authorization method reporting

https://gerrit.wikimedia.org/r/1174420

Change #1174420 merged by Vgutierrez:

[operations/puppet@production] varnish: Expand authorization method reporting

https://gerrit.wikimedia.org/r/1174420

Change #1174726 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] cache::haproxy: Deploy public JWT key used by MW

https://gerrit.wikimedia.org/r/1174726

Change #1174726 merged by Vgutierrez:

[operations/puppet@production] cache::haproxy: Deploy public JWT key used by MW

https://gerrit.wikimedia.org/r/1174726

Change #1174731 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] cache::haproxy:: Create /etc/haproxy/jwt explicitly

https://gerrit.wikimedia.org/r/1174731

Change #1174731 merged by Vgutierrez:

[operations/puppet@production] cache::haproxy: Fix /etc/haproxy/jwt on upload

https://gerrit.wikimedia.org/r/1174731

Change #1174738 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] cache::haproxy: Get rid of recurselimit warning on upload

https://gerrit.wikimedia.org/r/1174738

Change #1174738 merged by Vgutierrez:

[operations/puppet@production] cache::haproxy: Get rid of recurselimit warning on upload

https://gerrit.wikimedia.org/r/1174738

Change #1175056 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] cache::haproxy: Validate JWT tokens issued by MW

https://gerrit.wikimedia.org/r/1175056

Change #1175056 merged by Vgutierrez:

[operations/puppet@production] cache::haproxy: Validate JWT tokens issued by MW

https://gerrit.wikimedia.org/r/1175056

Change #1175099 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] cache::haproxy: Fix JWT exp date ACL

https://gerrit.wikimedia.org/r/1175099

Change #1175099 merged by Vgutierrez:

[operations/puppet@production] cache::haproxy: Fix JWT exp date ACL

https://gerrit.wikimedia.org/r/1175099

We are currently successfully validating JWT tokens for api.wm.o, action API and rest API:

-   ReqHeader      host: api.wikimedia.org
-   ReqHeader      x-provenance: cloud=hetzner
-   ReqHeader      x-trusted-request: C

-   ReqURL         /w/api.php?action=edit&format=json&tags=BotSDC&bot=&maxlag=2
-   ReqHeader      host: commons.wikimedia.org                  
-   ReqHeader      x-provenance: isp=vodafone germany
-   ReqHeader      x-trusted-request: C

-   ReqURL         /w/rest.php/wikibase/v1/entities/items/Q1650915/labels/en
-   ReqHeader      x-provenance: cloud=gcp                                              
-   ReqHeader      x-trusted-request: C

Change #1225598 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] cache::haproxy: Consider sessionJwt cookie for JWT validation purposes

https://gerrit.wikimedia.org/r/1225598

Change #1225598 merged by Vgutierrez:

[operations/puppet@production] cache::haproxy: Consider sessionJwt cookie for JWT validation purposes

https://gerrit.wikimedia.org/r/1225598

Change #1225633 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] cache::haproxy: Support sxp JWT field

https://gerrit.wikimedia.org/r/1225633

Change #1225633 merged by Vgutierrez:

[operations/puppet@production] cache::haproxy: Support sxp JWT field

https://gerrit.wikimedia.org/r/1225633

Change #1225636 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] cache::haproxy: Fix X-JWT-Sub value

https://gerrit.wikimedia.org/r/1225636

Change #1225636 merged by Vgutierrez:

[operations/puppet@production] cache::haproxy: Fix X-JWT-Sub value

https://gerrit.wikimedia.org/r/1225636

Change #1305923 had a related patch set uploaded (by BCornwall; author: BCornwall):

[operations/puppet@production] haproxy: Enable jwt for upload clusters

https://gerrit.wikimedia.org/r/1305923

Change #1305923 merged by BCornwall:

[operations/puppet@production] cache::haproxy: Enable jwt for upload clusters

https://gerrit.wikimedia.org/r/1305923

JWT support has been enabled for upload as well - this will help craft request limits that can be more relaxed for authenticated users.