I've noticed that if I go to beta meta, which logs me in to beta meta, and then revisit beta API Portal, I then see my clients.

Cindy and I are thinking that this is likely as issue with Central Auth and cookies. @Pchelolo would you be willing to help troubleshoot?

As far as I can tell, the steps to reproduce are:

1. Create an OAuth 2.0 client on the beta API Portal or on beta meta
2. Make sure you are logged out of beta meta
3. Log in to API Portal beta and go to Special:AppManagement
4. See that your client isn't displayed
5. Go back to beta meta and log in
6. Go back to the beta portal and see that your client is now displayed

It would be helpful if when this happens you could open a browser developer console and see what's going on with requests and if there's any errors. And post it here

Need to add the portal to $wgCentralAuthAutoLoginWikis

Change 632322 had a related patch set uploaded (by Cicalese; owner: Cicalese):
[operations/mediawiki-config@master] Add API Portal to $wgCentralAuthAutoLoginWikis - beta

https://gerrit.wikimedia.org/r/632322

Change 632323 had a related patch set uploaded (by Cicalese; owner: Cicalese):
[operations/mediawiki-config@master] Add API Portal to $wgCentralAuthAutoLoginWikis - prod

https://gerrit.wikimedia.org/r/632323

Change 632322 merged by jenkins-bot:
[operations/mediawiki-config@master] Add API Portal to $wgCentralAuthAutoLoginWikis - beta

https://gerrit.wikimedia.org/r/632322

Additionally, we need to unblock subpages of Special:CentralAutoLogin/, in particular Special:CentralAutoLogin/start

Change 632484 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/mediawiki-config@master] Allow read of CentralAuth special pages on api portal

https://gerrit.wikimedia.org/r/632484

Change 632484 merged by jenkins-bot:
[operations/mediawiki-config@master] Allow read of CentralAuth special pages on api portal

https://gerrit.wikimedia.org/r/632484

Mentioned in SAL (#wikimedia-operations) [2020-10-06T18:12:28Z] <ppchelko@deploy1001> Synchronized wmf-config/InitialiseSettings.php: gerrit:632484 T264637 (duration: 00m 58s)

Change 632323 merged by jenkins-bot:
[operations/mediawiki-config@master] Add API Portal to $wgCentralAuthAutoLoginWikis - prod

https://gerrit.wikimedia.org/r/632323

Mentioned in SAL (#wikimedia-operations) [2020-10-06T18:15:44Z] <ppchelko@deploy1001> Synchronized wmf-config/InitialiseSettings.php: gerrit:632323 T264637 (duration: 00m 58s)

@tstarling do you have any thoughts / input for this ticket?

In T264637#6518601, @apaskulin wrote:

3. Log in to API Portal beta and go to Special:AppManagement
4. See that your client isn't displayed

That's a bit different to the task description -- so you are logged in to the wiki, according to the navigation etc., but the special page still fails? That sounds like a problem with the special page.

That's a bit different to the task description -- so you are logged in to the wiki, according to the navigation etc., but the special page still fails? That sounds like a problem with the special page.

This particular special page is a dynamic JS page which makes cross-origin requests to REST API, only available on meta-wiki, with allow-credentials. When the user is logged into meta, it all works well - the meta cookie gets sent alongside with the request. However, it sometimes does not work because logging into api-portal wiki does not log you onto meta automatically.

Is there a good reason for doing it that way?

In T264637#6589236, @tstarling wrote:

Is there a good reason for doing it that way?

TLDR: the users should be able to manage their OAuth clients from the api-portal wiki per @eprodromou. On the backend, OAuth consumers exist in the database in the context of meta wiki. Doing cross-domain writes in OAuth extension, so it could be enabled on api-portal was dismissed by Brad as too hard, so we went for CORS requests in JS.

There's a thing called mw.ForeignApi which is apparently a fully-worked solution for this problem. There is ext.centralauth.ForeignApi.js which is purportedly a plugin for mw.ForeignApi which handles cross-wiki API requests while ensuring that the user will be logged in to the remote wiki.

It's non-trivial to port it from the action API to the REST API, but the method it uses seems like the right way to do it. It fetches a login token using action=centralauthtoken on the local action API. Then it adds a centralauthtoken parameter to the foreign request. On the foreign side, if MW_API is defined, CentralAuthTokenSessionProvider reads the centralauthtoken parameter and uses it to log in.

$wgCentralAuthAutoLoginWikis would probably work, but it's inefficient, adding an extra request to every login, and it's unreliable, since the meta cookie may expire or be removed, or the user may navigate away before the post-login requests complete.

In T264637#6589237, @Pchelolo wrote:

In T264637#6589236, @tstarling wrote:

Is there a good reason for doing it that way?

TLDR: the users should be able to manage their OAuth clients from the api-portal wiki per @eprodromou.

Could the client-side code call a proxy on api.w.o that passes through to meta.w.o on the server-side?

(EDIT: I don't think this is necessary; see comment below.)

Is this our problem?

https://stackoverflow.com/questions/42803394/cors-credentials-mode-is-include

https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS/Errors/CORSNotSupportingCredentials

I don't think we can turn off the "withCredentials" flag, since that's necessary to identify the user.

If we turn on automated central auth for both sites, and change the Access-Control-Allow-Origin header in the pre-flight request from * to the actual referring site (api.beta.wmflabs.org or api.wikimedia.org for beta and production respectively), will it work?

At least for the beta sites, it looks like the OPTIONS request will send back the explicit origin, but the GET request returns *. It would be interesting to see what happens if the GET request returns the whitelisted origin instead.

# With OPTIONS
 
Evans-MBP:~ eprodromou$ curl -i -X OPTIONS -H 'Origin: https://api.wikimedia.beta.wmflabs.org' 'https://meta.wikimedia.beta.wmflabs.org/w/rest.php/oauth2/client?limit=5&oauth_version=2&sort=%7B%22property%22%3A%22registration%22%2C%22direction%22%3A%22DESC%22%7D'
HTTP/2 204 
date: Tue, 17 Nov 2020 18:16:15 GMT
server: deployment-mediawiki-07.deployment-prep.eqiad1.wikimedia.cloud
x-content-type-options: nosniff
cache-control: no-cache
p3p: CP="See https://meta.wikimedia.beta.wmflabs.org/wiki/Special:CentralAutoLogin/P3P for more info."
access-control-allow-headers: Authorization, Content-Type, If-Mach, If-None-Match, If-Modified-Since
access-control-allow-methods: HEAD, GET, POST
vary: Origin
access-control-allow-credentials: true
access-control-allow-origin: https://api.wikimedia.beta.wmflabs.org
x-request-id: X7QTb6wQBHcAAF1ukOEAAAAD
content-type: text/html; charset=UTF-8
age: 0
x-cache: deployment-cache-text06 pass, deployment-cache-text06 pass
x-cache-status: pass
server-timing: cache;desc="pass"
report-to: { "group": "wm_nel", "max_age": 86400, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] }
nel: { "report_to": "wm_nel", "max_age": 86400, "failure_fraction": 0.05, "success_fraction": 0.0}
set-cookie: WMF-Last-Access=17-Nov-2020;Path=/;HttpOnly;secure;Expires=Sat, 19 Dec 2020 12:00:00 GMT
set-cookie: WMF-Last-Access-Global=17-Nov-2020;Path=/;Domain=.wikimedia.beta.wmflabs.org;HttpOnly;secure;Expires=Sat, 19 Dec 2020 12:00:00 GMT
x-client-ip: 66.130.124.134
set-cookie: GeoIP=CA:QC:Boisbriand:45.61:-73.84:v4; Path=/; secure; Domain=.beta.wmflabs.org
 
# With GET
 
Evans-MBP:~ eprodromou$ curl -i -X GET -H 'Origin: https://api.wikimedia.beta.wmflabs.org' 'https://meta.wikimedia.beta.wmflabs.org/w/rest.php/oauth2/client?limit=5&oauth_version=2&sort=%7B%22property%22%3A%22registration%22%2C%22direction%22%3A%22DESC%22%7D'
HTTP/2 404 
date: Tue, 17 Nov 2020 18:17:05 GMT
server: deployment-mediawiki-07.deployment-prep.eqiad1.wikimedia.cloud
x-content-type-options: nosniff
cache-control: no-cache
p3p: CP="See https://meta.wikimedia.beta.wmflabs.org/wiki/Special:CentralAutoLogin/P3P for more info."
access-control-allow-origin: *
x-request-id: X7QToawQBHcAAF1ukOUAAAAR
content-type: application/json
vary: Origin, Accept-Encoding
age: 0
x-cache: deployment-cache-text06 miss, deployment-cache-text06 pass
x-cache-status: pass
server-timing: cache;desc="pass"
report-to: { "group": "wm_nel", "max_age": 86400, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] }
nel: { "report_to": "wm_nel", "max_age": 86400, "failure_fraction": 0.05, "success_fraction": 0.0}
set-cookie: WMF-Last-Access=17-Nov-2020;Path=/;HttpOnly;secure;Expires=Sat, 19 Dec 2020 12:00:00 GMT
set-cookie: WMF-Last-Access-Global=17-Nov-2020;Path=/;Domain=.wikimedia.beta.wmflabs.org;HttpOnly;secure;Expires=Sat, 19 Dec 2020 12:00:00 GMT
x-client-ip: 66.130.124.134
set-cookie: GeoIP=CA:QC:Boisbriand:45.61:-73.84:v4; Path=/; secure; Domain=.beta.wmflabs.org
content-length: 123
 
{"messageTranslations":{"en":"The specified user (66.130.124.134) does not exist"},"httpCode":404,"httpReason":"Not Found"}

Authenticated GET requests will return origin. anon get requests will return '*'

In T264637#6628649, @Pchelolo wrote:

Authenticated GET requests will return origin. anon get requests will return '*'

That would explain it! So, the problem isn't that the CORS header that's being returned is wrong; it's that the authentication isn't being passed from the browser?

So, the problem isn't that the CORS header that's being returned is wrong; it's that the authentication isn't being passed from the browser?

The problem is that meta and portal auth cookies expire separately, so it's possible that while being logged in on portal you'd be logged out on meta. The solution proposed by Tim fixes that issue.

@AMooney asked me to work this up a bit further. It seems to me that the use of query strings in the action API doesn't fit well with the style of the REST API. The thing that would seem to make the most sense for the REST API would be to use the Authorization header.

Anomie and I had a bit of a discussion on T234665 about internal clients of this kind. I suggested putting access tokens for internal clients in the page, to which he responded:

We could just as well have a SessionProvider looking for "Authorization: MediaWiki $token", or "MediaWiki-Auth: $token". Which itself is basically just a version of "Cookie: foowiki_session=$token" that isn't subject to browser-based CSRF.

Well OK, I can see the desire to not add complexity to OAuth\SessionProvider. And the OAuth extension does not seem to have a working implementation of access token expiry or temporary access tokens in general, whereas CentralAuth has a central session store for this.

Here's what it would look like if it were loosely coupled, closely following the Action API approach:

• In CentralAuth, add another session provider, say CentralAuthRestSessionProvider, which looks at the Authorization header.
• In core, have a new JS library responsible for making cross-wiki REST requests, say mediawiki.ForeignRestApi. As with mediawiki.ForeignApi.core.js, the core implementation would not actually add headers.
• In CentralAuth, override mediawiki.ForeignRestApi with a version that requests a token from the local action API. Or a new REST route could be added to CentralAuth to achieve the same thing. When the token is received, the original request is extended by adding an Authorization header and then dispatched.
• In WikimediaApiPortalOAuth, instead of calling $.ajax() directly, use the new core REST client wrapper.

I'd like to know what @ItSpiderman thinks of this idea.

Would it be easier from this point to make the API call to meta on the server instead, and have a more static output page?

Would it also be possible to add an endpoint in api.w.o that only accepts session authorization, and returns a short-lived (session-length, like O(hours), or even request-length, like O(minutes)) OAuth 2.0 token? The client could call it before calling meta.w.o, then pass it along in the Authorisation header.

It wouldn't require changes to the m.w.o REST API endpoints, and would work as long as you're logged into api.w.o.

@tstarling says that we haven't set up short-term expiration in OAuth 2.0 before, but I think it's possible?

In T264637#6630516, @eprodromou wrote:

Would it be easier from this point to make the API call to meta on the server instead, and have a more static output page?

No, I think it's too late for that. And it doesn't seem to solve the problem, it just moves it to the server.

In T264637#6630526, @eprodromou wrote:

Would it also be possible to add an endpoint in api.w.o that only accepts session authorization, and returns a short-lived (session-length, like O(hours), or even request-length, like O(minutes)) OAuth 2.0 token? The client could call it before calling meta.w.o, then pass it along in the Authorisation header.

It wouldn't require changes to the m.w.o REST API endpoints, and would work as long as you're logged into api.w.o.

@tstarling says that we haven't set up short-term expiration in OAuth 2.0 before, but I think it's possible?

Yes it's possible, I just think it's harder than using CentralAuth tokens. The plan I've outlined follows an existing architecture and should not be difficult to review from a security point of view. It's complicated to add a concept of internal clients to the OAuth extension. The existing code in the OAuth extension does not provide much of value for this use case. The code for internal clients would not share much with the existing OAuth\SessionProvider, so we would effectively be duplicating the CentralAuth token feature into the OAuth extension.

In T264637#6632258, @tstarling wrote:

Would it be easier from this point to make the API call to meta on the server instead, and have a more static output page?

No, I think it's too late for that. And it doesn't seem to solve the problem, it just moves it to the server.

That's fair, but since the problem is authentication of cross-site API calls from the browser, moving it to the server would take the problem mostly away. With a server-to-server API call, we could use much simpler authentication, like a shared secret.

All that said, I'm happy with whatever works that gets the problem solved quickly.