Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Restricted Task | |||||
| Resolved | • Bstorm | T246122 Upgrade the Toolforge Kubernetes cluster to v1.16 | |||
| Resolved | • Bstorm | T211096 PAWS: Rebuild and upgrade Kubernetes | |||
| Open | None | T192237 Support administrative actions in PAWS | |||
| Open | None | T243200 Move PAWS to OAuth 2.0 | |||
| Open | Feature | None | T323849 pywikibot: Support OAuth2 token |
Event Timeline
They are not, if this will be implemented by modifying OAuth 1 implementation (extension of methods and classes) in Pywikibot. But for completely new separate implementation Pywikibot login needs to be refactored.
Removing task assignee due to inactivity as this open task has been assigned for more than two years. See the email sent to the task assignee on August 22nd, 2022.
Please assign this task to yourself again if you still realistically [plan to] work on this task - it would be welcome!
If this task has been resolved in the meantime, or should not be worked on ("declined"), please update its task status via "Add Action… 🡒 Change Status".
Also see https://www.mediawiki.org/wiki/Bug_management/Assignee_cleanup for tips how to best manage your individual work in Phabricator. Thanks!
As mentioned in T192237 it would appear that this can go in when pywikibot 8 is released
in https://github.com/toolforge/paws/pull/284
minikube gets as far as some new, to me, errors (kubectl logs deployment.apps/hub)
[I 2023-05-17 20:47:53.664 JupyterHub log:186] 302 GET /hub/oauth_login?next=%2Fhub%2F -> https://meta.wikimedia.org/w/rest.php/oauth2/authorize?response_type=code&redirect_uri=http%3A%2F%2Fhub.paws.local%2Fhub%2Foauth_callback&client_id=b4736b99b3b0c7c0831572529d7ea6f8&state=[secret]&scope=highvolume+editpage+createeditmovepage+patrol+uploadfile+uploadeditmovefile+rollback+viewmywatchlist+editmywatchlist (@192.168.49.1) 1.14ms
[E 2023-05-17 20:47:56.750 JupyterHub oauth2:386] Error fetching 400 POST https://meta.wikimedia.org/w/rest.php/oauth2/access_token: {
"error": "invalid_request",
"error_description": "The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed.",
"hint": "Check the `grant_type` parameter",
"message": "The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed."Anyone, perhaps @bd808, have any thoughts on if the request is malformed or perhaps something else is afoot?
When you register for oauth authentication (https://www.mediawiki.org/wiki/OAuth/For_Developers#Registration), you have to give also a callback url, if that callback does not match the parameter you are passing when authenticating you'll get an error similar to what you are showing there.
You can check the callback by going to https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration/list, search for the one you are using then click details and there will be an OAuth "callback URL" field there.
Oh I see yes I think that I have that https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/b4736b99b3b0c7c0831572529d7ea6f8
The request goes from minikube, then out to oauth where it gives the:
Hi VRook (WMF),
In order to complete your request, oauth2 test for PAWS dev needs permission to perform the following actions on your behalf on all projects of this site:
...
Page.
After which if I reload the hub.paws.local page I get:
400 : Bad Request OAuth state missing from cookies
I'm seeing the error from minikube. Not sure if there is somewhere we can view errors directly from oauth? Perhaps oauth2 requires https?
That 400 error is given by paws right?
Does that happen the first time you get redirected from meta.wikimedia.org to paws (when accepting the permission request)? Or does it happen after reloading?
It looks like somehow the cookie data is lost (maybe set for the wrong domain, or paws is unable to set it or something).
If you are able to capture the whole thing might help you debug (on firefox, control+shift+i -> network -> to the little gear on the right side -> persist logs).
Then you can check which cookies get set for which domains and where are they lost and such.
Correct, from paws. And yes after a reload, gives 500 : Internal Server Error before a reload. Sorry I forgot to add all the steps above.
Oh, ok! then I thin that there's some error when going to the callback and then paws is unable to set the cookie, so when reloading you don't have it.
Are there any logs on paws side about the 500?
I think that the issue is probably there, paws might be expecting some data back from meta that is not there or similar and panics.
I believe this is the whole error. I haven't been able to get much from it. Perhaps it offers you guidance?
[E 2023-05-23 13:09:54.290 JupyterHub oauth2:386] Error fetching 400 POST https://meta.wikimedia.org/w/rest.php/oauth2/access_token: {
"error": "invalid_request",
"error_description": "The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed.",
"hint": "Check the `grant_type` parameter",
"message": "The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed."
}
[E 2023-05-23 13:09:54.290 JupyterHub web:1798] Uncaught exception GET /hub/oauth_callback?code=<long code>&state=<state base64> (192.168.49.1)
HTTPServerRequest(protocol='http', host='hub.paws.local', method='GET', uri='/hub/oauth_callback?code=<long code>&state=<state base64>', version='HTTP/1.1', remote_ip='192.168.49.1')
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/tornado/web.py", line 1713, in _execute
result = await result
File "/usr/local/lib/python3.9/site-packages/oauthenticator/oauth2.py", line 222, in get
user = await self.login_user()
File "/usr/local/lib/python3.9/site-packages/jupyterhub/handlers/base.py", line 801, in login_user
authenticated = await self.authenticate(data)
File "/usr/local/lib/python3.9/site-packages/jupyterhub/auth.py", line 491, in get_authenticated_user
authenticated = await maybe_future(self.authenticate(handler, data))
File "/usr/local/lib/python3.9/site-packages/oauthenticator/auth0.py", line 107, in authenticate
resp_json = await self.fetch(req)
File "/usr/local/lib/python3.9/site-packages/oauthenticator/oauth2.py", line 387, in fetch
raise e
File "/usr/local/lib/python3.9/site-packages/oauthenticator/oauth2.py", line 366, in fetch
resp = await self.http_client.fetch(req, **kwargs)
tornado.httpclient.HTTPClientError: HTTP 400: Bad Request
[E 2023-05-23 13:09:54.292 JupyterHub log:178] {
"Upgrade-Insecure-Requests": "1",
"Cookie": "host=[secret]; oauthenticator-state=[secret]",
"Referer": "https://meta.wikimedia.org/",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "en-US,en;q=0.5",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0",
"X-Scheme": "http",
"X-Forwarded-Scheme": "http",
"X-Forwarded-Proto": "http,http",
"X-Forwarded-Port": "80,80",
"X-Forwarded-Host": "hub.paws.local",
"X-Forwarded-For": "192.168.49.1,::ffff:172.17.0.4",
"X-Real-Ip": "192.168.49.1",
"X-Request-Id": "09b343171a13e9ccc904b4940fbd21e4",
"Host": "hub.paws.local",
"Connection": "close"
}
[E 2023-05-23 13:09:54.292 JupyterHub log:186] 500 GET /hub/oauth_callback?code=[secret]&state=[secret] (@192.168.49.1) 201.18msHmm, it hints to a grant_type parameter that you don't seem to be sending.
From an app I have that does it "bare" (no libraries), I'm sending that parameter after the first callback, when trying to get the access_token for future requests:
https://github.com/david-caro/wm-what/blob/main/wm_what/app.py#LL182C1-L192C57
That makes sense, the step it's failing on is https://jupyterhub.readthedocs.io/en/stable/explanation/oauth.html#oauth-client-handles-callback-request
Not sure what code are you using, but it might be adding those params to https://github.com/jupyterhub/oauthenticator/blob/main/oauthenticator/generic.py#LL18C1-L18C52
Oh, there's a mediawiki specific one :) https://oauthenticator.readthedocs.io/en/latest/reference/api/gen/oauthenticator.mediawiki.html
The mediawiki one had been working for oauth1, it didn't seem like it would work for oauth2? I only saw mention of using jupyter itself to act as an oauth provider, perhaps I misread that though. I had been using https://oauthenticator.readthedocs.io/en/latest/reference/api/gen/oauthenticator.auth0.html Perhaps that simply won't work.
Code is in https://github.com/toolforge/paws/pull/284 Though it is not very pretty as there are still plenty of artifacts
Ack, in that one, you can try setting the token_params config: https://oauthenticator.readthedocs.io/en/latest/reference/api/gen/oauthenticator.auth0.html#oauthenticator.auth0.Auth0OAuthenticator.token_params
You might only need to add the 'grant_type' one, I'd start with the value of 'authorization_code'.
Sorry, I suspect I'm being dense. Where do I learn more about the grant_type and authorization_code. I see the former in the errors, though I do not know what it would look like when set, I assume as part of c.Auth0OAuthenticator.token_params. I'm assuming authorization_code is either another token, or a subset of grant_type?
No problem, asking is better than not xd
Where do I learn more about the grant_type and authorization_code.
Here you can see some python code using it https://github.com/david-caro/wm-what/blob/main/wm_what/app.py#LL182C1-L192C57
Here there's some docs, though they don't explain in detail why that value: https://www.mediawiki.org/wiki/OAuth/For_Developers#Authorization_2 (point 2)
Here's some docs from the standard itself: https://oauth.net/2/grant-types/
I see the former in the errors, though I do not know what it would look like when set, I assume as part of c.Auth0OAuthenticator.token_params. I'm assuming authorization_code is either another token, or a subset of grant_type?
It's the value for it, I think that something like this will do:
c.Auth0OAuthenticator.token_params = {
"grant_type": "authorization_code",
}Note that this is literal, the grant_type value is the string authorization_code.
I went back to the mediawiki library. Though still not getting too far
Sending requests with default User-Agent. Set 'user_agent' on mwoauth.flask.MWOAuth to quiet this message.
[E 2023-05-24 15:17:20.542 JupyterHub web:1798] Uncaught exception GET /hub/oauth_login?next=%2Fhub%2F (192.168.49.1)
HTTPServerRequest(protocol='http', host='hub.paws.local', method='GET', uri='/hub/oauth_login?next=%2Fhub%2F', version='HTTP/1.1', remote_ip='192.168.49.1')
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/tornado/web.py", line 1713, in _execute
result = await result
File "/usr/local/lib/python3.9/site-packages/oauthenticator/mediawiki.py", line 48, in get
redirect, request_token = await wrap_future(
File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.9/site-packages/mwoauth/handshaker.py", line 87, in initiate
return initiate(self.mw_uri, self.consumer_token,
File "/usr/local/lib/python3.9/site-packages/mwoauth/functions.py", line 93, in initiate
request_token = process_request_token(r.text)
File "/usr/local/lib/python3.9/site-packages/mwoauth/functions.py", line 109, in process_request_token
raise OAuthException(
mwoauth.errors.OAuthException: Expected x-www-form-urlencoded response from MediaWiki, but got something else: '{"messageTranslations":{"en":"The request method (POST) was not one of the allowed methods for this path (HEAD, GET)"},"httpCode":405,"httpReason":"Method Not Allowed"}'
[E 2023-05-24 15:17:20.567 JupyterHub log:178] {
"Upgrade-Insecure-Requests": "1",
"Cookie": "host=[secret]",
"Referer": "http://hub.paws.local/hub/login?next=%2Fhub%2F",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "en-US,en;q=0.5",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0",
"X-Scheme": "http",
"X-Forwarded-Scheme": "http",
"X-Forwarded-Proto": "http,http",
"X-Forwarded-Port": "80,80",
"X-Forwarded-Host": "hub.paws.local",
"X-Forwarded-For": "192.168.49.1,::ffff:172.17.0.6",
"X-Real-Ip": "192.168.49.1",
"X-Request-Id": "c230c543eeb3f3379db52fa54a198e1b",
"Host": "hub.paws.local",
"Connection": "close"
}
[E 2023-05-24 15:17:20.567 JupyterHub log:186] 500 GET /hub/oauth_login?next=%2Fhub%2F (@192.168.49.1) 160.39msI'm finding information in the docs about switching the POST method in a token request but not finding as much about the key request. Code is updated on the PR. Any guidance?
I suspect that the POST request might not be to the right path, the request is done here I think:
https://github.com/mediawiki-utilities/python-mwoauth/blob/master/mwoauth/functions.py#LL88C23-L91C58
what is the mw_uri value there? Maybe you can add a print or similar xd
I think though that it's the setting:
c.MWOAuthenticator.mw_index_url = 'https://meta.wikimedia.org/w/rest.php/oauth2/authorize'
from your PR, that seems that it should point to 'https://meta.wikimedia.org/index.php' instead (the index.php), can you try with this?
c.MWOAuthenticator.mw_index_url = 'https://meta.wikimedia.org/w/index.php'
btw. the value I got from the default here https://oauthenticator.readthedocs.io/en/latest/reference/api/gen/oauthenticator.mediawiki.html#oauthenticator.mediawiki.MWOAuthenticator.mw_index_url
[15:46] < bd808> I think the general problem is that oauthenticator.mediawiki uses https://github.com/mediawiki-utilities/python-mwoauth to actually handle the handshake. mwoauth is an OAuth 1.0a client, not an OAuth 2.0 client. [15:47] < bd808> We can try to use https://github.com/jupyterhub/oauthenticator/blob/main/oauthenticator/oauth2.py to make a custom client. That would be roughly what we did for superset.
oauth2 doesn't seem to be importing to jupyter.
[C 2023-05-26 12:02:21.886 JupyterHub application:112] Bad config encountered during initialization: The 'authenticator_class' trait of <jupyterhub.app.JupyterHub object at 0x7f9e9bf65430> instance must be a type, but 'oauth2' could not be imported
Trying with @dcaro's examples I get caught by flask not being included in the hub container. I guess it could be introduced, but I'm not immediately sure how, and it seems kind of odd to do.
Yuvi introduced https://github.com/jupyterhub/oauthenticator/blob/main/oauthenticator/mediawiki.py upstream to support MediaWiki's OAuth 1.0a service. I think the broadly correct thing to do to implement support for MediaWiki's OAuth 2.0 service would be to upstream another MediaWiki specific OAuthenticator class. The new one should extend the classes from https://github.com/jupyterhub/oauthenticator/blob/main/oauthenticator/oauth2.py and would likely be very similar to the existing upstream authenticators for other OAuth 2.0 services such as their GitHub and GitLab classes.