Pywikibot: Implement support for OAuth
Closed, ResolvedPublic

Description

It would be useful if bots on Tool-Labs did not need to store a password on the shared servers. The worst case for a stolen password is someone logging in and changing e-mail address, password etc. to lock the user out of their own account.

MediaWiki supports OAuth as a method of authentication. More information about https://www.mediawiki.org/wiki/OAuth/For_Developers

With OAuth, only a token needs to be stored in the shared environment, and in worst case scenario someone can make a few edits with the token, but it can be revoked at any time, and the malicious user can not lock out the rightful user.

This task is to implement OAuth support in pywikibot. To complete this task, a unit test should be added to the test suite to perform a login and logout using OAuth with assertions that verify APISite._userinfo is correct, and a second unit test should login, edit a userpage, and confirm the edit was performed using the OAuth-authenticated account. The unit test should be configured to run on travis-ci when the secret key is available in the Travis configuration, and skipped when it isnt.

To get started testing your OAuth integration, register at https://www.mediawiki.org/wiki/Special:OAuthConsumerRegistration/propose while logged in as your mediawiki.org user. You will probably want to use https://www.mediawiki.org/wiki/Special:OAuth/verified for the callback url. Leave the "Public RSA key" field blank. You will be able to use the key and secret that you receive when you complete the registration, as long as you approve it using the same mediawiki.org user that you used to register the application.

Suggested micro-task: Fix one of the bugs in Pywikibot-login.py (not many easy ones left) Pywikibot-network (T72965 is easy), Pywikibot-tests (e.g. T73817)

Possible-Tech-Projects checklist:

  • Primary mentor: @jayvdb (John Vandenberg)
  • Co-mentor: @Halfak (Aaron Halfaker)
  • Skills: OAuth, Python, MediaWiki, Travis-CI
  • Estimated project time for a senior contributor: 2-3 weeks
  • Community consensus: Yes

Details

Reference
bz72065

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes
jayvdb moved this task from Backlog to pywikibot on the Google-Code-in-2014 board.Dec 13 2014, 2:18 AM

I've drafted a Google-Code-in-2014 task for this at http://www.google-melange.com/gci/task/view/google/gci2014/5824634897301504 . I'd like a co-mentor with experience using MW OAuth before this is published.

Maybe @Halfak, @FiloSottile or @yuvipanda are interested (cf. https://github.com/wikimedia/MediaWiki-OAuth )?

MediaWiki-OAuth is *probably* the project you want to use for this.

A rough outline of how this could work:

  1. the user needs to register their own application. There is no API for this, so the user has to do this on-site, and has to select which permissions will be given.. This gives us consumer_key and consumer_secret, which need to be set somewhere.
  2. The user then needs to log in with that token, using the OAuth handshake shown on https://github.com/wikimedia/MediaWiki-OAuth
  3. We then have to sign requests. I'm not sure how to do this -- the underlying OAuth library ( https://pypi.python.org/pypi/oauthlib ) probably provides this?

This is not really convenient, though, so I'm not sure if @Krinkle's use case is actually covered well by the existing MediaWiki OAuth implementation.

Alternatively, we can provide a consumer_key and not-so-secret consumer_secret in pywikibot itself (that's how Google suggests to solve this workflow in their API)

jayvdb added a subscriber: csteipp.Jan 6 2015, 11:23 PM

I'd like to support this work on top of the MediaWiki-OAuth library. I'll commit to responding quickly re. necessary changes to support what's needed.

As for signing requests, https://github.com/wikimedia/MediaWiki-OAuth/blob/master/examples/request-oauthlib.py shows how you can do that with requests_oauthlib and MediaWiki-OAuth. Note "auth=auth1" on line 41. 'requests' handles the rest.

@jayvdb I'm interested in co-mentoring. Please contact me about what kind of commitment you'd need.

The Google Code-in is over, so the next opportunity will be #GSOC2015 (https://www.mediawiki.org/wiki/Google_Summer_of_Code_2015).

I've added this on https://www.mediawiki.org/wiki/Outreach_programs/Possible_projects#OAuth_support , with @Halfak as a co-mentor.

Halfak added a comment.EditedJan 30 2015, 3:42 PM

Looks good. Thanks for adding the project @jayvdb.

jayvdb updated the task description. (Show Details)Feb 10 2015, 1:28 AM
jayvdb set Security to None.
Qgil added a subscriber: Qgil.Feb 10 2015, 9:45 AM

@jayvdb, as part of T76341, I have documented the requirements for Featured Possible-Tech-Projects here. This is why I have moved this one to "Need Discussion". In every GSoC / OPW round we have seen that projects with a low score in that list of requirements had a lot more difficulties during the program, and I believe we need to be more strict for everybody's benefit.

Please assess this project accordingly.

@Qgil, which part of that definition of Featured are you concerned about?

fwiw, I feel I could do this task in 2-3 weeks. There are already existing implementations in Python, in use by other MW python libraries. I've purposely designed this task so it is only a MVP with very clear acceptance criteria (see task sentence "To complete this task ..."), as a full blown implementation is a much bigger task. I'd like to hear from @Halfak whether he feels that would be a reasonable period of time for him to implement this feature, as a bit of a sanity check.

Qgil added a comment.Feb 10 2015, 10:25 AM

I'm only concerned about all Featured projects having that checklist spelled and complete in the task description.

valhallasw updated the task description. (Show Details)Feb 10 2015, 10:33 AM
Halfak added a comment.EditedFeb 10 2015, 3:49 PM

@jayvdb I'm not very familiar with pywikibot's inner workings, but I expect that we'll have little trouble working with mwoauth. State is easy to manage with tokens and the whole thing is pickleable.

My primary concern is the use of httplib2. It doesn't seem to have any support for oauth, so we'd need to either implement a request signing ourselves or switch to 'requests' and 'requests-oauthlib' which are used inside of 'mwoauth'. At a glance, making the switch doesn't seem too difficult. requests.Session closely matches the behavior of httplib2.Http.

Sitic added a subscriber: Sitic.Feb 10 2015, 3:55 PM

@Halfak, re httplib2 , it doesnt have native support for oauth, but there are full-blown libraries around ( https://github.com/google/oauth2client/tree/master/oauth2client ), and also simple clients we could use as a template for a custom implementation (e.g.
https://github.com/tswicegood/httplib2-oauth/blob/master/oauthclient/base.py ).

Given your involvement, I think it would be wise to try to use requests-oauthlib . I've tested it in py2.6 and provided a patch for one of its dependency bugs. https://github.com/requests/requests-oauthlib/pull/167

If we use requests-oauthlib, could we isolate the use of requests to only the Oauth negotiation, and copy the credentials from the requests session into a httplib2 session?
Then only the login phase needs to use requests, and then httplib2 would be used for the rest of the workload. That would be simpler than a complete switch to requests.

I think that our biggest concern will be signing new requests with the AccessToken. requests-oauthlib/requests provides native support for this. I don't see how we can do that easily with httplib2.

It seems that the base oauthlib would be our best bet for getting onto the platform that is post popular and well documented. See https://oauthlib.readthedocs.org/en/latest/ requests-oauthlib is built on top of that. Also note that MediaWiki supports OAuth 1.0, so that oauth2client lib won't work for us.

If we do go the route of implementing our own request signing strategy with oauthlib, we'll probably want to start something like httplib2-oauthlib. I suspect this will be hard to do (maybe more difficult than writing out the httplib2 bits) and need some help from someone like @csteipp to get it reviewed.

That would be simpler than a complete switch to requests.

This sounded like a challenge ;-) I *think* this is pretty doable; I hacked up a change from httplib2 to requests in https://gerrit.wikimedia.org/r/#/c/189821/ .

Nemo_bis removed a subscriber: Nemo_bis.Feb 11 2015, 8:48 AM
Qgil added a comment.Feb 11 2015, 1:44 PM

Wikimedia will apply to Google Summer of Code and Outreachy on Tuesday, February 17. If you want this task to become a featured project idea, please follow these instructions.

Hello,

I plan to attempt this task for Google Summer of Code. I'm just now getting used to the code, getting used to GIT, and gerrit. I have next week off classes so I'll probably put some time into this then, and I'm going to do a little bit today as well. I've also looked over some of the reported bugs for login.py, and looks like some of the easy ones already have patches proposed for them.

Hi @Evanontario, yes we've had a few other people interested in attempting this task, and doing the microtasks. There are still tasks which do not have good solutions for them, and dont have unit tests, so feel free to participate in code reviews, or build unit tests. However, Pywikibot-network is also a suitable place to find microtasks for this oauth project.

Hi,

I am planning to start contributing to mediawiki and participate in GSoC this year. I found this project very interesting and have tried using pywikibot. Also, now I am going through the code base and looking at some bugs to solve.

I would be grateful if someone could recommend me some simple tasks from Pywikibot-network or elsewhere to get started. Also, if there is anything else I could do to understand the project better, please let me know.

Hello,

I am planning on participating in the GSoC 2015 and came across this project. I would really appreciate some pointers to get started with. Please provide me some pointers for getting started with.

jayvdb updated the task description. (Show Details)Mar 12 2015, 3:37 AM
Qgil added a comment.Mar 18 2015, 9:27 AM

@jayvdb, @Halfak, a proposal has been submitted by Sampad (only mentors and the candidate can access to it). Please request to become mentors in Google Melange.

Sampa, thank you for your interest in this project idea. You need to create a subtask here, as explained in our GSoC 2015 page.

Nv1620 added a subscriber: Nv1620.Mar 19 2015, 8:25 PM

We're holding an IRC meeting on March 25, at 1700 UTC for prospective GSoC and Outreachy participants with Wikimedia, on #wikimedia-office channel. Do join us!

Hello! The IRC meeting tomorrow has been shifted to #wikimedia-ect channel. Looking forward to seeing you there. :)

@Halfak we need you to sign up on Melange as a mentor in order to evaluate student proposals. Thanks!

Halfak added a comment.Apr 9 2015, 2:46 PM

I've signed up as 'halfak' and requested to be added to "Wikimedia Foundation".

@Halfak Thanks! I added you as a mentor. There seem to be three proposals in for this project. Your comments and ratings are welcome. Ratings are explained at: https://www.mediawiki.org/wiki/Outreach_programs/Possible_mentors#Before

Hello. The weekly reports task for this project has still not been created. Please check T100998: [tracking] GSoC 2015 and Outreachy 10 Weekly reports and create it asap.

Not to be seen in compat.

Hello!

End of GSoC is fast approaching. 17 August is "Suggested pencils down" deadline and 21 August is "Firm pencils down" deadline. It is expected that you don't dive into new features which might take longer than two weeks to complete and instead work on polishing up your project, testing thoroughly and getting your code merged into the main branch. I hope this project is almost complete so you can merge it and make it available to everyone as quickly as possible. :)

A few questions (for both mentors and student):

  • Are you confident in completing the project on time?
  • By when do you think you can merge the code, if at all?
  • Are there any major blockers or important missing features?

We are looking for projects which are (nearly) complete to feature on our post on Wikimedia and Google OSPO's blogs (for example: http://google-opensource.blogspot.in/2015/02/google-summer-of-code-wrap-up-processing.html). If you're interested in getting yours up there, hurry up and get this finished!

The hard deadline on getting code merged is September. T101393: Goal: All completed GSoC and Outreachy projects have code merged and deployed by September for details.

We'll be asking the students to demo their projects towards the end of the program as well.

Good luck!

Hi, I have associated two blocked-by tasks with this project.

For the student:

  1. Please go through the checklist in the end-term evaluation and fill out the fields which require any links. The checkboxes are for the mentor(s) only. Adding information on the past projects page is your task.
  2. Ensure that you have completed all the items listed in the end-term evaluation task. If there's a strong reason about why a particular item was not completed, please comment on the task and we shall look into it.
  3. Wrap-up report is mandatory and so is a demo-able link to the project (either in production or in a demo server).
  4. If you want your project to be featured in the blogpost on the Google OSPO blog, kindly comment back with a short, catchy description of the project along with a screenshot.