Page MenuHomePhabricator

Enhancements to gdrive-to-commons uploader tool
Open, Needs TriagePublic

Description

This tool allows uploading pictures in bulk from your Google drive directly to Wikimedia Commons. See similar upload tools here: https://tools.wmflabs.org/admin/tools#!/search/upload%20

Current tool workflow

  1. User authorizes our app via Wikimedia OAuth.
  2. User opens Google drive through the app, authorizing read only access to our app.
  3. User select photographs to upload.
  4. User modifies information such as description, title of the image.
  5. User clicks on an upload button and the app uploads the images to Wikimedia Commons. We also return a list of URLs to the user to see their uploads on the wiki.

Feature requests (sorted from high to low priority)

  • Add more metadata to the image before upload. Date, categories, captions, license etc. See related tasks: https://phabricator.wikimedia.org/T241596, https://phabricator.wikimedia.org/T241425
  • Ensure upload through the app works exactly as it would look like an upload from Media uploader.
  • Build a stats counter to track the number of uploads processed by the app. Would help in building trust in the community. https://github.com/tonythomas01/gdrive-to-commons/issues/17
  • Other UI changes (improve the editing button, ability to preview thumbnail images, add logo, etc).
  • Infrastructure: Navigate through the Google app verification process and see what we can do to get the application verified by Google. See related task: https://phabricator.wikimedia.org/T226052
  • It would be interesting to run a feedbak/usability campaign within the community to figure out the usefulness/adoptability of the app. Probably some sort of survey which we can do on the village pump or something like that.

Skills required python-django, JS, html, CSS, python

Mentors @01tonythomas, @srishakatux

Tool link https://tools.wmflabs.org/google-drive-photos-to-commons/

Github link https://github.com/tonythomas01/gdrive-to-commons

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 22 2020, 9:01 PM

@01tonythomas That sounds wonderful :) I may not have time to mentor, but I am happy to look into the current state of the tool and suggest some ideas to you in the next few days. Also, if you can consider mentoring the project via Outreachy, that'd be great! This time we want to promote more projects via Outreachy than GSoC as with the former, we are able to pool in diverse candidates, and we have more budget for Outreachy this time. What do you think?

I may not have time to mentor, but I am happy to look into the current state of the tool and suggest some ideas to you in the next few days.

Great. We have some outstanding pull requests in the machinery though. I would suggest you look at it after I merge them, or feel free to merge them yourself. I will probably do it during the weekend.

Also, if you can consider mentoring the project via Outreachy, that'd be great!

Sure. Once we have a project, applying to either would not be a problem.

@01tonythomas I looked at the tool today after a long time, happy to see it works flawlessly <3

I think there are a few tasks mostly related to the user experience of the tool (some are already filed as issues in the Github repo) that will make a good Outreachy project. Some small tasks that could be a part of a big project something like "Further improvements to the .... tool ":

  • Moving the agreement to a separate screen
  • Hosting the tool to a new location
  • Adding a logo (if we got one via GCI) (could be one of the microtasks)
  • Improve the permissions dialog – the one you see at the time of logging into the drive via the app for the first time. It currently is scary and discourages a user from using the app.
  • Improve the user experience around editing the image details:
    • Add edit button
    • Add remove button (though I see there is a related pull request pending)
    • Ability to preview thumbnail images
    • Add fields such as date, categories, captions, etc
    • Add release rights option
  • I see that there is an issue in the repo related to compressing images, but I've no idea if that happens on Wikimedia Commons too and if this is something we should consider

Plus, user research could be made part of this project as well. Maybe, in the beginning, as well as during the mid-point phase, some feedback could be collected from the community on wikis or via survey, and necessary iterations could then be made :) And, this research work is also part of the project, done solely by the intern. What do you think? I'm happy to help with some of this stuff if needed.

@01tonythomas Thank you! I will get back to you in a day or so.. :)

@01tonythomas I've suggested some changes in your doc. As and when you are ready, feel free to modify this Phabricator task's description and upload the proposal on the Outreachy site https://www.outreachy.org/.

srishakatux renamed this task from Plan for Google drive to Commons and GSoC 2020 ? to Enhancements to gdrive-to-commons uploader tool.Feb 18 2020, 10:51 PM
srishakatux updated the task description. (Show Details)
srishakatux changed the visibility from "Public (No Login Required)" to "Outreachy Mentors (Project)".
srishakatux moved this task from Backlog to Featured projects on the Outreachy (Round 20) board.
srishakatux changed the visibility from "Outreachy Mentors (Project)" to "Public (No Login Required)".Mar 5 2020, 6:15 PM

Hello, i am an outreachy aspirant and I chose to work only on wikimedia projects. just a quick question, are we allowed to start working immediately on these features or we should focus on fixing existing bugs for now?

@Raymond_Ndibe Hi! See point 8 in the guide here: https://www.mediawiki.org/wiki/Outreachy/Participants. Between now and the application deadline, you work only on the microtasks of the project to get familiar with it. You work on planned features during the internship (if you get accepted).

Hi all. I am Eesha from India, an Outreachy applicant and I would love to contribute to this project.

I was trying to set up the project and encountered some issues with obtaining the development keys. When I visit the site https://www.mediawiki.org/wiki/OAuth/For_Developers to obtain SOCIAL_AUTH_MEDIAWIKI_KEY & SOCIAL_AUTH_MEDIAWIKI_SECRET keys, I am asked to register an application. Not sure how to proceed with this. Could someone please help me with this?

Thanks

Hi @EeshaDutta! Could you please elaborate why you are trying to obtain these keys? Which docs are you following where? Which steps did you perform before? Thanks!

Hi @Aklapper I am following the installation guide here - https://github.com/tonythomas01/gdrive-to-commons/blob/master/README.md in the 'Pre-Requisites for local development' section. I was able to build the project without these keys as well.

However, when I run the project on my local machine, I encounter an error when I click the button - 'Login to wikimedia commons' [Error: Reverse for '<Wiki to authenticate to>' not found. '<Wiki to authenticate to>' is not a valid view function or pattern name] - attaching screenshot for reference.

This is because of the line 14 in local_settings.py in gdrive_to_commons folder. I think that some of these keys or other variable need to given appropriate values. It would be great if someone could share these values.

Hi @EeshaDutta
Paste the secrets keys correctly in your local settings which are given here : https://phabricator.wikimedia.org/T235969
Also if you are redirected to a page which shows "Access Denied: Restricted Paste" please join the project using the following link: https://phabricator.wikimedia.org/project/members/4068/
Thanks!

Hi @Rishikabhatia17 Thanks for your inputs. I have used the secret keys as specified in https://phabricator.wikimedia.org/P10014.

When I login and provide access, I am redirected to the following page

[http://localhost:8000/oauth/complete/mediawiki/?oauth_verifier=6e431cc839cf62c9c86da1444b0291af&oauth_token=ae5c7fe1f1ecfaa31344bb1084e9c0fd]. Not sure why I am getting this error? Do I need to make any changes other than pasting the secret keys and the urls?

Thanks!

Hi @EeshaDutta
I would like you to once check the following link https://phabricator.wikimedia.org/T247372
I have added a comment in the last about how to setup the keys and try following it.
Thanks!

Hi all,
I was making changes in the code to add features and while testing the tool, I was not able to upload the photos by clicking on the upload button and was facing the following errors:

  1. Internal Server Error: /api/v1.0/upload/
  1. self, "Failed to establish a new connection: %s" % e)

urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7fd0c8112278>: Failed to establish a new connection: [Errno -2] Name or service not known

  1. raise MaxRetryError(_pool, url, error or ResponseError(cause))

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='https', port=443): Max retries exceeded with url: //test.wikipedia.org/w/api.php/w/api.php?action=query&meta=siteinfo%7Cuserinfo%7Cuserinfo&continue=&uiprop=groups%7Crights%7Cblockinfo%7Chasmsg&format=json&siprop=general%7Cnamespaces (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fd0c8112278>: Failed to establish a new connection: [Errno -2] Name or service not known',))

  1. raise ConnectionError(e, request=request)

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='https', port=443): Max retries exceeded with url: //test.wikipedia.org/w/api.php/w/api.php?action=query&meta=siteinfo%7Cuserinfo%7Cuserinfo&continue=&uiprop=groups%7Crights%7Cblockinfo%7Chasmsg&format=json&siprop=general%7Cnamespaces (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fd0c8112278>: Failed to establish a new connection: [Errno -2] Name or service not known',))

What i understood according to my knowledge is that I have exceeded maximum entries to upload.
Hence I would request anyone to please guide me in solving this problem to test my code.
Thanks!

Sorry for the late reply. From the log,

  1. raise ConnectionError(e, request=request)

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='https', port=443): Max retries exceeded with url: //test.wikipedia.org/w/api.php/w/api.php?action=query&meta=siteinfo%7Cuserinfo%7Cuserinfo&continue=&uiprop=groups%7Crights%7Cblockinfo%7Chasmsg&format=json&siprop=general%7Cnamespaces (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fd0c8112278>: Failed to establish a new connection: [Errno -2] Name or service not known',))

I get a feeling that the WIKI URL might not be set correctly. can you see if it is set to WIKI_URL = "test.wikipedia.org" ? I see it somehow hit //test.wikipedia.org/w/api.php/w/api.php which looks troubling. (it should be https://test.wikipedia.org/w/api.php in any case)