Page MenuHomePhabricator

[GSoC 2019 Proposal] Categorization and Detection of Guidelines Violation using Tensorflow Lite for Commons Android App
Closed, DeclinedPublic

Description

Categorization and Detection of Guidelines Violation using Tensorflow Lite for Commons Android App

Relevent Skills:

✓ Github Workflow and Git Version Control

✓ Linux and CLI

✓ Android SDK

✓ Android UI design

✓ Understanding of RxJava

✓ Understanding of Tensorflow Lite

✓ Basic knowledge of Wikimedia APIs

✓ Familiarity with Kotlin

✓ ML and Image Classification using Python

Profile Information:

Name: Madhur Gupta

Email: madhurgupta10@gmail.com

University: Chitkara University, Punjab, India

IRC nickname on Freenode: madhurgupta10

Web Profile/Linkedin: madhur-gupta-in

Resume: Click to View

Github Link: madhurgupta10

Other Communication Medium: Zulip, Gitter, Slack, Google hangouts

Location (country or state): Punjab, India (UTC +5:30)

Typical working hours (include your timezone): Between 5 pm to 2 am UTC +5:30

Mentors

Synopsis

The Commons Android app allows users to upload pictures from their Android phone/tablet to Wikimedia Commons. Commons is not only the image repository for Wikipedia but an independent project that seeks to document the world with photos, videos, and recordings.

What is the current state of the project?

People around the world use the Commons app to upload images of various things. Sometimes unintentionally or intentionally a user might upload some images which are against the community guidelines, for example, selfies, photos depicting graphic violence, adult content, screenshots, etc. These images are worst for the community and the user should be immediately be warned about his/her action during the upload process.

What is your goal for this project and how it will benefit Wikimedia projects?

This project aims to detect such images during the upload process and warns the user about it. By implementing this feature, there will be very less violations of community guidelines and users will be aware of which image(s) they can upload.

To detect which images are against the guidelines, pre-trained machine learning models are used along with the Tensorflow Lite Library. As per the discussion with my possible mentors, I will run these pre-trained networks on all the Commons categories and assign tags to them.

Does this violate any privacy policy?

No, the best part of this project is that it does not violate any privacy policy as the classification works locally on the device and even works offline.

Have you contacted your mentors already?

Yes, I have discussed this idea and implementation with my possible mentors, based upon their feedback, I adjusted the implementation steps. The sample app MobileNet Example was tested and approved by the possible mentors, as it was able to provide satisfactory results.

About Me

I am a pre-final year Computer Science Undergraduate student from Chitkara University, India. I have been doing Android Development from past 2 years.

I have contributed to Commons Android app since March 2018. The contribution I am most proud of is the Home Screen Pic of the Day App Widget for the Commons Android App.

In my first year of college, I was awarded Dr APJ Abdul Kalam IGNITE Award from the President of India for my innovation. I love open source and participating in Hackathons.

I have participated in many Hackathons and won prizes in various National and International Hackathons, few of them are:-

And many others…

Do you have any other plans this summer?

No, I don’t have any other internship or plans for this summer, I will dedicate a minimum of 40 hours each week for this project, I am willing to dedicate more if needed. I am not applying for any other program. Google Summer of Code is the top most priority for me this summer.

What does making this project happen mean to you?

Completing this project means a lot to me, I am amazed by the fact that on the completion of this project, people around the world will be able to discover and contribute images which do not violate any guidelines. With the proper categorization of images, the Commons will partly remove the language barrier of categorization.

Past Experience

I have completed Udacity’s Android Developer Nanodegree Program for which I received the Graduation Scholarship. I also completed Python Foundation Nanodegree.

I also worked as Udacity’s Mentor for Android Developer Nanodegree where I help students with their doubts and guide them to follow best practices.

I have developed many android apps and published some to Google Play Store.

  • Popular Movies App - This app helps is similar to IMDB App, it helps you discover popular and top rated movies.
  • Baking App - With this app, you can learn about various baking recipes and can follow step by step guide to make them.
  • WalaAlarm - This app is an alarm clock with a twist which works with Walabot, where a user can set a goal for each morning like doing 10 PushUps, or Leave the Room, or Meditate to stop the alarm.

And many more…

Apart from developing apps, I have conducted various workshops/seminars with Cisco and Topcoder for many students of the college.

I also have experience in developing Websites (Frontend and Backend) using HTML, CSS, Javascript, Python, and NodeJs, Databases (SQL and NoSQL), Amazon Alexa and Google Assistant Skills, Arduino and Raspberry Pi.

Participation

I will be actively communicating with my mentors and community over Github, Zulip, Hangouts, etc. In case, I need any help, I won't hesitate to politely ask for it. I will also frequently write blog post on Medium and uploading demos on YouTube, explaining the code and feature(s) that I have implemented. I will document my code properly and will follow the best practices, code of conduct and community guidelines. I will publish my code over Github and will make a public gist for easy lookup. My Summer break will start from 1st May and will last till 31st August.

Timeline

TimeframeStart DateEnd DateTask
Community Bonding Period6 May 201927 May 2019Getting Familiarized with app structure, Wikimedia Api and Community. Familiarize myself with JavaRx. Go through Documentation once again. Learning about Tensorflow Lite in Detail and Exploring Various Pre-Trained Model and choosing the best model for each classification and testing them thoroughly with various sample images.
Week 128 May 20193 June 2019Associating Labels for commons images uploaded via Mobile by running pretrained network
(Subtask)28 May 201930 May 2019Fetching the Commons Images Uploaded via Mobile using the Wikimedia API
(Subtask)31 May 20193 June 2019Running the Pretrained MobileNet 2 Network on all these commons and associating labels for each.
Week 24 June 201910 June 2019Linking the image classification model with the Commons app using Tensorflow Lite
(Subtask)4 June 20195 June 2019Adding the Tensorflow Lite dependency in gradle, adding the labels and model file to the assets.
(Subtask)6 June 20197 June 2019Adding Structured Commons "depicts" property (or equivalent into description if the feature is not deployed yet on Commons at that time) and suggesting categories the user #2097 from the classifier
(Subtask)8 June 201910 June 2019Writing the function which recognises the images and suggests categories/tags.
Week 311 June 201917 June 2019Detection of Screenshots (Mobile/Web)
(Subtask)11 June 201913 June 2019Integrating the Screenshot Detection Classifier with the Commons app
(Subtask)14 June 201917 June 2019Writing the function which detects screenshots and warning to the user about the wikimedia policy
Week 4 & Remaining Time18 June 201928 June 2019Writing Unit Tests in Kotlin and testing the overall workflow
(Subtask)18 June 201921 June 2019Writing the unit tests in kotlin for the category suggestion and screenshot detection
(Subtask)22 June 201923 June 2019Expanding the scope of unit tests to test the overall workflow on various physical and virtual android mobiles and tablets with various sample commons images
(Subtask)24 June 201928 June 2019Evaluation of Code and Writing the Blog Post about this feature and explaining the technology and use case in detail
First Evaluation
Week 529 June 20195 July 2019Detection of Selfies and warning to the user about the image
(Subtask)29 June 20191 July 2019Integrating the Selfie Detection Classifier with the Commons app
(Subtask)2 July 20195 July 2019Writing the function which detects selfies and warning to the user about the wikimedia policy.
Week 66 July 201912 July 2019Detection of Adult Content and warning to the user about the image
(Subtask)6 July 20199 July 2019Integrating the Adult Content Classifier with the Commons app
(Subtask)9 July 201912 July 2019Writing the function which detects adult content and warning to the user about the wikimedia policy.
Week 7 & 8 & Remaining time13 July 201926 July 2019Writing Unit Tests for Adult Content Detection and Selfie Detection in Kotlin and testing the overall workflow
(Subtask)13 July 201915 July 2019Writing the unit tests for Selfie Detection System in Kotlin
(Subtask)16 July 201918 July 2019Writing the unit tests for Adult Content Detection System in Kotlin
(Subtask)19 July 201921 July 2019Expanding the scope of unit tests to test the overall workflow on various physical and virtual android mobiles and tablets with various sample images
(Subtask)22 July 201926 July 2019Evaluation of Code and Writing the Blog Post about these feature and explaining the technology and use case in detail
Second Evaluation
Week 927 July 20192 August 2019Detection of Graphic Violence Promotion Content and warn the user about the image
(Subtask)27 July 201929 July 2019Integrating the Violence Detection Classifier with the Commons app
(Subtask)30 July 20192 August 2019Writing the function which detects graphic violence and warning to the user about the wikimedia policy.
Week 103 August 20199 August 2019Writing Unit Tests for Graphic Violence Detection in Kotlin and testing the overall workflow
(Subtask)3 August 20196 August 2019Writing the unit tests for Graphic Violence Detection System in Kotlin
(Subtask)7 August 20199 August 2019Expanding the scope of unit tests to test the overall workflow on various physical and virtual android mobiles and tablets with various sample images
Week 11 & Remaining Time10 August 201918 August 2019Reducing the size of over all package by removing the unused classes from the tensorflow lite libraries as per discussion #75(comment)
(Subtask)10 August 201913 August 2019Finding the unused classes in the tensorflow lite library
(Subtask)14 August 201916 August 2019Removing these resources and trimming the over all apk size
(Subtask)17 August 201918 August 2019Writing the Blog Post about these feature and explaining the technology and use case in detail
Week 1219 August 201926 August 2019Over all Evaluation
(Subtask)19 August 201926 August 2019Evaluation of Code
Final Evaluation

Previous Contributions

So far I have made over 35 Pull Requests to the project out of which 21 are already merged. You can see all my Pull Requests here.

DatePR NumberTitleIssue NumberStatus
2-4-192834Fixed #2825 where user was able to add one lang twice2825Merged
31-3-192826Fixed Night Mode Problem in Spinner #28092809Merged
29-3-192799Removed duplicate File: from deletion notification2794Merged
28-3-192787Fixed App Crash on Tablets2766Merged
28-3-192780Fixed Padding for FAB in nearby #27792779Merged
28-3-192775Refactor LoginActivity #26902690Merged
28-3-192772Fixed Location of Checkbox in Permission Request Dialog2771Merged
27-3-192768Added Click To View Photo on Home Screen Widget2767Merged
27-3-192758Added Title to the Pic of the Day App Widget2757Merged
21-3-192687Added SignUp Button Unit Test2686Merged
20-3-192683Added information dialog to Statistics #25992599Merged
15-3-192600Fixed overlapping text bug for tablets #25922592Merged
14-3-192587Fixed bug #2324 - Number of Contributions not updating unless app is restarted2324Merged
12-3-192579Moved Images via Nearby Places to top, fixes #25242524Merged
21-2-192499Add Device Support Information to Feedback Intent2494Merged
20-2-192491Fix Bug #24892489Merged
19-2-192488Fix Bug #24772477Merged
16-5-181523Added Basic Pic Of the Day App Widget1296Merged
13-3-181304Fix Underline Tag Bug in About Page1303Merged
5-3-181259Revamp the Settings page1039Merged
3-3-181242Add Forgot Password Option in LoginActivity1063Merged
Open PRs
6-4-192866Added Appetize badge2842Open
5-4-192862Added Quick Launch Icon to the Home Screen Widget #12961296Open
4-4-192859Fixed 2857 where achievements activity was getting recreated on rotation2857Open
4-4-192856Fixed Width of Navigation Drawer for Tablets2854Open
4-4-192851Moved hardcoded URLs to build.grade Fixed2850Open
4-4-192849Adjusted the gravity of SKIP THIS IMAGE Fixed2848Open
3-4-192841Fixed the issue #2770 where incomplete list was shown in landscape mode2770Open
30-3-192808Completed TODO init views using bind view #28042804Open
30-3-192807Fixed Navigation UI Problem in Split Screen mode # 27882788Open
27-3-192765Added Toast on Successful Logout2764Open
23-3-192725Updated Navigation Base Activity Unit Tests2724Open

Issues

So far I have opened 40 issues out of which 22 are solved/closed and 18 are open: Click here to view all issues

DateIssue NumberTitleStatus
30-3-192825The user should not be allowed to add the same language twice.Solved
30-3-192809Night Mode: Language selection dialog is not readableSolved
28-3-192779Upload FAB Missing Padding in NearbySolved
28-3-192778Moving GPS FAB to bottom right cornerClosed
28-3-192776Replace Search Button with FABClosed
28-3-192771Location Permission Required Dialog's UI FixSolved
27-3-192776Add Click to View Photo to Home Screen WidgetSolved
27-3-192766App Crashing on TabletsSolved
27-3-192757Add title of the picture to the App WidgetSolved
26-3-192756Pic of the Day Widget API not working on betaClosed
21-3-192690Refactor LoginActivity code for better readabilitySolved
21-3-192686Write Unit Test for Login ActivitySolved
15-3-192599Adding information dialog to StatisticsSolved
15-3-192592Overlapping text in login screen in landscape modeSolved
23-2-192509Count on Contributions does not update on deletionClosed
21-2-192502Swipe to delete specific recent searchesClosed
20-2-192494Collection of device's essential information when user want to share feedbackSolved
19-2-192489Click event not showing button layout without scrolling for last item in the listSolved
13-3-181305Replace New Image Upload with Floating Action Button (Material Design)Closed
13-3-181303Fix Underline Tag in "About Page"Solved
13-3-181295Navigation Drawer too large for landscape modeClosed
5-3-181260Nothing happens on clicking a notificationClosed
Open Issues
5-4-192865Blur the background when FAB is openOpen
5-4-192864Using Outlined Edit Text from Material Design for Login ScreenOpen
4-4-192857Achievements Activity getting recreated on rotationOpen
4-4-192854Navigation Drawer is huge in TabletsOpen
4-4-192850Moving URLs from About Activity to build.gradleOpen
3-4-192848Skip this image title should be in center in landscape modeOpen
30-3-192842Hosting app on AppetizeOpen
29-3-192810Option to Remove a new language description while uploadingOpen
29-3-192804Initialize all views using bind view in NearbyMapFragmentOpen
28-3-192788Navigation UI problem in split screen modeOpen
28-3-192770Incomplete Nearby List shown in Landscape modeOpen
27-3-192764Handle and Fix the Delay in LogoutOpen
23-3-192724Complete the NavigationBaseActivityTestOpen
16-3-192616Wrong bookmarked item expanding onResumeOpen
5-4-192590Adding bubble scroll view to the appOpen
23-2-192508Popup to get more user rating on app storeOpen
13-3-181296WidgetOpen
3-3-181247Add Photo Challenge Activity to appOpen

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 8 2019, 10:43 AM
Madhurgupta10 renamed this task from [GSoC 2019 Proposal] Add Structured Commons support to Commons Android App to [GSoC 2019 Proposal] Categorization and Detection of Policy Violation using Tensorflow Lite for Commons Android App.Apr 8 2019, 6:26 PM
Madhurgupta10 updated the task description. (Show Details)
Madhurgupta10 renamed this task from [GSoC 2019 Proposal] Categorization and Detection of Policy Violation using Tensorflow Lite for Commons Android App to [GSoC 2019 Proposal] Categorization and Detection of Guidelines Violation using Tensorflow Lite for Commons Android App.Apr 8 2019, 7:12 PM
Madhurgupta10 updated the task description. (Show Details)Apr 8 2019, 7:14 PM
Madhurgupta10 updated the task description. (Show Details)Apr 8 2019, 8:30 PM

If you would like us to consider your proposal for review, please move it to the submitted column on Google-Summer-of-Code (2019) board.

(look for next steps in the email you'll receive shortly with an option to request for a debrief on why your proposal was not accepted)

srishakatux closed this task as Declined.May 7 2019, 12:24 AM