Categorization and Detection of Guidelines Violation using Tensorflow Lite for Commons Android App
✓ Github Workflow and Git Version Control
✓ Linux and CLI
✓ Android SDK
✓ Android UI design
✓ Understanding of RxJava
✓ Understanding of Tensorflow Lite
✓ Basic knowledge of Wikimedia APIs
✓ Familiarity with Kotlin
✓ ML and Image Classification using Python
Name: Madhur Gupta
University: Chitkara University, Punjab, India
IRC nickname on Freenode: madhurgupta10
Web Profile/Linkedin: madhur-gupta-in
Resume: Click to View
Github Link: madhurgupta10
Other Communication Medium: Zulip, Gitter, Slack, Google hangouts
Location (country or state): Punjab, India (UTC +5:30)
Typical working hours (include your timezone): Between 5 pm to 2 am UTC +5:30
The Commons Android app allows users to upload pictures from their Android phone/tablet to Wikimedia Commons. Commons is not only the image repository for Wikipedia but an independent project that seeks to document the world with photos, videos, and recordings.
What is the current state of the project?
People around the world use the Commons app to upload images of various things. Sometimes unintentionally or intentionally a user might upload some images which are against the community guidelines, for example, selfies, photos depicting graphic violence, adult content, screenshots, etc. These images are worst for the community and the user should be immediately be warned about his/her action during the upload process.
What is your goal for this project and how it will benefit Wikimedia projects?
This project aims to detect such images during the upload process and warns the user about it. By implementing this feature, there will be very less violations of community guidelines and users will be aware of which image(s) they can upload.
To detect which images are against the guidelines, pre-trained machine learning models are used along with the Tensorflow Lite Library. As per the discussion with my possible mentors, I will run these pre-trained networks on all the Commons categories and assign tags to them.
Have you contacted your mentors already?
Yes, I have discussed this idea and implementation with my possible mentors, based upon their feedback, I adjusted the implementation steps. The sample app MobileNet Example was tested and approved by the possible mentors, as it was able to provide satisfactory results.
I am a pre-final year Computer Science Undergraduate student from Chitkara University, India. I have been doing Android Development from past 2 years.
I have participated in many Hackathons and won prizes in various National and International Hackathons, few of them are:-
- First Prize in Smart India Hackathon 2019- World’s largest hackathon with over 40,000 participants.
- Third Prize in Topcoder/IBM Call for Code Global Hackathon
- First Prize in Microsoft MS Hacks 2.0
- First Prize in MMU Hackathon
- First Prize in SAP Cloud Hackathon
And many others…
- I was also the recipient of Udacity and Google India’s Challenge Scholarship and Facebook’s PyTorch Scholarship.
Do you have any other plans this summer?
No, I don’t have any other internship or plans for this summer, I will dedicate a minimum of 40 hours each week for this project, I am willing to dedicate more if needed. I am not applying for any other program. Google Summer of Code is the top most priority for me this summer.
What does making this project happen mean to you?
Completing this project means a lot to me, I am amazed by the fact that on the completion of this project, people around the world will be able to discover and contribute images which do not violate any guidelines. With the proper categorization of images, the Commons will partly remove the language barrier of categorization.
I also worked as Udacity’s Mentor for Android Developer Nanodegree where I help students with their doubts and guide them to follow best practices.
I have developed many android apps and published some to Google Play Store.
- Popular Movies App - This app helps is similar to IMDB App, it helps you discover popular and top rated movies.
- Baking App - With this app, you can learn about various baking recipes and can follow step by step guide to make them.
- WalaAlarm - This app is an alarm clock with a twist which works with Walabot, where a user can set a goal for each morning like doing 10 PushUps, or Leave the Room, or Meditate to stop the alarm.
And many more…
Apart from developing apps, I have conducted various workshops/seminars with Cisco and Topcoder for many students of the college.
I will be actively communicating with my mentors and community over Github, Zulip, Hangouts, etc. In case, I need any help, I won't hesitate to politely ask for it. I will also frequently write blog post on Medium and uploading demos on YouTube, explaining the code and feature(s) that I have implemented. I will document my code properly and will follow the best practices, code of conduct and community guidelines. I will publish my code over Github and will make a public gist for easy lookup. My Summer break will start from 1st May and will last till 31st August.
|Timeframe||Start Date||End Date||Task|
|Community Bonding Period||6 May 2019||27 May 2019||Getting Familiarized with app structure, Wikimedia Api and Community. Familiarize myself with JavaRx. Go through Documentation once again. Learning about Tensorflow Lite in Detail and Exploring Various Pre-Trained Model and choosing the best model for each classification and testing them thoroughly with various sample images.|
|Week 1||28 May 2019||3 June 2019||Associating Labels for commons images uploaded via Mobile by running pretrained network|
|(Subtask)||28 May 2019||30 May 2019||Fetching the Commons Images Uploaded via Mobile using the Wikimedia API|
|(Subtask)||31 May 2019||3 June 2019||Running the Pretrained MobileNet 2 Network on all these commons and associating labels for each.|
|Week 2||4 June 2019||10 June 2019||Linking the image classification model with the Commons app using Tensorflow Lite|
|(Subtask)||4 June 2019||5 June 2019||Adding the Tensorflow Lite dependency in gradle, adding the labels and model file to the assets.|
|(Subtask)||6 June 2019||7 June 2019||Adding Structured Commons "depicts" property (or equivalent into description if the feature is not deployed yet on Commons at that time) and suggesting categories the user #2097 from the classifier|
|(Subtask)||8 June 2019||10 June 2019||Writing the function which recognises the images and suggests categories/tags.|
|Week 3||11 June 2019||17 June 2019||Detection of Screenshots (Mobile/Web)|
|(Subtask)||11 June 2019||13 June 2019||Integrating the Screenshot Detection Classifier with the Commons app|
|(Subtask)||14 June 2019||17 June 2019||Writing the function which detects screenshots and warning to the user about the wikimedia policy|
|Week 4 & Remaining Time||18 June 2019||28 June 2019||Writing Unit Tests in Kotlin and testing the overall workflow|
|(Subtask)||18 June 2019||21 June 2019||Writing the unit tests in kotlin for the category suggestion and screenshot detection|
|(Subtask)||22 June 2019||23 June 2019||Expanding the scope of unit tests to test the overall workflow on various physical and virtual android mobiles and tablets with various sample commons images|
|(Subtask)||24 June 2019||28 June 2019||Evaluation of Code and Writing the Blog Post about this feature and explaining the technology and use case in detail|
|Week 5||29 June 2019||5 July 2019||Detection of Selfies and warning to the user about the image|
|(Subtask)||29 June 2019||1 July 2019||Integrating the Selfie Detection Classifier with the Commons app|
|(Subtask)||2 July 2019||5 July 2019||Writing the function which detects selfies and warning to the user about the wikimedia policy.|
|Week 6||6 July 2019||12 July 2019||Detection of Adult Content and warning to the user about the image|
|(Subtask)||6 July 2019||9 July 2019||Integrating the Adult Content Classifier with the Commons app|
|(Subtask)||9 July 2019||12 July 2019||Writing the function which detects adult content and warning to the user about the wikimedia policy.|
|Week 7 & 8 & Remaining time||13 July 2019||26 July 2019||Writing Unit Tests for Adult Content Detection and Selfie Detection in Kotlin and testing the overall workflow|
|(Subtask)||13 July 2019||15 July 2019||Writing the unit tests for Selfie Detection System in Kotlin|
|(Subtask)||16 July 2019||18 July 2019||Writing the unit tests for Adult Content Detection System in Kotlin|
|(Subtask)||19 July 2019||21 July 2019||Expanding the scope of unit tests to test the overall workflow on various physical and virtual android mobiles and tablets with various sample images|
|(Subtask)||22 July 2019||26 July 2019||Evaluation of Code and Writing the Blog Post about these feature and explaining the technology and use case in detail|
|Week 9||27 July 2019||2 August 2019||Detection of Graphic Violence Promotion Content and warn the user about the image|
|(Subtask)||27 July 2019||29 July 2019||Integrating the Violence Detection Classifier with the Commons app|
|(Subtask)||30 July 2019||2 August 2019||Writing the function which detects graphic violence and warning to the user about the wikimedia policy.|
|Week 10||3 August 2019||9 August 2019||Writing Unit Tests for Graphic Violence Detection in Kotlin and testing the overall workflow|
|(Subtask)||3 August 2019||6 August 2019||Writing the unit tests for Graphic Violence Detection System in Kotlin|
|(Subtask)||7 August 2019||9 August 2019||Expanding the scope of unit tests to test the overall workflow on various physical and virtual android mobiles and tablets with various sample images|
|Week 11 & Remaining Time||10 August 2019||18 August 2019||Reducing the size of over all package by removing the unused classes from the tensorflow lite libraries as per discussion #75(comment)|
|(Subtask)||10 August 2019||13 August 2019||Finding the unused classes in the tensorflow lite library|
|(Subtask)||14 August 2019||16 August 2019||Removing these resources and trimming the over all apk size|
|(Subtask)||17 August 2019||18 August 2019||Writing the Blog Post about these feature and explaining the technology and use case in detail|
|Week 12||19 August 2019||26 August 2019||Over all Evaluation|
|(Subtask)||19 August 2019||26 August 2019||Evaluation of Code|
So far I have made over 35 Pull Requests to the project out of which 21 are already merged. You can see all my Pull Requests here.
|Date||PR Number||Title||Issue Number||Status|
|2-4-19||2834||Fixed #2825 where user was able to add one lang twice||2825||Merged|
|31-3-19||2826||Fixed Night Mode Problem in Spinner #2809||2809||Merged|
|29-3-19||2799||Removed duplicate File: from deletion notification||2794||Merged|
|28-3-19||2787||Fixed App Crash on Tablets||2766||Merged|
|28-3-19||2780||Fixed Padding for FAB in nearby #2779||2779||Merged|
|28-3-19||2775||Refactor LoginActivity #2690||2690||Merged|
|28-3-19||2772||Fixed Location of Checkbox in Permission Request Dialog||2771||Merged|
|27-3-19||2768||Added Click To View Photo on Home Screen Widget||2767||Merged|
|27-3-19||2758||Added Title to the Pic of the Day App Widget||2757||Merged|
|21-3-19||2687||Added SignUp Button Unit Test||2686||Merged|
|20-3-19||2683||Added information dialog to Statistics #2599||2599||Merged|
|15-3-19||2600||Fixed overlapping text bug for tablets #2592||2592||Merged|
|14-3-19||2587||Fixed bug #2324 - Number of Contributions not updating unless app is restarted||2324||Merged|
|12-3-19||2579||Moved Images via Nearby Places to top, fixes #2524||2524||Merged|
|21-2-19||2499||Add Device Support Information to Feedback Intent||2494||Merged|
|20-2-19||2491||Fix Bug #2489||2489||Merged|
|19-2-19||2488||Fix Bug #2477||2477||Merged|
|16-5-18||1523||Added Basic Pic Of the Day App Widget||1296||Merged|
|13-3-18||1304||Fix Underline Tag Bug in About Page||1303||Merged|
|5-3-18||1259||Revamp the Settings page||1039||Merged|
|3-3-18||1242||Add Forgot Password Option in LoginActivity||1063||Merged|
|6-4-19||2866||Added Appetize badge||2842||Open|
|5-4-19||2862||Added Quick Launch Icon to the Home Screen Widget #1296||1296||Open|
|4-4-19||2859||Fixed 2857 where achievements activity was getting recreated on rotation||2857||Open|
|4-4-19||2856||Fixed Width of Navigation Drawer for Tablets||2854||Open|
|4-4-19||2851||Moved hardcoded URLs to build.grade Fixed||2850||Open|
|4-4-19||2849||Adjusted the gravity of SKIP THIS IMAGE Fixed||2848||Open|
|3-4-19||2841||Fixed the issue #2770 where incomplete list was shown in landscape mode||2770||Open|
|30-3-19||2808||Completed TODO init views using bind view #2804||2804||Open|
|30-3-19||2807||Fixed Navigation UI Problem in Split Screen mode # 2788||2788||Open|
|27-3-19||2765||Added Toast on Successful Logout||2764||Open|
|23-3-19||2725||Updated Navigation Base Activity Unit Tests||2724||Open|
So far I have opened 40 issues out of which 22 are solved/closed and 18 are open: Click here to view all issues
|30-3-19||2825||The user should not be allowed to add the same language twice.||Solved|
|30-3-19||2809||Night Mode: Language selection dialog is not readable||Solved|
|28-3-19||2779||Upload FAB Missing Padding in Nearby||Solved|
|28-3-19||2778||Moving GPS FAB to bottom right corner||Closed|
|28-3-19||2776||Replace Search Button with FAB||Closed|
|28-3-19||2771||Location Permission Required Dialog's UI Fix||Solved|
|27-3-19||2776||Add Click to View Photo to Home Screen Widget||Solved|
|27-3-19||2766||App Crashing on Tablets||Solved|
|27-3-19||2757||Add title of the picture to the App Widget||Solved|
|26-3-19||2756||Pic of the Day Widget API not working on beta||Closed|
|21-3-19||2690||Refactor LoginActivity code for better readability||Solved|
|21-3-19||2686||Write Unit Test for Login Activity||Solved|
|15-3-19||2599||Adding information dialog to Statistics||Solved|
|15-3-19||2592||Overlapping text in login screen in landscape mode||Solved|
|23-2-19||2509||Count on Contributions does not update on deletion||Closed|
|21-2-19||2502||Swipe to delete specific recent searches||Closed|
|20-2-19||2494||Collection of device's essential information when user want to share feedback||Solved|
|19-2-19||2489||Click event not showing button layout without scrolling for last item in the list||Solved|
|13-3-18||1305||Replace New Image Upload with Floating Action Button (Material Design)||Closed|
|13-3-18||1303||Fix Underline Tag in "About Page"||Solved|
|13-3-18||1295||Navigation Drawer too large for landscape mode||Closed|
|5-3-18||1260||Nothing happens on clicking a notification||Closed|
|5-4-19||2865||Blur the background when FAB is open||Open|
|5-4-19||2864||Using Outlined Edit Text from Material Design for Login Screen||Open|
|4-4-19||2857||Achievements Activity getting recreated on rotation||Open|
|4-4-19||2854||Navigation Drawer is huge in Tablets||Open|
|4-4-19||2850||Moving URLs from About Activity to build.gradle||Open|
|3-4-19||2848||Skip this image title should be in center in landscape mode||Open|
|30-3-19||2842||Hosting app on Appetize||Open|
|29-3-19||2810||Option to Remove a new language description while uploading||Open|
|29-3-19||2804||Initialize all views using bind view in NearbyMapFragment||Open|
|28-3-19||2788||Navigation UI problem in split screen mode||Open|
|28-3-19||2770||Incomplete Nearby List shown in Landscape mode||Open|
|27-3-19||2764||Handle and Fix the Delay in Logout||Open|
|23-3-19||2724||Complete the NavigationBaseActivityTest||Open|
|16-3-19||2616||Wrong bookmarked item expanding onResume||Open|
|5-4-19||2590||Adding bubble scroll view to the app||Open|
|23-2-19||2508||Popup to get more user rating on app store||Open|
|3-3-18||1247||Add Photo Challenge Activity to app||Open|