Profile Information

Name: Rishan P
Email: Rishan.pgowda@gmail.com
Github : rovertrack
Other communication modes: Discord, Zulip, Slack.
Location during GSoC: Bangalore India.
Typical Working hours (include your timezone) during GSoC: 6:00 pm to 2:00 am (IST) UTC+5:30

Synopsis

App

Wikimedia Commons is an Android application developed by the Wikimedia Foundation that enables users to browse, upload, and contribute media directly to the Wikimedia Commons repository using their mobile devices. Users can easily select images from their device gallery or capture photos using the camera and upload them to the platform. The app aims to simplify the media contribution workflow and make it easier for contributors to share freely licensed images with the Wikimedia ecosystem.

Project

Project Objectives

• Improve the existing lossless crop/rotate feature to not lose any information, preserving exact picture colors
• Lossless blur: JPG images are made of many 8x8 pixel blocks. The idea here is to blur only the JPG blocks showing what the users wants to blur, without recompressing the other blocks.
• Option to automatically blur people faces and car number plates.

Possible Mentor(s)

@Nicolas_Raoul
@RitikaPahwa4444

Have you contacted your mentors already?

Yes.

Implementation

Github issue/discussion links

• Improve the existing lossless crop/rotate feature to not lose any information, preserving exact picture colors: #6659
• Lossless blur: JPG images are made of many 8x8 pixel blocks. The idea here is to blur only the JPG blocks showing what the users wants to blur, without recompressing the other blocks: #190

I have no prior knowledge about JPG format , except messing around with photoshop in my school days i got to know about key things like RGB, pixel rendering , pixel blocks grid. That's what got me interested in this project

Phase 1 - Improve existing crop feature

Problem
Issue #6659
The current crop feature takes in the Jpeg runs LLJTran (reads the compressed JPEG blocks, rearranges them mathematically for rotation) here there is no loss in quality.In this step there is no loss in image quality.
LLJTran only handles the IMAGEDATA, not the METADATA.
Metadata is handled by the android's built-in metadata handler
ExifInterface.saveAttributes()
but this handler in the process strips off ICC profile as it ignores the APP2 marker throughout the transformation.
This results in loss in image quality

Solution
Instead of the above two step process we can achieve the lossless crop by passing the image to Jpegtran which is in native C library which on rotation copies every metadata byte-for-byte , rearranges DCT blocks for rotation.NO decoding, NO recompression in the process
This results in :

• Single step process
• No metadata such as EXIF , ICC is lost in the process
• No recompression of the image

As Jpegtran is a C library it would require a JNI (Java Native Interface) for the kotlin application layer to communicate with the Jpegtran library.

Phase 2 - Lossless Block-Level Blur

Problem
Issue #190
Present normal blur make quality loss in the process.
When normal blur is applied to a JPEG image it decodes all pixels , applies blur then recompresses everything which make the image lose quality , metadata and increase in file size.

Solution
If the recompression happens to only relevant blur required pixel blocks then we would not lose quality of the pixel blocks which are untouched then we wont have loss in quality of unblurred regions.

Process:

1. Get bounding box to the region the user wants to apply the blur :
   • User selects the region by drawing over which gives us the pixel coordinates.
   • Map pixel coordinates to the nearest Minimum Coded Unit (MCU) boundaries.
   • Jump to nearest block boundary if needed.
2. Transmit the calculated MCU indices to the native layer via JNI. This allows jpegtran to perform high-speed, direct manipulation of the DCT coefficients within the target blocks.
3. Now that jpegtran has the target blocks it can easily apply them to JPEG 8×8 DCT blocks.

• B = background blocks (untouched)
• F = target blocks (to be blurred)
• Each block is independent and blurring one block has no effect on another , making sure blur is only applied to required regions.

The Result

• Original metadata and header preserved
• Unblurred regions are preserved to their original quality without loss.
• blur applied to only selected region.

Phase 3 - Auto-detect and blur faces & car number plates.

Problem
No privacy for random people and car number plates captured in the uploading photo.
solution
Display an option for users which upon selecting detects people faces and car number plates in the image, shows to user to which block/region the blur is going to be applied.
The user can confirm all region , remove some of the regions or add regions manually to which the blur needs to be applied.

Process
Uses OpenCV to detect faces/plates.

1. The selected image is passed to OpenCV DNN via a JNI bridge which runs locally on the phone which runs pretrained lightweight deep learning models trained for object detection.

Preferring DNN over haar cascade as it detects faces in most of the angles compared to later one.
Faces - SSD , MobileNet.
Plates - Lightweight detection model.

2. Low confidence detections are filtered out using a confidence threshold before being shown to the user.

The bounding box coordinates returned by OpenCV is shown as regions to the user to which the user can

• confirm all region
• remove some of the regions
• add regions manually
• After the coordinates are finalized it is passed to the same image editing feature which the phase 2 improves the image blurring which applies blur to only required region and remaining region untouched.

Results:

• User always has final control over what gets blurred
• Confirmed bounding boxes passed directly to Phase 2
• Missed regions can be added manually by user
• False positives can be removed by user
• Faces , plates and user selected region is blurred successfully.

Deliverables

Timeline

This roadmap is designed to be flexible. If technical debt or implementation blockers arise, work can continue beyond the 12-week GSoC period to ensure code quality is never sacrificed for speed. I plan to continue contributing to the Wikimedia Commons Android App even after the GSoC program ends, maintaining the implemented features, addressing feedback, and contributing to other areas of the codebase.

About Me

• How did you hear about this program?

X (twitter)

• Will you have any other time commitments, such as school work, another job, planned vacation, etc, during the duration of the program?

Have enrolled in one exam happening July 10 - July 20

• We advise all candidates eligible for Google Summer of Code and Outreachy to apply for both programs. Are you planning to apply to both programs and, if so, with what organization(s)?

Yes , Wikimedia Foundation i dont know why but thats the one organization which i get along well.

• What does making this project happen mean to you?

Means gaining knowledge under guidance of some of the best mentors .Knowledge which will help me improve more Giving to open source, Providing values

Availability

Eligible for Google Summer of Code and Outreachy?
I am eligible for Google Summer of Code and am applying only for GSoC.

Do you plan to submit any other proposal apart from this one?
Yes.

Do you have any other plans this summer?
No I will have an exam which i have enrolled in from July 10 - July 20 , during time i will test the app manually for proper working and implement bug fixes , after this my full time is for Gsoc .

How many hours per week can you dedicate to this project?
I will dedicate a minimum of 44+ hours per week to this project and am willing to invest additional time if required to meet project goals.

Have you been accepted to GSoC before?
No, this is my first time applying for Google Summer of Code.

Contributions to Wikimedia Commons

Pull request

Link PRs

• Total: 25
• Merged: 13
• Pending: 11
• Draft: 1

Pull Requests Under Review

Draft Pull Requests

Issues

Link Issue

• Created: 13
• Closed: 10
• Open: 3

PR Review