Profile Information

Name : Mohd. Afzal Khan
Email : ak341668@gmail.com
Zulip Chat (nick): AFZL210
Resume :

Synopsis

VideoCutTool is a video editing tool for Wikimedia Commons. It is a simple tool that allows for quick video editing without the need to download any third-party software. we can just paste the video link or upload the video and edit it within the tool. It is not resource-intensive and runs smoothly on all modern browsers and mobile devices.

This Proposal aims to complete the following goals :

Goal of the project:

• Fix Bugs: Fix existing bugs related to frontend and backend some of them are
  • Improve keyboard navigation T333564
  • upload to commons throwing error T333060
• Improve Mobile View
  • since mobile widths are not very large, accessing the timeline and trim feature becomes difficult. Improving the landscape view would enhance the workflow and overall experience of the tool.
  • I will fix the mobile landscape view (max-width: 740px). Currently, when switching to landscape mode, the video takes up the full height of the device. To access the options, we have to scroll and then go back to the top to see the changes, which is not very convenient

• Expected Design (Mobile Landscape)

• Add more useful features

• write unit and end-to-end tests
  • currently, VideoCutTool only have unit test for UrlBox.jsx, Home.jsx and Notification.jsx
  • write unit and end-to-end tests for all other remaining components

I strongly believe that as this tool is used by volunteers, so it must have a Good user-friendly UI and all the necessary features that can help create more informative and engaging videos, some of the features I would like to work on are :

• take a screenshot of the video frame
• video speed control (at least 4 levels: .25x, .5x, .75x)
• redo/undo changes
• brightness control: add a brightness control slider or a set of predefined values to control the brightness

Expected Features Design (i will update them after discussion with the mentor in the community bonding period)

• Speed control : add fixed slider with states (.25x, .75x, .5x and 1x)

Deliverables

Mid Term Evaluation

By the midterm evaluation, the tool will be having:

• Improved UI for Desktop and Mobile (Portrait + Landscape Orientation)
• Improved Keyboard Navigation
• Reduced build time and bundle size
• persist video even after tab refresh
• Fix Bitrate drop (quality degrading) while editing videos.

Final Evaluation

Most of the goals will be completed by midterm evaluation after that I will be adding some additional features :

• Undo/Redo Feature
• unit and end-to-end test for frontend and backend using Jest, React-testing library, and integrating with Jenkins/CI.
• Updating the libraries/dependencies to avoid deprecation warnings during build time.
• Add static type checking using JSDoc.
• fix uploading to commons issue (converting other file types to .webm)

Additional Deliverables:

• Video stabilization
• Text Overlay
• Speed Control
• Video Brightness Control
• Color/Saturation/Hue Control

Timeline

Community Bonding Period

May 4 to May 15: Planning
In the initial phase of the program, I will share some designs I have in mind regarding the mobile view, as mentioned above. For the desktop, I have some ideas similar to T315143 and will finalize the UI after discussing it with the mentors. I will also explain and discuss all possible approaches to solve future issues. I plan to divide the timeline into four parts. In the first part, I will focus on fixing the existing bugs in the tool and improving its UI/UX. In the second part, I will work on development-related issues, including reducing the bundle size of the project and resolving the issue of running development(client and backend) on the same port (T332517). In the third part, my highest priority will be fixing existing issues in the backend, improving performance, and adding additional features after discussing them with the mentors. and I will finish the project by writing unit and end-to-end tests for the frontend and backend.

Part-1: Client/Frontend Improvements

May 16-May 28 : UI Improvement

as VideoCutTool is mostly used for sharing videos on Wikimedia commons and not by professional video editors, the tool must be easy to use, and the learning curve should be minimal. Firstly, I would implement the UI changes as discussed with mentors, and the most important UI change would be the mobile landscape orientation because when editing video in portrait the options and timeline are present on the same page, but as we go in landscape orientation the video takes the full height of app and the trim timeline and options are shifted to next page which makes it inconvenient to see changes.

a design something like this will definitely improve the user experience and overall workflow

Keyboard Navigation Improvement
Currently, in VideoCutTool we can only navigate through the options with arrow keys, I will be adding some other keyboard navigations as mentioned in T333564

• play/pause the video on the spacebar click
• control video timeline with left and right arrow keys

In addition to this, it would be great if the tool had some shortcuts like SHIFT C for Crop and SHIFT A for Audio. After finalizing the shortcut keys with mentors, I will incorporate them into the tool.

Fix Video Player Window Size I would like to fix the video player window size in VideoCutTool. Currently, when we upload any video, the size of the video player is dependent on the ratio/dimensions of the uploaded video. I intend to standardize the player window size to 16:9 or 16:10, whichever works best.

currently, when I upload a 4:3 video the video takes full-screen height and shift other feature to the next page even though I am on a desktop.

Fix
something similar to this design

May 29 - June 5

Part-2: Performace/Dev environment Improvements

The VideoCutTool uses react-bootstrap and react-icons to render Buttons and icons, which is not very light instead of using this we can shift to react-icons which has a bundle size of only 6kB whereas the bundle size of react-bootstrap-icons is 3MB which is a lot, and considering this is only used to render Buttons and icons, instead of this we can create a custom component and shift to react-icons, this will help to reduce the bundle size and build time while development. and to reduce it even more we can avoid using Axios and use react fetch (if necessary) and we can also avoid using Toast from react-bootstrap instead create a custom component for it, this will help us get rid of the whole react-bootstrap package and give more space and freedom to work.
reference: ( https://bundlephobia.com/ ) , ( https://bundlejs.com/ )

When we run the development server, the frontend code is available on port 3000 and the API is live on port 4000, we can use Nginx to run both on the single port as mentioned in T332517, to do so we have two different approaches :

• write command to build the client code and serve it statically on the same port
• use hot-reloading to do the same

the second approach is definitely better for development because the changes will be reflected as we change the code and the first approach is good for manual end-to-end testing and deployment because in production we most probably deploy the final build.

Part-3: Backend/new features Improvements

June 6 - July 9: Fixing existing issues

The first priority will be fixing existing issues in the backend and then I will focus on adding new features to the tool. first, I will fix the bitrate drop issue mentioned in T262892, to solve this issue first I have to identify the reason for this drop, most probably the root cause for this issue could be.

• drop in bitrate while copying the file ( while downloading video to server)
• upload different files and try varying the -b:v parameter which is responsible for changing the output video's bitrate

Upload to commons throwing error more details here T333060: to solve this we can check the file extension of the input video and convert it to .webm using the two pass encoding method using FFmpeg (reference: https://trac.ffmpeg.org/wiki/Encode/VP9#twopass)

Undo/redo:This feature is very important and can save a lot of time. If a user makes some unwanted changes, they would have to start from scratch, which can be very time-consuming. Therefore, I am planning to add an Undo/Redo feature.

To do so, we can store all the possible changes that can be made on a video (such as trimming, adjusting audio, changing speed, etc.) and create two different stacks to store the states of applied changes in the form of an object that contains all the possible details about a video's state. The state will look something like this:

speed: ".5x",
audio: "on",
trim: [[s1,e1], [s2,e2]]
...other attributes

As a user makes changes to the video, the states are pushed into the UndoStack. When the user clicks the undo button, we can send the topmost edit state to the server via a POST request and apply those changes. After successfully applying those changes, we push this state to the RedoStack.

flowchart of Undo/Redo Feature

Persist Video after tab refresh: When a new editing session is initiated, due to limitations in the size of the localStorage, we cannot store the entire video on the client side. Instead, I will create editing checkpoints and store them in localStorage. Whenever the user refreshes the tab, if we have any checkpoints stored, we will first apply those changes before proceeding further.

Midterm Evaluation (July 10 - July 14 )

I Will present the completed task to mentors and take feedback and discuss/plan for future goals.

July 15 - July 20: Working on Feedback

Here, I will work on the feedback given by the mentors, and while solving it, I will complete any remaining tasks until now. I will also prepare UI designs for additional features and get reviews on them, making changes accordingly.

Part-4: Adding New Feature to tool

July 21 - August 10

I will be adding the following additional features to VideoCutTool :

• Video stabilization
• text overlay
• video speed control (at least 4 levels: .25x, .5x, .75x)
• redo/undo changes
• Color Filter and Brightness control

Speed Control : the video speed can be manipulated by varying the presentation timestamp (PTS) of the video by adding setpts filter in FFmpeg command.

The filter works by changing the presentation timestamp (PTS) of each video frame. For example, if there are two succesive frames shown at timestamps 1 and 2, and you want to speed up the video, those timestamps need to become 0.5 and 1. and due to this the output video has some frame drops and to avoid this we can explictly pass the desired output video framrate also.

• Slowdown the Video : setpts=[value<=1]*PTS
• Speedup the Video : setpts=[value>=1]*PTS

Example (Slow down the video 8 times the original speed) :
ffmpeg -i fastVideo.mkv -filter:v "setpts=8.0*PTS" slowVideo.mkv

The output video may not be as smooth because frames are being skipped. To ensure a smoother output video, the minterpolate filter can be added and the desired output FPS can be explicitly passed.

minterpolate='mi_mode=mci:mc_mode=aobmc:vsbmc=1:fps=60

Demo Video(https://www.veed.io/view/7c3ed804-81ae-4db7-9748-621dae16bfdc?panel=share)

Video stabilization : This is one of the most useful features because most cameras don't have good stabilization. we can use VidStab to focus the video on the center and create a stable video effect, which helps to reduce shakiness in the video. FFmpeg comes with the vidstab plugin

Step 1 : Process the video and generate a so called transform file, which describes camera movements

ffmpeg -i ShakyVideo.mp4 -vf vidstabdetect=stepsize=32:shakiness=10:accuracy=10:result=transforms.trf -f null -

Step 2 : Stabilize video by passing filters such as smoothing, zoom and unsharp

ffmpeg -y -i ShakyVideo.mp4 -vf vidstabtransform=input=transforms.trf:zoom=0:smoothing=10,unsharp=5:5:0.8:3:3:0.4 -vcodec libx264 -tune film -acodec copy -preset slow output.mp4

Demo Video(https://www.veed.io/view/7b856d9a-6943-4f81-b55f-b0c2e0c5a14a?panel=share)

Adding Text Layer in Video : We can add a text layer to a video using the drawtext filter, which makes it easy to manipulate the text. You can change the font, text size, position, add a background, adjust the color, and most importantly, set the time frame.

ffmpeg -i input.mp4 -vf drawtext="fontfile=testFont.ttf: text='Super Heavy Test Flight #1':x=75:y=50:fontsize=32:fontcolor=red" -codec:a copy videoWithText.mp4

To display text only within a specific time frame, we can add
:enable='between(t,startTime, endTime)'filter in drawtext

x : margin from left in px
y : margin from top in px

Demo Video(https://www.veed.io/view/97bb23ca-5db8-4818-a116-973d4cb6a329?panel=share)

Video Color Filters :
There are two ways to achieve this:

1 -  Create a slider in the frontend that ranges from (-1 to 1 for brightness) and (0 to 3 for saturation).
2 -  Create a predefined selector in the frontend (with only a few allowed values or simple 3-4 states).

Although approach 2 is better because the video filter layer processing is done on the server side, not on the client machine. therefore, it is better to create states like 0%, 25%, 50%, 75%, and 100% for these color filters.

flowchart for applying color filters

August 11 - August 20

We have almost finished fixing bugs, improving UI/UX, and enhancing performance/development workflow. Now, I am planning to use JSDoc, which will be useful for developers, as rewriting the whole project with TypeScript in this time frame is not practical.

August 21 - August 27: Testing Using React testing library + Jest and integrate it with Jenkins-bot/CI.

In the final week, I will be addressing any remaining backlog and finishing the project by writing unit and end-to-end tests for both the frontend and backend. This will include testing API endpoints as well.

Unit Test: Here, I will write separate unit tests for each component, testing all possible interactions with the component. These tests will check if the component is rendering at the intended time or not, and I will also conduct micro-tests related to data fetching associated with the component using Jest and React Testing Library.

E2E Test: This test will mimic user interactions with the app from start to end, including all edge cases. It will also compare the correct error messages associated with them and data fetching throughout the app.

API Testing: I will manually test whether VideoCutTool interacts correctly with the Wikimedia Commons API by mocking the API calls and checking the fetched data. Apart from manual API testing, I will be using a library like nock (https://mswjs.io/docs/comparison) which will help in covering the edge cases.

In the end, I will test the compatibility with all modern browsers and operating systems using the jest-puppeteer library.

Reference to the Wikimedia API : https://www.mediawiki.org/wiki/API:Upload

August 28 - September 4: Final Evaluation

Wrapping Up
I plan to complete all the tasks within their respective time frames. However, if some tasks remain unfinished, I will aim to complete them In this time frame and address any bugs or issues in the code or new features.

September 5 Initial results of Google Summer of Code 2023 announced

September 4 - November 6 : As this is a large project, the timeline can be extended if required. Additionally, since this is an editing tool, there may be other improvements related to dev/client that can be made in the future. For example, writing the project in TypeScript and updating the packages to the newest versions. During this period, I would like to take input from the mentors and work accordingly.

November 6 - Novermber 13 Final Submission and Evaluation

Past Experience

I have been contributing to open source for the last 2-3 months. I have contributed to Backpack, Moja-community, and recently started contributing to VideoCutTool.

• Contributions made in the VideoCutTool by me :
  • T332988: allow the user to paste video from the clipboard
    • https://gerrit.wikimedia.org/r/c/904603 In-Review
  • T333736: Fixed alignment of icons in trim container
    • https://gerrit.wikimedia.org/r/c/labs/tools/VideoCutTool/+/904601/ Merged

• T334059: Prevent mobile title overflow on long video titles
  • https://gerrit.wikimedia.org/r/c/labs/tools/VideoCutTool/+/906700/ Merged

contribution made by me in coral-xyz/backpack : ETH and Solana Crypto wallet

• #3412 : Add support for pasting images from the clipboard
  • https://github.com/coral-xyz/backpack/pull/3412/ Merged

• #3309 : Clicking reply should auto-focus the messaging input field
  • https://github.com/coral-xyz/backpack/pull/3309 Merged

• #3437 : only accept video/image file type in chat file upload
  • https://github.com/coral-xyz/backpack/pull/3437 Merged

• #3179 : Fix : checkbox not visible in dark mode
  • https://github.com/coral-xyz/backpack/pull/3179 Merged

• #3141 : Fix : Send button on the contact profile page is not functional
  • https://github.com/coral-xyz/backpack/pull/3141 Merged

• #3333 : FIX: Notification profile pic should open the user profile page
  • https://github.com/coral-xyz/backpack/pull/3333 Active

Issues Authered

• T333564 : Improve Keyboard Navigation: play/pause the video with the spacebar key press
  • https://phabricator.wikimedia.org/T333564 Open

• T334059 : Make UI adaptive/responsive (work better on mobile)
  • https://phabricator.wikimedia.org/T334059 Working

Expected Designs

Home

Add Text Layer

Speed Control

Rotate Video

Previous Projects Around Collaboration

Fitmon

I have created a live Yoga Trainer in a 48 Hours Hackathon using Mediapipe. This trainer uses the MediaPipe Pose Model to track the human body in real time and allocates points to 32 different joints. With the help of these points, I have calculated various angles for a particular Yoga pose. These angles are then compared with the standard/correct set of angles. If there is any discrepancy, the trainer guides the user with voice commands.

This Yoga Trainer provides real-time feedback to the user and helps them to maintain the correct posture while performing Yoga. The use of MediaPipe Pose Model makes it more accurate and efficient as it tracks the human body movements precisely. With the integration of voice commands, it becomes easier for the user to follow the instructions without constantly looking at the screen.

This project also secured 2nd position HTC 2.0 Hackathon organized by BVP-ISTE

Bloggo

Bloggo is a full-stack blogging app, anyone can create and publish their own blog posts.

try bloggo(https://blog-c40q.onrender.com/)

About Me

I am currently pursuing B.Tech in Information Technology from Maharaja Agrasen Institute of Technology. I started my open-source journey with Hacktoberfest 2021, and since then, I have enjoyed working on open-source projects. I have contributed to coral-xyz/Backpack, and by far, this is the best feeling because I use Backpack myself, and seeing my changes in the app feels great. This is my first participation in GSoC, and during the summer, GSoC will be my first priority since I won't have any other commitments during this period.

and in this time frame, I'm only planning to contribute to open-source projects and GSoC and I'm not applying for any internship or any other program

Recently, I also started contributing to VideoCutTool, and I am looking forward to improving the tool and making it easier to use.

How did you hear about this program?

I got to know about GSoC in my freshman year when I contributed to Hacktoberfest 2021 but at that time I didn't have that much technical knowledge, so I just fixed some minor typos and other good-first-issue in some open-source projects but didn't have the confidence to apply for GSoC.

Will you have any other time commitments, such as school work, another job, planned vacation, etc, during the duration of the program?

No, GSoC is my topmost priority, that's why I haven't applied for any summer internship or summer training. During summers I will be able to invest my complete time towards this project by fixing existing issues and bugs and will be adding some additional valuable features to it after the approval by Mentors. and I have applied for only one organization so I can be loyal to one project and commit my time here.

What does making this project happen mean to you?

the project is both exciting and challenging. additionally, I am grateful to the Wikimedia Foundation for their invaluable work in supporting the community. Contributing to VideoCutTool is a way for me to give back to the community while also enhancing my skills as a developer. and apart from this, I really enjoy contributing to open-source projects and this will be a whole new experience for me.

Participation

I am active on Zulip, Phabricator and email. I'll be using Gerrit to push my code and will work on suggestions given by mentors in both Phabricator and Gerrit comment section.