Page MenuHomePhabricator

GSoC 2020 Proposal : Improve Wiki Education Dashboard JavaScript build pipeline and reduce bundle size
Closed, DeclinedPublic

Description

Profile Information

Name Shashwat Kathuria
GitHub shashwatkathuria
Personal Website shashwatkathuria.github.io
LinkedIn shashwat-kathuria-053796182
Resume Click to view
Location New Delhi, India
Typical working hours 12PM to 3 AM UTC+5.30

Synopsis

  • Short summary describing your project and how it will benefit Wikimedia projects

WikiEduDashboard is a web application which provides tools that help instructors and students share their knowledge on different wikis like Wikipedia, Wikidata, etc and also be able to track their contributions. Instructors are able to organize different types of courses like Edit-a-thons, Article Scoped and Basic, along with the flexibility to also let program leaders and instructors run campaigns having multiple courses. Keeping track of contributing on a vast platform like Wikipedia, or other wikis is itself a task and this application helps in managing that. With the backbone of such a strong implementation, there are many articles on Wikipedia which ultimately help people across the globe in having access to the knowledge shared. It is implemented with Ruby on Rails on the backend and mostly ReactJS on the frontend and runs on two websites: https://dashboard.wikiedu.org/ and https://outreachdashboard.wmflabs.org/

Over the last several years, evolution in the JavaScript trends have led to new features and some of them are to be implemented in the build tooling of the application. Currently, the project uses Gulp, a toolkit for running, automating time-consuming tasks and Webpack, a static module bundler for JavaScript (mostly) along with HTML, CSS and images which generates a dependency graph for helping developers in modular approach.

Webpack is basically an advanced build process, being similar to Gulp or Grunt, taking files, running them through some sort of conversion and spitting them out into something the browser can understand. It splits things in a modular way, such that things are no longer global, relieving us of certain collisions in namespaces, and making things very explicit which is very much required in an application made in React. It also handles conversion from ESNext to the current version of JavaScript that works in the browser using Babel. So using it with something like Babel or Sass or autoprefixer basically converts the code to something the browser can read, also making the code very organized. It can do dead code elimination, bundle up smaller modules of JavaScript so that all JavaScript of the application doesn’t end up reaching a user using only a small part of the application. The current usage of Gulp in our project can be completely replaced by Webpack, giving a significant performance boost and modularity.

  • Possible Mentor(s)

@Ragesoss

  • Have you contacted your mentors already?

I have communicated with my mentor Sage Ross many times, via GitHub and Slack and have been able to figure out a common time for discussion.

Implementation Details

The current implementation of the build tooling and bundle needs improvement. Removal of jQuery (2 libraries relating to it in package.json) and replacing it with vanilla JavaScript(which can work well when used with Code Splitting), and removal of other unnecessary dependencies needs to be done.

Currently the tasks Gulp is used for can be performed by Webpack itself. So, keeping in mind the advantages and disadvantages, removal of Gulp needs to be done, and the tasks be performed by Webpack itself, after weighing multiple factors like layering in running the tasks, I/O Operations in the processes, number of available plugins, number of tasks, compatibility with other operating systems like Windows in which npm doesn’t work nicely, etc.

Now coming to the tasks, following tasks can be found in gulp/tasks :

  • clean.js
  • copy-static.js
  • i18n.js
  • icon-font.js
  • jquery.uls.js
  • lint.js
  • minify.js
  • set-development.js
  • stylesheets.js
  • watch.js
  • and ultimately webpack.js

All of these tasks relate to asset compilation (which are present in app/assets, consisting of fonts, images, javascripts, stylesheets and svgs) which can be handled by Webpack pretty nicely in a modular and well organized fashion, which is why it has become an industry standard these days. Currently, in webpack.config.js we make use some of the libraries (configured as entry points) defined in app/assets/javascripts consisting of :

  • main.js
  • raven.js
  • styleguide.js
  • survey.js
  • survey-admin.js
  • survey-results.js
  • campaigns.js
  • charts.js
  • tinymce.js
  • embed_course_stats.js

Some of these are standard libraries, while others are user defined/project specific.

When the asset compilation will be shifted to Webpack along with Code Splitting, we will observe a significant performance boost as this extra layer of Gulp, which is not recommended when the number of tasks goes into two or more digits (which here is 11 tasks), will be removed. Also many of the extra dependencies involved with Gulp in package.json Line 38 - Line 61 can be removed and the related (and required) Webpack plugins, loaders, etc can be added to webpack.config.js, which will definitely be less in size and number. One of such instances can be pointed by looking at the i18n.js task in Gulp itself, which can be integrated into Webpack by adding i18n-loader or i18WebpackPlugin.

A significant performance boost can be obtained by Code Splitting, i.e., chunking the bundle along several entry points which are to be identified and configured, prevention of duplication of libraries by separating shared modules like lodash and Prefetching & Preloading modules. A useful feature can be implemented to split code into various bundles which can be loaded as required - on demand / parallel. Also implementing dynamic importing wherever applicable. Prefetching & Preloading of modules can be done where usage hints are there for future use in application, for example, Prefetching & Preloading libraries required for Login Module when a user clicks Log In With Wikipedia Link.

A Webpack bundle analyzer tool webpack-bundle-analyzer will also be added to visualize the size of the asset compiled files (output) of Webpack.

Along with this, Tree Shaking will be done, i.e., removal of code exported but not imported, which can be solved by importing by module name as much as possible. A useful feature of Webpack, Scope Hoisting will also be implemented which can help in reducing the Webpack bundle size and make JavaScript execute faster in the browser.

We can observe that a heavy library lodash is being used in many places in the code. It can be observed in package.json that 6 libraries relating to lodash are being used, so the removal of unused features of some libraries like lodash can be a boost. Addition of lodash-webpack-plugin can be done to remove lodash features that are not being used. This would significantly help in reducing bundle size.

Moreover, there are a few Ruby gems which can be eliminated, some which are deprecated and some which can be replaced with others. They include figaro which can be replaced by dotenv Link to Comparison, removal of validates_email_format_of, will_paginate which can be replaced by kaminari Link to Comparison and timecop (Related Link) (linked to issue #3864) with rails in built features if linked PR is not merged.

Deliverables

Describe the timeline of your work with deadlines and milestones, broken down week by week. Make sure to include time you are planning to allocate for investigation, coding, deploying, testing and documentation

1 April - 3 May
  • Study more about bundle analyzer tools and possible ways in which bundle size can be reduced
  • Gather more knowledge about how the usage of some heavy libraries like jQuery can be replaced with vanilla JavaScript (plain JavaScript)
  • Continue solving more project related bugs on WikiEduDashboard
May 4 - May 30
  • Community bonding period
  • Finalize the bundle analyzer tool to be used
  • Discuss the advantages, disadvantages and differences between Gulp and Webpack (and npm) with the mentors
  • Finalize the libraries (and dependencies) and gems which are to be added or removed from the build pipeline and rails bundle respectively as per the current trends
  • Mark the bugs to be solved during the internship
Week 1 (1st June - 7th June)
  • Perform Tree Shaking
  • Remove dead code from the bundle in places where code is exported but not imported anywhere
Week 2 (8th June - 14th June)
  • Perform Scope Hoisting
  • Add a bundle analyzer tool.
  • Integrate the use of i18n internationalization files into Webpack
  • Add fingerprints to the i18n files
Week 3 (15th June - 21st June)
  • Replace the use of jQuery with vanilla Javascript
  • Remove unnecessary dependencies wherever applicable
Week 4 (22nd June - 28th June)
  • Testing
  • Documentation
  • Analysis using bundle analyzer, performance metrics
  • Solve bugs
Week 5(29th June - 3rd July)
  • Phase 1 Evaluations
  • Start doing Week 6 work alongside
Week 6 ( 3rd July - 9th July) & Week 7(10th July - 16th July)
  • Removal of Gulp
  • Transition into Webpack
Week 8 (17th July - 23rd July)
  • Perform Code Splitting
  • Split code into various bundles
Week 9 (24th July - 27th July)
  • Testing
  • Documentation
  • Analysis using bundle analyzer, performance metrics
  • Solve bugs
Week 10 (27th July - 31st July)
  • Phase 2 Evaluations
  • Start doing Week 11 work alongside
Week 11 (1st August - 6th August) & Week 12 (7th August - 13th August)
  • Continue performing Code Splitting
  • Split code into various bundles
  • Prefetching & Preloading of modules
  • Removal of unused features of lodash
Week 13 (14th August - 20th August)
  • Replacement of deprecated gems and removal of unnecessary gems in the rails bundle
Week 14 (21st August - 24th August)
  • Testing
  • Documentation
  • Analysis using bundle analyzer, performance metrics
  • Solve bugs
24th August - 31st August
  • Final Project Submission
  • Final Evaluation Submission
31st August - 7th September
  • Final Evaluations by Mentors
8th September
  • Final Results Announced

Additionally, I will post about weekly updates and experiences on my personal website and meta wiki user page.

Participation

Describe how you plan to communicate progress and ask for help, where you plan to publish your source code, etc

  • Joined Slack group for WikiEduDashboard
  • Regularly communicating with mentors and asking for help as required
  • I am going to work on separate branches on git (GitHub) and upload code on a regular basis. Also, create pull requests as and when a feature is completed. I have followed this process since a while now in my PRs.
  • Online on Slack and active on GitHub during my working hours.
  • I will also communicate on Phabricator via comments on the related project subtasks.
  • I will submit my weekly reports on my meta wiki user page
  • Weekly publish my experience on my personal website

About Me

Tell us about a few:

  • Your education (completed or in progress)

I am currently pursuing Bachelor of Technology, Computer Science & Engineering from Indian Institute of Technology (IIT), Jodhpur and I am in my 3rd year (6th Semester) currently. The program spans four years (8 Semesters).

  • How did you hear about this program?

I heard about this program through students in my college. A few of them have already participated in the program. I have gained a lot of insight regarding the same from them, and was encouraged to take part in the same as I am really interested in open source development and participating in this program would give me a kick start for the same.

  • Will you have any other time commitments, such as school work, another job, planned vacation, etc, during the duration of the program?

I will be able to dedicate 40 hours per week to this project, and more if required.

I don’t have any other time commitments during the duration of the program apart from a brief period of time other than vacations in my college. Also, my summer vacations span from 3rd of May to 19th of July and I will be able to work during that time fully. In the remaining small part of the college days, I will still be able to work in the working hours specified as mostly all of my classes are during early morning hours only. Also, I have mostly worked while attending college till now in the WikiEduDashboard GitHub repository and do not face any issues regarding time commitments.

  • We advise all candidates eligible for Google Summer of Code and Outreachy to apply for both programs. Are you planning to apply to both programs and, if so, with what organization(s)?

I am applying only for Google Summer of Code and in 2 projects of Wikimedia Organization regarding WikiEduDashboard project.

  • What does making this project happen mean to you?

My inspiration to work on this project is due to the reach and free knowledge that this project provides, being used in many universities worldwide and collectively enhancing our knowledge, so I want to be a part of this initiative by contributing and try to make it even better with more features. Apart from that, there are technical reasons also due to which I am extremely excited to work here, first and foremost, WikiEduDashboard being a Web Development project, which is my favourite field, along with the real life applications of how an application is implemented to serve people communicating in different languages, the design of the application, database, classes, testing, performance, etc put together to give one piece of a software which is used by so many people around the globe.

Past Experience

Describe any relevant projects that you've worked on previously and what knowledge you gained from working on them. Describe any open source projects you have contributed to as a user and contributor (include links). If you have already written a feature or bugfix for a Wikimedia technology such as MediaWiki, link to it here; we will give strong preference to candidates who have done so

I have been working with Ruby on Rails, Django, Flask and JavaScript frameworks like ReactJS, AngularJS since 3 years and have done several web development related projects, implementing all of them from scratch as a full-stack (both frontend and backend) developer. I have also worked with several databases like MongoDB, PostgreSQL, SQLite, MySQL and also used several hosting services like Heroku, Amazon Web Services, Microsoft Azure, MongoDB Atlas, GitHub Pages, etc. along with testing tools like Rspec, Jest, Capybara, Selenium, Django testing framework, etc.

I use Linux Ubuntu, have worked with virtual environments in my Django projects and also, I am very comfortable with git VCS (Version Control System) and have been using it in all of my projects since the past 2 years.

Relevant Projects
Social Cloud

An Instagram like website for users to post images and follow people. Implemented with Ruby on Rails, MongoDB/SQLite/PostgreSQL, Bootstrap and AngularJS. Uses Rspec and related testing tools. Hosted on Heroku and database on MongoDB Atlas.

Drawing Canvas

A drawing application for users to draw stuff and save them. Implemented with Django, SQLite/PostgreSQL, Bootstrap and a ReactJS & d3JS frontend. Uses Django testing framework. Hosted on Heroku.

IITJ HealthCare

An online medical prescription portal for patients to get prescriptions from doctors. Also includes a medical store for inventory information about medicines and an emergency portal. Implemented with Django, SQLite/PostgreSQL, Bootstrap and Javascript. Uses Django testing framework. Hosted on Heroku.

Disaster Safety App

An online portal which broadcasts alerts about natural calamities like earthquakes, hurricanes, etc. as and when they happen through user interaction. It also has an android application apart from the web application. Implemented with Flask framework in Python, Android Studio, SQLite, Bootstrap. Used Microsoft Azure VM for deploying.

Pull Requests in WikiEduDashboard (Started contributing around the starting of March 2020)
Issues in WikiEduDashboard

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 30 2020, 8:43 PM
Shashwatkathuria renamed this task from Improve Wiki Education Dashboard JavaScript build pipeline and reduce bundle size to GSoC 2020 Proposal : Improve Wiki Education Dashboard JavaScript build pipeline and reduce bundle size.Mar 30 2020, 8:44 PM
Pavithraes closed this task as Declined.May 5 2020, 5:45 PM
Pavithraes added a subscriber: Pavithraes.

@Shashwatkathuria Congratulations on getting accepted for T248918!!! ^_^