Improve Wiki Education Dashboard's Error Monitoring
Project Repository: https://github.com/WikiEducationFoundation/WikiEduDashboard
GSoC Idea Link: https://phabricator.wikimedia.org/T244428
Profile Information:
Name: Hardik Chugh
Email: hardychugh34@gmail.com
University: Chitkara University, Punjab, India
Linkedin: https://www.linkedin.com/in/hardik-chugh-977a7b14b/
Resume: Click to View
Github Profile: hardy1334
Slack Username: _hardy_boy
Other Communication Method: Google Hangouts, Slack
Location (country or state): Punjab, India (UTC +5:30)
Typical working hours: Between 6 pm to 3 am UTC +5:30
About Me
I'm a Final year student at Chitkara University, Punjab, India pursuing my B.E in Computer Science.
Currently, I'm working with Truecaller as a Web Developer Intern since 27th May,2019 where I handle stuff across various web technologies and help in development of amazing functionalities . My Internship ends on 27th May,2020.
I provide my services at Fiverr on Git issues, Actions on Google and Website Development.
I have written quite a few tech blogs at Medium
I was also an Instructor at Chegg Tutors where I helped students across the globe with my knowledge of Computer Science domain, whether its web technologies related challenges or regarding programming languages.
Past Experience
I have experience in following Web Development technologies - React JS, Redux, Vue JS Node JS,MongoDB,Express JS, Django,Ruby on Rails, etc. A part from this I have also some experience in working with Docker and Kubernetes, which I gained during my internship while working on a project. I use Git as version Control System for my Day to Day work and prefer working on Linux based environments. I have also worked with Redis, on a project during one of my project on internship and also wrote a blog about redis.
While working on this proposal, as I was knew to background processing jobs, I started studying about the Sidekiq, how it works, how can we track the status of Sidekiq jobs, knowing about Sidekiq web ui and much more about it. Then when I was going through the codebase, I became familiar with Sentry how its being used in Wikiedudashboard to keep track of the errors but the problem with that is sometimes it doesn't keep track of all the errors. Then while going through CourseDataUpdateWorker I got to know more about the UpdateCourseStats service it was using, then I started understanding about this service what all its doing and gradually understood its working.
Projects
Devview : This project is uses technologies like React Js,Redux for Frontend, Node Js for server- side rendering and MongoDB for storing data.
Technologies Used: React JS, Redux, Node JS, MongoDB
Chefapp: This web app helps you create new recipes like suppose a chef wants to create a recipe for Chicken Soup ,in this he can create that recipe and upload that which others can view and he can also view other chefs recipes.
Technologies Used: Ruby On Rails, Maria DB
Locationapp: This app allows you to add hostels with its name,description and GPS location using Google Maps Api. You can search your location from Google Maps in this app and then you can accordingly drag your location on Google Maps with Marker and save that as Hostel Location which automatically selects state,city,locality of hostel from the location you selected.
Technologies Used: React JS, Google Maps API, Node JS, MongoDB
Possible Mentor(s)
Mentor: @Ragesoss (Sage Ross)
Synopsis
Wiki Education Dashboard is web based software satisfying the needs of both the instructors and the students. In it, instructors can create a course alongside keeping track of the progress of the students enrolled in that course, and students can complete their assignments, work with articles and much more stuff. So What I intend to do with this project is create an Error Monitoring System for the capturing errors in the course update process and then showing them to the user in appropriate manner. This error monitoring system will keep track of all the jobs linked to course which failed, store them and then will send notifications to the user in the course dashboard. Once this is done I intend to create a Dashboard which will have list of all the failed jobs in a particular course, alongside failed jobs, it will also contain error message corresponding to that particular job with its respective timestamp.
Deliverables
- Implement a functionality to keep track of all the failed jobs corresponding to a course and save them in our database.
- Implement a functionality to send Notifications to the user with an appropriate error message for the jobs which failed in the respective course.
- Develop a dashboard which shows all the failed jobs, alongside its reason for failure, timestamp corresponding to a particular course alongside user account on which it failed.
Implementation Details
Our First step in creating this error monitoring system is keeping track of all the jobs which have failed. So to achieve this goal -
As we know that Sidekiq web dashboard shows us what are the status of various jobs, if some has failed, it shows us which job has failed alongside the reason for failure.Something like this:
So to access the status of jobs i.e failed jobs , there are various gems like Sidekiq-status or Sidekiq-metrics which can help us in showing the status and metrics of our jobs.
SideKiq also has public api’s with the help of which we can check the status of a job. After all this, we can surely know which worker failed, and what was the reason for that. Like in our case its CourseDataUpdateWorker which is using services from UpdateCourseStats which handles all the updation of data. So if, any problem comes in it, CourseDataUpdateWorker will fail. So what we can do is as soon as this failure happens we can use Sidekiq apis or the above mentioned gems, we can retrieve properties like, which worker failed, reason of failure, when did it failed, etc.
After all this, we can save it in our database alongside, the course and user for the job. Now the next step is to implement api which can be used to send data to client side about this job failure containing appropriate error message for the error message which came for the reason of job failure.
Then I’ll implement functionality to send notification to the user regarding a particular job failure and will be using two way socket communication to make sure that notification is sent to the user in real time. It will be keeping track which failed earlier and are currently in retry phase, if they execute successfully, then it will also send notification to user that so and so job which failed earlier is now successful. Its a demo message which we will decide at the development what to put there.
After this entire system gets developed then I’ll move to displaying all the failed jobs in a particular course alongside all the details like,cause of failure, timestamp,etc, on some dashboard kind of thing.
FLOW DIAGRAM
Timeline
| Phase | Timeframe | Start Date | End Date | Task |
| Community Bonding Period | 4-May-2020 | 1-June-2020 | Research more on storing failing Sidekiq jobs in database, make designs ready for showing error message of failing jobs, make dashboard design ready for showing all the failing jobs. | |
| Phase 1 | Week 1 | 1-June-2020 | 7-June-2020 | Work on getting all the failing jobs for course and then saving them to DB |
| Phase 1 | Week 2 | 8-June-2020 | 14-June-2020 | writing rspec test cases for the functionality, if anything left over from previous week as per feedback given by mentor |
| Phase 1 | Week 3 | 15-June-2020 | 21-June-2020 | Work on creating api's which are to be used for client interaction |
| Phase 1 | Week 4 | 22-June-2020 | 28-June-2020 | Getting the functionality done from server side code if left any as per feedback given by mentor |
| Phase 2 | Week 5 | 29-June-2020 | 5-July-2020 | Developing UI and showing the appropriate error message for a job failure. |
| Phase 2 | Week 6 | 6 July-2020 | 12-July-2020 | Writing Unit test cases for the functionality being developed on client side. |
| Phase 2 | Week 7 | 13-July-2020 | 19-July-2020 | Finalise the content to be shown on dashboard for all the job failures and discuss with mentors for the same |
| Phase 2 | Week 8 | 20-July-2020 | 26-July-2020 | Work on development of the dashboard |
| Phase 3 | Week 9 | 27-July-2020 | 2-August-2020 | Finish the dashboard work alongside the feedback given by mentor |
| Phase 3 | Week 10 | 3-August-2020 | 9-August-2020 | Write unit test cases for the dashboard functionality being developed. |
| Phase 3 | Week 11 & Week 12 | 10-August-2020 | 23-August-2020 | Finish the remaining work if any and work on https://github.com/WikiEducationFoundation/WikiEduDashboard/issues/3194 |
| Phase 3 | Week 13 - Final | 24-August-2020 | 31-August-2020 | Resolving all the leftover bug fixes, issues , preparing for final evaluation and Writing blog about the overall project and experience. |
Participation
- How did You hear about this Program ?
I heard about this program from my college seniors in my sophomore year, then I applied in the GSOC-2019 to the Wikimedia Foundation to work on WikiEduDashboard project, but was not selected. Now in GSOC-2020 I'm again applying as now I feel I'm deserving to be given chance to work on this project.
- Have you discussed this GSoC Task with your mentors already?
Yes, I have discusses about this project with the mentor.
- What does making this GSoC Task happen mean to you?
I love to empower people, from my work and there could be no better opportunity than this. As this piece of software is used by thousands of people worldwide so it’s a great way to empower people with my skills. Not just that this will also help Wikimedia Foundation keeping track of all the track of errors which might come and indeed help in making this amazing product better in the near future.
- Will you have any other time commitments, such as school work, another job, planned vacation, etc, during the duration of the program?
No I don't have any other time commitments, as my current internship ends on 27th May,2020 which is before the official coding period starts (1st June,2020).
- Eligible for Google Summer of Code and Outreachy?
I am only Eligible for Google Summer of Code, so I'm only applying for Google Summer of code.
- Do you plan to submit any other proposal apart from this one?
No, I am not submitting any proposal apart from this one.
- How many hours per week can you dedicate for this?
I can dedicate a around 35 hours each week for this project, and I am willing to dedicate more if required.
- Have you been accepted to GSoC before?
No, I applied for the same project last year as well but was not selected.
Previous Contributions
I have made quite a few contributions to this project, from minor bugs to developing a new feature from scratch, and have become decently comfortable with the codebase now:
I also created some issues in Wiki Education Dashboard which are listed below: