== Profile Information
Name: Formasit Chijoh Fokunang
Nickname: formasit
Github: Formasitchijoh
Email: formasitf@gmail.com/manoica2023@gmail.com
Phone Number: +237-675-718-328
Location: Cameroon
Time Zone: Cameroon (GMT + 1)
== Meeting with mentors:
I am reachable anytime through email, slack, and any communication channel
GMT + 1, 6:00 AM to 10:00 PM
= Synopsis
== [WikiEducation Dashboard] Improve the Wiki Education Dashboard test suite
Wiki Education Dashboard is a Ruby on Rails application with a server-rendered frontend, incorporating both React and Redux. It tracks and provides statistics for hundreds of thousands of editors on Wikipedia and other Wikimedia wikis. The codebase includes a comprehensive suite of Ruby tests, such as unit and feature tests, to ensure functionality. However, the current test suite frequently fails and takes approximately 30 to 35 minutes to run on GitHub CI. It is also fragile, with intermittent test failures. This project will focus on improving the quality and performance of the test suite, optimizing execution times, and increasing test coverage. Strategies like parallelizing and refactoring slow tests will be used to create a more reliable and efficient platform. Enhancing the Dashboard’s test suite will significantly improve the developer experience while making the platform more scalable and performant
== Mentors
Sage Ross(@Ragesoss), Sulagna Saha(@saha23s)
== Deliverables
1. Identifying and fix all test that fail intermittently
2. Optimizing Workflow and Rspec Test Suite
3. Establish a test suite that passes 20 consecutive runs on CI
4. Refactor, rewrite, and improve code quality, filling gaps in test coverage and writing tests for untested lines of code (LOC)
== Implementation Details
The major scope of this project focuses on the Ruby RSpec test suite for the Ruby on Rails backend. This suite includes unit tests for individual class files as well as browser-driven feature (integration) tests that interact with both the client-side UI and server-side behavior. Improving this section will significantly enhance the performance of the entire test suite.
While the primary objective will be to evaluate the Ruby test suite and its recent CI logs to identify intermittently failing tests, their underlying causes, and tests that contribute to long execution times, I will also address the JavaScript test suite. However, the main focus will remain on Ruby. Proper evaluation of the external APIs that the system interacts with will be conducted alongside these efforts. The project will target areas of the Ruby codebase that exhibit gaps in test coverage and lines of code (LOC) that lack tests. Both the tests and the underlying code will be rewritten as needed
= STEP-I
The current API of the system throws errors intermittently. To address this:
* Unit tests will be ran on each controller, model, and service in isolation to ensure they work as expected.
* Identify patterns of test failures, and analyze logs to trace the causes.
* Document each failure and its cause.
Features relying on external APIs such as the LiftWing API, ReferenceCounter API, Commons Wiki, and WikiApi often fail intermittently. These issues stem from
* Inconsistencies in API responses and network reliability.
* LiftWing API: The API returns responses that differ from the ones the test specs were written around. Tests need to be updated to handle these variations in responses.
* Perform extensive testing under different conditions to analyze the responses from these third-party APIs and refactor existing RSpec tests and related code to account for the observed behavior.
* Investigate why these APIs interact inconsistently with the system and document the conditions that cause failures.
The current CI/CD pipeline takes approximately 30 to 35 minutes to complete, which is inefficient for development.
* Review recent Sentry logs and development logs to identify failure patterns and their evolution over time.
* Analyze the slowest API requests and test cases identified by the test suite, using timers to determine the execution time of each request.
Based on this data, implement strategies to optimize performance and reduce pipeline runtime.
I am currently addressing issues in both the JavaScript and Ruby test suites. Comprehensive tests are being written to cover gaps identified during previous contributions.
Failures encountered during the contribution stage are being systematically reviewed and resolved. I will continue to fix these issues through the Outreachy result declaration and the bonding phase.
= STEP-II
After analyzing the entire test suite and identifying the causes of intermittent failures, the next step is to implement solutions to optimize the test suite and improve performance. This will involve a range of technical strategies and improvements:
RSpec Optimizations
---------------------
* Separate Unit Tests and Integration Tests
* Write logic to run unit tests on every push, pull_request, and Integration Tests less often
Based on the results from the isolated unit tests, refactor and update tests to handle breaking changes. Focus will be on:
* Controller and Model Tests: Adjusting test logic to handle edge cases and undefined behaviors identified in STEP-I.
* External Service Tests: Improving mock strategies for third-party services (LiftWing, ReferenceCounter, etc.) by implementing enhanced stubbing/mocking mechanisms for unpredictable API responses.
* New unit tests will be written to cover previously untested code paths, ensuring all models, services, and controllers are fully covered.
* Enhancing External API Handling
Following the analysis in STEP-I, all API-related tests will be restructured:
* Update specs and tests to accommodate the changing response structure. Use dynamic response validation mechanisms such as JSON Schema validation to account for different responses.
* Improve the current retry mechanisms in API calls to handle intermittent failures due to network issues, ensuring more consistent API interactions.
* Integration Tests: Perform additional integration tests where the system interacts with the API, simulating real-world scenarios such as rate limiting, slow responses, and server errors.
End-to-End System Testing (JavaScript & Ruby)
* The server-rendered parts of the system, including both Ruby backend and JavaScript/JSX frontend, will be tested end-to-end using Capybara and Selenium WebDriver. This will:
* Simulate user interactions across the entire system, ensuring that controllers, views, and front-end logic function seamlessly together.
* Test cross-system functionality like user flows, login, data processing, and saving features.
Workflow Optimizations on the CI/CD pipeline
---------------------
* Running Steps Concurrently
While executing parallel steps is not supported by Github, a custom script will be written to run some steps parallel to reduce workflow time
* Separate workflow steps depending on file changes
There is no need to run RSpec tests if only a javascript file or some .md file has changed. We can filter the steps based on file changes using dorny/paths-filter@v3 workflow action.
* Handling API rate limits and timeouts will be addressed by implementing throttling, rate-limiting, and retry mechanisms, such as exponential backoff, to minimize failures from API rate limits
* API latency monitoring and fallback mechanisms will ensure external API latency is monitored with tools like Sentry or New Relic, and fallback mechanisms will be introduced to handle delayed responses effectively.
* Create new test Data configurations and ensure existing once are improved to work
= STEP-III
* Execute comprehensive end-to-end tests across the Ruby backend and JavaScript frontend to confirm all components interact seamlessly and fulfill user flows,
* Validate that unit tests have been successfully refactored and cover edge cases, ensuring all controllers, models, and services are functioning correctly without breaking changes.
* Monitor the performance of external API interactions to ensure stability and reliability, verifying that the retry mechanisms effectively handle intermittent network issues and that API responses conform to the updated specs.
* Confirm that the CI/CD pipeline runtime has been successfully reduced to under 20 minutes by analyzing execution times for all test suites and ensuring parallel testing is optimized.
* Create a developer guide detailing troubleshooting steps for common test failures and highlight the strategies implemented to enhance the testing process.
* Collect feedback from developers regarding the test suite and documentation to identify areas for further improvement and refinement.
== Future Goals:
Will be implemented if all above-mentioned milestones are achieved and completed successfully and approved by the mentors, also desired to be taken up after Outreachy.
*Increase the JavaScript/JSX statement coverage to 90%
*Continue monitoring the system platform to ensure all test pass properly and are effective and handle new code
= Timeline
October 29 - November 26:
---------------------
* Continue making contributions to the WikiEducation Dashboard
* Ongoing analysis of the existing test suite, focusing on documenting flaky tests
* Run unit tests for different controllers, models, and services in isolation to ensure they work as expected and identify patterns of failures.
* Document the causes of failures identified during testing
=== November 26 - December 9
* Community bonding period
* Begin with user research to identify further areas of improvement in the existing test suites.
* Ongoing analysis of the existing test suite, focusing on documenting flaky tests
December 9 - December 14 (Week 1 )
---------------------
Third-Party API Analysis:
* Analyze failures in system api and features that utilize third-party APIs (LiftWing, ReferenceCounter, Commons Wiki, WikiApi).
* Test these APIs under different conditions to analyze responses and update existing RSpec tests to handle varying response formats.
* Review recent CI , Sentry and development logs to identify failure patterns and performance issues.
* Analyze slow API requests and tests to determine execution times and strategies for optimization..
December 16 - December 21 (Week 2)
---------------------
* Refactor unit test for test that fails intermittently and write new test for uncovered section of the ruby rspec test suite
December 23 - Jan 5 (Week 3 & 4)
---------------------
* Continue Refactor unit test for test that fails intermittently and write new test for uncovered section of the ruby rspec test suite
* Create new test data configurations and ensure existing ones are improved.
Jan 7 - Jan 12 (Week 5)
---------------------
* Analyze previous work and incorporate feedback from the mentor
* Discuss potential improvements to existing tests and gather feedback on recent work.
* Mid-point feedback gathering
Jan 14 - Jan 19 (Week 6)
---------------------
* Perform additional integration tests where the system interacts with the API, simulating real-world scenarios such as rate limiting, slow responses, and server errors.
* Improve the current retry mechanisms in API calls to handle intermittent failures due to network issues, ensuring more consistent API interactions.
* Update specs and tests to accommodate the changing response structure.
Jan 21 - Jan 26 (Week 7)
---------------------
Feedback and External API Testing:
* Discuss improvements to existing tests and gather feedback on recent work.
* Continue working on the improving the test for external api
* Use Sentry logs to improve test suites, focusing on reducing runtime and optimizing performance
January 28 - Feb 3 (Week 8 )
---------------------
Focused Testing on Problem Areas and reducing test run below 20 min:
* Parallel testing by dividing the test suite into smaller groups that run concurrently across multiple CPU cores or machines, reducing the total CI/CD pipeline runtime
* Write scripts that track codebase changes and trigger only the necessary tests based on dependencies, reducing redundant executions both in development and on the CI/CD pipeline
* Handling API rate limits and timeouts will be addressed by implementing throttling, rate-limiting, and retry mechanisms, such as exponential backoff, to minimize failures from API rate limits
Feb 5 - Feb 10 (Week 9)
---------------------
* Analyze previous work and incorporate feedback from the mentor
* Ensure all existing modules are fully tested and cover all functionalities.
* Write new integration tests for new code and features, ensuring everything works as expected.
Feb 12 - Feb 17 (Week 10 )
---------------------
Stress Testing and End-to-End Testing:
* Perform stress tests on each API endpoint to evaluate performance under load.
* Use Capybara to conduct end-to-end tests, ensuring the server-rendered sections function properly across all system flows.
Feb 26 - March 2 (Week 11 )
---------------------
Final Testing and Documentation:
* Run end-to-end tests to ensure all intermittent test failures are resolved.
* Write a developer guide to assist in diagnosing issues in failing tests.
* Implement a user-friendly reporting strategy for error logs.
March 4- March 9 (Week 12 )
---------------------
JavaScript Coverage Improvement:
* Increase JavaScript/JSX test coverage to 90%
March 11- March 16 (Week 13 )
---------------------
Final Monitoring and Wrap-Up:
* Continue monitoring the system platform to ensure all tests pass and handle new code effectively.
* Wrap up the research and finalize the documentation.
* Conduct code cleanup in preparation for submission.
== Participation
I intend to use the project’s slack channel as the main communication channel and also through emails, as per the mentor’s convenience. I’ll equally github as the main means of contributing code where I will push commits to my own clone of the repository. I’ll be using GitHub’s issue tracker for feature requests and bugs and communicating through Phabricator task, I would equally help other contributors and users solve their issues and aid the process of reviewing my Pull request by writing good and understandable commit and pull request messages.
=About Me
== Education
I am a software engineering graduate from the University of Buea, set to graduate this December. During my studies, I gained proficiency in Python and Rust, test driven development, which enabled me have a strong knowledge software development. I was introduced to and began contributing to free and open-source software during my sophomore year, and I have continued ever since. The open-source community provides a collaborative and welcoming environment where I can practice and improve my skills without bias or limitations. I am particularly passionate about learning and ensuring that everyone has access to free, quality education. Contributing to Wikimedia and the Wiki Education Dashboard allows me to support this initiative in my own small way
== How did you hear about this program?
I learned about Outreachy from a mentor who has contributed to wikimedia through Outreachhy and GSOC over the years and suggested it was a great opportunity for me to learn and apply my skills in an industry project. I have applied to Outreachy for five times , the first two times I was accepted but didn't go pass the contribution phase and the last 3 I wasn't accepted.
== Will you have any other time commitments, such as school work, another job, planned vacation, etc, during the duration of the program?
I have no other commitments as i am done with my degree program. I would like to spend time with my family from 24th of December to the 1st of January but I can accommodate any important task during this time.
== Will you have any other time commitments, such as school work, another job, planned vacation, etc, during the duration of the program?
I don't have any other commitments, I don’t have any summer plan other than Outreachy. The Outreachy project phase takes the highest position on my priority list
== What does making this project happen mean to you?
I strongly believe in contributing in my own way to improve the challenges I have faced, enabling others to benefit from my experiences, my first PR to the platform ran for over 50 minutes and it failed. I struggled to understand the reason for the failure, as I did not clearly identify where the error originated and found it difficult to interpret the error messages. This process took me an entire day to resolve after seeking help from my mentors, Sageross and Abishecks, who were very approachable and provided valuable guidance. on what the issue was. Over the course of the development i have realized other contributors also have this challenge and for a newcomer it is very overwhelming, So improving the test suites of the dashboard will help improve the developer experience and help newcomers better understand the error they are having and what is the possible cause and how to fix them.
Furthermore, these improvements will enhance the overall performance of the dashboard, ensuring a seamless user experience and enabling students and instructors to maximize their use of the platform for education and knowledge transmission.
From a professional point of view, I have recently begun learning Ruby on Rails as part of my development skill set. Additionally, I am focused on writing comprehensive and efficient tests to improve system performance, maintainability, and scalability. I believe this project will allow me to apply my knowledge, enhance my skills as a tester, and contribute to significant FOSS projects while gaining direct access to mentorship.
== Past Experience
I am passionate about performant system that are maintainable and scalable, having been passionate about Software development since my second year in university since then I have been working continuously on personal projects and some open source projects. I have always been interested in building software that are reliable and has practical scale implications, and Open source has provided me with the platform to do so. I make commits with clean commit messages and well structured Pull Requests, I believe this helps me communicate my understanding of the task and help my mentors in the review process
= Coding Skills:
== Programming Languages and Frameworks:
Fluent in HTML, CSS, JavaScript, Ruby, Python
Proficient in Ruby on Rails and experienced with Django (Python) and Java.
Strong knowledge of OOP and MVC architecture.
Experienced with front-end frameworks such as React, Next.js, and Redux.
Proficient in testing tools: RSpec, Jest, and Capybara.
Familiar with CI/CD practices and tools.
Strong concepts of Git; can adapt to other version control tools if required.
Good analytical and problem-solving skills.
Effective communication skills for collaboration in a team.
Understanding of Agile methodologies
I have earned a great amount of experience with the codebase of the project. Also, I have made quite a few pull requests now and most of them have been merged. My contributions were focused towards writing unit test , fixing existing bugs,
Here is a list of my contributions:
== Wikimedia Contributions:
https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/5973
https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/5965
https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/5974
https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/5978
https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/5985
https://github.com/WikiEducationFoundation/WikiEduDashboard/pull/5986
= Other Contributions
== Unstructured Studio
https://github.com/unstructuredstudio/zubhub/pull/736
https://github.com/unstructuredstudio/zubhub/pull/778