Page MenuHomePhabricator

[Outreachy 2020 - 2021 Proposal] Evaluate Microsoft Playwright as replacement for our browser automation
Closed, DeclinedPublic

Description

Profile information

Name: Hulya Karakaya
Timezone: UTC-7
Web Profile: https://hulya.netlify.app/
Github: https://github.com/hulyak
Location: Seattle, WA
Working Hours: 9 am - 4 pm UTC-7, 8 pm - 12 am UTC-7

Contributions

634296
634301
634296
P12994
634994

Synopsis

Wikimedia supports different projects and repositories, and they value testing and automated end-to-end tests running as part of their CI process.
Currently, they use WebdriverIO as their browser automation framework for implementing end to end tests across multiple repos along with Puppeteer and Cypress. However, Cypress needed to be dropped because it couldn't run properly on CI. Among so many different technologies, a new automation framework, Microsoft Playwright, came into play.

Playwright is written by some of the same people who authored Puppeteer and it is maintained by Microsoft and Wikimedia would like to evaluate it against WebdriverIO as a potential replacement for browser automation framework.

Possible Implementation Methods

This is an iterative process, every feature must be checked and tested very well against Microsoft Playwright and existing testing tools. We need to have realistic expectations, and start testing small parts of the project first and decide how to replace existing code with Playwright.

Why Is This Necessary?

Playwright utilizes the same architecture as Puppeteer and is a thin WebSocket client. It uses a very similar syntax and language but there are a few differences—namely that Playwright supports more browsers (Safari) and that Playwright feels like a test automation tool rather than just an automation tool. It is built into to enable cross-browser web automation that is ever-green, capable, reliable, and fast.

Playwright is simple to set up, it has test framework stability features. Installs Chrome, Firefox, or WebKit (Safari) in a working version automatically. It has a thin wrapper and JavaScript first approach, so the code feels very natural.

How is Playwright different?
  1. Playwright delivers reliable, timeout-free automation.

Modern web apps are rich and responsive, issuing network requests and DOM changes based on user interactions. This asynchronous behavior makes modern apps harder to predictably automate. Traditional automated tests rely on sleep timeouts to manage this complexity, but timeouts often lead to unpredictable failures.

Playwright automatically waits for the UI to be ready, which ensures tests are reliable to execute and simpler to author. Under the hood, Playwright uses an event-driven architecture that can listen to precise browser events like DOM changes, network requests, and page navigations.

  1. Playwright is built for the modern web.

The web platform is continuously evolving and adding newer capabilities every year. Playwright is built to automate newer web features, including emulation of mobile viewports, geolocation, and web permissions. Playwright scripts can even intercept and modify network activity and automate scenarios across multiple pages.

  1. Playwright works on all modern browsers.

With Playwright, you can author automated tests for all modern browser engines: Chromium (for Google Chrome and the new Microsoft Edge), WebKit (for Apple Safari), and Mozilla Firefox. WebKit is supported on all platforms, which enables you to test rendering on Safari, even on Windows and Linux machines.

How Will The Proposed Features Benefit Wikimedia Projects?

Wikimedia can integrate Playwright in different repositories.

Do You See Any Risks/Concerns Involved In implementing The Planned Features?

With a plethora of automation tools out there, sometimes it becomes challenging to choose the best. Improvement of the overall testing procedure and meeting real requirements is the end goal. Automation testing is, without a doubt, highly dependent on the tool you decide to go ahead with. Every tool has specific capabilities. But it can be hard to have the level of expertise needed to get the best out of these capabilities. If the team works well, and check test reports, Playwright can be a success. But again, even if it fails, any other technology can be implemented.

Mentor(s)

@zeljkofilipin , @Soham , @Vidhi-Mody

Have you contacted your mentors already? Yes

Deliverables

Internships period: Dec. 1, 2020 - March 2, 2021

PeriodTask
Dec. 1, 2020Dec. 7, 2020Community bonding period - Communicating with mentors and exploring repositories to implement Microsoft Playwright testing. - Reference this blog post. Read stories across the Wikimedia movement on Medium. Setup up my MediaWiki user page and keep it up to date with my project work and reports. Watch previous videos on Wikimedia's technical topics.
Dec. 8, 2020Dec. 22, 2020Prepare weekly reports and start writing a blog post and add it to my MediaWiki user page. Attend meetings and learn about code practices and clean testing with Playwright. Watch previous videos on Wikimedia's technical topics. Pick a repository, get a better understanding of Puppeteer, Playwright, and WebDriverIO and how it is implemented in different repositories.
Dec. 23, 2020Jan. 06, 2021Introduce video recording and screenshots of tests, according to my research and the feedback from mentors. Work on a blog post and Playwright testing to the repository.
Jan. 07, 2021Jan. 21, 2021Finish the blog post, and communicate with mentors, and reimplement tests on the project. Make screenshots and video recordings of the tests.
Jan. 22, 2020Feb. 5, 2021Write a blog post and work on a different file to implement the tests and test against Puppeteer, Web DriverIO, and Playwright, evaluate the tests with Cypress to check if the issues are resolved. Try to solve the failing tests, screenshot the tests and do it again.
Feb. 06, 2021Feb. 20, 2021Get feedback from mentors, and test other parts of the project. Reevaluate the tests against Cypress and reiterate the whole process. Write a blog post.
Feb. 21, 2021March 2, 2021Wrap up the project and write about the experience.

Other Deliverables

Blog post on my progress every week
Regular communication with my mentor(s) and other community members

Participation

Describe how you plan to communicate progress and ask for help, where you plan to publish your source code, etc

I will select the repository to work on.
I will be online during my working hours( 9 am - 4 pm UTC-7, 8 pm - 12 am UTC-7) to collaborate with mentors and community members using Zulip messenger and other channels
I will use Phabricator for managing bugs and subtasks.
I will be available on Gmail/Zulip to be contacted when needed during the non-working hour

About me

Your education (completed or in progress)
My background was in education, I have got a diploma in education and international relationships; however, I had a curiosity for technology and decided to learn programming, I have learned through boot camps and Udacity, also got Cloud Engineering with Google Cloud certification and Microsoft's certificates. I have got a Scholarship from Linux Foundation and my interest in Open Source Sofware emerged, I have contributed to small projects like in Hactoberfest. I love joining Hackathons and learning from others.

How did you hear about this program?

I came across a blog post from a previous Outreachy applicant this year, and found the Twitter page and searched about the organization. I was so happy to find such a community that is helping underrepresented people in tech. I have signed up and got a mail that applications are opening and I have applied for the program.

Will you have any other time commitments, such as school work, another job, planned vacation, etc, during the duration of the program?
I am free during the time of the internship.

What does making this project happen to mean to you?

Wikimedia Foundation is a large organization and they have a number of different projects and repositories. The most important thing about this project is that it will help me to work with a real-life codebase. Learning is good; but when it comes to job search, the only thing matters is experience. I am lacking experience and Wikimedia would help me to get better at my skills and I would have some experience to show off to employers.

Past Experience

I am a junior front end developer specializing in HTML, CSS, Javascript, React, Node Python, Git, Google Cloud, SQL.
I am a member of Women Who Code Seattle and Facebook Developer Circles Seattle.

Skills

Languages: Javascript
Frameworks: React, Bootstrap, TypeScript, Express
Web Technologies: HTML, CSS, SASS
Version Control Systems: Git
Databases: SQLite, Postgresql, MongoDB, Firebase
Libraries and tools: React, React-Redux, Selenium, Jest, Mocha
Cloud Platforms: Google Cloud, Heroku, DigitalOcean

Open-source Contributions

I have contributed to ZTM and CODING organization through Github.

Event Timeline

@Hulya We are sorry to say that we could not allocate a slot for you this time. Please do not consider the rejection to be an assessment of your proposal. We received over 28 quality applications, and we could only accept 7 interns. We were not able to give all applicants a slot that would have deserved one, and these were some very tough decisions to make. Please know that you are still a valued member of our community and we by no means want to exclude you. Many interns who we did not accept in 2019 have become Wikimedia maintainers, contractors and even Outreachy interns and mentors this year!

Your ideas and contributions to our projects are still welcome! As a next step, you could consider finishing up any pending pull requests or inform us that someone has to take them over. Here is the recommended place for you to get started as a newcomer: https://www.mediawiki.org/wiki/New_Developers.

If you would still be eligible for Outreachy next year, we look forward to your participation!

Thank you so much for the feedback! I will try to contribute however I can.

If you're looking for a task take a look at T210726: Selenium framework maintenance. Let us know if you need help.