Page MenuHomePhabricator

Evaluate Microsoft Playwright as replacement for our browser automation
Closed, ResolvedPublic

Description

Overview

As you are aware, currently Wikimedia supports a large number of repositories as well as extensions. In order to ensure good code practices across all these repos, we perform an extensive amount of testing.

Currently, we use WebdriverIO as our browser automation framework for implementing end to end tests across multiple repos. A list of all such repos running WebdriverIO tests can be found here.

However with the recent increase in awareness about end to end testing, a number of equally competitive non-selenium solutions have been introduced. Two such include Puppeteer and Cypress. As a part of this years Google Summer of Code program, an extensive evaluation of Puppeteer was performed. Unfortunately owing to Cypress not being able to run properly on CI. We had to drop evaluating Cypress.

One of the contenders being considered at the time was Microsoft Playwright. Seeing the recent developments in the framework and the rapid surge in popularity. We would like to evaluate it against WebdriverIO as a potential replacement for our browser automation framework.

Here is a tentative list of what this project aims to achieve:

Evaluate Playwright as an end-to-end testing framework

  • Implement existing tests across moderately sized repo's in Microsoft Playwright
  • Introduce video recording and screenshots of tests
  • Benchmark Playwright against Puppeteer (using previous implementation) and WebdriverIO.
  • Re-run a short evaluation on Cypress (Depending upon whether the issues with Cypress are resolved)

Skills required:

  • Node.js (Javascript)
  • Previous knowledge on automation frameworks and end to end testing, while not necessary would surely be a plus.

Mentors:
@zeljkofilipin @Vidhi-Mody @Soham

Microtasks

See T265011

Event Timeline

Soham updated the task description. (Show Details)
srishakatux changed the visibility from "Public (No Login Required)" to "Outreachy Mentors (Project)".

Thank you for creating this task! It looks like a solid Outreachy project :) I've restricted the visibility to Outreachy Mentors for now as per Step 2 here https://www.mediawiki.org/wiki/Outreachy/Mentors#_Before_the_program.

You can also start adding microtasks to this project's description, and as and when you are ready, you may upload the project proposal on the Outreachy website.

Vidhi-Mody renamed this task from QTE: Evaluate Microsoft Playwright as our playwright for our browser automation. to QTE: Evaluate Microsoft Playwright as replacement for our browser automation..Sep 18 2020, 10:05 AM
zeljkofilipin renamed this task from QTE: Evaluate Microsoft Playwright as replacement for our browser automation. to QTE: Evaluate Microsoft Playwright as replacement for our browser automation.Sep 21 2020, 2:29 PM
Soham renamed this task from QTE: Evaluate Microsoft Playwright as replacement for our browser automation to Evaluate Microsoft Playwright as replacement for our browser automation.Sep 23 2020, 5:49 AM
srishakatux changed the visibility from "Outreachy Mentors (Project)" to "Public (No Login Required)".Oct 7 2020, 6:20 PM