Overview

As you are aware, currently Wikimedia supports a large number of repositories as well as extensions. In order to ensure good code practices across all these repos, we perform an extensive amount of testing.

Currently, we use WebdriverIO as our browser automation framework for implementing end to end tests across multiple repos. A list of all such repos running WebdriverIO tests can be found here.

However with the recent increase in awareness about end to end testing, a number of equally competitive non-selenium solutions have been introduced. Two such include Puppeteer and Cypress. As a part of this years Google Summer of Code program, an extensive evaluation of Puppeteer was performed. Unfortunately owing to Cypress not being able to run properly on CI. We had to drop evaluating Cypress.

One of the contenders being considered at the time was Microsoft Playwright. Seeing the recent developments in the framework and the rapid surge in popularity. We would like to evaluate it against WebdriverIO as a potential replacement for our browser automation framework.

Here is a tentative list of what this project aims to achieve:

Evaluate Playwright as an end-to-end testing framework

• Implement existing tests across moderately sized repo's in Microsoft Playwright
• Introduce video recording and screenshots of tests
• Benchmark Playwright against Puppeteer (using previous implementation) and WebdriverIO.
• Re-run a short evaluation on Cypress (Depending upon whether the issues with Cypress are resolved)

Skills required:

• Node.js (Javascript)
• Previous knowledge on automation frameworks and end to end testing, while not necessary would surely be a plus.

Mentors:
@zeljkofilipin @Vidhi-Mody @Soham

Microtasks

See T265011