As you are aware, currently Wikimedia supports a large number of repositories as well as extensions. In order to ensure good code practices across all these repos, we perform an extensive amount of testing.
Currently, we use [WebdriverIO](https://webdriver.io) as our browser automation framework for implementing end to end tests across multiple repos. A list of all such repos running WebdriverIO tests can be found [[https://www.mediawiki.org/wiki/Selenium/Reference/Example_Code | here]].
However with the recent increase in awareness about end to end testing, a number of equally competitive non-selenium solutions have been introduced. Two such include [[https://pptr.dev | Puppeteer ]] and [[https://cypress.io | Cypress ]]. As a part of this years Google Summer of Code program, [[ https://www.sohamp.dev/blog/2020-08-24-wmf-gsoc2020/ | an extensive evaluation of Puppeteer]] was performed. Unfortunately owing to Cypress not being able to run properly on CI. We had to drop evaluating Cypress.
One of the contenders being considered at the time was [[ https://github.com/microsoft/playwright | Microsoft Playwright ]]. Seeing the recent developments in the framework and the rapid surge in popularity. We would like to evaluate it against WebdriverIO as a potential replacement for our browser automation framework.
Here is a tentative list of what this project aims to achieve:
### Evaluate Playwright as an end-to-end testing framework
- Implement existing tests across moderately sized repo's in Microsoft Playwright
- Introduce video recording and screenshots of tests
- Benchmark Playwright against Puppeteer (using previous implementation) and WebdriverIO.
- Re-run a short evaluation on Cypress (Depending upon whether the issues with Cypress are resolved)
- Previous knowledge on automation frameworks and end to end testing, while not necessary would surely be a plus.
@zeljkofilipin @Vidhi-Mody @Soham