# Status
1. The data below was collected by running Puppeteer and WebdriverIO tests for mediawiki/core and extensions/EntitySchema.
2. Each repository has 15 test runs spanning over 2 separate machines totalling to a 30 tests overall.
3. For measuring reliability, % tests failures for each run were computed and averaged over 30 test runs.
**mediawiki/core**
| tool | reliability | speed (s) |
| Puppeteer | 100% | 214 |
| WebdriverIO | 100% | 316 |
| WebdriverIO Async | 100% | 322 |
**extensions/EntitySchema**
| tool | reliability | speed (s) |
| Puppeteer | 100% | 342 |
| WebdriverIO | 100% | 517 |
4. Additionally, a short PoC to test for behaviour of the frameworks when running tests in parallel was also evaluated. This was done for an additional of 10 tests with 2 workers. Totalling to 20 tests per framework.
**mediawiki/core**
| tool | reliability | speed |
| Puppeteer | 100% | 193 |
| WebdriverIO | 100% | 297 |
| WebdriverIO Async | 82% | 311 |
---
# Links
- https://www.sohamp.dev/blog/2020-08-24-wmf-gsoc2020/