Page MenuHomePhabricator

"Stress testing" the e2e tests for flakiness
Closed, ResolvedPublic

Description

"Stress testing" refers to running e2e tests repeatedly to identify any potential flakiness or instability. Flakiness refers to tests that sometimes pass and sometimes fail without any changes to the code or test cases. It is essential to ensure that E2E tests are reliable and provide consistent results.

The process typically involves running the E2E tests multiple times. By counting the failures, you can assess the stability of the tests and identify any patterns of flakiness. This approach helps in understanding whether the failures are due to legitimate issues or if there are any inconsistencies causing failures.

Event Timeline

Change 934455 had a related patch set uploaded (by Nik.xyz.in; author: Nik.xyz.in):

[mediawiki/extensions/WikiLambda@master] e2e: Add command to run a spec file multiple times

https://gerrit.wikimedia.org/r/934455

Change 934455 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] e2e: Add command to run specs file multiple times

https://gerrit.wikimedia.org/r/934455