Background
The Pywikibot test suite currently takes several minutes to complete in some environments. Its runtime includes live wiki requests, authenticated tests, subprocess-based script checks, deliberately exhaustive tests, and ordinary offline unit tests.
There are some opportunities to shorten the feedback loop while preserving meaningful coverage:
- Run independent offline tests in parallel.
- Remove unnecessary wall-clock sleeps.
- Separate offline, recorded, live, login, and write tests more clearly.
- Make cached API tests deterministic and strictly offline.
- Reduce repeated subprocess startup and excessive live API requests.
- Avoid running coverage and the complete live suite across every CI configuration.
Objective
Improve the performance, determinism, and organization of the test suite without reducing meaningful test coverage or increasing load on public services.