Task to track a couple of things I want to try out (and revert/abandon if not compatible and faster):
Test runner:
- Upgrade from Wdio v4 to v5. T213268
- Use native async-await instead of the wdio-sync/Fibers hack. T256626
- Let Wdio use the devtools protocol instead of the webdriver protocol to connect with browsers locally and in CI. This is a new feature in Wdio 5 and should be significantly faster for test execution. It also removes the need for the chromedriver executable.
Test configuration:
- Determine what overhead video recording is having on test execution.
-
If more than a little bit, consider disabling by default and opt-ing in via a "check <something>" command in Gerrit instead (which Quibble could detect via an environment variable like ZUUL_PIPELINE to decide to run something like npm run selenium-video, which we'd create as variant to npm run selenium).- Impact measured at https://gerrit.wikimedia.org/r/541055. Before and after is within normal variance. No obvious win.
Test execution:
- Figure out why the API requests we run with MWBot from Wdio tests are slow (5 seconds is not unusual). This makes no sense to me because MediaWiki in CI has been hyper-optimised, as demonstrated by Fresnel and QUnit tasks running MediaWiki with super low latencies (almost everything in pure RAM, full MW page loads in Fresnel with Chrome take under 150ms, incl load.php etc.).
- Talked with @kostajh and @Reedy about this, they suggested that maybe 1) the login session keeps getting lost between test cases and 2) login might be very slow because of too strong password crypto requirements by default which are intentionally CPU intensive.