This epic collects the work to speed up the quibble-for-mediawiki-core-browser-tests-only-vendor-mysql-php83 job.
Here are the metrics for the job (cache hits and failures filtered out):
| Month | count | min | max | average | stddev | median | p75 | p90 | p95 | p99 |
| January | 363 | 01:50 | 05:03 | 03:16 | 00:47 | 03:12 | 03:36 | 04:02 | 04:15 | 04:42 |
| February | 419 | 00:28 | 14:14 | 03:12 | 01:43 | 03:17 | 03:47 | 04:14 | 04:31 | 05:07 |
| March | 1756 | 01:23 | 54:45 | 02:35 | 02:04 | 02:18 | 02:43 | 03:16 | 03:38 | 04:34 |
| April | 1511 | 00:54 | 08:36 | 01:32 | 00:47 | 01:23 | 01:43 | 02:14 | 02:37 | 03:46 |
| May | 1604 | 01:03 | 15:06 | 01:58 | 01:50 | 01:34 | 02:03 | 02:57 | 04:11 | 08:24 |
| June | 1758 | 01:07 | 13:06 | 02:15 | 01:46 | 01:49 | 02:27 | 03:37 | 04:48 | 07:31 |
We've been rolling out changes to reduce CI overhead and increased webdriverIO speed.
- Make sure xvfb/videos work for multiple instances (T344754 - https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1147173)
- Enable --headless by default and disable video T411784
- Only take screenshots on failures T416704
- Increase number of maxInstances for core to 6 (max 6 tests will run in parallel) T414904
- Update documentation for how to record videos and take screenshots in CI T417131
- Write a changelog and release notes and create a new package with our new default settings for wdio-mediawiki T417132
- Change order of how the tests runs in core. Start with the slowest to make sure the slowest tests define the total runtime T417640
- Split test suites in core to make them faster and better use parallelization T417535
- Update extensions/examples with the new package
Fix performance issues in our tests/better use of parallelization:
- Wrong usage of not.toExist patterns make tests slow T419803
- Simplify admin login to make tests faster (core) T420081
- Make webdriverIO tests in CI run amount of parallel tests depending on number of CPUs T420185
And as the last step: