This epic collects the work to speed up the quibble-for-mediawiki-core-browser-tests-only-vendor-mysql-php83 job.
[[ https://releng-data.wmcloud.org/jobs?sql=SELECT%0D%0A++%22%3Ca+href%3D%2F-%2Fdashboards%2Fjenkins%3Frepo_name%3D%22+%7C%7C+repo+%7C%7C+%22%26date_start%3D%22+%7C%7C+%3Adate_start+%7C%7C+%22%3E%22+%7C%7C+repo+%7C%7C+%22%3C%2Fa%3E%22+as+%27Repo+name%27%2C%0D%0A++count%28*%29+as+count%2C%0D%0A++printf%28%0D%0A++++%27%2502d%3A%2502d%27%2C%0D%0A++++cast%28min%28time+%2F+1000%29+%2F+60+as+integer%29%2C%0D%0A++++cast%28min%28time+%2F+1000%29+%25+60+as+integer%29%0D%0A++%29+as+min%2C%0D%0A++printf%28%0D%0A++++%27%2502d%3A%2502d%27%2C%0D%0A++++cast%28max%28time+%2F+1000%29+%2F+60+as+integer%29%2C%0D%0A++++cast%28max%28time+%2F+1000%29+%25+60+as+integer%29%0D%0A++%29+as+max%2C%0D%0A++printf%28%0D%0A++++%27%2502d%3A%2502d%27%2C%0D%0A++++cast%28avg%28time+%2F+1000%29+%2F+60+as+integer%29%2C%0D%0A++++cast%28avg%28time+%2F+1000%29+%25+60+as+integer%29%0D%0A++%29+as+average%2C%0D%0A++printf%28%0D%0A++++%27%2502d%3A%2502d%27%2C%0D%0A++++cast%28%0D%0A++++++%28%0D%0A++++++++SQRT%28%0D%0A++++++++++AVG%28time+%2F+1000+*+time+%2F+1000%29+-+AVG%28time+%2F+1000%29+*+AVG%28time+%2F+1000%29%0D%0A++++++++%29%0D%0A++++++%29+%2F+60+as+integer%0D%0A++++%29%2C%0D%0A++++cast%28%0D%0A++++++%28%0D%0A++++++++SQRT%28%0D%0A++++++++++AVG%28time+%2F+1000+*+time%29+-+AVG%28time+%2F+1000%29+*+AVG%28time+%2F+1000%29%0D%0A++++++++%29%0D%0A++++++%29+%25+60+as+integer%0D%0A++++%29%0D%0A++%29+as+stddev%2C%0D%0A++printf%28%0D%0A++++%27%2502d%3A%2502d%27%2C%0D%0A++++cast%28median%28time+%2F+1000%29+%2F+60+as+integer%29%2C%0D%0A++++cast%28median%28time+%2F+1000%29+%25+60+as+integer%29%0D%0A++%29+as+median%2C%0D%0A++printf%28%0D%0A++++%27%2502d%3A%2502d%27%2C%0D%0A++++cast%28percentile%28time+%2F+1000%2C+75%29+%2F+60+as+integer%29%2C%0D%0A++++cast%28percentile%28time+%2F+1000%2C+75%29+%25+60+as+integer%29%0D%0A++%29+as+p75%2C%0D%0A++printf%28%0D%0A++++%27%2502d%3A%2502d%27%2C%0D%0A++++cast%28percentile%28time+%2F+1000%2C+90%29+%2F+60+as+integer%29%2C%0D%0A++++cast%28percentile%28time+%2F+1000%2C+90%29+%25+60+as+integer%29%0D%0A++%29+as+p90%2C%0D%0A++printf%28%0D%0A++++%27%2502d%3A%2502d%27%2C%0D%0A++++cast%28percentile%28time+%2F+1000%2C+95%29+%2F+60+as+integer%29%2C%0D%0A++++cast%28percentile%28time+%2F+1000%2C+95%29+%25+60+as+integer%29%0D%0A++%29+as+p95%2C%0D%0A++printf%28%0D%0A++++%27%2502d%3A%2502d%27%2C%0D%0A++++cast%28percentile%28time+%2F+1000%2C+99%29+%2F+60+as+integer%29%2C%0D%0A++++cast%28percentile%28time+%2F+1000%2C+99%29+%25+60+as+integer%29%0D%0A++%29+as+p99%0D%0AFROM%0D%0A++builds%0D%0A++JOIN+jobs+on+jobs.id+%3D+builds.job_id%0D%0AWHERE%0D%0A++time+IS+NOT+NULL%0D%0A++and+time+%3E+0.0%0D%0A++AND+datetime%28timestamp+%2F+1000%2C+%27unixepoch%27%29+%3E%3D+date%28%3Adate_start%29%0D%0A++AND+datetime%28timestamp+%2F+1000%2C+%27unixepoch%27%29+%3C%3D+date%28%3Adate_end%29%0D%0A++AND+result+%3D+%27SUCCESS%27%0D%0A++AND+success_cache_hit+%3D+false%0D%0A++AND+name+%3D+%3Ajob_name%0D%0A++AND+repo+%3D+%3Arepo_name%0D%0Agroup+by%0D%0A++repo%0D%0Ahaving%0D%0A++count%28*%29+%3E+10%0D%0Aorder+by%0D%0A++avg%28time%29+desc%2C%0D%0A++count&date_start=2026-05-01&date_end=2026-06-01&job_name=quibble-for-mediawiki-core-browser-tests-only-vendor-mysql-php83&repo_name=mediawiki%2Fcore | Here ]] are the metrics for the job (cache hits and failures filtered out):
|Month |count |min |max |average |stddev| median| p75| p90| p95| p99
|January| 363 |01:50 |05:03 |03:16| 00:47| 03:12| 03:36| 04:02 |04:15| 04:42
|February|419| 00:28 |14:14 |03:12 |01:43 |03:17 |03:47| 04:14 |04:31| 05:07
|March| 1756| 01:23 |54:45| 02:35 |02:04| 02:18| 02:43| 03:16| 03:38 |04:34
|April|1511 |00:54| 08:36| 01:32| 00:47| 01:23 |01:43 |02:14 |02:37| 03:46
|May|1604| 01:03 |15:06 |01:58 |01:50| 01:34| 02:03 |02:57| 04:11| 08:24
We've been rolling out changes to reduce CI overhead and increased webdriverIO speed.
[x] Make sure xvfb/videos work for multiple instances (T344754 - https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1147173)
[x] Enable --headless by default and disable video T411784
[x] Only take screenshots on failures T416704
[x] Increase number of maxInstances for core to 6 (max 6 tests will run in parallel) T414904
[x] Update documentation for how to record videos and take screenshots in CI T417131
[x] Write a changelog and release notes and create a new package with our new default settings for wdio-mediawiki T417132
[x] Change order of how the tests runs in core. Start with the slowest to make sure the slowest tests define the total runtime T417640
[x] Split test suites in core to make them faster and better use parallelization T417535
[x] Update extensions/examples with the new package
Fix performance issues in our tests/better use of parallelization:
[x] Wrong usage of not.toExist patterns make tests slow T419803
[x] Simplify admin login to make tests faster (core) T420081
[x] Make webdriverIO tests in CI run amount of parallel tests depending on number of CPUs T420185
And as the last step:
[ ] Measure and collect metrics and try to quantify the difference T417654
[ ] Write a blog post about the job done T417532