Let's rollout the changes we plan for reducing CI overhead and increased webdriverIO speed. Some work has already been done (see parent task) and in this epic we plan to do the last parts:
- Make sure xvfb/videos work for multiple instances (T344754 - https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1147173)
- Enable --headless by default and disable video T411784
- Only take screenshots on failures T416704
- Increase number of maxInstances for core to 6 (max 6 tests will run in parallel) T414904
- Update documentation for how to record videos and take screenshots in CI T417131
- Write a changelog and release notes and create a new package with our new default settings for wdio-mediawiki T417132
- Change order of how the tests runs in core. Start with the slowest to make sure the slowest tests define the total runtime T417640
- Split test suites in core to make them faster and better use parallelization T417535
- Update extensions/examples with the new package
Update all gated extensions to use the new wdio package and make sure local extension wdio config is set for max parallel tests:
- extensions/AbuseFilter
- extensions/CampaignEvents
- extensions/Echo
- extensions/Math
- extensions/PageTriage
- extensions/ProofreadPage
- extensions/TemplateData
- extensions/VisualEditor
- extensions/Wikibase
- skins/MinervaNeue
And as the last step:
- Measure and collect metrics and try to quantify the difference T417654
- Write a blog post about the job done T417532
Rough plan:
- 20 Feb the new wdio-mediawiki package released
- 6 March all extensions upgraded and rolled out
After that we can hopefully see the changes in the metrics.