Page MenuHomePhabricator

Implement changes for reduced CI overhead/increased speed for webdriverIO tests
Open, Needs TriagePublic

Description

Let's rollout the changes we plan for reducing CI overhead and increased webdriverIO speed. Some work has already been done (see parent task) and in this epic we plan to do the last parts:

  • Make sure xvfb/videos work for multiple instances (T344754 - https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1147173)
  • Enable --headless by default and disable video T411784
  • Only take screenshots on failures T416704
  • Increase number of maxInstances for core to 6 (max 6 tests will run in parallel) T414904
  • Update documentation for how to record videos and take screenshots in CI T417131
  • Write a changelog and release notes and create a new package with our new default settings for wdio-mediawiki T417132
  • Change order of how the tests runs in core. Start with the slowest to make sure the slowest tests define the total runtime T417640
  • Split test suites in core to make them faster and better use parallelization T417535
  • Update extensions/examples with the new package

Update all gated extensions to use the new wdio package and make sure local extension wdio config is set for max parallel tests:

  • extensions/AbuseFilter
  • extensions/CampaignEvents
  • extensions/Echo
  • extensions/Math
  • extensions/PageTriage
  • extensions/ProofreadPage
  • extensions/TemplateData
  • extensions/VisualEditor
  • extensions/Wikibase
  • skins/MinervaNeue

And as the last step:

  • Measure and collect metrics and try to quantify the difference T417654
  • Write a blog post about the job done T417532

Rough plan:

  • 20 Feb the new wdio-mediawiki package released
  • 6 March all extensions upgraded and rolled out

After that we can hopefully see the changes in the metrics.

Related Objects

Event Timeline

Update all gated extensions to use the new wdio package and make sure local extension wdio config is set for max parallel tests:

That is/was T226869: Run browser tests in parallel and we had multiple extensions moved to use parallelism:

$ git grep --recurse-submodules maxInstances: '*/tests/selenium/wdio.conf.js'
AbuseFilter/tests/selenium/wdio.conf.js:        maxInstances: 1
AchievementBadges/tests/selenium/wdio.conf.js:  maxInstances: 4
AdvancedSearch/tests/selenium/wdio.conf.js:     maxInstances: 4
CirrusSearch/tests/selenium/wdio.conf.js:       maxInstances: 1,
CirrusSearch/tests/selenium/wdio.conf.js:               maxInstances: 1,
CodeMirror/tests/selenium/wdio.conf.js: maxInstances: 2
CommunityRequests/tests/selenium/wdio.conf.js:  maxInstances: 4
Echo/tests/selenium/wdio.conf.js:       maxInstances: 4
PageTriage/tests/selenium/wdio.conf.js: maxInstances: 4,
Popups/tests/selenium/wdio.conf.js:     maxInstances: 4
ReportIncident/tests/selenium/wdio.conf.js:     maxInstances: 4,
Sanctions/tests/selenium/wdio.conf.js:  maxInstances: 4
TwoColConflict/tests/selenium/wdio.conf.js:     maxInstances: 4
VisualEditor/tests/selenium/wdio.conf.js:       maxInstances: 4,
Wikibase/client/data-bridge/tests/selenium/wdio.conf.js:        maxInstances: 1,
Wikibase/client/data-bridge/tests/selenium/wdio.conf.js:                maxInstances: 1,

And for skins:

MinervaNeue/tests/selenium/wdio.conf.js:        maxInstances: 4

But as you can see some did not get moved because there are tests racing with each others (eg T402927, T345149). While investigating some of those issues I found out each wdio instance share the same Xvfb screen and the browser windows from each tests overlap each other making it hard/impossible to debug. I filed it as T344754, I have it a wall at some point and stopped pursuing further. I guess that can of blocked raising maxInstances on other repos.

I've fixed the XVFB issue in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1147173 it's good if that gets merged before we start. With the changes we do, we will disable videos by default and make it easier to enable videos when needed. And then if you run multiple instances, it will be great that the videos actually works.

Adding QS-Test-Automation tag for visibility into this task on our board. Thank you Test Platform team!

Change #1240403 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[mediawiki/extensions/AbuseFilter@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240403

Change #1240404 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[mediawiki/extensions/CampaignEvents@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240404

Change #1240407 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[mediawiki/extensions/Echo@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240407

Change #1240408 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[mediawiki/extensions/Math@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240408

Change #1240409 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[mediawiki/extensions/PageTriage@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240409

Change #1240468 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[mediawiki/extensions/TemplateData@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240468

Change #1240470 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[mediawiki/extensions/VisualEditor@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240470

Change #1240473 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[mediawiki/skins/MinervaNeue@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240473

Change #1240407 merged by jenkins-bot:

[mediawiki/extensions/Echo@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240407

Change #1240404 merged by jenkins-bot:

[mediawiki/extensions/CampaignEvents@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240404

Change #1240468 merged by jenkins-bot:

[mediawiki/extensions/TemplateData@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240468

Change #1240473 merged by jenkins-bot:

[mediawiki/skins/MinervaNeue@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240473

Change #1240625 had a related patch set uploaded (by Zfilipin; author: Phedenskog):

[mediawiki/extensions/ProofreadPage@master] selenium: Upgrade to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240625

Change #1240403 merged by jenkins-bot:

[mediawiki/extensions/AbuseFilter@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240403

Change #1240408 merged by jenkins-bot:

[mediawiki/extensions/Math@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240408

Change #1240409 merged by jenkins-bot:

[mediawiki/extensions/PageTriage@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240409

Change #1240625 merged by jenkins-bot:

[mediawiki/extensions/ProofreadPage@master] selenium: Upgrade to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240625

Peter updated the task description. (Show Details)

Change #1240470 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] selenium: Update to wdio-mediawiki 6.3.0

https://gerrit.wikimedia.org/r/1240470