I have enable wdio test to run concurrently by setting { maxInstances: 4 } in MediaWiki core config.
When crafting a small change to Selenium test https://gerrit.wikimedia.org/r/c/mediawiki/core/+/842391 the specs/recentchanges.js failed, I have looked at the video and it displayed the creation of page with the title prefix BeforeEach-name- which comes from a different suite specs/page.js.
The reason is the the specs are run concurrently over 4 instances, but they all share the same X window display which is from where we capture the video. Hence the video displays over tests output which is really confusing.
I don't quite know how to fix it since there are a few bits involved:
- Quibble spawns Xvfb with a single display and single screen. That is passed as the DISPLAY environment variable when spawning chromedriver which then reuse that when spawning the chromium process
- chromium does support starting on a specific display and screen (example: `chromium --display=:1.1
- Quibble only spawns a single display with a single screen
Ideally we would have webdriver.io to spawn a Xvfb with as many screen as the specified maxInstances, however:
- Xvfb is spawned by Quibble (and that should probably be upstreamed to wdio-mediawiki)
- I could not find how to specific the DISPLAY environment variable in a Selenium capability. Maybe something is doable via the onWorkerStart hook or via goog:chromeOptions.
This is part of T226869: Run browser tests in parallel