Page MenuHomePhabricator

Browser tests video capture is shared between tests
Open, Needs TriagePublic

Description

I have enable wdio test to run concurrently by setting { maxInstances: 4 } in MediaWiki core config.

When crafting a small change to Selenium test https://gerrit.wikimedia.org/r/c/mediawiki/core/+/842391 the specs/recentchanges.js failed, I have looked at the video and it displayed the creation of page with the title prefix BeforeEach-name- which comes from a different suite specs/page.js.

The reason is the the specs are run concurrently over 4 instances, but they all share the same X window display which is from where we capture the video. Hence the video displays over tests output which is really confusing.

I don't quite know how to fix it since there are a few bits involved:

  • Quibble spawns Xvfb with a single display and single screen. That is passed as the DISPLAY environment variable when spawning chromedriver which then reuse that when spawning the chromium process
  • chromium does support starting on a specific display and screen (example: `chromium --display=:1.1
  • Quibble only spawns a single display with a single screen

Ideally we would have webdriver.io to spawn a Xvfb with as many screen as the specified maxInstances, however:

  • Xvfb is spawned by Quibble (and that should probably be upstreamed to wdio-mediawiki)
  • I could not find how to specific the DISPLAY environment variable in a Selenium capability. Maybe something is doable via the onWorkerStart hook or via goog:chromeOptions.

This is part of T226869: Run browser tests in parallel

Related Objects

StatusSubtypeAssignedTask
ResolvedNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
StalledNone
OpenNone
Resolvedzeljkofilipin
ResolvedRickiJay-WMDE
ResolvedRickiJay-WMDE
OpenNone
Resolvedvaughnwalters
ResolvedEBernhardson
ResolvedEBernhardson
DeclinedNone
Resolvedhashar
DeclinedNone
OpenNone
DeclinedNone
ResolvedOsamaahmed17
ResolvedNone
ResolvedNone
DuplicateBUG REPORTzeljkofilipin
ResolvedOsamaahmed17
Resolvedzeljkofilipin
ResolvedNone
ResolvedNone
ResolvedKrinkle
OpenNone
ResolvedOllie.Shotton_WMDE
ResolvedJakob_WMDE
DeclinedNone
OpenNone
ResolvedOsamaahmed17
Resolvedvaughnwalters
Resolvedzeljkofilipin
Resolvedzeljkofilipin
Resolvedzeljkofilipin
Resolvedzeljkofilipin
Resolvedkostajh
ResolvedOsamaahmed17
ResolvedOsamaahmed17
OpenNone
Resolvedpwangai
ResolvedWMDE-Fisch
ResolvedNone
ResolvedOsamaahmed17
ResolvedGehel
Resolvedpwangai
ResolvedNone
Invalidpwangai
ResolvedNone
InvalidNone
ResolvedOsamaahmed17
Resolvedvaughnwalters
Resolvedpwangai
ResolvedNone
InvalidNone
ResolvedOllie.Shotton_WMDE
ResolvedDreamy_Jazz
OpenNone
OpenNone
OpenNone
DuplicateNone
DeclinedNone
DeclinedNone
Resolvedzeljkofilipin
Resolvedzeljkofilipin
Resolvedzeljkofilipin
ResolvedOsamaahmed17
OpenNone
OpenNone
ResolvedMhmohona
OpenNone
ResolvedAghaSaad04
ResolvedMhmohona
ResolvedLykarungi
Resolvedzeljkofilipin
Resolvedzeljkofilipin
Resolvedzeljkofilipin
Resolvedzeljkofilipin
OpenNone
OpenNone
Resolvedzeljkofilipin
Resolvedzeljkofilipin
OpenNone
ResolvedOsamaahmed17
OpenNone
ResolvedOsamaahmed17
OpenNone
ResolvedReedy
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone

Event Timeline

Change 951587 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@master] wdio-mediawiki: use an Xvfb screen per browser instance

https://gerrit.wikimedia.org/r/951587

That can be implemented with https://webdriver.io/docs/shared-store-service/

In onPrepare (runs in the in main process), I can spin up the Xvfb server and register all the screen numbers in a pool. Then in beforeSession (which runs in the worker thread) retrieve a display and free it up in afterSession.

That requires webdriver.io v8.7.0 (feature is https://github.com/webdriverio/webdriverio/issues/10010 ).

:-(

Change 951587 abandoned by Hashar:

[mediawiki/core@master] wdio-mediawiki: use an Xvfb screen per browser instance

Reason:

The right approach is to use the shared store service and the data pool which allow sharing data between threads. That requires Webdriver.io 8.7.0+

See T344754#9115734

Abandoning this change for now.

https://gerrit.wikimedia.org/r/951587

That requires webdriver.io v8.7.0 (feature is https://github.com/webdriverio/webdriverio/issues/10010 ).

We're planning on resuming work on T324685: Upgrade WebdriverIO to v8 in all repositories now that T256626: Refactor WebdriverIO tests from sync to async mode is almost done. I'll try updating core to v8 soon so we can test your patch.

Note: I have abandoned my patch, but it can be restored later on and get amended to add the bits to use @wdio/shared-store-service :]