Page MenuHomePhabricator

Browser tests video capture is shared between tests
Open, Needs TriagePublic5 Estimated Story Points

Description

I have enable wdio test to run concurrently by setting { maxInstances: 4 } in MediaWiki core config.

When crafting a small change to Selenium test https://gerrit.wikimedia.org/r/c/mediawiki/core/+/842391 the specs/recentchanges.js failed, I have looked at the video and it displayed the creation of page with the title prefix BeforeEach-name- which comes from a different suite specs/page.js.

The reason is the the specs are run concurrently over 4 instances, but they all share the same X window display which is from where we capture the video. Hence the video displays over tests output which is really confusing.

I don't quite know how to fix it since there are a few bits involved:

  • Quibble spawns Xvfb with a single display and single screen. That is passed as the DISPLAY environment variable when spawning chromedriver which then reuse that when spawning the chromium process
  • chromium does support starting on a specific display and screen (example: `chromium --display=:1.1
  • Quibble only spawns a single display with a single screen

Ideally we would have webdriver.io to spawn a Xvfb with as many screen as the specified maxInstances, however:

  • Xvfb is spawned by Quibble (and that should probably be upstreamed to wdio-mediawiki)
  • I could not find how to specific the DISPLAY environment variable in a Selenium capability. Maybe something is doable via the onWorkerStart hook or via goog:chromeOptions.

This is part of T226869: Run browser tests in parallel

Event Timeline

Change 951587 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@master] wdio-mediawiki: use an Xvfb screen per browser instance

https://gerrit.wikimedia.org/r/951587

That can be implemented with https://webdriver.io/docs/shared-store-service/

In onPrepare (runs in the in main process), I can spin up the Xvfb server and register all the screen numbers in a pool. Then in beforeSession (which runs in the worker thread) retrieve a display and free it up in afterSession.

That requires webdriver.io v8.7.0 (feature is https://github.com/webdriverio/webdriverio/issues/10010 ).

:-(

Change 951587 abandoned by Hashar:

[mediawiki/core@master] wdio-mediawiki: use an Xvfb screen per browser instance

Reason:

The right approach is to use the shared store service and the data pool which allow sharing data between threads. That requires Webdriver.io 8.7.0+

See T344754#9115734

Abandoning this change for now.

https://gerrit.wikimedia.org/r/951587

That requires webdriver.io v8.7.0 (feature is https://github.com/webdriverio/webdriverio/issues/10010 ).

We're planning on resuming work on T324685: Upgrade WebdriverIO to v8 in all repositories now that T256626: Refactor WebdriverIO tests from sync to async mode is almost done. I'll try updating core to v8 soon so we can test your patch.

Note: I have abandoned my patch, but it can be restored later on and get amended to add the bits to use @wdio/shared-store-service :]

Summary from two years ago

The implementation I did at https://gerrit.wikimedia.org/r/c/mediawiki/core/+/951587 was expecting a way to share the screen assignments so that each parallelized session would pick a screen number from a shared pool. But the const is not shared, for that we would need https://webdriver.io/docs/shared-store-service/ which in turn requires webdriver.io v8.7.0. This task is thus blocked on T324685.

We can either:

  • resurrect the patch and move forward in preparation for the v8 upgrade
  • find an alternate way to share the screen numbers? Maybe file based with locks? Maybe prespawn Xvfb instances and uses the session numbers if any to lock to a given instance
  • find another way to record videos?

From a quick read of https://ffmpeg.org/ffmpeg-devices.html#x11grab, it seems that we could pass a window_id, which can be obtained via xwininfo. I found some examples of that, e.g. https://alexjgriffith.com/.

With that in place, I'm wondering if we might then also drop the recorded window size and do T314115.

I don't know if that would work though. Also, the same page says:

If the window is later enlarged, the new area is not recorded. Video ends when the window is closed, unmapped (i.e., iconified) or shrunk beyond the video size (which defaults to the initial window size).

I guess that might be fine with the normal setup, but it wouldn't work as well if someone calls browser.setWindowSize() as suggested in T314115.

@zeljkofilipin this task is indeed blocked on upgrading to WebdriverIO v8 see my summary above: T344754#10683362 and I don't think there is much we can do with WebdriverIO v7.

I'm thinking that all workers have their own id (1, 2, 3 etc) that is accessible when we move to wdio9 through an environment variable https://webdriver.io/docs/api/environment/#wdio_worker_id and then we could probably start/stop XVFB based on that and we do not need to use the shared store service. Another way could be to move the video functionality into a custom reporter https://webdriver.io/docs/customreporter/ that also have runner start/end hooks that have access to the worker id. I'll have a look when we upgraded to 9.

Slack-bot changed the task status from Open to In Progress.May 16 2025, 4:47 PM
Slack-bot closed this task as Resolved.

Thank you Slack-bot.

Change #1147173 had a related patch set uploaded (by Hashar; author: Phedenskog):

[mediawiki/core@master] selenium: Start one xvfb per maxInstance (NodeJS instance)

https://gerrit.wikimedia.org/r/1147173

Peter removed Peter as the assignee of this task.Aug 12 2025, 10:37 AM

The patch been ready for a while.

Peter set the point value for this task to 5.