Change 951587 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@master] wdio-mediawiki: use an Xvfb screen per browser instance

https://gerrit.wikimedia.org/r/951587

That can be implemented with https://webdriver.io/docs/shared-store-service/

In onPrepare (runs in the in main process), I can spin up the Xvfb server and register all the screen numbers in a pool. Then in beforeSession (which runs in the worker thread) retrieve a display and free it up in afterSession.

That requires webdriver.io v8.7.0 (feature is https://github.com/webdriverio/webdriverio/issues/10010 ).

:-(

Change 951587 abandoned by Hashar:

[mediawiki/core@master] wdio-mediawiki: use an Xvfb screen per browser instance

Reason:

The right approach is to use the shared store service and the data pool which allow sharing data between threads. That requires Webdriver.io 8.7.0+

See T344754#9115734

Abandoning this change for now.

https://gerrit.wikimedia.org/r/951587

In T344754#9115734, @hashar wrote:

That requires webdriver.io v8.7.0 (feature is https://github.com/webdriverio/webdriverio/issues/10010 ).

We're planning on resuming work on T324685: Upgrade WebdriverIO to v8 in all repositories now that T256626: Refactor WebdriverIO tests from sync to async mode is almost done. I'll try updating core to v8 soon so we can test your patch.

Note: I have abandoned my patch, but it can be restored later on and get amended to add the bits to use @wdio/shared-store-service :]

Summary from two years ago

The implementation I did at https://gerrit.wikimedia.org/r/c/mediawiki/core/+/951587 was expecting a way to share the screen assignments so that each parallelized session would pick a screen number from a shared pool. But the const is not shared, for that we would need https://webdriver.io/docs/shared-store-service/ which in turn requires webdriver.io v8.7.0. This task is thus blocked on T324685.

We can either:

• resurrect the patch and move forward in preparation for the v8 upgrade
• find an alternate way to share the screen numbers? Maybe file based with locks? Maybe prespawn Xvfb instances and uses the session numbers if any to lock to a given instance
• find another way to record videos?

In T390097#10683270, @Daimona wrote:

From a quick read of https://ffmpeg.org/ffmpeg-devices.html#x11grab, it seems that we could pass a window_id, which can be obtained via xwininfo. I found some examples of that, e.g. https://alexjgriffith.com/.

With that in place, I'm wondering if we might then also drop the recorded window size and do T314115.

I don't know if that would work though. Also, the same page says:

If the window is later enlarged, the new area is not recorded. Video ends when the window is closed, unmapped (i.e., iconified) or shrunk beyond the video size (which defaults to the initial window size).

I guess that might be fine with the normal setup, but it wouldn't work as well if someone calls browser.setWindowSize() as suggested in T314115.

@zeljkofilipin this task is indeed blocked on upgrading to WebdriverIO v8 see my summary above: T344754#10683362 and I don't think there is much we can do with WebdriverIO v7.

Per above ^

I'm thinking that all workers have their own id (1, 2, 3 etc) that is accessible when we move to wdio9 through an environment variable https://webdriver.io/docs/api/environment/#wdio_worker_id and then we could probably start/stop XVFB based on that and we do not need to use the shared store service. Another way could be to move the video functionality into a custom reporter https://webdriver.io/docs/customreporter/ that also have runner start/end hooks that have access to the worker id. I'll have a look when we upgraded to 9.

Change #1147173 had a related patch set uploaded (by Hashar; author: Phedenskog):

[mediawiki/core@master] selenium: Start one xvfb per maxInstance (NodeJS instance)

https://gerrit.wikimedia.org/r/1147173

Change #1239320 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/core@master] WIP selenium: maxInstances: 2

https://gerrit.wikimedia.org/r/1239320

Change #1147173 merged by jenkins-bot:

[mediawiki/core@master] selenium: Start one xvfb per maxInstance (NodeJS instance)

https://gerrit.wikimedia.org/r/1147173

Change #1239320 abandoned by Zfilipin:

[mediawiki/core@master] WIP selenium: maxInstances: 2

https://gerrit.wikimedia.org/r/1239320

Thanks for doing this! Could we get a wdio-mw release with the fix? I'm eager to parallelize selenium tests in CampaignEvents, but the video artefacts make new failures hard to debug.

@Daimona we will be releasing new version of the npm package and updating gated extensions and skins soon, probably this week.

@Daimona I think this will be released on Wednesday this week. One thing though we are focusing on making the feedback loop faster so we are disabling videos by default (but you will be able to enable them and see the result for multiple tests if you need). I prepared most of that in the changelog https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1239870

Thanks both!

In T344754#11621606, @Peter wrote:

One thing though we are focusing on making the feedback loop faster so we are disabling videos by default (but you will be able to enable them and see the result for multiple tests if you need).

This sounds a bit problematic for CI, because selenium failures are harder to reproduce even between different CI runs, let alone locally. Videos are basically the only way to more or less see why something isn't working in CI. I imagine that turning video off would come with better performance, but I would also want to be able to somewhat-easily debug failures. As a compromise, could it be made so that video recording is turned on for spec file retries only? (Also, happy to move this conversation elsewhere if there's a better task)

@Daimona let me create a task for that. We've been discussing that as feature but we said we wanted to see how this turns out first. I'll add you to that task and we can discuss it there.

The things is that this bug has been here for so long, so if you need to verify the video, you need change maxInstances to 1, so that the video on CI really works. And I'm think since has been a bug for such a long time, maybe its ok to make it configurable.

There were some issues with recording videos in fresh-node so I didn't do the release today, I want to have that work first, so probably tomorrow.