This is the current documentation about the different test run modes in core’s tests/selenium/README.md:
- Headless. It's the default. You will not see the browser while tests are running because it's running in a headless mode. This mode should run fine on all supported platforms.
- Headless recording. Set DISPLAY environment variable to a value that starts with colon (:) and video of each test will be recorded. Browser will run headless. Recording videos works only on Linux.
- Visible. If you want to see the browser, set DISPLAY environment variable to any value that does not start with colon. This mode will not work in a headless environment like MediaWiki-Vagrant.
I found this very confusing, since as far as I can tell it doesn’t match what the tests actually do.
- It’s not very helpful to describe headless as “the default”. Headless mode is used if the $DISPLAY variable is unset or empty, but on a desktop system, that is not the default – the user would have to unset DISPLAY or export DISPLAY='' manually.
- As far as I can tell, it’s not true that DISPLAY=:0, for example, would use headless mode. wdio.conf.js only passes --headless to Chrome if process.env.DISPLAY is falsy, it doesn’t check whether the value begins with a colon or not.
- It’s also not clear to me how “visible” mode is supposed to work – which display will it use then? – but I can’t try it out anyways for unrelated reasons.
The whole paragraph seems to assume a MediaWiki-Vagrant setup, whereas the rest of the README.md is more careful to mention the changes necessary to support other setups (MW_SERVER, MW_SCRIPT_PATH, etc.). The upshot of it seems to be that, in fact, the “default” mode on system (no containerization of any kind, just a plain MediaWiki install on a desktop Linux system) would work perfectly well (DISPLAY=:0, not headless, recording), except for the fact that I use Wayland, whereas ffmpeg -f x11grab requires X11 (which should be documented alongside the Linux requirement, I think).