TLDR: Docker defaults to 64MB of shared memory, which is insanely low.
For example the first run here: https://gerrit.wikimedia.org/r/#/c/VisualEditor/VisualEditor/+/558531/ (https://integration.wikimedia.org/ci/job/visualeditor-node10-browser-docker/1196/console)
I've seen this at least 3 times in the past week.
14:32:16 Running "karma:firefox" (karma) task 14:32:16 17 12 2019 14:32:16.097:WARN [watcher]: All files matched by "/src/node_modules/qunit/qunit/qunit.js" were excluded or matched by prior matchers. 14:32:16 17 12 2019 14:32:16.179:INFO [karma-server]: Karma v3.1.3 server started at http://0.0.0.0:9876/ 14:32:16 17 12 2019 14:32:16.179:INFO [launcher]: Launching browsers FirefoxHeadless with concurrency unlimited 14:32:16 17 12 2019 14:32:16.182:INFO [launcher]: Starting browser FirefoxHeadless 14:33:16 17 12 2019 14:33:16.186:WARN [launcher]: FirefoxHeadless have not captured in 60000 ms, killing. 14:33:16 17 12 2019 14:33:16.267:INFO [launcher]: Trying to start FirefoxHeadless again (1/2). 14:34:02 17 12 2019 14:34:02.255:INFO [Firefox 68.0.0 (Linux 0.0.0)]: Connected on socket C2YBo2DbLQsFFg3PAAAB with id 94570126 14:34:37 17 12 2019 14:34:37.260:WARN [Firefox 68.0.0 (Linux 0.0.0)]: Disconnected (0 times)reconnect failed before timeout of 5000ms (ping timeout) 14:34:37 Firefox 68.0.0 (Linux 0.0.0) ERROR 14:34:37 Disconnectedreconnect failed before timeout of 5000ms (ping timeout) 14:34:37 Firefox 68.0.0 (Linux 0.0.0): Executed 0 of 0 DISCONNECTED (35.005 secs / 0 secs) 14:34:37 17 12 2019 14:34:37.263:INFO [karma-server]: Restarting Firefox 68.0.0 (Linux 0.0.0) (1 of 2 attempts) 14:35:37 17 12 2019 14:35:37.490:WARN [launcher]: FirefoxHeadless have not captured in 60000 ms, killing. 14:35:37 17 12 2019 14:35:37.620:INFO [launcher]: Trying to start FirefoxHeadless again (2/2). 14:36:37 17 12 2019 14:36:37.626:WARN [launcher]: FirefoxHeadless have not captured in 60000 ms, killing. 14:36:37 17 12 2019 14:36:37.769:ERROR [launcher]: FirefoxHeadless failed 2 times (timeout). Giving up. 14:36:37 Warning: Task "karma:firefox" failed. Use --force to continue.
Happened again when the image got updated by mistake T259925. It seems to be related to Firefox 68.
Image | Firefox package |
---|---|
docker-registry.wikimedia.org/releng/node10-test-browser:0.6.0-s1 | 60.8.0esr-1~deb9u1 |
docker-registry.wikimedia.org/releng/node10-test-browser:0.6.2 | 68.11.0esr-1~deb9u1 |
docker-registry.wikimedia.org/releng/node10-test-browser:0.6.3-s3 | 78.10.0esr |
References:
- https://github.com/karma-runner/karma-firefox-launcher/issues/104#issuecomment-748140630
- https://bugzilla.mozilla.org/show_bug.cgi?id=1245239 Shared memory allocations can SIGBUS if there's not enough space in /dev/shm
- https://bugzilla.mozilla.org/show_bug.cgi?id=1464690 Reproducible crash when running in memory-constrained Docker container