Page MenuHomePhabricator

WikibaseQualityConstraints browser tests are failing in CI: Cypress binary is missing
Closed, ResolvedPublic

Description

This is currently breaking CI for Wikibase, WikibaseQualityConstraints and possibly other extensions.

Error log output:

09:07:39 The cypress npm package is installed, but the Cypress binary is missing.
09:07:39 
09:07:39 We expected the binary to be installed here: /cache/Cypress/15.8.2/Cypress/Cypress
09:07:39 
09:07:39 Reasons it may be missing:
09:07:39 
09:07:39 - You're caching 'node_modules' but are not caching this path: /cache/Cypress
09:07:39 - You ran 'npm install' at an earlier build step but did not persist: /cache/Cypress
09:07:39 
09:07:39 Properly caching the binary will fix this error and avoid downloading and unzipping Cypress.
09:07:39 
09:07:39 Alternatively, you can run 'cypress install' to download the binary again.
09:07:39 
09:07:39 https://on.cypress.io/not-installed-ci-error

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php83-selenium/25340/console
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php83-selenium/25348/console

Explicitly setting CYPRESS_CACHE_FOLDER for WBQC does not seem to help.

Event Timeline

The expected behavior without a custom $CYPRESS_CACHE_FOLDER is that Cypress gets installed to $XDG_CACHE_HOME/Cypress/ (i.e. /cache/Cypress/ in the CI containers) as part of a postinstall script, and locally this still happening AFAICT. Why is it not working in CI?

Is the cache folder somehow getting wiped between the npm install (or npm ci) and npm run selenium-test?

Hm, this seems suspicious:

lucaswerkmeister-wmde@integration-castor05:~$ ls /srv/castor/castor-mw-ext-and-skins/master/quibble-vendor-mysql-php83-selenium/Cypress/15.8.2/Cypress/
chrome_100_percent.pak  resources  resources.pak

Compared to a Fresh container, after npm ci:

i-have-no-name@30c39368ffdd:/WikibaseQualityConstraints$ ls /cache/Cypress/15.8.2/Cypress/
Cypress  LICENSE.electron.txt  LICENSES.chromium.html  browser_v8_context_snapshot.bin  chrome-sandbox  chrome_100_percent.pak  chrome_200_percent.pak  chrome_crashpad_handler  icudtl.dat  libEGL.so  libGLESv2.so  libffmpeg.so  libvk_swiftshader.so  libvulkan.so.1  locales  resources  resources.pak  snapshot_blob.bin  v8_context_snapshot.bin  vk_swiftshader_icd.json

The cached Cypress folder in Castor seems to be missing most of its contents, including the Cypress binary itself. I wonder if this could be related to T295351: npm cache saved by castor get corrupted for unknown reason?

I guess we could try deleting the cached Cypress folder on the castor host and see if that helps, similar to the workaround for corrupted npm caches…

Mentioned in SAL (#wikimedia-releng) [2026-03-02T10:16:26Z] <Lucas_WMDE> ssh integration-castor05.integration.eqiad1.wikimedia.cloud sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/quibble-vendor-mysql-php83-selenium/Cypress/15.8.2/ # T418718

After deleting the Cypress cache (see SAL above), this build (for this change) succeeded. So I think that was indeed the reason – Castor cache corruption, with different symptoms than T295351 but probably the same underlying cause.

This also explains why only WikibaseQualityConstraints was affected. In most other Cypress-using extensions, we override the $CYPRESS_CACHE_FOLDER; this brings about its own problems, including inefficiency (wastefully redownloading Cypress all the time) and higher susceptibility to cdn outages (see T383325#11383506), but it effectively shields those extensions from a corrupted cache shared across builds, as happened in this case.