"The cypress npm package is installed, but the Cypress binary is missing" error prevents merging changes
Open, HighPublic
Actions

Assigned To

Authored By

	matmarex
	Apr 1 2024, 8:50 PM

Project Tags

Referenced Files

	F44164930: image.png
	Apr 2 2024, 12:59 PM

Subscribers

• Lucas_Werkmeister_WMDE

View All 18 Subscribers

Description

The following error prevents merging changes in at least MobileFrontend and DiscussionTools, maybe other repositories:

> selenium-test
> cypress run

The cypress npm package is installed, but the Cypress binary is missing.

We expected the binary to be installed here: /cache/Cypress/13.6.1/Cypress/Cypress

Reasons it may be missing:

- You're caching 'node_modules' but are not caching this path: /cache/Cypress
- You ran 'npm install' at an earlier build step but did not persist: /cache/Cypress

Properly caching the binary will fix this error and avoid downloading and unzipping Cypress.

Alternatively, you can run 'cypress install' to download the binary again.

https://on.cypress.io/not-installed-ci-error

Example failures:

Details

Related Changes in Gerrit:

	Subject	Author	Repo	Branch	Lines +/-
	Configure CYPRESS_CACHE_FOLDER	Lucas Werkmeister (WMDE)	mediawiki/extensions/Cite	master	+4 -2

Customize query in gerrit

Related Objects

Mentioned In: T361624: Running `cypress` in Wikimedia CI requires unusual env variables
Mentioned Here: T385940: Parsoid CI is broken because of Cypress builds failing for selenium tests
T361624: Running `cypress` in Wikimedia CI requires unusual env variables

Event Timeline

matmarex created this task.Apr 1 2024, 8:50 PM

Restricted Application added a subscriber: • Aklapper. · View Herald TranscriptApr 1 2024, 8:50 PM

matmarex triaged this task as Unbreak Now! priority.Apr 1 2024, 8:51 PM

I think this test is defined in Cite extension if I'm reading the CI log correctly?

Also affects ContentTranslation, https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ContentTranslation/+/1015683

• hashar removed a project: Continuous-Integration-Infrastructure.Apr 2 2024, 9:19 AM

Tgr subscribed.Apr 2 2024, 9:37 AM

jhsoby subscribed.Apr 2 2024, 9:48 AM

• Muhammad_Yasser_Jazirahly_WMDE subscribed.Apr 2 2024, 10:41 AM

Silvan_WMDE subscribed.Apr 2 2024, 11:56 AM

Also affects Wikibase (example change), as well as Cite itself (example change).

In EntitySchema, where we also use Cypress, it seems to be working still (example change and console); Cypress seemingly autoinstalls itself:

> selenium-test
> CYPRESS_CACHE_FOLDER=./cypress/.cache cypress run

It looks like this is your first time using Cypress: 13.6.6

[STARTED] Task without title.
[SUCCESS] Task without title.

Opening Cypress...

I don’t know why it’s not working for Cite, but it could be because Cite doesn’t customize the CYPRESS_CACHE_FOLDER?

The path /cache/Cypress looks suspicious anyways. I’m guessing that it’s partially the result of the test user having their home directory set to /, but according to the cypress docs, the default should be ~/.cache/Cypress, i.e. with a dot before the word “cache”…

Change #1016337 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Cite@master] Configure CYPRESS_CACHE_FOLDER

https://gerrit.wikimedia.org/r/1016337

gerritbot added a project: Patch-For-Review.Apr 2 2024, 12:55 PM

Looking good so far…

Quibble sets XDG_CACHE_HOME=/cache. I guess that directory then gets cleared between running npm and running the tests?

Change #1016337 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Configure CYPRESS_CACHE_FOLDER

https://gerrit.wikimedia.org/r/1016337

Physikerwelt subscribed.Apr 2 2024, 2:40 PM

In T361520#9680281, @Tgr wrote:

Quibble sets XDG_CACHE_HOME=/cache. I guess that directory then gets cleared between running npm and running the tests?

Could be, yeah. (Bit annoying that the Cypress docs doesn’t mention that they respect $XDG_CACHE_HOME, but that seems like a plausible source for the path at least.)

I think we can close this task, as the CI blocker is resolved; I don’t know if we should have a follow-up task for not clearing the cache or not.

Thank you @Lucas_Werkmeister_WMDE! I filed a follow-up: T361624: Running `cypress` in Wikimedia CI requires unusual env variables

ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.26; 2024-04-09).Apr 3 2024, 2:02 PM

Seems not properly fixed? https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php81-docker/20433/console

17:26:07 npm ERR! [FAILED] The Cypress App could not be downloaded.
17:26:07 npm ERR! [FAILED] 
17:26:07 npm ERR! [FAILED] Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: https://on.cypress.io/proxy-configuration
17:26:07 npm ERR! [FAILED] 
17:26:07 npm ERR! [FAILED] Otherwise, please check network connectivity and try again:
17:26:07 npm ERR! [FAILED] 
17:26:07 npm ERR! [FAILED] ----------
17:26:07 npm ERR! [FAILED] 
17:26:07 npm ERR! [FAILED] URL: https://download.cypress.io/desktop/13.6.1?platform=linux&arch=x64
17:26:07 npm ERR! [FAILED] Error: Failed downloading the Cypress binary.
17:26:07 npm ERR! [FAILED] Response code: 502
17:26:07 npm ERR! [FAILED] Response message: Bad Gateway
17:26:07 npm ERR! [FAILED] 
17:26:07 npm ERR! [FAILED] ----------
17:26:07 npm ERR! [FAILED] 
17:26:07 npm ERR! [FAILED] Platform: linux-x64 (Debian - 10.13)
17:26:07 npm ERR! [FAILED] Cypress Version: 13.6.1

Didn't reappear during recheck.

Recurred in T385940: Parsoid CI is broken because of Cypress builds failing for selenium tests.

Maintenance_bot removed a project: Patch-For-Review.Feb 10 2025, 4:30 PM

I am reopening this due to CYPRESS_CACHE_FOLDER causing the local cache to be bypassed and it to not be cached on CI.

In T361624#9682983 I wrote:

Here is some context about CI:

the Docker images have the environment variables:
- XDG_CACHE_HOME=/cache , that is a bind mount from the host and anything in that path ends up being saved in the central cache (nicknamed Castor)
- XDG_CONFIG_HOME=/tmp/xdg-config-home
containers run as user nobody which has the home directory set to the directory /nonexistent which is intentionally not existent

At a quick glance, Cypress invokes return cachedir('Cypress') which is cachedir which does:

const cacheHome = process.env.XDG_CACHE_HOME || path.join(os.homedir(), '.cache')

The binary should thus be somewhere under /cache/Cypress. Some possibilities to assist debug is to hack your npm script to add:

finding the path find /cache -type f -name Cypress || : : will show the file if it exists
use DEBUG=cypress:cli which enables its debug log and would output something such as:

cypress:cli checking if executable exists /cache/Cypress/13.6.1/Cypress/Cypress +2ms
cypress:cli Binary is executable? : true +2ms
cypress:cli binaryDir is  /cache/Cypress/13.6.1/Cypress +0ms
cypress:cli Reading binary package.json from: /cache/Cypress/13.6.1/Cypress/resources/app/package.json +0ms
cypress:cli Found binary version 13.6.1 installed in: /cache/Cypress/13.6.1/Cypress +4ms
cypress:cli { verified: true } +5ms
cypress:cli is Verified ? true +2ms

Maybe Cypress uses XDG_CONFIG_HOME which is under /tmp and indeed is not saved/restored.

• hashar merged a task: T361624: Running `cypress` in Wikimedia CI requires unusual env variables.Apr 14 2025, 9:59 AM

• hashar added subscribers: Peter, SDunlap, zeljkofilipin and 2 others.

I went to use Fresh and the Cite extension as a playground. I get Cypress 13.17.0 installed.

I have removed CYPRESS_CACHE_FOLDER, ran npm ci:

$ find /cache /tmp . -type f -name Cypress
/cache/Cypress/13.17.0/Cypress/Cypress

That is under /cache which is XDG_CACHE_HOME`.

When I run cypress, it does find the binary:

$ DEBUG=cypress:cli cypress open --config-file tests/cypress/cypress.config.js
  cypress:cli checking if executable exists /cache/Cypress/13.17.0/Cypress/Cypress +3ms
  cypress:cli Binary is executable? : true +1ms
  cypress:cli binaryDir is  /cache/Cypress/13.17.0/Cypress +1ms
  cypress:cli Reading binary package.json from: /cache/Cypress/13.17.0/Cypress/resources/app/package.json +0ms
  cypress:cli Found binary version 13.17.0 installed in: /cache/Cypress/13.17.0/Cypress +3ms

I have send a dummy change which removes CYPRESS_CACHE_FOLDER and turns on DEBUG mode https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1136340

A concerning side note is the Cypress caches takes 680MBytes:

$ du -s -h /cache/Cypress/
679M	/cache/Cypress/

A 128M browser_v8_context_snapshot.bin which I guess is Chrome, and a 66M binary of ffmpeg (which we already have) (due to @ffmpeg-installer/ffmpeg). For jobs running browser tests for multiple repos, there will be multiple versions stored in cache and we upgrade Cypress, they will pill up even more, that is itself problematic.

When I remove CYPRESS_CACHE_FOLDER it passes fine now, I don't know what could have gone wrong previously but I doubt it is necessary given Cypress obey XDG_CACHE_HOME.

Given accumulating Cypress versions in the cache will certainly outgrow it, setting CYPRESS_CACHE_FOLDER is a way to have a copy local to the build at the expense of having to download it entirely on every single build :( That is far from being ideal specially given the large size of Cypress. Maybe we can keep well known copies in a standalone cache, but that sounds like an extra layer of complexity. Looks like not caching it is the easiest path for now :/

@hashar do you need help from Test Platform on this task?

matmarex unsubscribed.Apr 14 2025, 3:51 PM

Jdlrobson-WMF moved this task from Untriaged to Tracking on the MobileFrontend board.Apr 24 2025, 3:16 PM

Jdlrobson-WMF edited projects, added: MobileFrontend (Tracking); removed: MobileFrontend.

Nikerabbit moved this task from Needs Triage to Maintenance on the ContentTranslation board.Jun 2 2025, 10:25 AM

In T361520#10738058, @hashar wrote:

When I remove CYPRESS_CACHE_FOLDER it passes fine now, I don't know what could have gone wrong previously but I doubt it is necessary given Cypress obey XDG_CACHE_HOME.

Given accumulating Cypress versions in the cache will certainly outgrow it, setting CYPRESS_CACHE_FOLDER is a way to have a copy local to the build at the expense of having to download it entirely on every single build :( That is far from being ideal specially given the large size of Cypress. Maybe we can keep well known copies in a standalone cache, but that sounds like an extra layer of complexity. Looks like not caching it is the easiest path for now :/

I just (randomly) stumbled over this ticket. In the Cite extension we still use CYPRESS_CACHE_FOLDER=./tests/cypress/.cache cypress run is there a problem with that, that needs to be solved?

"The cypress npm package is installed, but the Cypress binary is missing" error prevents merging changesOpen, HighPublicActions

Description

Details

Related Objects

Event Timeline

"The cypress npm package is installed, but the Cypress binary is missing" error prevents merging changes
Open, HighPublic
Actions