Page MenuHomePhabricator

Jenkins failing everything due to npm being screwed up
Closed, ResolvedPublic

Description

I don't know if this has to do with the WMCS outage (T216030), but in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/490308#message-c5e8a6d7c284267e7a8f531e5c6b6eea9ce4547d most of the tests failed. The failures are all due to npm, as far as I can tell its cache is corrupted. Some extracts from the various tests there:

11:22:56 + npm install --no-progress
11:22:59 npm ERR! Linux 4.9.0-0.bpo.8-amd64
11:22:59 npm ERR! argv "/usr/bin/nodejs" "/usr/local/bin/npm" "install" "--no-progress"
11:22:59 npm ERR! node v6.11.0
11:22:59 npm ERR! npm  v3.8.3
11:22:59 npm ERR! file /cache/json-stable-stringify-without-jsonify/1.0.1/package/package.json
11:22:59 npm ERR! code EJSONPARSE
11:22:59 
11:22:59 npm ERR! Failed to parse json
11:22:59 npm ERR! Unexpected token 'b' at 1:1
11:22:59 npm ERR! b9761e09d397bdda699fee454c327946ec1240aa
11:22:59 npm ERR! ^
11:22:59 npm ERR! File: /cache/json-stable-stringify-without-jsonify/1.0.1/package/package.json
11:22:59 npm ERR! Failed to parse package.json data.
11:22:59 npm ERR! package.json must be actual JSON, not just JavaScript.
11:22:59 npm ERR! 
11:22:59 npm ERR! This is not a bug in npm.
11:22:59 npm ERR! Tell the package author to fix their package.json file. JSON.parse
11:25:42 npm ERR! Failed to parse json
11:25:42 npm ERR! Unexpected token '\u0000' at 1:2752
11:25:42 npm ERR! ted-git-info@>=2.1.4 <3.0.0","readme":"ERROR: No README data found!"}
11:25:42 npm ERR!                                                                      ^
11:25:42 npm ERR! File: /cache/npm/espree/5.0.0/package/package.json
11:25:42 npm ERR! Failed to parse package.json data.
11:25:42 npm ERR! package.json must be actual JSON, not just JavaScript.
11:25:42 npm ERR! 
11:25:42 npm ERR! This is not a bug in npm.
11:25:42 npm ERR! Tell the package author to fix their package.json file. JSON.parse
11:23:48 npm ERR! Linux 4.9.0-0.bpo.8-amd64
11:23:48 npm ERR! argv "/usr/bin/nodejs" "/usr/local/bin/npm" "install"
11:23:48 npm ERR! node v6.11.0
11:23:48 npm ERR! npm  v3.8.3
11:23:48 npm ERR! code EISDIR
11:23:48 npm ERR! errno -21
11:23:48 npm ERR! syscall read
11:23:48 
11:23:48 npm ERR! eisdir EISDIR: illegal operation on a directory, read
11:23:48 npm ERR! eisdir This is most likely not a problem with npm itself
11:23:48 npm ERR! eisdir and is related to npm not being able to find a package.json in
11:23:48 npm ERR! eisdir a package you are trying to install.
11:23:51 npm ERR! addLocal Could not install /workspace/src/tests/selenium/wdio-mediawiki
11:23:51 npm ERR! Linux 4.9.0-0.bpo.8-amd64
11:23:51 npm ERR! argv "/usr/bin/nodejs" "/usr/local/bin/npm" "install"
11:23:51 npm ERR! node v6.11.0
11:23:51 npm ERR! npm  v3.8.3
11:23:51 npm ERR! path /cache/npm/wdio-mediawiki/0.3.0/package
11:23:51 npm ERR! code ENOTDIR
11:23:51 npm ERR! errno -20
11:23:51 npm ERR! syscall mkdir
11:23:51 
11:23:51 npm ERR! ENOTDIR: not a directory, mkdir '/cache/npm/wdio-mediawiki/0.3.0/package'

Event Timeline

There are also similar errors in the unrelated change https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/490358. That said, the core change @Anomie mentioned has now been merged, so this might have been fixed in the meantime – I’ll recheck the Wikibase one.

Castor (our caching middlelayer for this stuff in CI) was one of the effected vms, we're removed it from service so nothing is using its cache but things should be working again.

Do you still see issues? I see eg: https://integration.wikimedia.org/ci/job/mwgate-npm-node-6-docker/78961/console working

Lucas_Werkmeister_WMDE claimed this task.

The Wikibase build went through, I was just about to close this when I saw your comment :) let’s call this resolved for now.

Several similar tasks got filled. Namely the central cache (that holds a copy of packagist/npm cache) ends up being corrupted due to the WMCS outage last week.

The instance got rebuild with a cold cache, so it is no more an issue and is definitely fixed now (reference: T216039#4951138 )