Page MenuHomePhabricator

Fails npm build failure "File exists: ../esprima/bin/esparse.js"
Closed, ResolvedPublic

Description

23:30:33 npm ERR! EEXIST, symlink '../esprima/bin/esparse.js'
23:30:33 File exists: ../esprima/bin/esparse.js
23:30:33 Move it away, and try again.

Caused by a bug in npm or a bug in the dev-dependencies in how they register themselves with npm.

This error happens when two builds run in parallel on the same ci slave host.

Upstream report:

A recheck usually "fixes" it for now.

Event Timeline

Krinkle raised the priority of this task from to High.
Krinkle updated the task description. (Show Details)
Krinkle subscribed.
Krinkle set Security to None.
Krinkle claimed this task.

I've upgraded npm from v2.4.1 to v2.7.6 on our slaves. I've worked with upstream and there've been several improvements to mitigate race conditions and ensure shared cache integrity. We've not been able to reproduce this exact bug so that may or may not be fixed. Re-open if this shows up again.

Krinkle reopened this task as Open.EditedApr 9 2015, 6:50 PM

Happened again.

https://integration.wikimedia.org/ci/job/npm/2194/console

18:15:08 Building remotely on integration-slave-trusty-1012 ( .. UbuntuTrusty) in workspace /mnt/jenkins-workspace/workspace/..
18:15:10 + node --version
18:15:10 v0.10.25
18:15:10 + npm --version
18:15:10 2.7.6
18:15:10 + rm -rf node_modules
18:15:10 + npm install
...
18:16:10 npm ERR! Linux 3.13.0-45-generic
18:16:10 npm ERR! argv "node" "/usr/local/bin/npm" "install"
18:16:10 npm ERR! node v0.10.25
18:16:10 npm ERR! npm  v2.7.6
18:16:10 npm ERR! path ../esprima-harmony-jscs/bin/esparse.js
18:16:10 npm ERR! code EEXIST
18:16:10 npm ERR! errno 47
18:16:10 
18:16:10 npm ERR! EEXIST, symlink '../esprima-harmony-jscs/bin/esparse.js'
18:16:10 File exists: ../esprima-harmony-jscs/bin/esparse.js
18:16:10 Move it away, and try again. 
18:16:11

Full npm-debug.log at P504.

Krinkle changed the task status from Open to Stalled.Apr 16 2015, 2:03 AM
Krinkle removed Krinkle as the assignee of this task.

Upstream JSCS (markelog) is working on solving it per my comment on node-jscs issue #883.

Change 207468 had a related patch set uploaded (by Jforrester):
build: Bump various devDependencies

https://gerrit.wikimedia.org/r/207468

Change 207471 had a related patch set uploaded (by Jforrester):
build: Bump various devDependencies

https://gerrit.wikimedia.org/r/207471

Change 207471 merged by jenkins-bot:
build: Bump various devDependencies

https://gerrit.wikimedia.org/r/207471

Change 207468 merged by jenkins-bot:
build: Bump various devDependencies

https://gerrit.wikimedia.org/r/207468

Another example, this time because our library was out-of-date: https://integration.wikimedia.org/ci/job/npm/31617/console

Side note, it would be nice if we didn't have to keep our CI glue up-to-date, e.g. it comes in via an independent repo.

Side note, it would be nice if we didn't have to keep our CI glue up-to-date, e.g. it comes in via an independent repo.

We intentionally moved away from maintaining dev dependencies in a central place as this does not scale and significantly stagnates productivity. It would encourage (un)intentional creation of bi-directional dependency links, thus making it impossible to upgrade anything unless every project ever upgrades code at once. Which is infeasible for major upgrades of CI frameworks that that have different priorities for different consuming projects. I'm happy to discuss the details by e-mail or IRC if you like.

One example is tools like JSHint or JSCS which cannot function correctly if the version is not specified within the repository.