Page MenuHomePhabricator

`npm ci` fails when using Fresh in MinervaNeue
Open, Needs TriagePublic

Description

npm ci works fine when I'm not using Fresh.

1~/Documents/gerrit/mediawiki/core/skins/MinervaNeue$ npm ci
2npm WARN prepare removing existing node_modules/ before installation
3
4> core-js-pure@3.10.1 postinstall /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/core-js-pure
5> node -e "try{require('./postinstall')}catch(e){}"
6
7Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!
8
9The project needs your help! Please consider supporting of core-js on Open Collective or Patreon:
10> https://opencollective.com/core-js
11> https://www.patreon.com/zloirock
12
13Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)
14
15
16> spawn-sync@1.0.15 postinstall /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/spawn-sync
17> node postinstall
18
19
20> pre-commit@1.2.2 install /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/pre-commit
21> node install.js
22
23pre-commit:
24pre-commit: Detected an existing git pre-commit hook
25pre-commit: Old pre-commit hook backuped to pre-commit.old
26pre-commit:
27
28> core-js@3.10.1 postinstall /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/core-js
29> node -e "try{require('./postinstall')}catch(e){}"
30
31
32> iltorb@2.4.5 install /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/iltorb
33> node ./scripts/install.js || node-gyp rebuild
34
35info looking for cached prebuild @ /Users/z/.npm/_cacache/_prebuilds/ee226e-iltorb-v2.4.5-node-v64-darwin-x64.tar.gz
36info found cached prebuild
37info unpacking @ /Users/z/.npm/_cacache/_prebuilds/ee226e-iltorb-v2.4.5-node-v64-darwin-x64.tar.gz
38info unpack resolved to /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/iltorb/build/bindings/iltorb.node
39info unpack required /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/iltorb/build/bindings/iltorb.node successfully
40info install Successfully installed iltorb binary!
41
42> fibers@5.0.0 install /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/fibers
43> node build.js || nodejs build.js
44
45 CXX(target) Release/obj.target/fibers/src/fibers.o
46 CXX(target) Release/obj.target/fibers/src/coroutine.o
47 CC(target) Release/obj.target/fibers/src/libcoro/coro.o
48 SOLINK_MODULE(target) Release/fibers.node
49Installed in `/Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/fibers/bin/darwin-x64-64/fibers.node`
50
51> fsevents@1.2.13 install /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/watchpack-chokidar2/node_modules/fsevents
52> node install.js
53
54 SOLINK_MODULE(target) Release/.node
55 CXX(target) Release/obj.target/fse/fsevents.o
56 SOLINK_MODULE(target) Release/fse.node
57
58> fsevents@1.2.13 install /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/fork-ts-checker-webpack-plugin/node_modules/fsevents
59> node install.js
60
61 SOLINK_MODULE(target) Release/.node
62 CXX(target) Release/obj.target/fse/fsevents.o
63 SOLINK_MODULE(target) Release/fse.node
64
65> ejs@2.7.4 postinstall /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/ejs
66> node ./postinstall.js
67
68Thank you for installing EJS: built with the Jake JavaScript build tool (https://jakejs.com/)
69
70
71> core-js@2.6.12 postinstall /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/node_modules/babel-runtime/node_modules/core-js
72> node -e "try{require('./postinstall')}catch(e){}"
73
74added 1969 packages in 47.675s

It fails in Fresh.

1~/Documents/gerrit/mediawiki/core/skins/MinervaNeue$ fresh-node -env -net
2# fresh: 21.04.1 (2021-04-29)
3# image: docker-registry.wikimedia.org/releng/node10-test-browser:0.6.3-s2
4# software: Debian GNU/Linux 9 (stretch)
5# Node.js v10.15.2 (npm 6.14.5)
6# Chromium 73.0.3683.75
7# Mozilla Firefox 68.12.0esr
8# JSDuck 5.3.4 (Ruby 2.3.3)
9# mount: /MinervaNeue ➟ /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue (read-write)
10# /MinervaNeue/.git ➟ /Users/z/Documents/gerrit/mediawiki/core/skins/MinervaNeue/.git (read-only)
11# env: MW_*, MEDIAWIKI_* (none found)
12# net: expose host
13
14🌱 Fresh!
15
16
17
18nobody@docker-desktop:/MinervaNeue$ npm ci
19npm WARN prepare removing existing node_modules/ before installation
20[..................] / : WARN prepare removing existin[..................] / : WARN prepare removing existin
21> core-js-pure@3.10.1 postinstall /MinervaNeue/node_modules/core-js-pure
22> node -e "try{require('./postinstall')}catch(e){}"
23
24Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!
25
26The project needs your help! Please consider supporting of core-js on Open Collective or Patreon:
27> https://opencollective.com/core-js
28> https://www.patreon.com/zloirock
29
30Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)
31
32
33> spawn-sync@1.0.15 postinstall /MinervaNeue/node_modules/spawn-sync
34> node postinstall
35
36
37> pre-commit@1.2.2 install /MinervaNeue/node_modules/pre-commit
38> node install.js
39
40pre-commit:
41pre-commit: Detected an existing git pre-commit hook
42fs.js:115
43 throw err;
44 ^
45
46Error: EROFS: read-only file system, open '/MinervaNeue/.git/hooks/pre-commit.old'
47 at Object.openSync (fs.js:439:3)
48 at Object.writeFileSync (fs.js:1190:35)
49 at Object.<anonymous> (/MinervaNeue/node_modules/pre-commit/install.js:38:6)
50 at Module._compile (internal/modules/cjs/loader.js:689:30)
51 at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
52 at Module.load (internal/modules/cjs/loader.js:599:32)
53 at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
54 at Function.Module._load (internal/modules/cjs/loader.js:530:3)
55 at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
56 at startup (internal/bootstrap/node.js:283:19)
57npm ERR! code ELIFECYCLE
58npm ERR! errno 1
59npm ERR! pre-commit@1.2.2 install: `node install.js`
60npm ERR! Exit status 1
61npm ERR!
62npm ERR! Failed at the pre-commit@1.2.2 install script.
63npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
64
65npm ERR! A complete log of this run can be found in:
66npm ERR! /cache/_logs/2021-05-07T14_28_27_311Z-debug.log

Workaround: remove pre-commit package from package.json, run npm i.

Event Timeline

Restricted Application added subscribers: Masumrezarock100, Aklapper. · View Herald Transcript
Krinkle moved this task from Inbox to Support & Meta on the Fresh board.

This is, unfortunately, working as expected.

The Minerva repository tries to install an executable script in your .git/hooks directory. This script would then insecurely execute Node.js code as part of your credentialed/authenticated git interactions outside your development environment (e.g. when staging your commit, or when pushing to Gerrit).

Fresh is essentially detecting this as if it were malware trying to escape the container, and correctly prevents it from doing that.

It'd be hard to support something like this in a secure manner, and I'm not sure how Reading Web team generally manage this day-to-day. On occasion when I've patched client-side code in this kind of repository, I've learned to modify package.json to remove the git hook, then run install again, and then let Git undo that change (so as to not commit it).

Personally, I generally use separate terminal tab anyway for running tests vs staging commit and writing commit messages (where the former tab might use a "watch" mode if the project has one, giving you real-time feedback regardless of when a commit is saved). In non-Wikimedia projects I've generally found such hooks obtrusive, usually running at times I specifically don't want or need them to.

In any event, you could for now use the same workaround as me. Though It'd be nice if was a documented recommendation in the repository readme that would do this automatically, e.g. NO_COMMIT_HOOK=1 npm ci. (Maybe something like that exists already?), whicih would skip this step, and Fresh (and WMF CI) could also set such env var by default.

Alternatively, if there's interest from Reading Web, we could prioritize T253924 and support shortcuts like fresh -- npm test which developers could then use as part of a command-line alias like npm-test-and-git-commit, e.g. as git npmtac or some other short or tab-completable phrase of your choosing. This can then be adopt as alternate to git commit for those using that workflow, without it affecting the standard "npm install", "npm test", "git commit", "git rebase" etc workflow that new contributors would generally know and expect to work a certain way.

Krinkle triaged this task as Low priority.
Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.
Krinkle subscribed.

I've learned to modify package.json to remove the git hook, then run install again, and then let Git undo that change (so as to not commit it).

I'm looking at package.json but I don't see anything related to git. What do I need to change?

I'm looking at package.json but I don't see anything related to git. What do I need to change?

For my future reference: remove pre-commit package from package.json, run npm i.

zeljkofilipin renamed this task from `npm ci` fails when using Fresh in MinervaNeue to `npm ci` fails when using Fresh in MinervaNeue and Popups.Jun 18 2021, 1:54 PM
zeljkofilipin added a project: Page-Previews.
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)
Jdlrobson moved this task from Tech debt to Tracking on the MinervaNeue board.
Jdlrobson edited projects, added MinervaNeue (Tracking); removed MinervaNeue.

@Jdlrobson I believe "Tracking" usually means there is the understanding that a different team will resolve the ticket instead. If so, which team did you have in mind?

Tracking for web team means we don't have bandwidth right now. We review tasks in tracking at end of projects and it can change, particularly if tasks are associated with epics. If you need web to work on this please send me or Olga an email asking for us to prioritize it.

I should note weeb team has had nothing to do with Fresh yet so not really sure we would be the right team or who would be.

zeljkofilipin renamed this task from `npm ci` fails when using Fresh in MinervaNeue and Popups to `npm ci` fails when using Fresh in MinervaNeue.Oct 7 2021, 11:18 AM
zeljkofilipin updated the task description. (Show Details)

npm ci works fine in Popups, but it still fails in MinervaNeue.

Having a hard time debugging T301575 because of this. :(

zeljkofilipin raised the priority of this task from Low to Needs Triage.Oct 31 2023, 4:18 PM