Page MenuHomePhabricator

CI mwext-node12-rundoc-docker job failing on repos using Storybook
Closed, ResolvedPublic

Description

sh: 1: build-storybook: not found

This is occurring on all repos that use storybook: Minerva, Vector blocking merges.

Example: https://gerrit.wikimedia.org/r/c/mediawiki/skins/Vector/+/730858

It was working fine early today, works fine locally (although haven't tested with fresh). It was working yesterday so something has likely changed upstream.

Unbreak now because it's blocking a train blocker.

Event Timeline

Jdlrobson triaged this task as Unbreak Now! priority.Oct 20 2021, 9:14 PM
Jdlrobson lowered the priority of this task from Unbreak Now! to High.Oct 20 2021, 9:59 PM

Doesn't seem to be impacting wmf5 as the docs command is not run there, which means we can at least unblock this week's train:
https://gerrit.wikimedia.org/r/q/project:mediawiki%252Fskins%252FMinervaNeue+branch:wmf%252F1.38.0-wmf.5

Jdlrobson raised the priority of this task from High to Needs Triage.Oct 21 2021, 4:13 PM

Happy to say this has resolved itself today. I'm guessing this was an upstream error.
FetchError: Invalid response body while trying to fetch https://registry.npmjs.org/@storybook%2fhtml:

Leaving open in case there is anything further to investigate here about making our CI more resilient.

hashar added a subscriber: hashar.
npm WARN old lockfile FetchError: Invalid response body while trying to fetch https://registry.npmjs.org/@storybook%2fhtml: ENOENT: no such file or directory, lstat '/cache/_cacache/content-v2/sha512/a0/b6/650d44d3252a72eb5e0c00a8d83f11789b2bded5c9debcbcbcfbc805ff3b82baaebd2088308dc7ee724bdf29cc26112d30e5b52f5161e4d5c11a8309b022'
npm WARN old lockfile     at /srv/npm/node_modules/minipass-fetch/lib/body.js:162:15
npm WARN old lockfile     at async Array.<anonymous> (/srv/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:691:9)
npm WARN old lockfile  Could not fetch metadata for @storybook/html@5.2.8 FetchError: Invalid response body while trying to fetch https://registry.npmjs.org/@storybook%2fhtml: ENOENT: no such file or directory, lstat '/cache/_cacache/content-v2/sha512/a0/b6/650d44d3252a72eb5e0c00a8d83f11789b2bded5c9debcbcbcfbc805ff3b82baaebd2088308dc7ee724bdf29cc26112d30e5b52f5161e4d5c11a8309b022'
npm WARN old lockfile     at /srv/npm/node_modules/minipass-fetch/lib/body.js:162:15
npm WARN old lockfile     at async Array.<anonymous> (/srv/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:691:9) {
npm WARN old lockfile   code: 'ENOENT',
npm WARN old lockfile   errno: 'ENOENT',
npm WARN old lockfile   syscall: 'lstat',
npm WARN old lockfile   path: '/cache/_cacache/content-v2/sha512/a0/b6/650d44d3252a72eb5e0c00a8d83f11789b2bded5c9debcbcbcfbc805ff3b82baaebd2088308dc7ee724bdf29cc26112d30e5b52f5161e4d5c11a8309b022',
npm WARN old lockfile   type: 'system'
npm WARN old lockfile }

Looks like something went wrong when transfering the centrally cached files, maybe they were being updated by another job and left in an awkward state that then got retrieved by the build.

It is definitely a one time error due to a race condition. Thank you for the task ;)

The same has been reported at T294426 and we will follow up there.

Mentioned in SAL (#wikimedia-releng) [2021-10-27T12:22:39Z] <hashar> integration-castor03: sudo rm -fR /srv/jenkins-workspace/caches/castor-mw-ext-and-skins/master/mwgate-node12-docker # T294426 T293937