Migrate all repos to lockfileVersion 2 to avoid "The package-lock.json file was created with an old version of npm"
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	kostajh
	Nov 4 2021, 1:14 PM

Description

I see this message in the npm ci step of various builds (e.g. https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/119033/console)

13:37:22 INFO:quibble.commands:>>> Start: npm install in /workspace/src
13:37:22 npm WARN old lockfile 
13:37:24 npm WARN old lockfile The package-lock.json file was created with an old version of npm,
13:37:24 npm WARN old lockfile so supplemental metadata must be fetched from the registry.
13:37:24 npm WARN old lockfile 
13:37:24 npm WARN old lockfile This is a one-time fix-up, please be patient...
13:37:24 npm WARN old lockfile 
13:37:24 npm WARN old lockfile wdio-mediawiki: No matching version found for wdio-mediawiki@1.2.0.
13:37:27 npm WARN old lockfile     at module.exports (/srv/npm/node_modules/npm-pick-manifest/index.js:209:23)
13:37:27 npm WARN old lockfile     at packument.then.packument (/srv/npm/node_modules/pacote/lib/registry.js:118:26)
13:37:27 npm WARN old lockfile  Could not fetch metadata for wdio-mediawiki@1.2.0 { wdio-mediawiki: No matching version found for wdio-mediawiki@1.2.0.
13:37:27 npm WARN old lockfile     at module.exports (/srv/npm/node_modules/npm-pick-manifest/index.js:209:23)
13:37:27 npm WARN old lockfile     at packument.then.packument (/srv/npm/node_modules/pacote/lib/registry.js:118:26)
13:37:27 npm WARN old lockfile   stack:
13:37:27 npm WARN old lockfile    'wdio-mediawiki: No matching version found for wdio-mediawiki@1.2.0.\n    at module.exports (/srv/npm/node_modules/npm-pick-manifest/index.js:209:23)\n    at packument.then.packument (/srv/npm/node_modules/pacote/lib/registry.js:118:26)',
13:37:27 npm WARN old lockfile   code: 'ETARGET',
13:37:27 npm WARN old lockfile   type: 'version',
13:37:27 npm WARN old lockfile   wanted: '1.2.0',
13:37:27 npm WARN old lockfile   versions:
13:37:27 npm WARN old lockfile    [ '0.1.7',
13:37:27 npm WARN old lockfile      '0.2.0',
13:37:27 npm WARN old lockfile      '0.3.0',
13:37:27 npm WARN old lockfile      '0.4.0',
13:37:27 npm WARN old lockfile      '0.5.0',
13:37:27 npm WARN old lockfile      '1.0.0',
13:37:27 npm WARN old lockfile      '1.1.0',
13:37:27 npm WARN old lockfile      '1.1.1' ],
13:37:27 npm WARN old lockfile   name: 'wdio-mediawiki',
13:37:27 npm WARN old lockfile   distTags: { latest: '1.1.1' },
13:37:27 npm WARN old lockfile   defaultTag: 'latest' }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/cli@7.4.6',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/config@7.4.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/dot-reporter@7.4.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/junit-reporter@7.4.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/local-runner@7.4.6',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/logger@7.4.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/mocha-framework@7.13.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/logger@7.7.0',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/types@7.13.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/utils@7.13.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/protocols@7.4.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/repl@7.4.6',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/reporter@7.4.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/runner@7.4.6',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/sync@7.4.6',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/types@7.4.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: '@wdio/utils@7.4.6',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: 'devtools@7.4.6',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: 'eslint-plugin-wdio@7.4.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: 'mocha@9.1.2',
13:37:27 npm WARN EBADENGINE   required: { node: '>= 12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: 'stylelint-no-unsupported-browser-features@5.0.1',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: 'webdriver@7.4.6',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN EBADENGINE Unsupported engine { package: 'webdriverio@7.4.6',
13:37:27 npm WARN EBADENGINE   required: { node: '>=12.0.0' },
13:37:27 npm WARN EBADENGINE   current: { node: 'v10.24.0', npm: '7.21.0' } }
13:37:27 npm WARN deprecated @stylelint/postcss-markdown@0.36.2: Use the original unforked package instead: postcss-markdown
13:37:32 npm WARN deprecated har-validator@5.1.5: this library is no longer supported
13:37:32 npm WARN deprecated formidable@1.2.2: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
13:37:32 npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
13:37:33 npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
13:37:33 
13:37:42 added 1096 packages, and audited 1098 packages in 19s
13:37:42 
13:37:42 123 packages are looking for funding
13:37:42   run `npm fund` for details
13:37:42 
13:37:42 3 moderate severity vulnerabilities
13:37:42 
13:37:42 To address all issues (including breaking changes), run:
13:37:42   npm audit fix --force
13:37:42 
13:37:42 Run `npm audit` for details.

It looks like there are two problems:

package-lock.json created with previous version of npm
some packages (wdio ones) not supported by current node engine (10, they want version 12)

Details

Related Changes in Gerrit:

Subject	Repo	Branch	Lines +/-
Regenerate package-lock.json to lockfileVersion 2	wikimedia-cz/tracker	master	+8 K -22
build: Drop package-lock.json, repo is broken(?)	mediawiki/services/service-scaffold-node	main	+0 -11 K
build: Re-generate package-lock.json with lockfileVersion 2	mediawiki/libs/WebIDL	main	+21 -1
build: Re-generate package-lock.json with lockfileVersion 2	mediawiki/libs/Dodo	master	+39 -3
build: update npm lock file to version 2	mediawiki/tools/api-testing	master	+3 K -12

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	Catrope	T295420 Add Codex to LibraryUpgrader monitored repos
Resolved	Jdforrester-WMF	T273785 Deal with release of npm 7
Resolved	Legoktm	T273810 Prepare libup for npm 7
Resolved	None	T295036 Migrate all repos to lockfileVersion 2 to avoid "The package-lock.json file was created with an old version of npm"

Event Timeline

kostajh created this task.Nov 4 2021, 1:14 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 4 2021, 1:14 PM

@hashar sorry I don't know which projects to tag this with exactly; please move around as you see fit.

I stumbled over this today as well. It seems that quibble was intentionally left out when most CI jobs were upgraded to Node12 in June: T284345#7139881. Part of the reason seems to have been that bullseye with Node12 was not available yet. Though, as I understand it, this should now be the case with T284346 being resolved.

Node12 should allow us upgrading the Browser Tests for WikibaseLexeme better and that _might_ help with very flaky daily browser test runs 🤞

Umherirrender mentioned this in T295114: quibble-fundraising-cldr-REL1_35-php73-docker is failing.Nov 5 2021, 5:22 PM

zeljkofilipin added a project: User-zeljkofilipin.Nov 10 2021, 2:34 PM

zeljkofilipin moved this task from Backlog 🪒 to Quarter 👔 on the User-zeljkofilipin board.

zeljkofilipin subscribed.

This might be fixed with T294931: Migrate quibble images from node10 to something modern being done, right?

It's the 5th item at T273785: Deal with release of npm 7. If you've got LibraryUpgrader active on your repo, it will always return to package-lock.json v1, no matter if you (on npm v7) have made dependency upgrades and locally have v2 that you'll commit.

Jdforrester-WMF added a parent task: T273810: Prepare libup for npm 7.Dec 2 2021, 7:04 PM

zeljkofilipin moved this task from Quarter 👔 to Watching 📺 on the User-zeljkofilipin board.Dec 22 2021, 12:18 PM

This does not cause a CI failure and is fully compatible. Individual project maintainers may want to update their lock file and/or LibUp needs to not downgrade, but it shouldn't be causing any issue afaik by itself, it's just warning noise.

The problem with lockfileVersion: 1 is npm 7 will revalidate all dependencies. For MobileFrontend that takes 34 seconds on my machine. Once that got upgraded to lockfileVersion: 2, npm ci no more does that whole revalidation.

On the CI builds the revalidation takes a bit of time and we redo it for any build test flow that has to run npm ci. We should migrate at least the master branches to lockfileVersion: 2 and enforce it (maybe via the job running npm test).

From @Volker_E comment above, it looks like we first need to teach LibUp to respect the currently defined lockfileVersion. Or maybe it is still using npm 6.

hashar mentioned this in T273785: Deal with release of npm 7.Mar 22 2023, 2:07 PM

Change 902094 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/tools/api-testing@master] build: update npm lock file to version 2

https://gerrit.wikimedia.org/r/902094

gerritbot added a project: Patch-For-Review.Mar 22 2023, 2:10 PM

Change 902094 merged by jenkins-bot:

[mediawiki/tools/api-testing@master] build: update npm lock file to version 2

https://gerrit.wikimedia.org/r/902094

hashar mentioned this in rMTAT0ff403bb4182: build: update npm lock file to version 2.Mar 22 2023, 2:25 PM

Maintenance_bot removed a project: Patch-For-Review.Mar 22 2023, 2:30 PM

Jdforrester-WMF renamed this task from The package-lock.json file was created with an old version of npm to Migrate all repos to lockfileVersion 2 to avoid "The package-lock.json file was created with an old version of npm".Mar 22 2023, 2:41 PM

Jdforrester-WMF updated the task description. (Show Details)

Change 902100 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/libs/Dodo@master] build: Re-generate package-lock.json with lockfileVersion 2

https://gerrit.wikimedia.org/r/902100

Change 902101 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/libs/WebIDL@main] build: Re-generate package-lock.json with lockfileVersion 2

https://gerrit.wikimedia.org/r/902101

Change 902105 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/services/service-scaffold-node@main] build: Drop package-lock.json, repo is broken(?)

https://gerrit.wikimedia.org/r/902105

Those three patches above apparently complete this migration: https://codesearch-beta.wmcloud.org/search/?q=%22lockfileVersion%22%3A+1&files=%5Epackage-lock%5C.json%24&excludeFiles=&repos=

Change 902100 merged by jenkins-bot:

[mediawiki/libs/Dodo@master] build: Re-generate package-lock.json with lockfileVersion 2

https://gerrit.wikimedia.org/r/902100

Change 902101 merged by jenkins-bot:

[mediawiki/libs/WebIDL@main] build: Re-generate package-lock.json with lockfileVersion 2

https://gerrit.wikimedia.org/r/902101

zeljkofilipin removed a project: User-zeljkofilipin.Mar 22 2023, 2:58 PM

Jdforrester-WMF mentioned this in rWLWI3cb8cf2c4fe7: build: Re-generate package-lock.json with lockfileVersion 2.Mar 22 2023, 3:01 PM

Jdforrester-WMF closed this task as Resolved.Mar 22 2023, 3:02 PM

Change 902105 merged by Jforrester:

[mediawiki/services/service-scaffold-node@main] build: Drop package-lock.json, repo is broken(?)

https://gerrit.wikimedia.org/r/902105

Maintenance_bot removed a project: Patch-For-Review.Mar 22 2023, 3:10 PM

Awesome. We certainly still have lockfile version 1 in release branches but I am willing to ignore those entirely.

Thank you for the code search link @Jdforrester-WMF :]

Jdforrester-WMF mentioned this in rMSSN3191484ef60a: build: Drop package-lock.json, repo is broken(?).Mar 22 2023, 6:32 PM

Change 954751 had a related patch set uploaded (by Hashar; author: Hashar):

[wikimedia-cz/tracker@master] Regenerate package-lock.json to lockfileVersion 2

https://gerrit.wikimedia.org/r/954751

gerritbot added a project: Patch-For-Review.Sep 4 2023, 8:43 PM

Change 954751 merged by jenkins-bot: