Hmm, the package has been touched since 2014. There's a wishlist bug to upgrade it (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=804633), but that hasn't seen a followup either...

Node 4.x ships with its own npm. One shouldn't install "npm" from Debian as its own package. Maybe we're shadowing it somehow, due to the separate npm package overwriting the bin link?

The nodejs package from Debian does not ship npm (see file list at https://packages.debian.org/sid/amd64/nodejs/filelist ). Instead, npm is split to its own npm package. To get NodeJs 4.2/4.3 we backported the nodejs package from sid, npm hasn't been updated either by us or upstream Debian.

So this task is about extracting npm from Nodejs 4.2/4.3, update the npm package and ship it.

Meanwhile, the Jessie instances will be stuck to npm 1.4.21. I am not going to have puppet to ship it via npm install -g. That has proven to be a nightmare to handle.

Will npm-node-4.3 CI jobs be fixed too? They actually run npm install...

In T124474#2114179, @Ricordisamoa wrote:

Will npm-node-4.3 CI jobs be fixed too? They actually run npm install...

That job runs on I instances that are using Jessie configured to get packages from apt.wikimedia.org. So yes, whenever a new version is available the will catch up.

@hashar helped me to figure out an easyer way of installing npm

Please do the following

Add this to package.json

"dependencies": { "npm": "~3" }

you can pick your own npm version to run.

:)

See instructions at https://www.mediawiki.org/wiki/Continuous_integration/Entry_points#Recent_npm_version

In T124474#2027410, @mobrovac wrote:

I'm setting this task to stalled, as this is only relevant for edge cases where we execute npm install directly on the hosts (some testing hosts and CI). I don't think there is much value in packaging it ourselves until the version is bumped by upstream.

NOTE: Since the nodejs package is backported and the versions differ quite considerably, this may cause problems with requirements for the npm and nodejs-legacy packages. The solution is to install nodejs=0.10 along with npm and nodejs-legacy first and then apt-get install nodejs separately.

We can use the version directly from npm. Plus npm is running 2.x on node 0.10 but is running npm 1.x on node 4.3 so it is a down grade. We have tried npm 3.x by adding npm 3 to dependencies and it worked so no problems.

In T124474#2135197, @Paladox wrote:

npm is running 2.x on node 0.10 but is running npm 1.x on node 4.3 so it is a down grade. We have tried npm 3.x by adding npm 3 to dependencies and it worked so no problems.

This statement is confusing. You refer to the current situation as is the case for our Jenkins jobs named "npm" (not the software called "npm"). Our test builds that run on persistent slave indeed have Ubuntu Trusty with Node.js 0.10 and npm 2.x (Trusty defaults to 1.x but we upgraded to 2.x due to various bugs in older npm. This is fine since npm 2 supports officially supports Node 0.10). Test builds that run on our new experimental Nodepool instances have Debian Jessie with Node.js 4.x and npm 1.x.

However note that this has nothing to do with official Node.js, Debian or Ubuntu. This is all due to our own actions. Here are the upstream versions:

Official Node.js:

• Node 0.10 ships with npm 1.x.
• Node 0.12 ships with npm 2.x
• Node 4.x ships with npm 2.x
• Node 6.x (in the future) will likely ship with npm 3.x.

Package manager:

• Ubuntu Trusty: Node 0.10 and npm 1.3 http://packages.ubuntu.com/trusty/nodejs, http://packages.ubuntu.com/trusty/npm
• Debian Jessie: Node 0.10 and npm 1.4 (https://packages.debian.org/jessie/nodejs, https://packages.debian.org/jessie/npm

The Node v4.3.0 we currently run in Nodepool was backported by Wikimedia. But due to the lack of a packaged npm, we now have a terribly wrong combination of Node and npm.

Please do not move forward with Nodepool until npm is upgraded to at least npm 2.14.12 (the version that ships with Node v4.3.0), or npm 3.5.2 (the equivalent 3.x branch release).

Either by packaging npm, or by installing it with puppet like we have been doing for the past year on the persistent slaves https://github.com/wikimedia/operations-puppet/blob/4b78fffdfb/modules/contint/manifests/packages/javascript.pp#L39-L43

See also:

• https://phabricator.wikimedia.org/T119143#2026030
• http://packages.ubuntu.com/search?keywords=npm It seems Ubuntu Xenial has an npm 3.5.2 package we could use.

In T124474#2135319, @Krinkle wrote:

In T124474#2135197, @Paladox wrote:

npm is running 2.x on node 0.10 but is running npm 1.x on node 4.3 so it is a down grade. We have tried npm 3.x by adding npm 3 to dependencies and it worked so no problems.

This statement is confusing. You refer to the current situation as is the case for our Jenkins jobs named "npm" (not the software called "npm"). Our test builds that run on persistent slave indeed have Ubuntu Trusty with Node.js 0.10 and npm 2.x (Trusty defaults to 1.x but we upgraded to 2.x due to various bugs in older npm. This is fine since npm 2 supports officially supports Node 0.10). Test builds that run on our new experimental Nodepool instances have Debian Jessie with Node.js 4.x and npm 1.x.

However note that this has nothing to do with official Node.js, Debian or Ubuntu. This is all due to our own actions. Here are the upstream versions:

Official Node.js:

• Node 0.10 ships with npm 1.x.
• Node 0.12 ships with npm 2.x
• Node 4.x ships with npm 2.x
• Node 6.x (in the future) will likely ship with npm 3.x.

Package manager:

• Ubuntu Trusty: Node 0.10 and npm 1.3 http://packages.ubuntu.com/trusty/nodejs, http://packages.ubuntu.com/trusty/npm
• Debian Jessie: Node 0.10 and npm 1.4 (https://packages.debian.org/jessie/nodejs, https://packages.debian.org/jessie/npm

The Node v4.3.0 we currently run in Nodepool was backported by Wikimedia. But due to the lack of a packaged npm, we now have a terribly wrong combination of Node and npm.

Please do not move forward with Nodepool until npm is upgraded to at least npm 2.14.12 (the version that ships with Node v4.3.0), or npm 3.5.2 (the equivalent 3.x branch release).

Either by packaging npm, or by installing it with puppet like we have been doing for the past year on the persistent slaves https://github.com/wikimedia/operations-puppet/blob/4b78fffdfb/modules/contint/manifests/packages/javascript.pp#L39-L43

See also:

• https://phabricator.wikimedia.org/T119143#2026030
• http://packages.ubuntu.com/search?keywords=npm It seems Ubuntu Xenial has an npm 3.5.2 package we could use.

Hi sorry that it was confusing. I didn't explain it as good as you did so thanks. Yes what you suggested but hashar was going to migrate all extensions but that did not happen not sure though but it might be this week. We could package nodejs 4.3 with npm 2.x that comes with it or use npm 3.

@Krinkle would a package meant for Ubuntu Xenial work on Jessie since that is debian.

@Krinkle thanks for your detailed explanation of node/npm version matching on T124474#2135319 . Yesterday I triggered the npm-node-4.3 on MediaWiki extensions and all of them passed just fine with the node 4.3 / npm 1.4.21. Most probably because for those repositories we just install very simple packages (grunt, banana, jshint etc).

Almost all repositories have been migrated now, there are still a few leftover but that is because they are invoking either jsduck or composer which are not yet provided in the Nodepool image.

npm 2.x would be nice to have, but it is currently not preventing npm jobs to run. If we end up having a blocker, we can revisit and bump this task priority. Meanwhile, it is de facto no more a blocker of T119143, migration has been almost completed and jobs are running fine.

@hashar I'm sorry, but I can't work with that.

It passing doesn't mean it worked. Running an old npm version may mean certain hooks/tests never run in the first place, giving the false illusion of a pass.

An outdated npm also means script hooks may work differently. It may cause a build script to run twice instead of once (making the build significantly slower). It may also cause new hooks to be ignored as they didn't exist yet. For example, the js documentation generator (not migrated to Nodepool yet) would not work. It will appear to pass, but the output will be incomplete for a browser.

npm 1.4.21 is EOL, unsupported, and insecure (in general, but more so in combination with Node 4). This means we have no upstream support when we encounter bugs and other issues.

Over the past 2 years, we encountered countless bugs and performance issues with npm. Many of which I worked hard with upstream to prioritise getting fixed and subsequently upgrading our CI slaves. By downgrading us to a two year old unsupported npm release you undo all that work. It's a miracle none of the tests failed so far, but I assure you: This is a recipe for disaster.

It's nice that the services team already packaged Node, but that's no excuse to stop there. We should finish the job and package npm too. Or install it through puppet like I did for the persistent slaves, or write a shell script as part of the image build process that installs npm.

Until this is fixed, I strongly recommend all npm jobs be reverted back to the persistent slaves.

I have looked at Debian source package from https://anonscm.debian.org/git/collab-maint/nodejs.git branch master-lts and it pass the config flag --without-npm .. Ubuntu has a Debian package for npm which is fairly recent, maybe it can be published on our apt.wikimedia.org ?

Per @Krinkle earlier comment: Node 4.x ships with npm 2.x . Maybe we can get the Nodepool image to just npm install -g npm@~2 and that would be sufficient. I am pretty sure it is going to have bad interactions with the npm deb package though :(

I agree with @Krinkle here. All tests pass now is due to (1) most npm jobs running just linters for the time being; and (2) no complex npm scripts in extensions. But that's no excuse. For the CI use case, how about using nvm?

@mobrovac That might work. Nvm is most known for managing multiple node versions, but it's also a useful tool for easily installing standard Node tarballs (which include npm). It'll need to happen as part of the image creation process as running it within the job would be much too slow.

We'll need to make sure to tell nvm to set the version as default since we can't use nvms' subshell feature (Jenkins gives each execution step a fresh bash environments).

It would additionally be interesting to explore actual multi-version node (nvm's primary feature) E.g. next time there is a Node LTS release we can add it alongside the current version in Nodepool base images and create experimental Jenkins jobs to select the newer version.

We can always use http://packages.ubuntu.com/xenial/npm which has a deb package. And since Ubuntu is based on debian it should work.

@hashar and @Krinkle what do you think of doing ^^

Or we can use https://github.com/nodejs/node/tree/master/deps/npm and add it in the deps/npm folder.

Change 280974 had a related patch set uploaded (by Hashar):
contint: stop installing grunt-cli

https://gerrit.wikimedia.org/r/280974

Change 280980 had a related patch set uploaded (by Hashar):
dib: provision npm 2.x

https://gerrit.wikimedia.org/r/280980

Change 280974 merged by Rush:
contint: stop installing grunt-cli

https://gerrit.wikimedia.org/r/280974

Mentioned in SAL [2016-04-01T20:56:58Z] <hashar> Refreshing Nodepool snapshot to hopefully get npm 2.x installed T124474

Change 280980 merged by jenkins-bot:
dib: provision npm 2.x

https://gerrit.wikimedia.org/r/280980

Mentioned in SAL [2016-04-04T10:04:02Z] <hashar> integration: salt -v '*-slave*' cmd.run 'npm -g uninstall grunt-cli' | T124474

Mentioned in SAL [2016-04-04T10:06:01Z] <hashar> integration: salt -v '*-slave*' cmd.run 'rm /usr/local/bin/grunt; rm -fR /usr/local/lib/node_modules/grunt-cli' | T124474

Change 281427 had a related patch set uploaded (by Hashar):
Fix qunit jobs pointing to non existent 'grunt'

https://gerrit.wikimedia.org/r/281427

Change 281427 merged by jenkins-bot:
Fix qunit jobs pointing to non existent 'grunt'

https://gerrit.wikimedia.org/r/281427

In T124474#2164007, @Krinkle wrote:

It would additionally be interesting to explore actual multi-version node (nvm's primary feature) E.g. next time there is a Node LTS release we can add it alongside the current version in Nodepool base images and create experimental Jenkins jobs to select the newer version.

Indeed. Even better, we can start testing some ECMAScript 2015 features only available in Node.js 5, right now.