Page MenuHomePhabricator

newer npm for nodejs Kubernetes instances
Closed, ResolvedPublic

Description

I've been playing with hackmd and found several bugs installing npm packages that seem to be resolved by using a newer npm than the 1.4.21 version that is installed via our Debian Jessie packages.

A workaround is to install npm locally in a tool and add it to the path before the system version:

$ webservice --backend=kubernetes nodejs shell
$ cd $HOME
$ mkdir npm
$ cd $npm
$ npm install npm
$ echo 'export PATH=$HOME/npm/node_modules/.bin:$PATH' >> $HOME/.bashrc
$ exec bash
$ cd $HOME/www/js
$ npm install

Rather than trying to find a more modern npm version in a deb package, it may be easiest to add RUN npm install npm -g as part of the Dockerfile for our nodejs images.

Event Timeline

bd808 created this task.Jul 1 2017, 11:26 PM
bd808 edited projects, added Kubernetes; removed Toolforge.Jul 28 2017, 11:01 PM

FYI, npm hasn't made it to stretch, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=857986 . The package is orphaned unfortunately.

bd808 moved this task from Triage to Tools on the Cloud-Services board.Sep 14 2017, 5:19 AM
bd808 edited projects, added Toolforge; removed Cloud-Services, Tools-Kubernetes.

I think we should use the jessie packaged npm as the bootstrap to install a newer version of npm - this is what we do for CI.

Change 453666 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[operations/docker-images/toollabs-images@master] Update npm to 6.4.0

https://gerrit.wikimedia.org/r/453666

This is a big jump for npm (1.x -> 6.x), what kind of announcements should we have?

bd808 added a comment.Aug 29 2018, 7:56 PM

This is a big jump for npm (1.x -> 6.x), what kind of announcements should we have?

This is a good question, but I don't have a great answer for it. The webservice tool only gives access to the "latest" image, so as soon as it is published to the index restarts will pick it up. Since this is mostly a helper tool for the nodejs folks it is probably sufficient to send an email to cloud-announce after publishing. If you want to be really careful you could send a warning email a few days before.

This is a big jump for npm (1.x -> 6.x), what kind of announcements should we have?

This is a good question, but I don't have a great answer for it. The webservice tool only gives access to the "latest" image, so as soon as it is published to the index restarts will pick it up. Since this is mostly a helper tool for the nodejs folks it is probably sufficient to send an email to cloud-announce after publishing. If you want to be really careful you could send a warning email a few days before.

My patch leaves the old npm around still (just lower on in $PATH), so if somehow this does break something, people will still have a workaround. So I think we can deploy the current patch, announce to cloud-announce, wait a month for no breakage, and then remove the old npm from the image (apt remove npm), and maybe re-announce at that time if we think it's needed. Does that sound good?

bd808 added a comment.Aug 30 2018, 2:28 AM

My patch leaves the old npm around still (just lower on in $PATH), so if somehow this does break something, people will still have a workaround. So I think we can deploy the current patch, announce to cloud-announce, wait a month for no breakage, and then remove the old npm from the image (apt remove npm), and maybe re-announce at that time if we think it's needed. Does that sound good?

Sounds good to me. If it is easy to swap between npm versions the worst thing that will happen is people will not see the email and look to irc or the wiki for a fix.

Legoktm raised the priority of this task from Normal to High.Sep 4 2018, 7:56 PM
[11:42:22] <Krinkle> I'm working on tools.perflogbot which was set up a while back using a Kubernetes deployment without webservice, with a yaml file and using kubectl directly.
[11:42:31] <Krinkle> It uses the nodejs-base image.
[11:42:55] <Krinkle> I'm finding it impossible to re-install the bot because the image appears to have an invalid combination of Node.js and NPM.
[11:43:02] <Krinkle> It has Node.js 6.x with npm 1.4.
[11:48:16] <Krinkle> During the 'npm install' command (run from kubectl exec -it podname -- /bin/bash), it stops at some point due to a "Method Not Allowed" which is likely because the npmjs.org registry no longer supports the 2014 npm-cli client
[12:42:23] <Krinkle> I've documented a workaround at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes#nodejs for the time being

OK, so I drafted an email announcement at https://etherpad.wikimedia.org/p/toolforge-k8s-npm-upgrade. Please edit. The one thing I'm not sure about is what steps existing tool maintainers need to take - do they need to rm -rf node_modules and re-install? (my assumption) ...or will everything just work and they can safely npm install over an existing, older directory?

Second, I spotted an issue in webservice that would have broken with newer npm, writing a patch now.

And finally, is there an example/dummy nodejs tool that I can play with? If not, I'll write one.

Change 458629 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[operations/software/tools-webservice@master] Prefer using npm from /usr/local/bin/npm

https://gerrit.wikimedia.org/r/458629

Change 458629 merged by jenkins-bot:
[operations/software/tools-webservice@master] Prefer using npm from /usr/local/bin/npm

https://gerrit.wikimedia.org/r/458629

bd808 added a comment.May 18 2019, 9:35 AM

Thanks to T195103: Add Support for Node.js 10 LTS to Toolforge we now have npm 6.5.0 when using the Kubernetes backend and the node10 type:

$ webservice --backend=kubernetes node10 shell
$ npm --version
6.5.0

On the bastions and the grid engine nodes, we have npm 5.8:

$ npm --version
5.8.0
bd808 closed this task as Resolved.May 19 2019, 9:50 AM