Page MenuHomePhabricator

Upgrade latest docker-registry.wikimedia.org/nodejs-devel to stretch
Closed, ResolvedPublic

Description

It seems that docker-registry.wikimedia.org/nodejs-slim has been updated recently with a build based on Debian stretch, but docker-registry.wikimedia.org/nodejs-devel remains a jessie based image. This discrepancy breaks the multi-stage production Mathoid build.

In general, these kinds of major discrepancies will be problematic for the new release pipeline. We should discuss ideas between our teams to mitigate such issues, perhaps with more detailed tagging (e.g. nodejs-slim:stretch and nodejs-slim:jessie in addition to nodejs-slim:latest) or cascading updates in the base image build system.

Event Timeline

dduvall created this task.Nov 14 2017, 8:20 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 14 2017, 8:20 PM

@dduvall the reason this is happening right now is that stretch doesn't have a package for npm and I didn't get around to tackle that issue properly.

I did leave nodejs-devel in a non-upgraded state because no one was using it. I think we have two solutions in the brief term:

  • find a quick and clean way to install npm on stretch
  • revert nodejs-slim to jessie

@MoritzMuehlenhoff any idea on how to do the first option?

Current npm releases are not packaged in Debian since the list of dependencies exploded. Efforts are tracked in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=794890 and https://wiki.debian.org/Javascript/Nodejs/Tasks/npm

Possible options:

  • We build a custom npm 5 package which doesn't depend on the split up dependencies (needs some deeper look why is needs 50 packages to begin with)
  • WMF funds the updated packaging of npm mentioned in #794890: https://www.generosity.com/fundraising/update-npm-package-in-debian
  • Try try to build the jessie package on stretch (from what I can tell it was booted from stretch because it was deemed too old, maybe there are some modules which cannot be pulled with 1.4.21 any longer?)

Thanks for the update @Joe and @MoritzMuehlenhoff!

What do you think about maintaining separate tags based on Debian release? Or perhaps maintaining separate images by release?

The reason I suggest such a scheme is because I think there's a greater issue here with how we're expecting users of Blubber to specify both the base image and system packages, more specifically how these two configuration pieces are coupled; the latter are not likely to be consistent between Debian releases and so if a dist upgrade occurs for the base image, it necessitates a very cumbersome update to all apt: { packages: ... } references in blubber.conf files among project repos. However, if we can allow users to specify base images with tags based on Debian release, the issue is avoided and the coordination to upgrade smoothly is left to the user—where I'd argue it should be in our brave new container-based world. Of course, this would require Ops to maintain multiple branches of each base image and I'd also understand any reluctance to do so.

Joe added a comment.Nov 15 2017, 6:58 PM

@MoritzMuehlenhoff did you ever took a look at the deb packages that are officially distributed by node?

They are distributed from https://deb.nodesource.com and peeking at the "install script" here they do proper signing of the packages and all.

Would adding that repo to what we mirror locally work? Or are those packages known to be horrible/harmful/is there any other reason not to use them?

Yeah, I guess that would be an alternative to consider.

Joe triaged this task as High priority.Nov 29 2017, 9:15 PM
Joe added a project: User-Joe.
Joe moved this task from Backlog to Blocking others on the User-Joe board.
Joe moved this task from Blocking others to Doing on the User-Joe board.Dec 1 2017, 11:48 AM

Change 394550 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/docker-images/production-images@master] Re-add nodejs-devel

https://gerrit.wikimedia.org/r/394550

Change 394550 merged by Giuseppe Lavagetto:
[operations/docker-images/production-images@master] Re-add nodejs-devel

https://gerrit.wikimedia.org/r/394550

Joe added a comment.Dec 1 2017, 12:27 PM

A new version of the nodejs-devel image based on nodesource's own package for node 6 installed. It is tagged as 0.1.0 and (currently) latest.

Joe closed this task as Resolved.Dec 1 2017, 12:27 PM