The more general T267891: Create Debian packages for Node.js 14 upgrade was declined (and then worked around for CI), but as @Esanders says in T267890#7884480, node12 is EOL which means a bunch of our tools are losing availability. Can this be discussed more directly?
Description
Description
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | None | T267890 Upgrade all CI jobs for WMF-deployed projects from Node 12 to Node 14 LTS | |||
| Open | None | T306995 Migrate node-based services in production to node14 | |||
| Resolved | Joe | T306996 Provide node14 and node16 images for running production node-based services |
Event Timeline
Comment Actions
From https://nodejs.org/en/about/releases/:
| Release | Status | Initial Release | Active LTS Start | Maintenance LTS Start | End-of-life |
|---|---|---|---|---|---|
| v12 | Maintenance LTS | 2019-04-23 | 2019-10-21 | 2020-11-30 | 2022-04-30 |
| v14 | Maintenance LTS | 2020-04-21 | 2020-10-27 | 2021-10-19 | 2023-04-30 |
| v16 | Active LTS | 2021-04-20 | 2021-10-26 | 2022-10-18 | 2024-04-30 |
| v18 | Current | 2022-04-19 | 2022-10-25 | 2023-10-18 | 2025-04-30 |
@Joe's arguments from T267891: Create Debian packages for Node.js 14 upgrade that Debian will be providing critical security patches to nodejs v12 as long as their LTS OS versions are still using it are true. That reasoning misses that the nodejs community is much more proactive than say the PHP community in dropping support for EOL versions. Our nodejs codebases are also more dependent on 3rd party libraries than our PHP codebases are. Within days of an LTS reaching EOL major nodejs libraries will be looking to remove support for it from their releases.
Getting to v14 (EOL 2023-04-30) is necessary. I believe we should really be on v16 already (EOL 2024-04-30) and working out how we will migrate to v18 (LTS 2022-10-25) in Q2 of next fiscal (October-December 2022).
I would propose https://github.com/nodesource/distributions as the upstream for us to use to get deb packages from in the production blessed images. I do not have a strong opinion either way on if these should be mirrored on apt.wikimedia.org as separate components or if we can just pull directly from deb.nodesource.com.
Comment Actions
While Debian has e.g. 14 and 16 in various development branches none of those are going to be continously updated (e.g. 14 will be replaced by 16 in testing soon).
We can import the nodesource packages into separate repository components, e.g. thirdparty/node14 and thirdparty/node16. This way applications have the flexibility to follow the LTS series that works best for their stack. I can have a look at setting up the repo syncs in the next days.
Comment Actions
Within days of an LTS reaching EOL major nodejs libraries will be looking to remove support for it from their releases.
Indeed, and many don't even wait for the EOL date. I've seen several dependencies drop Node 12 support in the last few months citing the "upcoming EOL".
Comment Actions
We can import the nodesource packages into separate repository components, e.g. thirdparty/node14 and thirdparty/node16. This way applications have the flexibility to follow the LTS series that works best for their stack. I can have a look at setting up the repo syncs in the next days.
That'd be great if that's a supportable way forward for SRE.
Using the external nodesource debs has various downsides (we have no control over the maintenance, they link various components statically that properly patched to use the system libs in Debian and a few more), but we also don't have the resources available to maintain quality packages of multiple LTS branches in parallel (and we need to work on higher priority tasks), so it's a tradeoff/compromise to be made.
One important precondition is the lifetime, though. In the past many of our applications missed to migrate to a new Node LTS in time, but it wasn't as much of an issue since we simply backported fixes to an otherwise EOLed LTS branch. That will no longer be possible when using the external nodesource packages, so any production service using these base images needs to commit to follow supported LTS series (so that we can cull EOLed images from the registry).
I can take care of the repo syncs next week and then we can figure out next steps.
Comment Actions
Change 789592 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):
[operations/puppet@production] Enable repo sync for node14
Comment Actions
Change 789592 merged by Muehlenhoff:
[operations/puppet@production] Enable repo sync for node14
Comment Actions
I've added repo sync definitions for node 14 on Bullseye (and imported 14.19.2 into it). I suppose 16 is needed as well?
Comment Actions
Change 789776 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):
[operations/puppet@production] Enable repo sync for Node 16
Comment Actions
Change 789776 merged by Muehlenhoff:
[operations/puppet@production] Enable repo sync for Node 16
Comment Actions
I suppose 16 is needed as well?
Yes, please.
Ok, thirdparty/node16 now also contains Node 16.15
Comment Actions
Change 790264 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):
[operations/docker-images/production-images@master] Add node14, node16 images
Comment Actions
Change 790264 merged by Giuseppe Lavagetto:
[operations/docker-images/production-images@master] Add node14, node16 images
Comment Actions
I've build and published the nodejs14-slim and the nodejs16-slim images, using the nodejs package from the components.
One importaant thing to stress is that both images also include npm, so they can be used in development as well.
Comment Actions
I can't find these images in https://docker-registry.wikimedia.org/ yet. Is there a build and push step that still needs to be done to publish them?