Hey @Jdforrester-WMF - did you need any assistance with this from the Security-Team? Patch review, poking maintainers, etc?

In T415590#11598133, @sbassett wrote:

Hey @Jdforrester-WMF - did you need any assistance with this from the Security-Team? Patch review, poking maintainers, etc?

I don't think so — @tchin, maybe we should do a release to make the security scanners happy?

@tchin - Any follow-up on this work on your end? Thanks.

Apologies, I’ve been out for the last few weeks. Will release once I’m back on a laptop

…ooor @Scott_French since you’re working on service-utils maybe you can try releasing it? The trigger_release pipeline in CI underneath the deployment step should automatically do a patch version bump without needing to change anything else

@tchin - Thanks for flagging. Sounds good, I'll take a look at the process tomorrow and follow up with you if I have any questions.

While reviewing the changes in this release for the CHANGELOG.md update, this turns out to be a bit more complicated than expected.

As alluded to in the MR desciption, c12 v2's switch to ESM-only will break CJS-based service-utils users whose version of node doesn't yet (transparently) support Loading ECMAScript modules using require() (while service-utils provides CJS dist artifacts for them to use, the require("c12") therein will fail with ERR_REQUIRE_ESM).

Spot-checking blubber.yaml files and referenced base images for the set of existing users, this would impact all three DPE services. Abstract Wikipedia services would be fine (nodejs22-slim runs a new-enough node).

In any case, I think this means:

1. We need major version bump for this release (in which case, I'll update the linked MR).
2. For DPE services, we'll first need to move those to a compatible version of node. Options include upgrading those to 22 as a vehicle for adopting the more-recent nodejs22-slim image (if that's something planned anyway) or updating the version of node in the nodejs20-slim image to at least v20.19.0.

@tchin @Jdforrester-WMF - Does that sound about right to you?

Bonus content: Why didn't we catch this in tests?

Even though the service-utils CI tests run on nodejs20-slim, they don't test the built artifacts: They're unit tests that (reasonably) test the source code directly. Meaning, it's all just typescript transpiled to ESM by vitest, and there's nothing in that path that tests CJS compatibility. I'll give some thought to how we can improve that.

There's already a request floating around somewhere in Phab (but after 10 minutes I still can't find it) to bump our prod Node 20 images to a later version, so doing that would be great if someone with packager rights can do it. IME Node 20->22 is pretty trivial, but I'd not want to insist on it for DPE, so if we can rev 20 that'd be great.

I'm not sure this is properly a major version change, as this feels like an accidental rather than intentional part of the feature/compatibility set, but in practice as it'll break the repos this code was originally built for (if not first used it), it's good to make a major version bump.

Thanks, @Jdforrester-WMF - I've adjusted the CHANGELOG.md update to reflect targeting v2.0.0. Also, agreed - this feels like odd case, since service-utils hasn't directly broken anything (e.g., via an incompatible API change), but on balance it seems like the clearest way to reflect that this can break (previously) working use cases.

There's already a request floating around somewhere in Phab (but after 10 minutes I still can't find it) to bump our prod Node 20 images to a later version, so doing that would be great if someone with packager rights can do it. IME Node 20->22 is pretty trivial, but I'd not want to insist on it for DPE, so if we can rev 20 that'd be great.

Hmmm ... Yeah, I can't seem to find it anywhere either. Thanks for looking!

I chatted with @MoritzMuehlenhoff earlier today, and it sounds like it should be possible to import a newer node 20 version into our apt component, to be picked up by nodejs20-slim. That should unblock moving this forward for the DPE use cases.

That said, Moritz rightly points out that node 20 is EOL in April, so migrating to 22 is something that will need to happen soon anyway :) (but indeed, that's best left up to DPE)

In T415590#11652702, @Scott_French wrote:

I chatted with @MoritzMuehlenhoff earlier today, and it sounds like it should be possible to import a newer node 20 version into our apt component, to be picked up by nodejs20-slim. That should unblock moving this forward for the DPE use cases.

I've just updated the repository component for node 20 to 20.20.0. This will get picked up by the weekly rebuild of images in the night from Sunday to Monday, but we can also trigger a manual rebuild to speed his up?

That said, Moritz rightly points out that node 20 is EOL in April, so migrating to 22 is something that will need to happen soon anyway :) (but indeed, that's best left up to DPE)

Node 20 will be end of life by 30 April, but given that a new LTS is out with Node 24 we should probably also add images for it. I'll take care of this, input at https://phabricator.wikimedia.org/T418440 welcome.

In T415590#11653063, @MoritzMuehlenhoff wrote:

In T415590#11652702, @Scott_French wrote:

I chatted with @MoritzMuehlenhoff earlier today, and it sounds like it should be possible to import a newer node 20 version into our apt component, to be picked up by nodejs20-slim. That should unblock moving this forward for the DPE use cases.

I've just updated the repository component for node 20 to 20.20.0.

Thank you!

This will get picked up by the weekly rebuild of images in the night from Sunday to Monday, but we can also trigger a manual rebuild to speed his up?

That said, Moritz rightly points out that node 20 is EOL in April, so migrating to 22 is something that will need to happen soon anyway :) (but indeed, that's best left up to DPE)

Node 20 will be end of life by 30 April, but given that a new LTS is out with Node 24 we should probably also add images for it. I'll take care of this, input at https://phabricator.wikimedia.org/T418440 welcome.

I'm not sure T393434: Migrate node-based services in production to node22 will get done by April, but I've created T418489: Migrate node-based services in production to node24 for keen teams once the base image exists. Thanks.

Awesome - thank you, Moritz!

v2.0.0 has now been published to the gitlab registry, so you should be good to upgrade the Abstract Wikipedia services, @Jdforrester-WMF. I'll follow up with you separately about OTel-related changes in this release (does not block your upgrading in the meantime, though).

cc: @Ahoelzl as essential work

Ah! great. Should we try eventstreams at node24 instead of T420257: Upgrade eventstreams and eventstreams-internal to node24 (or node22)? Any reason not to?

Oh! T420257#11716049, yes we should :)

Any update on this? Thanks.

In T415590#11861974, @sbassett wrote:

Any update on this? Thanks.

Hey all - any update here? Thanks.

@Ottomata any news on this?

DE will pick up work shortly on this. Thanks @sbassett and @Rsilvola for the ping and sorry for the delay.

Thanks all for getting this done. Any reason to keep this task private? I think most/all of these fixes were done publicly, just looking around in gerrit, etc.

I think it's fine to make it public now, if that's how it's usually done