Page MenuHomePhabricator

CI is broken for lots of MW repos due to extension/Popups explicitly requiring node 18
Closed, ResolvedPublic

Description

For reasons that I don't really fathom, the Popups extension has an explicit dependency on node 18.20.2. But CI has been updated to node 20 today (T343827), so any extension that depends on Popups and runs its browser test has broken CI now.

O tempora, o mores.

errors
npm error code 1
npm error path /workspace/src/extensions/Popups/node_modules/iltorb
npm error command failed
npm error command sh -c node ./scripts/install.js || node-gyp rebuild
npm error make: Entering directory '/workspace/src/extensions/Popups/node_modules/iltorb/build'
npm error   CC(target) Release/obj.target/iltorb/brotli/c/common/dictionary.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/common/transform.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/dec/bit_reader.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/dec/decode.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/dec/huffman.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/dec/state.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/backward_references.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/backward_references_hq.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/bit_cost.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/block_splitter.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/brotli_bit_stream.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/cluster.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/compress_fragment.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/compress_fragment_two_pass.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/dictionary_hash.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/encode.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/encoder_dict.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/entropy_encode.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/histogram.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/literal_cost.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/memory.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/metablock.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/static_dict.o
npm error   CC(target) Release/obj.target/iltorb/brotli/c/enc/utf8_util.o
npm error   CXX(target) Release/obj.target/iltorb/src/common/allocator.o
npm error make: Leaving directory '/workspace/src/extensions/Popups/node_modules/iltorb/build'
npm error info install installing standalone, skipping download.
npm error gyp info it worked if it ends with ok
npm error gyp info using node-gyp@10.1.0
npm error gyp info using node@20.16.0 | linux | x64
npm error gyp info find Python using Python version 3.7.3 found at "/usr/bin/python3"
npm error gyp http GET https://nodejs.org/download/release/v20.16.0/node-v20.16.0-headers.tar.gz
npm error gyp http 200 https://nodejs.org/download/release/v20.16.0/node-v20.16.0-headers.tar.gz
npm error gyp http GET https://nodejs.org/download/release/v20.16.0/SHASUMS256.txt
npm error gyp http 200 https://nodejs.org/download/release/v20.16.0/SHASUMS256.txt
npm error gyp info spawn /usr/bin/python3
npm error gyp info spawn args [
npm error gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm error gyp info spawn args 'binding.gyp',
npm error gyp info spawn args '-f',
npm error gyp info spawn args 'make',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/workspace/src/extensions/Popups/node_modules/iltorb/build/config.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/cache/node-gyp/20.16.0/include/node/common.gypi',
npm error gyp info spawn args '-Dlibrary=shared_library',
npm error gyp info spawn args '-Dvisibility=default',
npm error gyp info spawn args '-Dnode_root_dir=/cache/node-gyp/20.16.0',
npm error gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp',
npm error gyp info spawn args '-Dnode_lib_file=/cache/node-gyp/20.16.0/<(target_arch)/node.lib',
npm error gyp info spawn args '-Dmodule_root_dir=/workspace/src/extensions/Popups/node_modules/iltorb',
npm error gyp info spawn args '-Dnode_engine=v8',
npm error gyp info spawn args '--depth=.',
npm error gyp info spawn args '--no-parallel',
npm error gyp info spawn args '--generator-output',
npm error gyp info spawn args 'build',
npm error gyp info spawn args '-Goutput_dir=.'
npm error gyp info spawn args ]
npm error gyp info spawn make
npm error gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm error In file included from ../../nan/nan.h:178,
npm error                  from ../src/common/allocator.cc:1:
npm error ../../nan/nan_callbacks.h:55:23: error: ‘AccessorSignature’ is not a member of ‘v8’
npm error  typedef v8::Local<v8::AccessorSignature> Sig;
npm error                        ^~~~~~~~~~~~~~~~~
npm error ../../nan/nan_callbacks.h:55:23: note: suggested alternative: ‘Signature’
npm error  typedef v8::Local<v8::AccessorSignature> Sig;
npm error                        ^~~~~~~~~~~~~~~~~
npm error                        Signature
npm error ../../nan/nan_callbacks.h:55:40: error: template argument 1 is invalid
npm error  typedef v8::Local<v8::AccessorSignature> Sig;
npm error                                         ^
npm error In file included from ../src/common/allocator.cc:1:
npm error ../../nan/nan.h: In function ‘void Nan::SetAccessor(v8::Local<v8::ObjectTemplate>, v8::Local<v8::String>, Nan::GetterCallback, Nan::SetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, Nan::imp::Sig)’:
npm error ../../nan/nan.h:2551:16: error: no matching function for call to ‘v8::ObjectTemplate::SetAccessor(v8::Local<v8::String>&, void (*&)(v8::Local<v8::Name>, const v8::PropertyCallbackInfo<v8::Value>&), void (*&)(v8::Local<v8::Name>, v8::Local<v8::Value>, const v8::PropertyCallbackInfo<void>&), v8::Local<v8::Object>&, v8::AccessControl&, v8::PropertyAttribute&, Nan::imp::Sig&)’
npm error      , signature);
npm error                 ^
npm error In file included from /cache/node-gyp/20.16.0/include/node/v8-function.h:15,
npm error                  from /cache/node-gyp/20.16.0/include/node/v8.h:33,
npm error                  from /cache/node-gyp/20.16.0/include/node/node.h:73,
npm error                  from ../../nan/nan.h:58,
npm error                  from ../src/common/allocator.cc:1:
npm error /cache/node-gyp/20.16.0/include/node/v8-template.h:809:8: note: candidate: ‘void v8::ObjectTemplate::SetAccessor(v8::Local<v8::String>, v8::AccessorGetterCallback, v8::AccessorSetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, v8::SideEffectType, v8::SideEffectType)’
npm error    void SetAccessor(
npm error         ^~~~~~~~~~~
npm error /cache/node-gyp/20.16.0/include/node/v8-template.h:809:8: note:   no known conversion for argument 7 from ‘Nan::imp::Sig’ {aka ‘int’} to ‘v8::SideEffectType’
npm error /cache/node-gyp/20.16.0/include/node/v8-template.h:816:8: note: candidate: ‘void v8::ObjectTemplate::SetAccessor(v8::Local<v8::Name>, v8::AccessorNameGetterCallback, v8::AccessorNameSetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, v8::SideEffectType, v8::SideEffectType)’
npm error    void SetAccessor(
npm error         ^~~~~~~~~~~
npm error /cache/node-gyp/20.16.0/include/node/v8-template.h:816:8: note:   no known conversion for argument 7 from ‘Nan::imp::Sig’ {aka ‘int’} to ‘v8::SideEffectType’
npm error make: *** [iltorb.target.mk:147: Release/obj.target/iltorb/src/common/allocator.o] Error 1
npm error gyp ERR! build error 
npm error gyp ERR! stack Error: `make` failed with exit code: 2
npm error gyp ERR! stack at ChildProcess.<anonymous> (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:209:23)
npm error gyp ERR! System Linux 6.1.0-0.deb11.7-cloud-amd64
npm error gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm error gyp ERR! cwd /workspace/src/extensions/Popups/node_modules/iltorb
npm error gyp ERR! node -v v20.16.0
npm error gyp ERR! node-gyp -v v10.1.0
npm error gyp ERR! not ok
npm error A complete log of this run can be found in: /cache/npm/_logs/2024-12-16T10_28_42_800Z-debug-0.log

Event Timeline

For reasons that I don't really fathom

The explicit dependency was introduced in rEPOP04afdd4741474a6c193ac34c7e2badafdc66f9b8; the commit message says “From now on, we will all have to use the same node version to build assets.”

In the short term, I guess this “same node version” could just as well be 20, so simply changing the Node version in package.json should unblock the CI.

In the long term, is building assets still necessary, or could it be replaced by the packaged files support of ResourceLoader?

For reasons that I don't really fathom

The explicit dependency was introduced in rEPOP04afdd4741474a6c193ac34c7e2badafdc66f9b8; the commit message says “From now on, we will all have to use the same node version to build assets.”

In the short term, I guess this “same node version” could just as well be 20, so simply changing the Node version in package.json should unblock the CI.

Thank you, makes sense I guess. Also, on a closer look, the error seems to be due to an incompatible dependency, not the overall version mismatch. A day may come when I'll properly understand nodejs error messages, but it is not this day.

Still, the check-built-assets step, and the whole thing of assets being dependent on a specific node version, is highly inconvenient. For example, I am allergic to nodejs and I want a single version of it on my machine. I also don't want to install Node Virus Manager because I don't need it. I could use docker but haven't bothered thus far. My fault? Definitely. But I can live with it because it's usually fine for MW development. This extension (and MobileFrontend) being the exception. And it's not just me: updating node in CI becomes permanently blocked on updating the requirements of said extensions. And LibUp, for example, can't make patches because it uses a different version of node. This resulted in several outdated packages, and most likely missed security updates.

The current issue seems to be fixed by just updating all dependencies (npm update). I would make a patch, and maybe also a couple more patches to update a few direct dependencies, but I can't (/ don't want to) because of the node version thing.

In the long term, is building assets still necessary, or could it be replaced by the packaged files support of ResourceLoader?

+1. This is tracked in T315929 and it seems to be happening, which I'm really happy to see.

Change #1104596 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Popups@master] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104596

Change #1104597 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Popups@master] Clean up bundlesize config

https://gerrit.wikimedia.org/r/1104597

Change #1104596 merged by jenkins-bot:

[mediawiki/extensions/Popups@master] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104596

Can we backport this to supported release versions? It is causing failures such as this one for 1104357 (A REL1_42 change).

Change #1104629 had a related patch set uploaded (by Michael Große; author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Popups@wmf/1.44.0-wmf.6] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104629

Good point. Let's deal with them after the backport window.

Change #1104629 merged by jenkins-bot:

[mediawiki/extensions/Popups@wmf/1.44.0-wmf.6] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104629

Mentioned in SAL (#wikimedia-operations) [2024-12-16T15:04:23Z] <ladsgroup@deploy2002> Started scap sync-world: Backport for [[gerrit:1104629|Kick bundlesize out of package.json (T382192 T360590)]], [[gerrit:1104633|fix(surfacing): Show highlights in lists as well (T381841)]], [[gerrit:1104624|stats(surfacing): track link recommendation api recommendations (T378536)]]

Mentioned in SAL (#wikimedia-operations) [2024-12-16T15:09:08Z] <ladsgroup@deploy2002> migr, ladsgroup: Backport for [[gerrit:1104629|Kick bundlesize out of package.json (T382192 T360590)]], [[gerrit:1104633|fix(surfacing): Show highlights in lists as well (T381841)]], [[gerrit:1104624|stats(surfacing): track link recommendation api recommendations (T378536)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-12-16T15:15:54Z] <ladsgroup@deploy2002> Finished scap sync-world: Backport for [[gerrit:1104629|Kick bundlesize out of package.json (T382192 T360590)]], [[gerrit:1104633|fix(surfacing): Show highlights in lists as well (T381841)]], [[gerrit:1104624|stats(surfacing): track link recommendation api recommendations (T378536)]] (duration: 11m 30s)

Change #1104673 had a related patch set uploaded (by Michael Große; author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Popups@REL1_41] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104673

Change #1104675 had a related patch set uploaded (by Michael Große; author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Popups@REL1_42] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104675

Change #1104676 had a related patch set uploaded (by Michael Große; author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Popups@REL1_43] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104676

Change #1104673 merged by jenkins-bot:

[mediawiki/extensions/Popups@REL1_41] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104673

Change #1104679 had a related patch set uploaded (by Jforrester; author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Popups@REL1_39] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104679

Change #1104676 merged by jenkins-bot:

[mediawiki/extensions/Popups@REL1_43] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104676

Change #1104675 merged by jenkins-bot:

[mediawiki/extensions/Popups@REL1_42] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104675

Change #1104679 merged by jenkins-bot:

[mediawiki/extensions/Popups@REL1_39] Kick bundlesize out of package.json

https://gerrit.wikimedia.org/r/1104679

Michael lowered the priority of this task from Unbreak Now! to High.Mon, Dec 16, 4:24 PM
Michael added a subscriber: Jdforrester-WMF.

Can we backport this to supported release versions? It is causing failures such as this one for 1104357 (A REL1_42 change).

This should now also be fixed on the most recent four REL1_XX branches (in addition to master and -wmf.6). Lowering the priority, because this should no longer block anything, but leaving it open because some cleanup/follow-up work seems necessary (see the open change by Lucas).

Thanks to @Lucas_Werkmeister_WMDE and @Jdforrester-WMF !

Change #1104597 merged by jenkins-bot:

[mediawiki/extensions/Popups@master] Clean up bundlesize config

https://gerrit.wikimedia.org/r/1104597

Change #1104972 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Popups@REL1_43] Clean up bundlesize config

https://gerrit.wikimedia.org/r/1104972

Change #1104972 merged by jenkins-bot:

[mediawiki/extensions/Popups@REL1_43] Clean up bundlesize config

https://gerrit.wikimedia.org/r/1104972

I backported the cleanup to REL1_43; beyond that it doesn’t make sense to backport because Popups BundleSizeTest was only added a few months ago. This means that the previous fix, backported to all release branches, leaves those release branches (other than REL1_43) with no bundle size checking for resources/dist/index.js / ext.popups.main; I assume that’s fine, since no feature development should take place on those branches anyway.

I’ll leave the task open for Popups developers to review / verify, but as far as I’m concerned this is done.

Let's call it Resolved.

Change #1105005 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/UniversalLanguageSelector@master] bundlesize.config.json: Declare ext.uls.pt for wikis where it's loaded in articles, like Wikifunctions

https://gerrit.wikimedia.org/r/1105005