Page MenuHomePhabricator

Keep npm debug logs out of the castor cache
Open, Needs TriagePublic

Description

npm writes its debug logs to _logs inside the cache. Every build adds ~10 files and deletes ~10 (npm keeps the last 10), churn with no caching value that, together with the verify rewrite, stops 1289314 from skipping saves. Details: T435973#12288198.

Change: set logs-dir to an absolute path outside /cache in the global npmrc added by 1335785 (dockerfiles/node24|node26/npmrc), so one change covers Quibble, Wikibase sub-invocations and codehealth.

Once every job runs on the new images, purge the existing _logs from the namespaces on castor06 once. Load restores them, so excluding them from the save alone is not enough.

Event Timeline

Change #1338389 had a related patch set uploaded (by Mhurd; author: Mhurd):

[integration/config@master] dockerfiles: keep npm debug logs out of the castor cache

https://gerrit.wikimedia.org/r/1338389

Change #1338389 merged by jenkins-bot:

[integration/config@master] dockerfiles: keep npm debug logs out of the castor cache

https://gerrit.wikimedia.org/r/1338389

Change #1341713 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: update npm jobs to drop debug logs from cache

https://gerrit.wikimedia.org/r/1341713

Change #1341713 merged by jenkins-bot:

[integration/config@master] jjb: update npm jobs to drop debug logs from cache

https://gerrit.wikimedia.org/r/1341713

Change #1342041 had a related patch set uploaded (by Mhurd; author: Mhurd):

[integration/config@master] jjb: run extension browser tests without npm --prefix

https://gerrit.wikimedia.org/r/1342041

Four namespaces on castor06 kept receiving npm debug logs after this morning's switch: the Wikibase and GrowthExperiments browser-tests-only jobs. Their job definitions run npm --prefix extensions/<Ext> ..., and --prefix also moves npm's global config lookup to <prefix>/etc/npmrc, so the image's /usr/etc/npmrc is never read. The log header shows it loading extensions/GrowthExperiments/etc/npmrc instead. Reproduced in quibble-bookworm-php83:1.21.0-s1: with --prefix the log lands in <cache>/_logs, with cd extensions/<Ext> && it lands in /tmp/npm-logs.

Fix: run npm from the extension directory instead, https://gerrit.wikimedia.org/r/c/integration/config/+/1342041. The one-time cleanup of the existing _logs directories waits until that is deployed, or those four namespaces refill.

Change #1342041 abandoned by Mhurd:

[integration/config@master] jjb: run extension browser tests without npm --prefix

Reason:

Superseded by the NPM_CONFIG_USERCONFIG approach in Hashar's comment: moving the npmrc to npm's user config fixes --prefix everywhere, including Wikibase's own nested npm --prefix calls, which this change would not have covered. Follow-up under T435975.

https://gerrit.wikimedia.org/r/1342041

Abandoned https://gerrit.wikimedia.org/r/c/integration/config/+/1342041: @hashar explained that Wikibase's own package.json also uses npm --prefix, so the cd change would have missed those nested calls. The fix instead is to move the npmrc to npm's user config, which --prefix does not relocate, tracked under T435975.

Still open for that change reaching the images and jobs, then a one-time removal of the ~223 existing _logs directories on castor06.