Page MenuHomePhabricator

Fresh lacks an in container npm cache
Open, Needs TriagePublic

Description

While inside a Fresh container, I could go doing several npm ci commands which wipe the node_modules and redownload dependencies. That could benefit a cache to be setup inside the container to avoid redownloading large dependencies.

I noticed that on performance/fresnel which depends on @mdn/browser-compat-data which is apparently 12 MBytes.

Event Timeline

It inherits XDG_CACHE_HOME from the CI images and npm does use /cache as an incontainer cache. Of course the cache vanishes when the container is terminated, so maybe the cache could be stored inside the current working directory (eg .fresh-cache ).

That could be a shared cache at XDG_CACHE_HOME/fresh / $HOME/.cache/fresh, then I guess it could be poisoned? Else it can be namespaced by the current working directory.

Based on the title and task description, I believe you're asking for an in-container cache. However, as far as I know, we allow npm to cache within a given session already, such that repeat commands etc do not need to re-download things.

I generally encourage long-running sessions, treating Fresh similar to how you might treat ssh to vagrant or ssh to production. This way makes it easy to iterate on a patch and periodically re-running a lint or unit test etc. It also avoids accidentally running commands outside the container as you won't mix the same Terminal tab between Fresh and non-Fresh. E.g. when working on repo X, I'll have two tabs one for Fresh and one for git/misc stuff.

When working on a project, I keep that open for quite a while, sometimes multiple days. It's pretty light/cheap in practice. I use npm install locally so that the current directory functions as a persistent cache between sessions as well. In current versions of npm, npm-install behaves very close to npm-ci with the exception that npm-ci distrusts the current working directory and re-populates it from central cache or network. I'm not sure that this is needed for local development, especially if we are considering storing the cache in that same working directory.

That could be a shared cache at XDG_CACHE_HOME/fresh / $HOME/.cache/fresh, then I guess it could be poisoned? Else it can be namespaced by the current working directory.

Yeah, I've intentionally not shared it in the past as that makes it no longer an isolated environment where I can trust it to be able to run on a untrusted code and not affect other projects I work on or contribute to. Namespacing the cache would bring up issues of gargage collection, and corruption/concurrent writes. I've personally did not that that worth the trade-off to invest in. (And I'd rather not introduce bugs or disk space regressions by releasing a version without those mitigations in place). So the speed trade-off has felt to me like the worthwhile choice, especially when combined with long-running sessions and using npm-install, you basically get a persistent per-project cache for free already.

There's some best practices / documentation issue in here for sure though. It might be worth polling how people use Fresh. E.g. do they keep long-running sesions like me? If not, why not? Would they benefit from it? Are there reasons not to? Do they use npm-ci? Why/Why not? Would they benefit from using npm-install instead?