This doesn't happen all that often, but occasionally projects will have multiple, nested package.json / package-lock.json files (sub-apps, if you will). For our tools which are checking against various node/npm vulnerabilities, which reference these files, we need to a way to search for all of them, iterate over them and run various security tooling for each individual file. This likely needs to be no more complicated than the something like this loop within the ci template yaml linter. We will also want to confirm that this only: changes: directive matching pattern will trigger on changes to nested package.json / package-lock.json files.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | sbassett | T289290 Design and Build Application Security Pipeline Components for Gitlab | |||
| Resolved | brennen | T289292 Create Security Team group within gitlab.wikimedia.org | |||
| Resolved | sbassett | T289293 Create initial proof of concept application security pipeline repository | |||
| Resolved | sbassett | T296806 Find optimal solution for projects with nested package.json files (within the context of Gitlab CI) |
Event Timeline
Comment Actions
Update:
- Confirmed via this commit and this commit that the logic currently employed via the only: changes: directives within some of our current ci templates does actually trigger on changes to both root-level and nested package.json and package-lock.json files. See respective pipeline triggers here and here. Though both pipelines did not complete due to a lack of runners being available right now, see also: T294050#7557027.
- As a basic proof-of-concept, using audit-ci as an example, this seems to solve this problem. The glob could stand some optimization, I'd imagine, but it works fine:
#!/bin/bash root_dir=$(pwd) file_list=$(git diff-tree --no-commit-id --name-only -r $CI_COMMIT_SHA -- \ {*package.json,*package-lock.json,*/package.json,*/package-lock.json}) for f in $file_list; do npm_dir="$(dirname "$f")" cd $root_dir/$npm_dir npm install audit-ci --moderate=true --skip-dev=true --report=true done
Comment Actions
hypothetical future gitlabbot: https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/commit/858a5865bb76658ba939d160cdd2e610bcaa85cf
Anyhow, the logic for this is done and can now be found within the current auditjs, npm audit and npm outdated ci templates. It ended up looking like this.
And here's an example of a ci config with all three of the node/npm templates configured, a pipeline run against two package.json files with known vulnerabilities, and the output for each tool: auditjs, npm audit and npm outdated. (note that npm outdated is allowed to fail as it is purely informational reporting).
So calling this done, for now.