Page MenuHomePhabricator

node packages should be installed upon startup
Closed, DeclinedPublic5 Estimated Story Points

Description

Attempting to build and start security-api results in the web server constantly restarting itself because packages (mariadb, nodemon) are missing. These are defined in the package.json and afaik should have been installed as part of docker compose up. The current solution is to add npm install into the init.sh script. Please investigate (and fix as part of your investigation):

  • If it's expected that the packages are missing (preliminary research suggests something around node_modules ownership, deleted node_modules, etc etc but not confirmed)
  • What the best practice for getting these packages installed on web on startup are

AC:

  • running docker compose up results in the web and db servers successfully starting up

Event Timeline

Niharika set the point value for this task to 5.Dec 20 2022, 5:14 PM
Niharika subscribed.

Notes:

  • Need to confirm if this is the right fix and implement it
  • If this is not the best practice, go with the best practice

I think it should be okay to fix this script as-is for dev environments only since the script depends on nodemon existing and that's definitely dev-only. iirc there'll be a way to make a prod-ready init (see T325628: Investigate: Ensure there's a production-ready init process) and if there isn't the whole thing will have to be removed later anyway when we find that out.

The current solution is to add npm install into the init.sh script.

This is already the case: https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia/security/security-api/+/refs/heads/master/init.sh#3

AC:

  • running docker compose up results in the web and db servers successfully starting up

This is already the case too - tested locally.

Given the above, and given T325626#8786823 is there any more to do here?

Can we do npm install && ./init.sh? Does docker-compose up actually run npm install?

Can we do npm install && ./init.sh? Does docker-compose up actually run npm install?

docker-compose.yml says that init.sh is the entrypoint, and that runs npm install. So I think this task is done?

For production, the docker image that we build every time a patch is merged should include the installed npm package contents.

When I ran docker compose up, npm install ran. The DB installed, however, I the "web" container then failed and wouldn't start up. This was due to the following error:

2023-07-20 11:59:06 npm ERR! code EACCES
2023-07-20 11:59:06 npm ERR! syscall mkdir
2023-07-20 11:59:06 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:06 npm ERR! errno -13
2023-07-20 11:59:06 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:06 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:06 npm ERR!   errno: -13,
2023-07-20 11:59:06 npm ERR!   code: 'EACCES',
2023-07-20 11:59:06 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:06 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:06 npm ERR! }
2023-07-20 11:59:06 npm ERR! 
2023-07-20 11:59:06 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:06 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:06 npm ERR! 
2023-07-20 11:59:06 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:06 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:06 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:06 
2023-07-20 11:59:06 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:06 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_05_755Z-debug-0.log
2023-07-20 11:59:06 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:06 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:06 /tmp/watch-64e436c6.sh: 1: nodemon: not found
2023-07-20 11:59:08 npm ERR! code EACCES
2023-07-20 11:59:08 npm ERR! syscall mkdir
2023-07-20 11:59:08 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:08 npm ERR! errno -13
2023-07-20 11:59:08 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:08 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:08 npm ERR!   errno: -13,
2023-07-20 11:59:06 
2023-07-20 11:59:06 > ipoid@0.1.0 watch
2023-07-20 11:59:06 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:06 
2023-07-20 11:59:08 
2023-07-20 11:59:08 > ipoid@0.1.0 watch
2023-07-20 11:59:08 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:08 
2023-07-20 11:59:10 
2023-07-20 11:59:10 > ipoid@0.1.0 watch
2023-07-20 11:59:10 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:10 
2023-07-20 11:59:12 
2023-07-20 11:59:12 > ipoid@0.1.0 watch
2023-07-20 11:59:12 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:12 
2023-07-20 11:59:14 
2023-07-20 11:59:14 > ipoid@0.1.0 watch
2023-07-20 11:59:14 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:14 
2023-07-20 11:59:08 npm ERR!   code: 'EACCES',
2023-07-20 11:59:08 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:08 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:08 npm ERR! }
2023-07-20 11:59:08 npm ERR! 
2023-07-20 11:59:08 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:08 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:08 npm ERR! 
2023-07-20 11:59:08 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:08 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:08 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:08 
2023-07-20 11:59:08 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:08 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_07_715Z-debug-0.log
2023-07-20 11:59:08 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:08 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:08 /tmp/watch-2a8153f9.sh: 1: nodemon: not found
2023-07-20 11:59:10 npm ERR! code EACCES
2023-07-20 11:59:10 npm ERR! syscall mkdir
2023-07-20 11:59:10 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:10 npm ERR! errno -13
2023-07-20 11:59:10 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:10 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:10 npm ERR!   errno: -13,
2023-07-20 11:59:10 npm ERR!   code: 'EACCES',
2023-07-20 11:59:10 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:10 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:10 npm ERR! }
2023-07-20 11:59:10 npm ERR! 
2023-07-20 11:59:10 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:10 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:10 npm ERR! 
2023-07-20 11:59:10 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:10 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:10 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:10 
2023-07-20 11:59:10 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:10 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_09_676Z-debug-0.log
2023-07-20 11:59:10 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:10 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:10 /tmp/watch-038b419c.sh: 1: nodemon: not found
2023-07-20 11:59:12 npm ERR! code EACCES
2023-07-20 11:59:12 npm ERR! syscall mkdir
2023-07-20 11:59:12 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:12 npm ERR! errno -13
2023-07-20 11:59:12 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:12 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:12 npm ERR!   errno: -13,
2023-07-20 11:59:12 npm ERR!   code: 'EACCES',
2023-07-20 11:59:12 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:12 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:12 npm ERR! }
2023-07-20 11:59:12 npm ERR! 
2023-07-20 11:59:12 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:12 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:12 npm ERR! 
2023-07-20 11:59:12 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:12 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:12 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:12 
2023-07-20 11:59:12 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:12 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_11_659Z-debug-0.log
2023-07-20 11:59:12 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:12 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:12 /tmp/watch-f87887db.sh: 1: nodemon: not found
2023-07-20 11:59:14 npm ERR! code EACCES
2023-07-20 11:59:14 npm ERR! syscall mkdir
2023-07-20 11:59:14 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:14 npm ERR! errno -13
2023-07-20 11:59:14 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:14 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:14 npm ERR!   errno: -13,
2023-07-20 11:59:14 npm ERR!   code: 'EACCES',
2023-07-20 11:59:14 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:14 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:14 npm ERR! }
2023-07-20 11:59:14 npm ERR! 
2023-07-20 11:59:14 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:14 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:14 npm ERR! 
2023-07-20 11:59:14 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:14 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:14 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:14 
2023-07-20 11:59:14 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:14 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_14_074Z-debug-0.log
2023-07-20 11:59:14 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:14 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:14 /tmp/watch-6f35e4d3.sh: 1: nodemon: not found
2023-07-20 11:59:17 npm ERR! code EACCES
2023-07-20 11:59:17 npm ERR! syscall mkdir
2023-07-20 11:59:17 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:17 npm ERR! errno -13
2023-07-20 11:59:17 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:17 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:17 npm ERR!   errno: -13,
2023-07-20 11:59:17 npm ERR!   code: 'EACCES',
2023-07-20 11:59:17 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:17 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:17 npm ERR! }
2023-07-20 11:59:17 npm ERR! 
2023-07-20 11:59:17 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:17 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:17 npm ERR! 
2023-07-20 11:59:17 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:17 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:17 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:17 
2023-07-20 11:59:17 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:17 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_17_222Z-debug-0.log
2023-07-20 11:59:17 
2023-07-20 11:59:17 > ipoid@0.1.0 watch
2023-07-20 11:59:17 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:17 
2023-07-20 11:59:22 
2023-07-20 11:59:22 > ipoid@0.1.0 watch
2023-07-20 11:59:22 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:22 
2023-07-20 11:59:17 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:17 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:17 /tmp/watch-d86cf956.sh: 1: nodemon: not found
2023-07-20 11:59:22 npm ERR! code EACCES
2023-07-20 11:59:22 npm ERR! syscall mkdir
2023-07-20 11:59:22 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:22 npm ERR! errno -13
2023-07-20 11:59:22 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:22 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:22 npm ERR!   errno: -13,
2023-07-20 11:59:22 npm ERR!   code: 'EACCES',
2023-07-20 11:59:22 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:22 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:22 npm ERR! }
2023-07-20 11:59:22 npm ERR! 
2023-07-20 11:59:22 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:22 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:22 npm ERR! 
2023-07-20 11:59:22 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:22 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:22 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:22 
2023-07-20 11:59:22 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:22 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_21_860Z-debug-0.log
2023-07-20 11:59:22 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:22 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:22 /tmp/watch-953d3ed6.sh: 1: nodemon: not found
2023-07-20 11:59:30 npm ERR! code EACCES
2023-07-20 11:59:30 npm ERR! syscall mkdir
2023-07-20 11:59:30 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:30 npm ERR! errno -13
2023-07-20 11:59:30 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:30 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:30 npm ERR!   errno: -13,
2023-07-20 11:59:30 npm ERR!   code: 'EACCES',
2023-07-20 11:59:30 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:30 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:30 npm ERR! }
2023-07-20 11:59:30 npm ERR! 
2023-07-20 11:59:30 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:30 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:30 npm ERR! 
2023-07-20 11:59:30 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:30 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:30 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:30 
2023-07-20 11:59:30 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:30 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_29_842Z-debug-0.log
2023-07-20 11:59:30 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:30 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:30 
2023-07-20 11:59:30 > ipoid@0.1.0 watch
2023-07-20 11:59:30 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:30 
2023-07-20 11:59:30 /tmp/watch-6709990f.sh: 1: nodemon: not found

I use Docker Desktop on Windows and manage the containers through WSL. I cannot access the associated log file as the web server keeps on rebooting.

This is now theoretically reproducible. @Dreamy_Jazz was able to reproduce the issue but I apparently still can't. You can try to recreate this problem by deleting node_modules and trying to docker compose up. You should get a similar error since runuser doesn't have permission to create the folder. It's hacked by manually creating node_modules with 777 permissions. The problem appears to be in local.Dockerfile:

RUN (getent group "65533" || groupadd -o -g "65533" -r "somebody") && (getent passwd "65533" || useradd -l -o -m -d "/home/somebody" -r -g "65533" -u "65533" "somebody") && mkdir -p "/srv/service" && chown "65533":"65533" "/srv/service" && mkdir -p "/opt/lib" && chown "65533":"65533" "/opt/lib"
RUN (getent group "900" || groupadd -o -g "900" -r "runuser") && (getent passwd "900" || useradd -l -o -m -d "/home/runuser" -r -g "900" -u "900" "runuser")

Here somebody is setting the ownership on the folder to themself and whatever the permission is, it doesn't seem like runuser can then perform any actions on or within those folders. Running as root or changing the permissions will subsequently fix followup runs since node_modules will exist in the repo.

According to @STran, this has an easy manual solution and only affects local, not production, so we will decline this. Anyone who wants to fix, please feel free to reopen and do so!