node packages should be installed upon startup
Closed, DeclinedPublic5 Estimated Story Points
Actions

Assigned To

None

Authored By

	STran
	Dec 20 2022, 12:54 PM

Description

Attempting to build and start security-api results in the web server constantly restarting itself because packages (mariadb, nodemon) are missing. These are defined in the package.json and afaik should have been installed as part of docker compose up. The current solution is to add npm install into the init.sh script. Please investigate (and fix as part of your investigation):

If it's expected that the packages are missing (preliminary research suggests something around node_modules ownership, deleted node_modules, etc etc but not confirmed)
What the best practice for getting these packages installed on web on startup are

AC:

running docker compose up results in the web and db servers successfully starting up

Related Objects

Mentioned In: T339325: "Duplicate entry '0' for key 'proxy'"
Mentioned Here: T325628: Investigate: Ensure there's a production-ready init process

Event Timeline

STran created this task.Dec 20 2022, 12:54 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 20 2022, 12:54 PM

Notes:

Need to confirm if this is the right fix and implement it
If this is not the best practice, go with the best practice

I think it should be okay to fix this script as-is for dev environments only since the script depends on nodemon existing and that's definitely dev-only. iirc there'll be a way to make a prod-ready init (see T325628: Investigate: Ensure there's a production-ready init process) and if there isn't the whole thing will have to be removed later anyway when we find that out.

Tchanders moved this task from AHaT Sprint 29: The Ikseongwan to AHaT Sprint 30: Avanton Gold Cone on the Anti-Harassment-Team board.Apr 25 2023, 1:03 PM

Tchanders edited projects, added Anti-Harassment-Team (AHaT Sprint 30: Avanton Gold Cone); removed Anti-Harassment-Team (AHaT Sprint 29: The Ikseongwan).

Tchanders claimed this task.May 8 2023, 6:37 PM

The current solution is to add npm install into the init.sh script.

This is already the case: https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia/security/security-api/+/refs/heads/master/init.sh#3

AC:

running docker compose up results in the web and db servers successfully starting up

This is already the case too - tested locally.

Given the above, and given T325626#8786823 is there any more to do here?

Can we do npm install && ./init.sh? Does docker-compose up actually run npm install?

In T325626#8851900, @Tchanders wrote:

Can we do npm install && ./init.sh? Does docker-compose up actually run npm install?

docker-compose.yml says that init.sh is the entrypoint, and that runs npm install. So I think this task is done?

For production, the docker image that we build every time a patch is merged should include the installed npm package contents.

ARamirez_WMF edited projects, added Anti-Harassment-Team (AHaT Sprint 31 - The Fez); removed Anti-Harassment-Team (AHaT Sprint 30: Avanton Gold Cone).Jun 6 2023, 5:14 PM

ARamirez_WMF moved this task from Ready 🎬 (ONLY IF YOU HAVE NO MORE CODE TO REVIEW) to In Progress 💪 on the Anti-Harassment-Team (AHaT Sprint 31 - The Fez) board.

Niharika moved this task from AHaT Sprint 31 - The Fez to AHaT Sprint 32 - Baseball Cap on the Anti-Harassment-Team board.Jun 20 2023, 5:56 PM

Niharika edited projects, added Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap); removed Anti-Harassment-Team (AHaT Sprint 31 - The Fez).

Niharika moved this task from Ready 🎬 (ONLY IF YOU HAVE NO MORE CODE TO REVIEW) to In Progress 💪 on the Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap) board.

Tchanders removed Tchanders as the assignee of this task.Jun 21 2023, 12:54 PM

Tchanders moved this task from In Progress 💪 to Ready 🎬 (ONLY IF YOU HAVE NO MORE CODE TO REVIEW) on the Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap) board.

Tchanders subscribed.

When I ran docker compose up, npm install ran. The DB installed, however, I the "web" container then failed and wouldn't start up. This was due to the following error:

2023-07-20 11:59:06 npm ERR! code EACCES
2023-07-20 11:59:06 npm ERR! syscall mkdir
2023-07-20 11:59:06 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:06 npm ERR! errno -13
2023-07-20 11:59:06 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:06 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:06 npm ERR!   errno: -13,
2023-07-20 11:59:06 npm ERR!   code: 'EACCES',
2023-07-20 11:59:06 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:06 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:06 npm ERR! }
2023-07-20 11:59:06 npm ERR! 
2023-07-20 11:59:06 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:06 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:06 npm ERR! 
2023-07-20 11:59:06 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:06 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:06 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:06 
2023-07-20 11:59:06 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:06 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_05_755Z-debug-0.log
2023-07-20 11:59:06 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:06 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:06 /tmp/watch-64e436c6.sh: 1: nodemon: not found
2023-07-20 11:59:08 npm ERR! code EACCES
2023-07-20 11:59:08 npm ERR! syscall mkdir
2023-07-20 11:59:08 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:08 npm ERR! errno -13
2023-07-20 11:59:08 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:08 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:08 npm ERR!   errno: -13,
2023-07-20 11:59:06 
2023-07-20 11:59:06 > ipoid@0.1.0 watch
2023-07-20 11:59:06 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:06 
2023-07-20 11:59:08 
2023-07-20 11:59:08 > ipoid@0.1.0 watch
2023-07-20 11:59:08 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:08 
2023-07-20 11:59:10 
2023-07-20 11:59:10 > ipoid@0.1.0 watch
2023-07-20 11:59:10 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:10 
2023-07-20 11:59:12 
2023-07-20 11:59:12 > ipoid@0.1.0 watch
2023-07-20 11:59:12 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:12 
2023-07-20 11:59:14 
2023-07-20 11:59:14 > ipoid@0.1.0 watch
2023-07-20 11:59:14 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:14 
2023-07-20 11:59:08 npm ERR!   code: 'EACCES',
2023-07-20 11:59:08 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:08 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:08 npm ERR! }
2023-07-20 11:59:08 npm ERR! 
2023-07-20 11:59:08 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:08 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:08 npm ERR! 
2023-07-20 11:59:08 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:08 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:08 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:08 
2023-07-20 11:59:08 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:08 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_07_715Z-debug-0.log
2023-07-20 11:59:08 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:08 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:08 /tmp/watch-2a8153f9.sh: 1: nodemon: not found
2023-07-20 11:59:10 npm ERR! code EACCES
2023-07-20 11:59:10 npm ERR! syscall mkdir
2023-07-20 11:59:10 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:10 npm ERR! errno -13
2023-07-20 11:59:10 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:10 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:10 npm ERR!   errno: -13,
2023-07-20 11:59:10 npm ERR!   code: 'EACCES',
2023-07-20 11:59:10 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:10 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:10 npm ERR! }
2023-07-20 11:59:10 npm ERR! 
2023-07-20 11:59:10 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:10 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:10 npm ERR! 
2023-07-20 11:59:10 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:10 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:10 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:10 
2023-07-20 11:59:10 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:10 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_09_676Z-debug-0.log
2023-07-20 11:59:10 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:10 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:10 /tmp/watch-038b419c.sh: 1: nodemon: not found
2023-07-20 11:59:12 npm ERR! code EACCES
2023-07-20 11:59:12 npm ERR! syscall mkdir
2023-07-20 11:59:12 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:12 npm ERR! errno -13
2023-07-20 11:59:12 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:12 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:12 npm ERR!   errno: -13,
2023-07-20 11:59:12 npm ERR!   code: 'EACCES',
2023-07-20 11:59:12 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:12 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:12 npm ERR! }
2023-07-20 11:59:12 npm ERR! 
2023-07-20 11:59:12 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:12 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:12 npm ERR! 
2023-07-20 11:59:12 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:12 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:12 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:12 
2023-07-20 11:59:12 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:12 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_11_659Z-debug-0.log
2023-07-20 11:59:12 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:12 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:12 /tmp/watch-f87887db.sh: 1: nodemon: not found
2023-07-20 11:59:14 npm ERR! code EACCES
2023-07-20 11:59:14 npm ERR! syscall mkdir
2023-07-20 11:59:14 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:14 npm ERR! errno -13
2023-07-20 11:59:14 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:14 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:14 npm ERR!   errno: -13,
2023-07-20 11:59:14 npm ERR!   code: 'EACCES',
2023-07-20 11:59:14 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:14 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:14 npm ERR! }
2023-07-20 11:59:14 npm ERR! 
2023-07-20 11:59:14 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:14 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:14 npm ERR! 
2023-07-20 11:59:14 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:14 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:14 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:14 
2023-07-20 11:59:14 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:14 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_14_074Z-debug-0.log
2023-07-20 11:59:14 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:14 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:14 /tmp/watch-6f35e4d3.sh: 1: nodemon: not found
2023-07-20 11:59:17 npm ERR! code EACCES
2023-07-20 11:59:17 npm ERR! syscall mkdir
2023-07-20 11:59:17 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:17 npm ERR! errno -13
2023-07-20 11:59:17 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:17 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:17 npm ERR!   errno: -13,
2023-07-20 11:59:17 npm ERR!   code: 'EACCES',
2023-07-20 11:59:17 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:17 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:17 npm ERR! }
2023-07-20 11:59:17 npm ERR! 
2023-07-20 11:59:17 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:17 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:17 npm ERR! 
2023-07-20 11:59:17 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:17 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:17 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:17 
2023-07-20 11:59:17 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:17 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_17_222Z-debug-0.log
2023-07-20 11:59:17 
2023-07-20 11:59:17 > ipoid@0.1.0 watch
2023-07-20 11:59:17 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:17 
2023-07-20 11:59:22 
2023-07-20 11:59:22 > ipoid@0.1.0 watch
2023-07-20 11:59:22 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:22 
2023-07-20 11:59:17 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:17 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:17 /tmp/watch-d86cf956.sh: 1: nodemon: not found
2023-07-20 11:59:22 npm ERR! code EACCES
2023-07-20 11:59:22 npm ERR! syscall mkdir
2023-07-20 11:59:22 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:22 npm ERR! errno -13
2023-07-20 11:59:22 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:22 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:22 npm ERR!   errno: -13,
2023-07-20 11:59:22 npm ERR!   code: 'EACCES',
2023-07-20 11:59:22 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:22 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:22 npm ERR! }
2023-07-20 11:59:22 npm ERR! 
2023-07-20 11:59:22 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:22 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:22 npm ERR! 
2023-07-20 11:59:22 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:22 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:22 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:22 
2023-07-20 11:59:22 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:22 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_21_860Z-debug-0.log
2023-07-20 11:59:22 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:22 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:22 /tmp/watch-953d3ed6.sh: 1: nodemon: not found
2023-07-20 11:59:30 npm ERR! code EACCES
2023-07-20 11:59:30 npm ERR! syscall mkdir
2023-07-20 11:59:30 npm ERR! path /srv/service/node_modules
2023-07-20 11:59:30 npm ERR! errno -13
2023-07-20 11:59:30 npm ERR! Error: EACCES: permission denied, mkdir '/srv/service/node_modules'
2023-07-20 11:59:30 npm ERR!  [Error: EACCES: permission denied, mkdir '/srv/service/node_modules'] {
2023-07-20 11:59:30 npm ERR!   errno: -13,
2023-07-20 11:59:30 npm ERR!   code: 'EACCES',
2023-07-20 11:59:30 npm ERR!   syscall: 'mkdir',
2023-07-20 11:59:30 npm ERR!   path: '/srv/service/node_modules'
2023-07-20 11:59:30 npm ERR! }
2023-07-20 11:59:30 npm ERR! 
2023-07-20 11:59:30 npm ERR! The operation was rejected by your operating system.
2023-07-20 11:59:30 npm ERR! It is likely you do not have the permissions to access this file as the current user
2023-07-20 11:59:30 npm ERR! 
2023-07-20 11:59:30 npm ERR! If you believe this might be a permissions issue, please double-check the
2023-07-20 11:59:30 npm ERR! permissions of the file and its containing directories, or try running
2023-07-20 11:59:30 npm ERR! the command again as root/Administrator.
2023-07-20 11:59:30 
2023-07-20 11:59:30 npm ERR! A complete log of this run can be found in:
2023-07-20 11:59:30 npm ERR!     /home/runuser/.npm/_logs/2023-07-20T10_59_29_842Z-debug-0.log
2023-07-20 11:59:30 wait-for-it.sh: waiting 30 seconds for host.docker.internal:3308
2023-07-20 11:59:30 wait-for-it.sh: host.docker.internal:3308 is available after 0 seconds
2023-07-20 11:59:30 
2023-07-20 11:59:30 > ipoid@0.1.0 watch
2023-07-20 11:59:30 > nodemon server.js --signal SIGTERM
2023-07-20 11:59:30 
2023-07-20 11:59:30 /tmp/watch-6709990f.sh: 1: nodemon: not found

I use Docker Desktop on Windows and manage the containers through WSL. I cannot access the associated log file as the web server keeps on rebooting.

This is now theoretically reproducible. @Dreamy_Jazz was able to reproduce the issue but I apparently still can't. You can try to recreate this problem by deleting node_modules and trying to docker compose up. You should get a similar error since runuser doesn't have permission to create the folder. It's hacked by manually creating node_modules with 777 permissions. The problem appears to be in local.Dockerfile:

RUN (getent group "65533" || groupadd -o -g "65533" -r "somebody") && (getent passwd "65533" || useradd -l -o -m -d "/home/somebody" -r -g "65533" -u "65533" "somebody") && mkdir -p "/srv/service" && chown "65533":"65533" "/srv/service" && mkdir -p "/opt/lib" && chown "65533":"65533" "/opt/lib"
RUN (getent group "900" || groupadd -o -g "900" -r "runuser") && (getent passwd "900" || useradd -l -o -m -d "/home/runuser" -r -g "900" -u "900" "runuser")

Here somebody is setting the ownership on the folder to themself and whatever the permission is, it doesn't seem like runuser can then perform any actions on or within those folders. Running as root or changing the permissions will subsequently fix followup runs since node_modules will exist in the repo.

• AGueyte mentioned this in T339325: "Duplicate entry '0' for key 'proxy'".Aug 14 2023, 3:09 PM

According to @STran, this has an easy manual solution and only affects local, not production, so we will decline this. Anyone who wants to fix, please feel free to reopen and do so!

STran moved this task from Backlog to Done on the iPoid-Service board.Oct 4 2023, 6:28 AM

node packages should be installed upon startupClosed, DeclinedPublic5 Estimated Story PointsActions

Description

Related Objects

Event Timeline

node packages should be installed upon startup
Closed, DeclinedPublic5 Estimated Story Points
Actions