Page MenuHomePhabricator

Fix application files/runtime permissions scheme
ClosedPublic

Authored by dduvall on Feb 22 2018, 12:20 AM.

Details

Maniphest Tasks
T187372: Blubber should implement a better file permissions convention
Reviewers
thcipriani
hashar
demon
Group Reviewers
Release-Engineering-Team
Commits
rGBLBR47526283fea7: Fix application files/runtime permissions scheme
Patch without arc
git checkout -b D984 && curl -L https://phabricator.wikimedia.org/D984?download=true | git apply
Summary

Introduces new lives configuration that provides the name/UID/GID of
the user that will own application files and installed dependencies.
This new configuration is distinct from runs in that the former
determines application file location ownership and the latter now only
determines runtime process ownership. Default configuration has also
been introduced for both config sections.

In addition to the new configuration, a new build.CopyAs instruction
has been introduced that ensures correct UID/GID ownership of files
copied into the container image, and all unqualified build.Copy
instructions are wrapped by the new build.CopyAs instruction using the
UID/GID appropriate for the current build phase. A new build.User
instruction is also introduced and injected into the build at the start
of certain phases to enforce ownership of build.Run processes.

This effective process/file ownership model is:

PhasePrivileged - "root"
PhasePrivilegedDropped - lives.as
PhasePreInstall - lives.as
PhaseInstall - lives.as
PhasePostInstall - runs.as

Fixes T187372

Test Plan

Run go test ./....

Diff Detail

Repository
rGBLBR Blubber
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

dduvall created this revision.Feb 22 2018, 12:20 AM
Restricted Application added a reviewer: Release-Engineering-Team. · View Herald TranscriptFeb 22 2018, 12:20 AM
Restricted Application added a project: Release-Engineering-Team. · View Herald Transcript
dduvall requested review of this revision.Feb 22 2018, 12:20 AM
dduvall updated this revision to Diff 2583.Feb 22 2018, 8:32 PM
dduvall retitled this revision from WIP Fix application files/runtime permissions scheme to Fix application files/runtime permissions scheme.
dduvall edited the summary of this revision. (Show Details)

Fixed lint errors, default config, and wrote a more coherent commit message

dduvall updated this revision to Diff 2584.Feb 22 2018, 8:35 PM

Fixed redundant USER root output

mmodell added a subscriber: mmodell.EditedFeb 26 2018, 5:49 PM
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestAptConfig
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestAptConfigInstructions
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestAptConfigValidation
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestArtifactsConfig
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestArtifactsConfigInstructions
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestArtifactsConfigValidation
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestCommonConfig
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestCommonConfigValidation
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestConfigValidation
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestFlagOverwrite
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestNodeConfig
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestNodeConfigInstructionsNoDependencies
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestNodeConfigInstructionsNonProduction
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestNodeConfigInstructionsProduction
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestNodeConfigInstructionsEnvironmentOnly
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestNodeConfigValidation
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestResolveIncludesPreventsInfiniteRecursion
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestMultiLevelIncludes
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestMultiIncludes
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestRunsConfig
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestRunsHomeWithUser
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestRunsHomeWithoutUser
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestRunsConfigInstructions
PASS   10ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestRunsConfigValidation
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestIsValidationError
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestVariantConfig
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestVariantDependencies
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestVariantLoops
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestVariantConfigInstructions
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::TestVariantConfigValidation
PASS   <1ms★  Go::Test::phabricator.wikimedia.org::source::blubber::config::ExampleResolveIncludes

Tests pass, 2 users are created, but I'm having a problem with dropped privileges using mathiod's blubber.yaml

FROM docker-registry.wikimedia.org/nodejs-devel
USER "root"
RUN apt-get update && apt-get install -y "librsvg2-2" "librsvg2-dev" "git" "python-pkgconfig" "build-essential" && rm -rf /var/lib/apt/lists/*
RUN groupadd -o -g "65533" -r "somebody" && useradd -o -m -d "/home/somebody" -r -g "somebody" -u "65533" "somebody" && mkdir -p "/srv/app" && chown "65533":"65533" "/srv/app" && mkdir -p "/opt/lib" && chown "65533":"65533" "/opt/lib"
RUN groupadd -o -g "666" -r "runuser" && useradd -o -m -d "/home/runuser" -r -g "runuser" -u "666" "runuser"
USER "somebody"
ENV HOME="/home/runuser"
ENV APP_BASE_PATH="/srv/service" LINK="g++"
WORKDIR /srv/app
COPY --chown=65533:65533 ["package.json", "/opt/lib"]
RUN cd "/opt/lib" && npm install
COPY --chown=65533:65533 [".", "."]
USER "runuser"
ENV NODE_ENV="" NODE_PATH="/opt/lib/node_modules" PATH="/opt/lib/node_modules/.bin:${PATH}"
ENTRYPOINT ["npm", "test"]
LABEL blubber.variant="test" blubber.version="0.2.0+c3fdb20"

Is the output dockerfile; however, there is a problem with the "somebody" user using the "/home/runuser" $HOME and then running npm install. I get:

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: librsvg@^0.7.0 (node_modules/librsvg):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Error: EACCES: permission denied, mkdir '/home/runuser/.npm'
npm ERR! Linux 4.14.0-3-amd64      
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "install"                             
npm ERR! node v6.12.0  
npm ERR! npm  v3.10.10                        
npm ERR! path /home/runuser/.npm               
npm ERR! code EACCES
npm ERR! errno -13                                                                                                                            
npm ERR! syscall mkdir                                                                                                                                                                                                                    
npm ERR! Error: EACCES: permission denied, mkdir '/home/runuser/.npm'                                       
npm ERR!     at Error (native)
npm ERR!  { Error: EACCES: permission denied, mkdir '/home/runuser/.npm'
npm ERR!     at Error (native)             
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',                           
npm ERR!   syscall: 'mkdir',    
npm ERR!   path: '/home/runuser/.npm',
npm ERR!   parent: 'mathoid' }
npm ERR!                                                                                   
npm ERR! Please try running this command again as root/Administrator.
                                                            
npm ERR! Please include the following file with any support request:
npm ERR!     /opt/lib/npm-debug.log

Is the output dockerfile; however, there is a problem with the "somebody" user using the "/home/runuser" $HOME and then running npm install. I get:

Ah ha! Yes, the setting of HOME for the current user will probably have to be moved into config/variant.go, alongside the setting of USER.

dduvall updated this revision to Diff 2603.Feb 28 2018, 8:23 PM

Moved home directory setting into config/variant.go alongside build.User instruction

thcipriani accepted this revision.Mar 5 2018, 4:46 PM

Works well, I like the macros. Took me a while to get my head around what was wrong with the current mathoid config. I wish there was a way to warn if there were extra fields in the yaml file as in the case of including an in field for runs.

config/user.go
8

maybe should be uint16

This revision is now accepted and ready to land.Mar 5 2018, 4:46 PM
dduvall added inline comments.Mar 5 2018, 9:21 PM
config/user.go
8

I think technically 32bit uids are, though not recommended, allowed.

thcipriani added inline comments.Mar 5 2018, 9:24 PM
config/user.go
8

true enough, seems to work on my computer.

Although uint64 is definitely too big:

sudo useradd -o -m -d "/home/reallytoohigh" -u "18446744073709551615" "reallytoohigh"

gives me:

useradd: invalid user ID '18446744073709551615'

Just a nitpick, feel free to ignore.

This revision was automatically updated to reflect the committed changes.