Page MenuHomePhabricator

Should we expose some JENKINS_ environment variables in docker?
Closed, ResolvedPublic

Event Timeline

Legoktm created this task.Oct 7 2017, 8:10 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 7 2017, 8:10 AM
hashar added a subscriber: hashar.Oct 11 2017, 10:01 PM

https://wiki.jenkins.io/display/JENKINS/Building+a+software+project has a list of some Jenkins built-in variables.

The way variables are passed to the container right now is by generating a file on the build host and invokng docker run with --env-file .env:

jjb/macro-docker.yaml 
- builder:
    name: docker-zuul-env
    builders:
     - shell: |
        rm -rf .env

        cat <<ZUUL > .env
        ZUUL_URL=$ZUUL_URL
        ZUUL_PROJECT=$ZUUL_PROJECT
        ZUUL_COMMIT=$ZUUL_COMMIT
        ZUUL_REF=$ZUUL_REF
        HOME=/var/lib/jenkins
        ZUUL

        # Finally, output for debugging help
        cat .env

HOME should be dropped but it is used by the operations/puppet container.

Seems ZUUL_PIPELINE is missing among others.

So I had to add a BUNCH of zuul and other env vars to the docker-zuul-env macro for the phan jobs.

It now looks like this.

- builder:
    name: docker-zuul-env
    builders:
     - shell: |
        #!/bin/bash -e

        set -x

        rm -rf .env

        cat <<ZUUL > .env
        ZUUL_URL=$ZUUL_URL
        ZUUL_PROJECT=$ZUUL_PROJECT
        ZUUL_COMMIT=$ZUUL_COMMIT
        ZUUL_REF=$ZUUL_REF
        ZUUL_BRANCH=$ZUUL_BRANCH
        ZUUL_CHANGES=$ZUUL_CHANGES
        ZUUL_CHANGE_IDS=$ZUUL_CHANGE_IDS
        ZUUL_CHANGE=$ZUUL_CHANGE
        ZUUL_PATCHSET=$ZUUL_PATCHSET
        ZUUL_VOTING=$ZUUL_VOTING
        ZUUL_PIPELINE=$ZUUL_PIPELINE
        ZUUL_UUID=$ZUUL_UUID
        EXT_NAME=$EXT_NAME
        SKIN_NAME=$SKIN_NAME
        EXT_DEPENDENCIES=$EXT_DEPENDENCIES
        SKIN_DEPENDENCIES=$SKIN_DEPENDENCIES
        ZUUL

        # Finally, output for debugging help
        cat .env

An alternative would be to blindly pass the whole environment to the Docker container?

/usr/bin/env > .env

Change 385204 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Expose JENKINS_URL in Docker

https://gerrit.wikimedia.org/r/385204

Change 385204 merged by jenkins-bot:
[integration/config@master] Expose JENKINS_URL in Docker

https://gerrit.wikimedia.org/r/385204

Change 390432 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Pass env to docker run

https://gerrit.wikimedia.org/r/390432

hashar claimed this task.Nov 10 2017, 4:45 PM
hashar triaged this task as Normal priority.Nov 10 2017, 4:45 PM

Mentioned in SAL (#wikimedia-releng) [2017-11-20T15:03:21Z] <hashar> integration: pass all environment variables to the docker run commands | https://gerrit.wikimedia.org/r/#/c/390432/ | T177684

hashar closed this task as Resolved.Nov 20 2017, 3:03 PM

Change 390432 merged by jenkins-bot:
[integration/config@master] Pass env to docker run

https://gerrit.wikimedia.org/r/390432

We now pass the whole environment which includes JENKINS_* variables as well as parameters dynamically injected by Zuul.

hashar reopened this task as Open.Nov 20 2017, 8:10 PM

Mentioned in SAL (#wikimedia-operations) [2017-11-20T20:11:00Z] <hashar> CI docker jobs were all broken due to a mistake. Should be back now. T177684

exec docker run --rm --env-file /dev/fd/63 --volume /srv/jenkins-workspace/workspace/mwext-php70-phan-docker/src:/src --volume /srv/jenkins-workspace/workspace/mwext-php70-phan-docker/cache:/cache --volume /srv/git:/srv/git --entrypoint bash wmfreleng/ci-src-setup:v2017.10.17.09.17 /srv/setup-mwext.sh
/srv/setup-mwext.sh: line 9: SKIN_DEPENDENCIES: unbound variable

eek

bash variables are not available to sub programs (eg /usr/bin/env) unless they are marked with export :-\

Change 392632 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: fix ci-src-setup with unbound variables

https://gerrit.wikimedia.org/r/392632

Mentioned in SAL (#wikimedia-releng) [2017-11-21T13:02:28Z] <hashar> docker push wmfreleng/ci-src-setup:v2017.11.21.12.57 && docker push wmfreleng/ci-src-setup:latest | https://gerrit.wikimedia.org/r/392632 | T177684

Change 392633 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Pass env to docker run [2]

https://gerrit.wikimedia.org/r/392633

Change 392632 merged by jenkins-bot:
[integration/config@master] docker: fix ci-src-setup with unbound variables

https://gerrit.wikimedia.org/r/392632

At least the mwext-php70-phan-docker seems to work now.

hashar closed this task as Resolved.Nov 21 2017, 1:22 PM

Change 392633 merged by jenkins-bot:
[integration/config@master] Pass env to docker run [2]

https://gerrit.wikimedia.org/r/392633

Change 393071 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Do not pass env variables set by login to Docker

https://gerrit.wikimedia.org/r/393071

hashar reopened this task as Open.Nov 23 2017, 6:47 PM

Reopening since we pass HOME=/mnt/home/jenkins-deploy from the Docker host, PATH and other unwanted env variables. They are set by login and should be stripped.

https://gerrit.wikimedia.org/r/393071 solves it

Change 393071 merged by jenkins-bot:
[integration/config@master] Do not pass env variables set by login to Docker

https://gerrit.wikimedia.org/r/393071

hashar closed this task as Resolved.Nov 23 2017, 10:18 PM