Page MenuHomePhabricator

Allow `doc-publish` to be run without zuul dependency
Closed, ResolvedPublic


Currently, there is some difficulty getting to publish from differential/phab (T117431).

The main roadblock seems that $ZUUL_UUID is used as a sort of shared random number from the {name}-tox-doc-publish job template.

$ZUUL_UUID is used in the docs-publish job (in publish.yaml):

set -u
echo "Making sure that destination does not contain unset variables (like DOC_SUBPATH, see zuul/ {docdest}"
rsync --recursive "{docsrc}/" "rsync://$ZUUL_UUID"
echo "Publishing to{docdest}/

The $ZUUL_UUID is then passed and used by the publish-on-gallium job that does the actual publishing of docs:

set -u
echo "Publishing $ZUUL_PROJECT from labs to prod"
echo "Zuul UUID: $ZUUL_UUID"
echo "..."

if [ ! -d $LOCAL_DOCROOT ]; then
   echo "Error: Invalid docroot. Directory $LOCAL_DOCROOT must exist, aborting."
   exit 1
mkdir -p "$LOCAL_DEST"
rsync --recursive --delete-after "rsync://$ZUUL_UUID"/ "$LOCAL_DEST"

LOCAL_VHOST=$(echo $WMF_CI_PUB_DOCROOOT | awk -F'/' '{ print $3"."$2"."$1 }')
echo "Published to https://$LOCAL_VHOST/$WMF_CI_PUB_DEST/"

The $ZUUL_UUID (and to a more trivial extent $ZUUL_PROJECT [just an echo]) are the only dependencies that the *-tox-doc-publish jobs have on zuul. By removing the $ZUUL_UUID dependency and replacing it with something ($RANDOM or uuidgen (from uuid-runtime?)) we could trivially generate docs from any scm.

Event Timeline

thcipriani raised the priority of this task from to Needs Triage.
thcipriani updated the task description. (Show Details)
thcipriani added subscribers: thcipriani, mmodell, demon, hashar.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 4 2015, 6:38 PM

Change 251118 had a related patch set uploaded (by Thcipriani):
Publish scap docs without $ZUUL_UUID

Change 251118 merged by jenkins-bot:
Publish scap docs without $ZUUL_UUID

hashar added a comment.Nov 5 2015, 5:01 PM

The change above got reverted. It was setting a UUID variable but shells in Jenkins are isolated and come up with a clean env.

The use case for ZUUL_UUID is to namespace the builds when they are pushed on integration-publisher. When the parent job doc-publish triggers the child job publish-on-gallium Jenkins copy the parent job parameters, hence the child has the ZUUL_UUID.


Jenkins set a bunch of env variables:

Maybe we can pass to publish-on-gallium a new variable such as:

predefined-parameters: |

Then in the publish-on-gallium job replace any occurrences of $ZUUL_UUID with the injected $PUBLISHER_PATH. Should work ™

Change 251442 had a related patch set uploaded (by Thcipriani):

@hashar implemented your suggestion from T117770#1785791

@dduvall and I tested this by creating two jobs, one of which can be deleted after the patch has been reviewed:

Docs for Scap now seem to be generating on SCM update!

Change 251442 merged by jenkins-bot:

thcipriani closed this task as Resolved.Feb 5 2016, 6:26 PM
thcipriani claimed this task.

Been working for a while, cleaning up task