Page MenuHomePhabricator

zuul-cloner does not support Ref events
Closed, DeclinedPublic

Description

When triggering a job with a Gerrit ref-updated event (such as the publish: pipeline), zuul-cloner bails out because the job is missing parameters:

$ zuul-cloner --version
Zuul version: 2.0.0.288.gf0e3688
$
$ zuul-cloner --color --verbose \  --map zuul-clonemap.yaml --workspace src \
  https://gerrit.wikimedia.org/r/p mediawiki/core mediawiki/vendor
usage: zuul-cloner [-h] [-m CLONE_MAP_FILE] [--workspace WORKSPACE] [-v]
                   [--color] [--version] [--cache-dir CACHE_DIR]
                   [--branch BRANCH] [--project-branch PROJECT=BRANCH]
                   [--zuul-branch $ZUUL_BRANCH] [--zuul-change $ZUUL_CHANGE]
                   [--zuul-patchset $ZUUL_PATCHSET]
                   [--zuul-pipeline $ZUUL_PIPELINE]
                   [--zuul-project $ZUUL_PROJECT] [--zuul-ref $ZUUL_REF]
                   [--zuul-url $ZUUL_URL]
                   git_base_url projects [projects ...]
zuul-cloner: error: Some Zuul parameters are not properly set:
    zuul_branch
    zuul_change 
    zuul_patchset
Define them either via environment variables or using options above.
$

Parameters used while pushing a fake tag test-hashar-for-ci on mediawiki/core:

BASE_LOG_PATH="6c/6c8aabbdff141872f6789eaec3f32982df5ef1c0g"
DOC_SUBPATH="test-hashar-for-ci"
LOG_PATH="6c/6c8aabbdff141872f6789eaec3f32982df5ef1c0/publish/mediawiki-core-doxygen-publish/e228fac"
ZUUL_UUID="e228fac55fa34eeab7f881c34df47417"
ZUUL_COMMIT="6c8aabbdff141872f6789eaec3f32982df5ef1c0"
ZUUL_NEWREV="6c8aabbdff141872f6789eaec3f32982df5ef1c0g"
ZUUL_OLDREV="0000000000000000000000000000000000000000"
ZUUL_PIPELINE="publish"
ZUUL_PROJECT="mediawiki/core"
ZUUL_REF="refs/tags/test-hashar-for-ci"
ZUUL_REFNAME="refs/tags/test-hashar-for-ci"
ZUUL_URL="git://zuul.eqiad.wmnet"

From Zuul zuul/launcher/gearman.py file:

# This is what we should be heading toward for parameters:

# required:
# ZUUL_UUID
# ZUUL_REF (/refs/zuul/..., /refs/tags/foo, master)
# ZUUL_COMMIT

# optional:
# ZUUL_PROJECT
# ZUUL_PIPELINE

# optional (changes only):
# ZUUL_BRANCH
# ZUUL_CHANGE
# ZUUL_CHANGE_IDS
# ZUUL_PATCHSET

# optional (ref updated only):
# ZUUL_OLDREV
# ZUUL_NEWREV

So zuul-cloner should gain the ability to be triggered with two different set of parameters, one for changes, the other for ref updates.

Event Timeline

hashar claimed this task.
hashar raised the priority of this task from to High.
hashar updated the task description. (Show Details)
hashar changed Security from none to None.
hashar added subscribers: Unknown Object (MLST), Ltrlg, greg and 2 others.

Command for later reference

ZUUL_NEWREV=e43b3670cf61cef003851f13309f1fd4dd85619d ZUUL_PIPELINE=publish ZUUL_REFNAME=refs/tags/1.19.22 ZUUL_PROJECT=mediawiki/core ZUUL_REF=refs/tags/1.19.22 ZUUL_COMMIT=83fbfb9261a17bdcc43f7011e0380c082a5154c5 ZUUL_URL=/Users/amusso/projects ZUUL_OLDREV=0000000000000000000000000000000000000000 tox -evenv -- zuul-cloner --workspace=/tmp/workspace /Users/amusso/projects mediawiki/core

It seems we're blocked on the strategy for composing multiple repositories (e.g. dependencies).

I suppose for generating documentation, we don't need other projects to be fetched. And if people only tag commits that are in a branch, then the commit has already been tested when it was originally committed to the branch.

I've seen some projects (outside Wikimedia) use commits that are only in the tag tree and not in a branch, but that seems like a bad practice to me. Though its sometimes used to distribute pre-compiled files via git.

I am not working this anymore.

@hashar Is this still a problem since zuul was updated to 2.1?

I think that got more or less fixed upstream. The idea was to trigger job on a reference update for example to generate documentation or whatever else.

Potentially we could need multiple repositories, but it seems easier to just git clone whatever we need as Krinkle pointed out.

So lets decline that task. Has been filled back in November 2014 and if we had a use for it it would have been solved by now.