CR +2 events not triggering gate-and-submit pipeline in zuul
Closed, ResolvedPublic


Likely related to today's zuul upgrade. Commenting CR+2 is not triggering the gate-and-submit pipeline. Other pipelines appear to work fine.

Running zuul enqueue --trigger gerrit --pipeline gate-and-submit --project mediawiki/extensions/Echo --change 226227,1 on gallium worked properly, so this appears to be something in the gerrit trigger?

In the debug.log, I see

2015-07-21 21:24:08,188 DEBUG zuul.Scheduler: Adding trigger event: <TriggerEvent comment-added mediawiki/extensions/Echo master 226227,1 Code-Review:2>
2015-07-21 21:24:08,190 DEBUG zuul.DependentPipelineManager: Starting queue processor: gate-and-submit
2015-07-21 21:24:08,190 DEBUG zuul.DependentPipelineManager: Finished queue processor: gate-and-submit (changed: False)

...and then nothing happens.

Legoktm created this task.Jul 21 2015, 10:25 PM
Legoktm updated the task description. (Show Details)
Legoktm raised the priority of this task from to Unbreak Now!.
Legoktm assigned this task to hashar.
Legoktm added a subscriber: Legoktm.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 21 2015, 10:25 PM

@chasemp has downgraded zuul to 2.0.0-304-g685ca22-wmf1precise1 for now.

Thanks @chasemp for the revert . I should have been more careful in doing this upgrade.

That is an oddity in Zuul BaseFilter which uses a lower case version of the label description instead of the label type. Given an event having the approval:


Zuul normalizes the description field to lower case and replaces spaces with dashes. From zuul/

def normalizeCategory(name):
    name = name.lower()
    return re.sub(' ', '-', name)

Hence we have the normalized form:

>>> import zuul.model
>>> zuul.model.normalizeCategory('Code-Review')

In BaseFilter.matchesRequiredApprovals() the relevant code is:

if (normalizeCategory(approval['description']) != k or
    int(approval['value']) not in v):
    found_approval = False

In I changed the case of all labels to fix another issue. That also changed the approval: field to be upper cased ( ) triggering this corner case.

So in our Zuul layout.yaml the approvals need to use the normalized description i.e. Code-Review -> code-review.

I have sent a documentation change to Zuul upstream at Needs a bit more work.

Change 226274 had a related patch set uploaded (by Hashar):
Workaround Zuul normalizing approval fields

Change 226274 merged by jenkins-bot:
Workaround Zuul normalizing approval fields

I deployed the layout change and reupgraded Zuul to zuul_2.0.0-327-g3ebedde-wmf2precise1 using the deb from

Gave it a try by crafting a dummy change against integration/config.git that fails the test. Removed jenkins-bot verified-1 and voted verified+2 / code-review+2 on

The change successfully entered the gate-and-submit pipeline and did trigger the jobs.

Sorry for the mess and again thank you very much to have handled the revert while I was shamelessly sleeping...