Page MenuHomePhabricator

CR +2 events not triggering gate-and-submit pipeline in zuul
Closed, ResolvedPublic

Description

Likely related to today's zuul upgrade. Commenting CR+2 is not triggering the gate-and-submit pipeline. Other pipelines appear to work fine.

Running zuul enqueue --trigger gerrit --pipeline gate-and-submit --project mediawiki/extensions/Echo --change 226227,1 on gallium worked properly, so this appears to be something in the gerrit trigger?

In the debug.log, I see

2015-07-21 21:24:08,188 DEBUG zuul.Scheduler: Adding trigger event: <TriggerEvent comment-added mediawiki/extensions/Echo master 226227,1 Code-Review:2>
...
2015-07-21 21:24:08,190 DEBUG zuul.DependentPipelineManager: Starting queue processor: gate-and-submit
2015-07-21 21:24:08,190 DEBUG zuul.DependentPipelineManager: Finished queue processor: gate-and-submit (changed: False)

...and then nothing happens.

Event Timeline

Legoktm assigned this task to hashar.
Legoktm raised the priority of this task from to Unbreak Now!.
Legoktm updated the task description. (Show Details)
Legoktm subscribed.

@chasemp has downgraded zuul to 2.0.0-304-g685ca22-wmf1precise1 for now.

Thanks @chasemp for the revert . I should have been more careful in doing this upgrade.

That is an oddity in Zuul BaseFilter which uses a lower case version of the label description instead of the label type. Given an event having the approval:

{"type":"Code-Review","description":"Code-Review","value":"-1"}

Zuul normalizes the description field to lower case and replaces spaces with dashes. From zuul/model.py:

def normalizeCategory(name):
    name = name.lower()
    return re.sub(' ', '-', name)

Hence we have the normalized form:

>>> import zuul.model
>>> zuul.model.normalizeCategory('Code-Review')
'code-review'
>>>

In BaseFilter.matchesRequiredApprovals() the relevant code is:

if (normalizeCategory(approval['description']) != k or
    int(approval['value']) not in v):
    found_approval = False

In https://gerrit.wikimedia.org/r/#/c/226220/ I changed the case of all labels to fix another issue. That also changed the approval: field to be upper cased ( https://github.com/wikimedia/integration-config/blob/4e31d4a94297a49146fc186a9c4c8b6b6f9f47ac/zuul/layout.yaml#L448-L449 ) triggering this corner case.

So in our Zuul layout.yaml the approvals need to use the normalized description i.e. Code-Review -> code-review.

I have sent a documentation change to Zuul upstream at https://review.openstack.org/#/c/204499/ Needs a bit more work.

Change 226274 had a related patch set uploaded (by Hashar):
Workaround Zuul normalizing approval fields

https://gerrit.wikimedia.org/r/226274

Change 226274 merged by jenkins-bot:
Workaround Zuul normalizing approval fields

https://gerrit.wikimedia.org/r/226274

I deployed the layout change https://gerrit.wikimedia.org/r/226274 and reupgraded Zuul to zuul_2.0.0-327-g3ebedde-wmf2precise1 using the deb from gallium.wikimedia.org/root

Gave it a try by crafting a dummy change against integration/config.git that fails the test. Removed jenkins-bot verified-1 and voted verified+2 / code-review+2 on https://gerrit.wikimedia.org/r/#/c/226275/

The change successfully entered the gate-and-submit pipeline and did trigger the jobs.

Sorry for the mess and again thank you very much to have handled the revert while I was shamelessly sleeping...