Unreviewed commits merged in gerrit
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	awight
	Jun 22 2015, 6:34 PM

Description

One of our main repos just became corrupted, there are two commits which were not merged normally but appear in the master branch anyway.

See the tree here, https://git.wikimedia.org/log/wikimedia%2Ffundraising%2Fcrm/refs%2Fheads%2Fmaster

Bad commits are:
https://gerrit.wikimedia.org/r/#/c/218438/
https://gerrit.wikimedia.org/r/#/c/217989/

They originally appeared in this order, and the second patch was CR+2'd. Somehow, this caused an insane cyclical rebase situation, I discovered after clicking the "Rebase" web UI button a few times. Now, our repo includes both patches as if they had been merged, but there's no record of a merge, only "Merge failed".

Related Objects

Mentioned Here: rWFCGdf81eb47d329: Use RAII for fixtures
rWFCG15c080c477dc: Write test for contact_groups import

Event Timeline

awight created this task.Jun 22 2015, 6:34 PM

awight raised the priority of this task from to Needs Triage.

awight updated the task description. (Show Details)

awight added a project: Release-Engineering-Team.

awight added subscribers: awight, greg, Ejegg, • cwdent.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 22 2015, 6:34 PM

greg added a project: Gerrit.Jun 22 2015, 6:35 PM

greg set Security to None.

hashar subscribed.Jun 22 2015, 6:35 PM

The "merge failed" message reported back in Gerrit is issued when zuul-merger on gallium.wikimedia.org is unable to merge the patch on the tip of the branch. There might be a server side issue with wikimedia/fundraising/crm being corrupted on gallium ( somewhere under /srv/ssd/zuul/git/ ).

The git log --oneline --graph looks like:

* 15c080c - (HEAD, gerrit/master, gerrit/HEAD, master) Write test for contact_groups import (3 days ago) <Adam Roses Wight>
* df81eb4 - Use RAII for fixtures (3 days ago) <Adam Roses Wight>
*   c145cf8 - Merge "Rename function, it's not specific to contributions" (3 days ago) <jenkins-bot>
|\  
| * 59731d6 - Rename function, it's not specific to contributions (4 days ago) <Adam Roses Wight>
* |   580d2c8 - Merge "Implement soft credit import" (3 days ago) <Ejegg>
|\ \  
| |/  
| * 53a8625 - Implement soft credit import (4 days ago) <Adam Roses Wight>
* | dde86b0 - Renumber 6000-series updates (4 days ago) <Elliott Eggleston>
|/  
* 0a1f82a - Port contribution import tests as pure fixture data (4 days ago) <Adam Roses Wight>
* c1490d1 - Don't use Name or Title fields if this is not an Organization contact. (4 days ago) <Adam Roses Wight>

I have fetched Gerrit review notes git fetch gerrit refs/notes/review:refs/notes/review

df81eb4 is https://gerrit.wikimedia.org/r/#/c/218438/5 (PS 5)

$ git show --show-notes=review --format=fuller --summary df81eb4
commit df81eb47d329908470090711470dab28e8da33a5
Author:     Adam Roses Wight <awight@wikimedia.org>
AuthorDate: Mon Jun 15 13:01:39 2015 -0700
Commit:     Adam Roses Wight <awight@wikimedia.org>
CommitDate: Fri Jun 19 12:39:19 2015 -0700

    Use RAII for fixtures
    
    The db records last for as long as the CiviFixtures object.
    
    Change-Id: I1e1a9dbbfd1b56310ef27def7dd5fe24a83c40a9

Notes (review):
    Verified+2: jenkins-bot
    Reviewed-on: https://gerrit.wikimedia.org/r/218438
    Project: wikimedia/fundraising/crm
    Branch: refs/heads/master

So apparently it has been merged in by Adam but the Gerrit database hasn't catched it. @Chad I think we need to manually set https://gerrit.wikimedia.org/r/#/c/218438/5 has merged, then one can propose a patch that revert it and get it merged. You can then resume with a new change number.

15c080c is https://gerrit.wikimedia.org/r/#/c/217989/8 patchset 8

$ git show --show-notes=review --summary --format=fuller 15c080c
commit 15c080c477dcad53c33b7e9981c9c5c41fb26475
Author:     Adam Roses Wight <awight@wikimedia.org>
AuthorDate: Fri Jun 12 18:21:47 2015 -0700
Commit:     Adam Roses Wight <awight@wikimedia.org>
CommitDate: Fri Jun 19 12:39:19 2015 -0700

    Write test for contact_groups import
    
    Change-Id: Iaabd4c13325495b6c65f0b439a16dc2c3ef7d092

Notes (review):
    Verified+2: jenkins-bot
    Reviewed-on: https://gerrit.wikimedia.org/r/217989
    Project: wikimedia/fundraising/crm
    Branch: refs/heads/master

You can see that one got CR+2, should have got Zuul to ask Gerrit to merge it in.

Looking in Zuul log files on gallium.wikimedia.org:/var/log/zuul I find:

2015-06-22 16:48:16,350 DEBUG zuul.Scheduler: Processing trigger event <TriggerEvent comment-added wikimedia/fundraising/crm master 217989,8 Code-Review:2>
2015-06-22 16:48:16,350 INFO zuul.Scheduler: Adding wikimedia/fundraising/crm, <Change 0x7f37acbba850 217989,8> to <Pipeline gate-and-submit>
2015-06-22 16:48:16,350 DEBUG zuul.DependentPipelineManager: Considering adding change <Change 0x7f37acbba850 217989,8>
2015-06-22 16:48:16,351 DEBUG zuul.DependentPipelineManager: Checking for changes needed by <Change 0x7f37acbba850 217989,8>:
2015-06-22 16:48:16,351 DEBUG zuul.DependentPipelineManager:   Change <Change 0x7f37acbba790 218438,5> is needed but can not be merged
2015-06-22 16:48:16,351 DEBUG zuul.DependentPipelineManager: Failed to enqueue changes ahead of <Change 0x7f37acbba850 217989,8>

So 217989,8 received a CR+2, Zuul added it to gate-and-submit, found the change depends on 218438,5 which can not be merged. Thus the change is removed from Zuul queue.

Just after that Zuul receives:

2015-06-22 16:48:16,351 DEBUG zuul.IndependentPipelineManager:
    Event <TriggerEvent ref-updated wikimedia/fundraising/crm>
    for change <Ref 0x7f37acbbafd0 master updated
      c145cf8fa8b3fdf30b3488f0795db139cb4e41aa..15c080c477dcad53c33b7e9981c9c5c41fb26475>
     matched <EventFilter types: ref-updated refs: ^(?!(devnull|refs/.*)).*$> in pipeline <IndependentPipelineManager post>

15c080c477dcad53c33b7e9981c9c5c41fb26475 being 218438,5 which got merged by Gerrit for some reason :-(

Not much ideas :-(

hashar moved this task from Untriaged to Externally Blocked on the Continuous-Integration-Infrastructure board.Jun 30 2015, 1:51 PM

Thanks for looking at this! FYI, we've worked around in the way you suggested, by merging reverts, and it seems perfectly stable again. I'll leave this ticket open for you to decide whether it's worth further investigation...

hashar removed a project: Continuous-Integration-Infrastructure.Jul 27 2015, 2:42 PM

awight closed this task as Resolved.Aug 11 2015, 6:42 AM

awight claimed this task.

@awight abandoned the fault changes in Gerrit. Not ideal but at least they are no more left open :-)

greg moved this task from INBOX to Done on the Release-Engineering-Team board.Sep 8 2015, 8:44 PM

Unreviewed commits merged in gerritClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Unreviewed commits merged in gerrit
Closed, ResolvedPublic
Actions