Page MenuHomePhabricator

operations/software repo commits always getting -1 from Jenkins-bot
Closed, ResolvedPublic

Description

Hello,

Today I tried to push this change: https://gerrit.wikimedia.org/r/#/c/338326/ and it quickly got -1 from Jenkins bot.
After a few tests trying to rebase the change manually, and remove the tabs and so forth, there was no way it would get +1.
@Volans and @jcrespo also tried to debug the issue without finding anything.

In order to discard issues with that specific commit and/or file we tried different test commits even on other files, and they always got -1:

https://gerrit.wikimedia.org/r/338335
https://gerrit.wikimedia.org/r/338334
https://gerrit.wikimedia.org/r/338331

We forced a +2 and it got merged successfully so looks like cobalt isn't the issue but the copy the jenkins bot is using.
After getting it merged, a git pull on another old copy shows the merge and the file without any issue.

The reason for setting this priority as High is because it might be really confusing as you wouldn't know if your commit is wrong and can get confused looking for issues on your code while the problem might not be it.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 17 2017, 11:09 AM
Marostegui triaged this task as High priority.Feb 17 2017, 11:11 AM
Marostegui updated the task description. (Show Details)
Marostegui added a subscriber: hashar.

I am lunching but I am 99% sure it is due to T157818. Fix is to rm -fr /srv/zuul/git/operations/software on both contint1001 and contint2001 and recheck patches.

I have yet to find a proper / long term solution to that problem.

If you mean that it happens from time to time, and that is already tracked on, T157818, this can be closed as soon as the specific issue (not voting -1) is gone.

hashar closed this task as Resolved.Feb 17 2017, 12:53 PM

The issue was on contint2001. The operations/software/cumin got cloned first and thus operations/software could not be cloned (because the directory exists). Applied on contint2001 the fix listed on T157818 / T138455#2401076:

ssh contint2001.wikimedia.org
sudo -u zuul -i
cd /srv/ssd/zuul/git/operations/software
git init .
git remote add origin ssh://jenkins-bot@gerrit.wikimedia.org:29418/operations/software
git remote set-head origin --auto

And I confirmed that fixed it for good. The proper fix, if any can be found, will be handled via parent task T157818.

Also thanks, with this documentation, we will be able to fix it ourselves with confidence if it happens again, until the parent is resolved!