Page MenuHomePhabricator

[Trebuchet] git fetch writes corrupted objects when its connection to the web service is lost during fetch
Closed, DeclinedPublic

Description

Author: rlane32

Description:
While git fetch is being run, if it loses its connection with the web server it'll write out a corrupted object and will refuse to run git fetch again until the corrupted object is restored. Fixing the objects is not a sane process, basically making the repo unusable.

We should determine if python-dulwich is a viable replacement or we should fix git cli upstream.


Version: wmf-deployment
Severity: critical
Whiteboard: deploysprint-13

Details

Reference
bz51142

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:10 AM
bzimport added a project: Deployments.
bzimport set Reference to bz51142.

I had a look at the git release notes between 1.7.9.5 (used on Precise) and 1.8.4 and have not found anything obvious.

Do you get a way to reproduce this reliably? I guess you want the issue reported upstream they might have some hints as to how to reproduce that.

The one person I could think of who would have addressed this already (Joey Hess, author of git-annex), hasn't. Booo.

Via google I found some behavior similar to ours. Tim didn't notice any http disconnects in the Apache logs, but the system was under very heavy load when this occurred.

http://stackoverflow.com/questions/7366907/git-clone-issue-clone-fails-with-corrupted-mac-on-input

http://blog.e-shell.org/270

There are two ways of addressing this:

  1. making it so git fetch never fails (heh)
  2. dealing with failures intelligently.

For 2, we now have http://git-repair.branchable.com/

I propose that git-repair be integrated with trebuchet (obviously as a recommended addition, not a hard requirement, since it is Haskell).

Reported upstream:
https://github.com/trebuchet-deploy/trebuchet/issues/8

It looks like this is packaged, but not in precise. Needs a backport: http://packages.ubuntu.com/trusty/utils/git-repair

(In reply to Ryan Lane from comment #5)

It looks like this is packaged, but not in precise. Needs a backport:
http://packages.ubuntu.com/trusty/utils/git-repair

As we're migrating to Trusty, this is probably less of a concern. What else is blocking this? (Is it still needed?)

(In reply to Ryan Lane from comment #5)

It looks like this is packaged, but not in precise. Needs a backport:
http://packages.ubuntu.com/trusty/utils/git-repair

As we're migrating to Trusty, this is probably less of a concern. What else is blocking this? (Is it still needed?)

Proposing to close this task as no reply to greg's comment above?
And should this really still be assigned to rlane32?

greg set Security to None.
demon added a subscriber: demon.

Nobody cares about Trebuchet anymore.