Faster source code fetches thanks to git protocol version 2

In 2015 I noticed git fetches from our most active repositories to be unreasonably slow, sometimes up to a minute which hindered fast development and collaboration. You can read some of the debugging details I have conducted at the time on T103990. Gerrit upstream was aware of the issue and a workaround was presented though we never went to implement it.

When fetching source code from a git repository, the client and server conduct a negotiation to discover which objects have to be sent. The server sends an advertisement that lists every single reference it knows about. For a very active repository in Gerrit it means sending references for each patchset and each change ever made to the repository, or almost 200,000 references for mediawiki/core. That is a noticeable amount of data resulting in a slow fetch, especially on a slow internet connection.

Gerrit originated at Google and has full time maintainers. In 2017 a team at Google went to tackle the problem and proposed a new protocol to address the issue, and they closely worked with git maintainers while doing so. The new protocol makes git smarter during the advertisement phase, notably to filter out references the client is not interested in. You can read Google introduction post at https://opensource.googleblog.com/2018/05/introducing-git-protocol-version-2.html

Since June 28th 2020, our Gerrit has been upgraded and now supports git protocol version 2. But to benefit from faster fetches, your client also needs to know about the newer protocol and have it explicitly enabled. For git, you will want version 2.18 or later. Enable the new protocol by setting git configuration protocol.version to 2.

It can be done either on an on demand basis:

git -c protocol.version=2 fetch

Or enabled in your user configuration file:

$HOME/.gitconfig
[protocol]
    version = 2

On my internet connection, fetching for mediawiki/core.git went from ~15 seconds to just 3 seconds. A noticeable difference in my day to day activity.

If you encounter any issue with the new protocol, you can file a task in our Phabricator and tag it with git-protocol-v2.

Written by hashar on Mon, Jul 6, 10:57 AM.
User
Projects
Subscribers
dancy
Tokens
"Orange Medal" token, awarded by Krinkle."Love" token, awarded by santhosh."Love" token, awarded by thcipriani."Like" token, awarded by mmodell.

Event Timeline

dancy added a comment.Jul 1 2020, 3:27 PM

Nice post! I made some minor edits.