Page MenuHomePhabricator

Upgrade git fleet wide to git 2.20
Closed, ResolvedPublic

Description

This task is to upgrade git on the production cluster to 2.20. It comes with support for git protocol v2 which dramatically reduces the initial server advertisement when clients fetch objects. That leads to faster git fetches and less load on the git server side.

Some related references:

T256810 determined we should go with git 2.20.

The CI images have already been switched.

  1. Add a global configuration setting to enable git protocol v2:
/etc/gitconfig
[protocol]
    version=2

That can then be disabled on an opt out basis if needed: git -c protocol.version=1.

  1. Install the git 2.20 package:
BusterComes with it by default
StretchConfigure the apt source list stretch-wikimedia component/git
JessieDo we still care?

List of versions can be found at https://debmonitor.wikimedia.org/packages/git

Event Timeline

hashar created this task.Tue, Sep 8, 7:52 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptTue, Sep 8, 7:52 AM

Change 625847 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] base: upgrade git on stretch to 2.20

https://gerrit.wikimedia.org/r/625847

Change 625848 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] git: allow multiple calls to git::systemconfig

https://gerrit.wikimedia.org/r/625848

Change 625849 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] base: enable git protocol version2 fleet wide

https://gerrit.wikimedia.org/r/625849

Mentioned in SAL (#wikimedia-releng) [2020-09-09T09:05:49Z] <hashar> deployment-prep upgrading git from 2.11 to 2.20 on all Stretch based hosts # T262244

Change 625847 abandoned by Hashar:
[operations/puppet@production] base: upgrade git on stretch to 2.20

Reason:
I have used it on deployment-prep to upgrade git there and scap works fine for mediawiki. We will upload git to 'main' for the production upgrade.

https://gerrit.wikimedia.org/r/625847

@MoritzMuehlenhoff I have upgraded on deployment-prep and the automatic deployment of MediaWiki with scap works all fine there :]

So I guess we can upload git 2.20 to main and process with the fleet upgrade. Potentially starting with just the deployment host and mwdebug machines for a start.

Mentioned in SAL (#wikimedia-operations) [2020-09-10T11:13:01Z] <moritzm> uploaded git 2.20.1-2+deb10u3~wmf1 to stretch-wikimedia/main T262244

Mentioned in SAL (#wikimedia-operations) [2020-09-10T12:01:25Z] <moritzm> upgrading deployment servers to git 2.20 T262244

Change 625848 merged by Jbond:
[operations/puppet@production] git: allow multiple calls to git::systemconfig

https://gerrit.wikimedia.org/r/625848

Change 626260 had a related patch set uploaded (by Jbond; owner: Jbond):
[operations/puppet@production] git: allow multiple calls to git::systemconfig

https://gerrit.wikimedia.org/r/626260

Change 626598 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] git: allow multiple calls to git::systemconfig [take 2]

https://gerrit.wikimedia.org/r/626598

Deployment servers and some other clusters (hadoop e.g.) have been upgraded to 2.20 yesterday, the rest will follow on Monday.

Deployment servers and some other clusters (hadoop e.g.) have been upgraded to 2.20 yesterday, the rest will follow on Monday.

Awesome thank you!

Actually enabling protocol v2 will be done via a git config change, but that is still in progress :]

Change 626598 merged by Jbond:
[operations/puppet@production] git: allow multiple calls to git::systemconfig [take 2]

https://gerrit.wikimedia.org/r/626598

Change 626757 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] deployment: migrate to git::systemconfig

https://gerrit.wikimedia.org/r/626757

Change 626758 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] phabricator: migrate to git::systemconfig

https://gerrit.wikimedia.org/r/626758

hashar added a subscriber: jbond.Fri, Sep 11, 9:28 PM

status update

@MoritzMuehlenhoff started upgrading git on some Stretch servers.

@jbond kindly assisted in polishing up the puppet magic to add protocol.version=2 in /etc/gitconfig. I caught a couple other places generating /etc/gitconfig which would need to be adjusted (the two patches I have send above).

Mentioned in SAL (#wikimedia-operations) [2020-09-14T08:42:39Z] <moritzm> upgrading remaining stretch systems to git 2.20 T262244

All production hosts using Stretch have been updated to 2.20 (Cloud VPS runs unattended-upgrades, so should also have been upgraded mostly). Only container images are missing, but it should trickle in slowly with rebuilds.

Change 625849 merged by Jbond:
[operations/puppet@production] base: enable git protocol version2 fleet wide

https://gerrit.wikimedia.org/r/625849

Change 626757 merged by Jbond:
[operations/puppet@production] deployment: remove obsolete Trebuchet git config

https://gerrit.wikimedia.org/r/626757

Change 626758 merged by Jbond:
[operations/puppet@production] phabricator: migrate to git::systemconfig

https://gerrit.wikimedia.org/r/626758

hashar closed this task as Resolved.Mon, Sep 14, 12:28 PM
hashar claimed this task.

Thank you @MoritzMuehlenhoff and @jbond !

Change 626260 abandoned by Jbond:
[operations/puppet@production] git: allow multiple calls to git::systemconfig

Reason:

https://gerrit.wikimedia.org/r/626260