Page MenuHomePhabricator

CI gitcache is not updated
Closed, ResolvedPublic

Description

The WMCS CI instances have git repositories cloned in and serving as mirrors. The repositories are supposedly updated on a daily basis:

modules/profile/manifests/ci/gitcache.pp
$minute = fqdn_rand(60)

systemd::timer::job { 'ci-gitcache-refresh':
    ensure      => present,
    description => 'Regular job to update the CI git cache',
    user        => 'root',
    command     => '/usr/bin/find /srv/git -type d -name \'*.git\' -exec git -C {} fetch origin --prune --prune-tags --force \;',
    interval    => {'start' => 'OnCalendar', 'interval' => "*-*-* 3:${minute}:00"},
}

The repositories HEAD branch point to commits from January 30 / 31st. That matches 7291ed4dd1228043a8c7182884f95c0ef7801aa5 which was made for T252310 / T356247 which changed the command:

- git fetch origin --prune '+refs/heads/*:refs/heads/*'
+ git fetch origin --prune --prune-tags --force

The refspec for heads got removed in order to fetch tags as well, assuming the default was to fetch both, but that is apparently not the case.

Trying on integration-agent-docker-1047.integration.eqiad1.wikimedia.cloud it does not fetch anything:

$ sudo git -C /srv/git/mediawiki/core.git fetch --prune --prune-tags --force
$

With --verbose it apparently only fetches tags.

The instances have git 2.34.1.

I can't reproduce locally with 2.39.5.

Details

Event Timeline

When using a bare repository, I see the same issue locally with git 2.39.5. In a bare repository there is no local reference and nothing get fetch, --prune-tags is documented as: a shorthand for providing tag refspec (refs/tags/*:refs/tags/*).

So +refs/heads/*:refs/heads/* should be added back. I probably removed it cause when specified that overrides/prevent --prune-tags from adding the tag refspec. Instead I should have appended the tag refspec and thus fetch both explicitly.

Change #1079472 had a related patch set uploaded (by Hashar; author: Hashar):

[operations/puppet@production] ci: fix git mirror not fetching branches

https://gerrit.wikimedia.org/r/1079472

Mentioned in SAL (#wikimedia-releng) [2024-10-11T14:04:20Z] <hashar> integration: force refresh the git mirrored repositories from integration-cumin: sudo cumin -b1 'name:docker' 'systemctl start ci-gitcache-refresh.service' # T376981

Change #1079472 merged by Dzahn:

[operations/puppet@production] ci: fix git mirror not fetching branches

https://gerrit.wikimedia.org/r/1079472

hashar claimed this task.