Figure out how to create and maintain replication on integration slaves. After T96629, we can do it from a scheduled job. But that is the "when", not the "how".
See also:
T96629: Convert pool from a few large slaves (4X) to more smaller slaves (1X)
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Implement git-cache-update script | integration/jenkins | master | +123 -0 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Krinkle | T91211 gallium and lanthanum disks full (tracking) | |||
| Declined | None | T91707 L10n-bot should not force-merge / override Jenkins (breaks the build) | |||
| Declined | hashar | T93703 reduce copies of mediawiki/core in workspaces | |||
| Resolved | PRODUCTION ERROR | Krinkle | T86730 Zuul-cloner failing to acquire .git lock sometimes | ||
| Declined | hashar | T96627 Jenkins jobs must wipe workspace | |||
| Resolved | hashar | T97098 Update jobs to use zuul-cloner with git cache | |||
| Declined | hashar | T96687 Set up git replication on integration slaves |
Event Timeline
Comment Actions
That is similar to T87294: Nodepool images need Gerrit mirror for git-clone performance. Though with Nodepool that would be done on image creations and for our current permanent slave we would probably need some kind of weekly cron to refresh them.
From that task @Krinkle pointed to OpenStack script https://github.com/openstack-infra/project-config/blob/59e04c19850a067e57b6304ee398b20438b295a1/nodepool/scripts/cache_git_repos.py
Comment Actions
Cron would interfere with builds and create races that corrupt git-clone. Per advice from people in #openstack-infra, we should use a Jenkins job scheduled to run on each instance. That way it will not run concurrently with other jobs.
From that task @Krinkle pointed to OpenStack script https://github.com/openstack-infra/project-config/blob/59e04c19850a067e57b6304ee398b20438b295a1/nodepool/scripts/cache_git_repos.py
That script is for creating the replication, it does not have logic for updating replication. We will need a list of commands to run for each repo. E.g.: git fetch origin. And maybe disable auto-gc, run gc after update, and maybe repack?
Comment Actions
Change 206074 had a related patch set uploaded (by Krinkle):
[WIP] Implement git-cache-update script
Comment Actions
This will be done as part of migrating to Nodepool instances. The specific task is T87294: Nodepool images need Gerrit mirror for git-clone performance