|Resolved||• hashar||T70480 Scale merge operations (tracking)|
|Resolved||• hashar||T70481 Zuul repositories have too many refs causing slow updates|
|Open||None||T103528 Run zuul-clear-refs.py daily on all our repositories to reclaim Zuul references|
|Resolved||• hashar||T103529 Package / puppetize zuul-clear-refs.py|
|Resolved||elukey||T145057 Upgrade Zuul on scandium.eqiad.wmnet (Jessie zuul-merger)|
The patch I have proposed upstream has been approved :-} https://review.openstack.org/#/c/109276/
The references created under refs/zuul are not garbage collected. Since git fetch send them all to Gerrit to sync the repositories, the time spent on merge will slightly grow overtime and start being noticeable.
To clean them you can use the tools/zuul-clear-refs.py script on each repositories. It will delete Zuul references that point to commits for which the commit date is older than a given amount of days (default 360):
We would want to include the utility in the Zuul Debian package.
I have included the script in our Debian package for Precise which ship it as /usr/bin/zuul-clear-ref. That is meant to be used on zuul-merger instance, the one we have is scandium.eqiad.wmnet which is running Jessie, so this patch is now pending for a port of the Zuul package to jessie.
2.5.0-8-gcbc7f62-wmf2jessie1 has been deployed on scandium.eqiad.wmnet (the zuul-merger) but I screwed it up. The python shebang needs to be adjusted to point to the venv:
$ zuul-clear-refs Traceback (most recent call last): File "/usr/bin/zuul-clear-refs", line 30, in <module> import git ImportError: No module named git
$ head -n 1 `which zuul-cloner` #!/usr/share/python/zuul/bin/python
The custom patch debian/patches/0002-Replace-python-shebang-with-python2.7.patch from the branch patch-queue/debian/precise-wikimedia needs to be rebased.
I have rebased the shebang patch. Rebuild package for both Precise and Jessie:
debdiff shows they are noop
comparing the package contents with previous version shows that three files get their shebang updated.
Will get them pushed to the zuul systems and apt.wikimedia.org
zuul_2.5.0-8-gcbc7f62-wmf4* fix it:
$ which zuul-clear-refs
usage: zuul-clear-refs [-h] [--until DAYS_AGO] [-n] [-v] gitrepo
zuul-clear-refs: error: too few arguments