@Volans mentioned to use the special refspec tag v2.0.0 instead of refs/tags/v2.0.0. That is an alias according to git-fetch(1):

tag <tag>
means the same as refs/tags/<tag>:refs/tags/<tag>; it requests fetching everything up to the given tag.

And when using +refs/tags/v2.0.0:refs/tags/v2.0.0 I end up with:

0c369cf (HEAD -> tag, tag: v2.0.0) CHANGELOG: add changelogs for release v2.0.0
3d4aa18 (grafted) Copyright notice: add 2018

Change 607990 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] jjb: checkout branch and tag properly

https://gerrit.wikimedia.org/r/607990

Change 608013 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: abstract common git operations

https://gerrit.wikimedia.org/r/608013

Change 607990 merged by jenkins-bot:
[integration/config@master] jjb: checkout branch and tag properly

https://gerrit.wikimedia.org/r/607990

Change 608013 merged by jenkins-bot:
[integration/config@master] dockerfiles: abstract common git operations

https://gerrit.wikimedia.org/r/608013

The implementation is complete, it is partly deployed. Need to generalize to the whole fleet of images and have all jobs updated.

I had all Stretch based images rebuild with https://gerrit.wikimedia.org/r/c/integration/config/+/620696 so they now all come with /utils/ci-src-setup.sh which fetches using: "+$ZUUL_REF:$ZUUL_REF".

The releng/tox image (and other images) use a slightly different logic in their entrypoint, so they all need to be adjusted to rely on that common script instead.

Change 622374 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: refresh images to use ci-src-setup.sh

https://gerrit.wikimedia.org/r/622374

Change 622374 merged by jenkins-bot:
[integration/config@master] dockerfiles: refresh images to use ci-src-setup.sh

https://gerrit.wikimedia.org/r/622374

Change 622417 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] jjb: update jobs to use /utils/ci-src-setup.sh

https://gerrit.wikimedia.org/r/622417

Change 622417 abandoned by Hashar:
[integration/config@master] jjb: update jobs to use /utils/ci-src-setup.sh

Reason:
faulty change :-(

https://gerrit.wikimedia.org/r/622417

Change 622535 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: fix tags fetch

https://gerrit.wikimedia.org/r/622535

Change 622787 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: rebuild images for I4431a09526

https://gerrit.wikimedia.org/r/622787

Change 622535 merged by jenkins-bot:
[integration/config@master] dockerfiles: fix tags fetch

https://gerrit.wikimedia.org/r/622535

Change 622798 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] jjb: update jobs for ci-src-setup update

https://gerrit.wikimedia.org/r/622798

Change 622798 merged by jenkins-bot:
[integration/config@master] jjb: update jobs for ci-src-setup update

https://gerrit.wikimedia.org/r/622798

Change 622787 merged by jenkins-bot:
[integration/config@master] dockerfiles: rebuild images for I4431a09526

https://gerrit.wikimedia.org/r/622787

Change 623762 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] jjb: update images to latest versions

https://gerrit.wikimedia.org/r/623762

Change 623762 merged by jenkins-bot:
[integration/config@master] jjb: update images to latest versions

https://gerrit.wikimedia.org/r/623762

@hashar that's great! Can I force a re-generation of some other docs in an easy way?

In T186496#6430999, @Volans wrote:

@hashar that's great! Can I force a re-generation of some other docs in an easy way?

Not directly unfortunately but we use the Zuul client to enqueue a reference update event. Something such as:

ssh contint.wikimedia.org
zuul enqueue-ref --trigger gerrit --pipeline publish --project <GERRIT PROJECT> --ref <GIT REF EX: refs/tags/v1.0> --newrev <SHA1 of commit pointed by tag>

Real example:

https://doc.wikimedia.org/cumin/v3.0.2/ has Cumin 0.1.dev2+gd1760d6 documentation

zuul enqueue-ref --trigger gerrit --pipeline publish --project operations/software/cumin --ref refs/tags/v3.0.2 --new d1760d6c769b7196fd262e87036483b8d6678578

But the job failed at https://integration.wikimedia.org/ci/job/cumin-tox-tag-publish/10/console with:

00:00:24.654 + tox -v -e py37-sphinx

00:00:24.869 using tox.ini: /src/tox.ini (pid 46)
00:00:24.944 using tox-3.10.0 from /usr/local/lib/python3.5/dist-packages/tox/__init__.py (pid 46)
00:00:24.944 py37-sphinx cannot reuse: no previous config /src/.tox/py36-tests/.tox-config1

00:00:24.945 py37-sphinx create: /src/.tox/py36-tests
00:00:24.945 SKIPPED: InterpreterNotFound: 
00:00:24.945 ___________________________________ summary ____________________________________
00:00:24.945 SKIPPED:  py37-sphinx: InterpreterNotFound:

Thanks, that works fine, I've regenerated https://doc.wikimedia.org/spicerack/v0.0.42/ :)
As for master will that work too? I tried using --ref refs/heads/master but no luck.
If there are no additional commits on master after the last tag it should show Spicerack 0.0.42 documentation, while if there is any additional commit it should show something like Spicerack 0.0.43.dev1+g73f3949 documentation.

Cool!

For refs/heads/master I looked at the build Zuul has updated its own git repo (on zuul-merger instance contint2001). That is done to ensure the branches are matching the "upstream" Gerrit repo:

2020-09-03 06:17:41,893 DEBUG zuul.MergeServer: Got update job: 84a23708a3344fc89186668165a80448
2020-09-03 06:17:41,893 INFO zuul.Merger: Updating local repository operations/software/spicerack
2020-09-03 06:17:41,902 DEBUG zuul.Repo: Updating repository /srv/zuul/git/operations/software/spicerack

Then the Jenkins job ( https://integration.wikimedia.org/ci/job/spicerack-tox-tag-publish/35/ ) lists the parameters:

Triggered by reference: refs/heads/master
Project: operations/software/spicerack
Old revision: 0000000000000000000000000000000000000000
New revision: b6bec1c87fd0499b3bc517dcfc337cc52e69728e
Pipeline: publish

The job then did:

+ git fetch --quiet --update-head-ok --depth 2 git://contint2001.wikimedia.org/operations/software/spicerack +refs/heads/master:refs/heads/master
+ git checkout -q FETCH_HEAD

b6bec1c87fd0499b3bc517dcfc337cc52e69728e is the tip of the master branch AND is also the target of the 0.0.42 tag. At this point, setuptools_scm invokes git to figure out the version (I am guessing with git describe) and resolves it to 0.0.42 cause well... that is 0.0.42 :] The next commit that merges should show up with 0.0.43.dev1 if everything works as intended.

A hack would be to hint setuptools_scm to recognizes it is being triggered for a master and not for the tag for example by looking at the value of ZUUL_REF then I don't quite now which version we should generate since for all purposes, that is the exact same as 0.0.42.

Yes, I was expecting 0.0.42 to appear on the master-generated doc too, but it got Spicerack 0.1.dev2+gb6bec1c documentation instead. Or maybe I'm missing something in your answer.

Bah I looking at https://doc.wikimedia.org/spicerack/v0.0.42/ which has 0.0.42 .... So my analyzes above is slightly off.

https://doc.wikimedia.org/spicerack/master shows 0.1.dev2+gb6bec1c, it has been generated in the postmerge pipeline which act after the change has been merged and NOT on the refs/heads/master being updated. Thus that doc is the old one.

The zuul enqueue-ref command we used earlier end up publishing the doc at https://doc.wikimedia.org/spicerack/refs/heads/master/ which does list 0.0.42. I was wrong when suggesting that command and it turns out we have some bug in the pipeline.

I have added back the latest change ( https://gerrit.wikimedia.org/r/c/operations/software/spicerack/+/623422 ) in postmerge by using zuul enqueue:

zuul enqueue --trigger gerrit --pipeline postmerge --project operations/software/spicerack --change 623422,1

And now https://doc.wikimedia.org/spicerack/master/ properly shows up 0.0.42 (you might need to drop the cache by adding a dummy query parameter to the URL).

Sorry for all the confusion and I guess I should remove the doc that ended up being pushed to refs/heads/master

@hashar thanks for the additional context and the fixes, and sorry for having played with the command changing refs. Yes defintely you can delete the docs generated at refs/heads/master. I guess we can just wait for the next commit in master to get the new doc generated with the correct numbers. Thanks again for the fix, really appreciated.

In T186496#6433890, @Volans wrote:

@hashar thanks for the additional context and the fixes, and sorry for having played with the command changing refs.

To be fair: I suggested refs/heads/master in the first place :]

Yes definitely you can delete the docs generated at refs/heads/master.

Done.

I guess we can just wait for the next commit in master to get the new doc generated with the correct numbers. Thanks again for the fix, really appreciated.

Note the version IS correct since master and 0.0.42 both point at the same commit :-]

A potential workaround is: immediately after cutting a release, a commit that adds an entry for the next release in the CHANGELOG.rst. Maven for Java does that and we tend to do that for Quibble, for example after we have cut 0.0.44: 73842ba

In T186496#6439775, @hashar wrote:

I guess we can just wait for the next commit in master to get the new doc generated with the correct numbers. Thanks again for the fix, really appreciated.

Note the version IS correct since master and 0.0.42 both point at the same commit :-]

A potential workaround is: immediately after cutting a release, a commit that adds an entry for the next release in the CHANGELOG.rst. Maven for Java does that and we tend to do that for Quibble, for example after we have cut 0.0.44: 73842ba

I meant for the other repositories, this is totally correct indeed.

@hashar for reasons unknown to me it seems that it didn't work for https://doc.wikimedia.org/cumin/master/ (I just made 2 commits and a new tag)

Congratulations on releasing cumin 4.0.0 ( https://doc.wikimedia.org/cumin/v4.0.0/ ) :]

The issue for the master branch is slightly different. Namely we use a shallow fetch without any tags fetched:

git fetch --quiet --update-head-ok --depth 2 "${ZUUL_URL}/${ZUUL_PROJECT}" "+$ZUUL_REF:$ZUUL_REF"

We could have it fetch the tags, but due to the shallow depth of 2 it is unlikely any tag will be connected to the current HEAD :\