E.g. "1.23.3" instead of "REL1_23".
For some reason those got lost https://doc.wikimedia.org/mediawiki-core/
We should have:
instead of
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Legoktm | T74454 OOjs UI's PHP docs should be auto-generated | |||
| Resolved | hashar | T73062 doc.wikimedia.org: Generate documentation for release tags (tracking) | |||
| Resolved | hashar | T75991 publish jobs should support branch filtering for postmerge and still be triggerable via publish | |||
| Declined | None | T76003 zuul-cloner does not support Ref events | |||
| Resolved | thcipriani | T96390 Allow ref-updated listener to filter out tag deletions |
@hashar I think we originally had something related to this (there is a ZUUL environmental variable that will contain refs/tags/.. and there is the publish pipeline in Zuul), but either it didn't work or we forgot about it.
This should be relatively easy to set up. Once done, we can do a few manual builds of the job filling in parameters for a recently created tags.
Rather long, at first offers a possible explanation as why some paths disappearing then summarize the work I have done on refactoring the doc jobs.
Some context:
I have no idea whether we had per version documentation under https://doc.wikimedia.org/mediawiki-core/ , one sure thing is that I dropped the whole docs tree on Nov 13th when migrating the Doxygen job to zuul-cloner and Trusty. There reason is that I triggered the job manually, forgetting to fill ZUUL_UID, that caused a rsync of an empty directory which deleted everything.
Changes related to the migration share the same Change-Id: https://gerrit.wikimedia.org/r/#/q/I1fd6d23917c2098dbdad37d9e718335616207f68,n,z That fixed:
The issue is no more going to happen since the job will now die whenever $ZUUL_UUID is missing.
Summary of what I have done so far:
When looking at the per version/branch doc generation, I noticed we had duplicate code in the Doxygen and Jsduck job templates, mostly related to extracting the version from either $ZUUL_BRANCH (ex: master, REL1_24) or $ZUUL_REF (ex: refs/tags/1.24.0). I thought it would be smarter to provide the doc subpath directly to the job instead of attempting to interpret it in different shell scripts / templates.
Turns out OpenStack already solved this problem with a hook in Zuul which let one craft injects additional parameters. So I created a tiny python script which provide a new variable $DOC_SUBPATH https://gerrit.wikimedia.org/r/#/c/173049/ the variable is only set for jobs those name match ^.*-publish$.
From there I started dropping the logic from the Doxygen job / script:
I have another patch pending to tweak Zuul configuration. The mediawiki-core-doxygen-publish job has a per branch filter applied which would discard ref-updated events: they don't have a branch and Zuul fallback to using the ref spec (ex: refs/tags/1.23.0) which would not pass. Should be fixed by: https://gerrit.wikimedia.org/r/#/c/175712/2/zuul/layout.yaml
Migrating the mediawiki/core JSDuck macro is proposed with https://gerrit.wikimedia.org/r/#/c/174120/2/jjb/mediawiki.yaml which gives a good idea of what need to be done for all other repositories.
An overview of $DOC_SUBPATH and the above is in the Zuul section of https://www.mediawiki.org/wiki/Continuous_integration/Documentation_generation . Might need to amend the wiki doc.
This bug actually triggered me in refactoring, sorry for not having updated anything beside the wiki page. I guess I now need to create a bunch of sub tasks and use this as a tracking.
Change 204791 had a related patch set uploaded (by Krinkle):
Split up misc *-publish from *-coverage
Change 204812 had a related patch set uploaded (by Krinkle):
Don't run mediawiki-core-doxygen-publish on wmf tags
Change 204812 merged by jenkins-bot:
Don't run mediawiki-core-doxygen-publish on wmf tags
Documentation for tags is available as well as multiple release branches. So that got fixed / regenerated at some point. Zuul has:
job: - name: mediawiki-core-doxygen-publish branch: ^(master|REL.*|refs/tags/(?!wmf).*)$ # REL is bug 50325 project: - mediawiki/core publish: - mediawiki-core-doxygen-publish