Page MenuHomePhabricator

MW core doxygen / jsduck build should run for git tags (again)
Closed, ResolvedPublic

Description

15:13:12 <Krinkle> James_F: Looks like jsduck/doxygen stopped running on tags
15:13:13 <Krinkle> https://doc.wikimedia.org/mediawiki-core/
15:13:25 <Krinkle> stopped at 1.30.0 and non-zero times for REL1_32
15:13:43 <Krinkle> REL1_33 as well.
15:13:46 <Krinkle> So I guess the branch trigger is working
15:13:49 <Krinkle> but tag stopped
15:14:27 <Krinkle> https://github.com/wikimedia/integration-config/commit/8f9b3528176eca6c60dee0bb8b2c65a3cd21ba2a
15:14:28 <Krinkle> Aha
15:14:38 <Krinkle> It was changed from postmerge/publish to an hourly cron
15:14:42 <Krinkle> which watches branches only

8f9b3528176eca6c60dee0bb8b2c65a3cd21ba2a

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 4 2019, 10:15 PM
hashar updated the task description. (Show Details)Sep 5 2019, 2:28 PM
hashar added a subscriber: hashar.

eek, note the job is explicitly made to NOT process tags. Either because I could not figure out how to trigger on tag change or I considered that having document per branch was sufficient.

Krinkle added a subscriber: Krinkle.
From T236600

See also T73062: doc.wikimedia.org: Generate documentation for release tags (tracking).
Looks like this has stopped working again, or rather, it seems the Zuul config was removed?

Krinkle renamed this task from MW core doxygen / jsduck build doesn't run for tags, just branches to MW core doxygen / jsduck build should run for git tags (again).Sat, Oct 26, 7:50 PM

Change 546338 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Enable 'publish' pipeline to generate js docs for mediawik/core

https://gerrit.wikimedia.org/r/546338

Change 546339 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Create regular job (not cron) for mwcore-doxygen and run for git tags

https://gerrit.wikimedia.org/r/546339

Change 546338 merged by jenkins-bot:
[integration/config@master] Enable 'publish' pipeline to generate js docs for mediawik/core

https://gerrit.wikimedia.org/r/546338

Change 546340 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] castor: Fallback to loading 'master' cache if ZUUL_BRANCH unset

https://gerrit.wikimedia.org/r/546340

From what I can tell, it should work just fine to use the same job for postmerge as for publish.

The code we use to compute DOC_SUBPATH (e.g. "master", "REL1_34" or "1.2.3") already supports refs/tags/.*.

I tried this out for unicodejs, which publishes to https://doc.wikimedia.org/unicodejs/ but has stopped working 2 years ago. The last doc publish was for v0.2.0 but there has since been v0.2.2 and v12.0.0.

I replayed the Gerrit event on contint1001 that created one of these newer tags, like so:

krinkle@contint1001:~$ zuul enqueue-ref --trigger gerrit --pipeline publish --project unicodejs --ref 'refs/tags/v0.2.2' --newrev '2b81e56f2da9c8d33c2ca1d0670d41d8a06b0716' --oldrev '0000000000000000000000000000000000000000'

This correctly triggered the job at https://integration.wikimedia.org/ci/job/generic-node10-docs-docker-publish/57/console.

However, it fails early when Castor runs, due to unconditionally using ZUUL_BRANCH:

+ exec docker run … docker-registry.wikimedia.org/releng/castor:0.2.3 load
…
castor/castor-define-namespace.bash: line 7: ZUUL_BRANCH: unbound variable

I temporarily edited the generic-node10-docs-docker-publish job in the Jenkins administration to set ZUUL_BRANCH to see if this is the only issue. After setting it and replaying the event again, it works as expected:

https://integration.wikimedia.org/ci/job/generic-node10-docs-docker-publish/59/console
https://doc.wikimedia.org/unicodejs/v0.2.2/

Yay :)

Change 546341 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] jjb: Upgrade castor-load macro to castor:0.2.4

https://gerrit.wikimedia.org/r/546341

Change 546340 merged by jenkins-bot:
[integration/config@master] castor: Fallback to loading 'master' cache if ZUUL_BRANCH unset

https://gerrit.wikimedia.org/r/546340

Change 546341 merged by jenkins-bot:
[integration/config@master] jjb: Upgrade castor-load macro to castor:0.2.4

https://gerrit.wikimedia.org/r/546341

Mentioned in SAL (#wikimedia-releng) [2019-10-26T20:47:17Z] <Krinkle> krinkle@contint1001$ zuul enqueue-ref --trigger gerrit --pipeline publish --project unicodejs --ref 'refs/tags/v12.0.0' --newrev '63ab6302019d3b6b1b762b5c0094370e3030e94c' --oldrev '0000000000000000000000000000000000000000' for T232055

Change 546342 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Enable publish pipeline for oojs

https://gerrit.wikimedia.org/r/546342

Change 546342 merged by jenkins-bot:
[integration/config@master] Enable publish pipeline for oojs

https://gerrit.wikimedia.org/r/546342

Mentioned in SAL (#wikimedia-releng) [2019-10-26T20:54:32Z] <Krinkle> krinkle@contint Replay oojs/core git tags to zuul 'publish' pipeline for T232055

Change 546343 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Convert 'mediawiki-core-jsduck-docker-publishæ filter to skip-if

https://gerrit.wikimedia.org/r/546343

The Castor issue is fixed, and – Tag published documentation now works fine for library repos!

Next: MediaWiki core.

Change 546338 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Enable 'publish' pipeline to generate js docs for mediawik/core
https://gerrit.wikimedia.org/r/546338

The mediawiki-core-jsduck-docker-publish job isn't starting when I replay a mediawiki/core git tag event because it is rejected by its branch filter in zuul/layout.yaml:

zuul/layout
jobs:
  - name: mediawiki-core-jsduck-docker-publish
    branch: ^(REL.*|master)$

projects:
  - name: mediawiki/core
    postmerge:
      - mediawiki-core-jsduck-docker-publish
    publish:
      - mediawiki-core-jsduck-docker-publish

This is because implicit in having a branch filter, Zuul also rejects any event that does not have a change.branch attribute. To fix this, I have converted it to a skip-if filter. So instead of only allowing events that have a branch and match the filter, it will now only reject events that have a branch and match the filter.

- name: mediawiki-core-jsduck-docker-publish
  skip-if:
    - branch: ^wmf/.*$

Change 546343 merged by jenkins-bot:
[integration/config@master] Convert 'mediawiki-core-jsduck-docker-publish' filter to skip-if

https://gerrit.wikimedia.org/r/546343

Mentioned in SAL (#wikimedia-releng) [2019-10-26T21:25:51Z] <Krinkle> krinkle@contint zuul enqueue-ref --trigger gerrit --pipeline publish --project mediawiki/core --ref 'refs/tags/1.30.1' --newrev 'a0b31d95610c8f415a2cc29349cdbf14c23c7d4b' --oldrev '0000000000000000000000000000000000000000' for T232055

Change 546339 merged by jenkins-bot:
[integration/config@master] Create regular job (not cron) for mwcore-doxygen and run for git tags

https://gerrit.wikimedia.org/r/546339

Mentioned in SAL (#wikimedia-releng) [2019-10-27T00:18:16Z] <Krinkle> krinkle@contint Replay various mediawiki/core git tags to zuul 'publish' pipeline for T232055

hashar awarded a token.Thu, Nov 7, 7:29 PM
DannyS712 added a subscriber: DannyS712.

[batch] remove patch for review tag from resolved tasks