Page MenuHomePhabricator

Update Doxygen in CI to 1.8.15 or greater
Closed, ResolvedPublic

Description

This should fix T239481

I believe a fix was merged in https://github.com/doxygen/doxygen/pull/6549
According to https://github.com/doxygen/doxygen/pull/6549/commits/97f2d220240a1f6692627171398a06a3fa238639
the commit is in Release_1_8_16 Release_1_8_15
And we run:

12:20:14 + doxygen --version
12:20:14 1.8.13

Details

Related Gerrit Patches:
integration/config : masterjjb: bump to doxygen:0.6.1 for php extensions
integration/config : masterdocker: add php extensions to doxygen
integration/config : masterjjb: upgrade Doxygen to 1.8.16
integration/config : masterdocker: upgrade Doxygen to 1.8.16
operations/puppet : productionAdd component/ci for buster-wikimedia
operations/debs/doxygen : debian/buster-backportsBackports for Buster
integration/config : masterMake Debian glue voting for operations/debs/doxygen
operations/debs/doxygen : debian/buster-backportsBackports for Buster
integration/config : masterAdd Debian glue for operations/debs/doxygen

Event Timeline

Addshore created this task.Nov 29 2019, 5:22 PM

I have added a couple other tasks as blocked by that upgrade:

The doxygen package in Debian is orphaned (it does not have a dedicated maintainer), and overall it is a pain to get any patch upstreamed to Debian and probably very hard to get one to update an old Debian version.

We most probably just want to fork the Debian repository and maintainer our own copy in Gerrit (ex: operations/debs/doxygen) then publish it on our apt.wikimedia.org under a specific component (ex: component/doxygen).

A backport of the package made available as a component would fit nicely. If we follow what we do for other CI containers we would so something like:

Dockerfile.template
FROM {{ "docker-registry.wikimedia.org/wikimedia-buster" | image_tag }}
USER root
RUN echo "deb http://apt.wikimedia.org/wikimedia stretch-wikimedia component/doxygen" \
    > /etc/apt/sources.list.d/buster-doxygen.list \
    && {{ "doxygen" | apt_install }}
USER nobody
ENTRYPOINT ["doxygen"]

Which is super easy :] If an upgrade is needed later on:

  • build the new Debian package
  • upload to apt
  • bump image changelog
  • docker-pkg
  • done \o/

Debian buster comes with php 7.3 which should not be a problem for MediaWiki doxygen helper ( maintenance/mwdoc-filter.php ).

Change 553779 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Add Debian glue for operations/debs/doxygen

https://gerrit.wikimedia.org/r/553779

Change 553779 merged by jenkins-bot:
[integration/config@master] Add Debian glue for operations/debs/doxygen

https://gerrit.wikimedia.org/r/553779

Change 553780 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/debs/doxygen@debian/buster-backports] Backports for Buster

https://gerrit.wikimedia.org/r/553780

Got it build locally for buster (but haven't tested it yet).

Eventually CI fails to build the package due to an obsolete pristine-tar package:

gbp:debug: /usr/bin/pristine-tar [] ['checkout', '/srv/jenkins-workspace/workspace/debian-glue-non-voting/doxygen_1.8.16.orig.tar.gz']
pristine-tar: delta is version 3, newer than maximum supported version 2
pristine-tar: failed to generate tarball
gbp:error: Couldn't checkout "doxygen_1.8.16.orig.tar.gz": it exited with 255
Build step 'Execute shell' marked build as failure
hashar triaged this task as Medium priority.Nov 29 2019, 10:13 PM
Addshore moved this task from incoming to in progress on the Wikidata board.Dec 5 2019, 9:08 AM

Change 553780 merged by Hashar:
[operations/debs/doxygen@debian/buster-backports] Backports for Buster

https://gerrit.wikimedia.org/r/553780

Change 554942 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/debs/doxygen@debian/buster-backports] Backports for Buster

https://gerrit.wikimedia.org/r/554942

Change 554944 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Make Debian glue voting for operations/debs/doxygen

https://gerrit.wikimedia.org/r/554944

Change 554944 merged by jenkins-bot:
[integration/config@master] Make Debian glue voting for operations/debs/doxygen

https://gerrit.wikimedia.org/r/554944

I have a working backport for Doxygen 1.8.16 but that is using Buster. Our releng/doxygen container is based on Stretch..

I have a working backport for Doxygen 1.8.16 but that is using Buster. Our releng/doxygen container is based on Stretch..

Let's bump over whilst we're at it, then?

The Doxygen CI container is build using: doxygen < php72 < ci-stretch < stretch, so we got php7.2.

The reason we inherit from php7.2 is that MediaWiki core doc is generated via maintenance/mwdocgen.php which generates the Doxygen configuration and also uses mwdoc-filter.php. But I guess that one will work fine with php 7.3 provided by Buster.

The backport of Doxygen to Buster has been straightforward.

So I guess we can simplify the dependency chain by dropping our custom php image (releng/php7.2) and use the php from Buster instead. So the chain becomes:

doxygen < ci-buster < buster

And have the releng/doxygen to:

  • Include the apt configuration for where ever doxygen will be uploaded
  • install php-cli from Buster

Change 554942 merged by Muehlenhoff:
[operations/debs/doxygen@debian/buster-backports] Backports for Buster

https://gerrit.wikimedia.org/r/554942

Change 556984 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Add component/ci for buster-wikimedia

https://gerrit.wikimedia.org/r/556984

Change 556984 merged by Muehlenhoff:
[operations/puppet@production] Add component/ci for buster-wikimedia

https://gerrit.wikimedia.org/r/556984

Mentioned in SAL (#wikimedia-operations) [2019-12-13T10:30:16Z] <moritzm> uploaded doxygen 1.8.16-1~exp4~deb10+wmf1 to buster-wikimedia/component/ci T239482

Change 557033 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: upgrade Doxygen to 1.8.16

https://gerrit.wikimedia.org/r/557033

Change 557033 merged by jenkins-bot:
[integration/config@master] docker: upgrade Doxygen to 1.8.16

https://gerrit.wikimedia.org/r/557033

2019-12-13 15:33:08,737 [docker-pkg-build] INFO - Successfully tagged docker-registry.discovery.wmnet/releng/doxygen:0.6.0 (image.py:179)

:)

Change 557046 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] jjb: upgrade Doxygen to 1.8.16

https://gerrit.wikimedia.org/r/557046

hashar closed this task as Resolved.Dec 13 2019, 3:38 PM

And all jobs have been updated to use 1.8.16!

Change 557046 merged by jenkins-bot:
[integration/config@master] jjb: upgrade Doxygen to 1.8.16

https://gerrit.wikimedia.org/r/557046

hashar reopened this task as Open.Dec 13 2019, 5:55 PM

The polling job for mediawiki/core https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/ fails:

Status: Downloaded newer image for docker-registry.wikimedia.org/releng/doxygen:0.6.0
Error: Missing one or more required components of PHP.
You are missing a required extension to PHP that MediaWiki needs.
Please install:
 * mbstring <https://www.php.net/mbstring>
 * xml <https://www.php.net/xml>

Build step 'Execute shell' marked build as failure

Even though we do not really need them :]

Change 557081 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: add php extensions to doxygen

https://gerrit.wikimedia.org/r/557081

Change 557081 merged by jenkins-bot:
[integration/config@master] docker: add php extensions to doxygen

https://gerrit.wikimedia.org/r/557081

Change 557082 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] jjb: bump to doxygen:0.6.1 for php extensions

https://gerrit.wikimedia.org/r/557082

hashar closed this task as Resolved.Dec 13 2019, 6:09 PM

Change 557082 merged by jenkins-bot:
[integration/config@master] jjb: bump to doxygen:0.6.1 for php extensions

https://gerrit.wikimedia.org/r/557082