At least PHP works since that generates https://doc.wikimedia.org/

The file /srv/docroot/org/wikimedia/doc/mediawiki-core/master/php/search.php has literally the same content:

<div id="main-nav"></div>
<script language="php">
require_once "search_functions.php";
main();
</script>
</div><!-- doc-content -->

Generated by Doxygen 1.8.13

Maybe that was interpreted by PHP 5 and is not since we switched to PHP 7?

Yes from http://php.net/manual/en/language.basic-syntax.phptags.php

Changelog

Doxygen comes from Debian Stretch https://packages.debian.org/stretch/doxygen

One would have to look at Doxygen upstream to check whether there is a commit fixing it. And then we need a patch to be send to Debian to patch the Stretch package.

If we don't want to wait for Upstream Doxygen or Debian, we can always fork the Debian package, include the fix and upload it on our apt.wikimedia.org (then rebuild the affected CI containers).

https://github.com/doxygen/doxygen/issues/3545

PHP was not parsed properly when it appeared in a <script language="php"> section.

doxygen closed this on 2 Jul 2018

Fixed by https://github.com/doxygen/doxygen/pull/680 / 22b67836d678cea695b977ec648c0aa013339c55.

Released in 1.8.15:

$ git tag --contains 22b67836d678cea695b977ec648c0aa013339c55
Release_1_8_15

Looks like all the Debian channels (jessie, stretch, buster, sid) are still on 1.8.13. Last update was last year (1.8.13-10) indicating that the package no longer has a Debian maintainer (discussion).

I suppose a cherry-pick to our own repo would be the quickest way for now. Alternatively, one of our own Debian maintainers might be able to patch in Debian instead.

I have send a bug to Debian but I have not received the automatic acknowledgement yet :(

My local email server configuration was screwed up. The bug is: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=925068

From: Paolo Greppi
Date: Wed, 20 Mar 2019 18:52:04 +0100

Hi Antoine,

and thanks for the good and well documented report.

I agree that for consistency doxygen search should work with the PHP version we ship in any given
Debian release.
And this issue also applies to buster as it will ship doxygen 1.8.13 and php 7.3.

Fixing it for buster is higher priority, and a solution could be to backport doxygen 1.8.15,
but before that we need it in testing: https://bugs.debian.org/919413
Then for stretch, backporting it to stretch too is an option if it is painless.

Another option is to just backport the upstream fix.

I propose that we keep this on hold for now.

Paolo

From: Antoine Musso
Date: Wed, 20 Mar 2019, 22:24

Hello Paolo,

Thank you for the quick reply and all the context with 1.8.15 status. I hope it makes it to testing and get included in buster!

Regarding this bug, I will get our team to rebuild the Debian package with the patch included (22b67836), test it, deploy it and monitor how our documentations behave. At least that will fix it for us.

Then maybe that will build enough confidence to have the patch included in the Stretch package?

In T218233#5036593, @Krinkle wrote:

Looks like all the Debian channels (jessie, stretch, buster, sid) are still on 1.8.13. Last update was last year (1.8.13-10) indicating that the package no longer has a Debian maintainer (discussion).

There is a maintainer but he is willing to step back since Doxygen has a lot of Javascript and it causes other packages to fail eventually. so that ends up being a burden. At least Paolo replied to my bug ;]

I suppose a cherry-pick to our own repo would be the quickest way for now.

Yes we can fork it to our Gerrit, pick the patch and get Operations to rebuild and upload the package. Then we would have to mess a bit with apt configuration to fetch it but it is not the end of the world usually.

Alternatively, one of our own Debian maintainers might be able to patch in Debian instead.

That is a nmu (non maintainer upload) and I have absolutely no idea what the policy is on that regard. Maybe after we tested/deployed the cherry pick that will give the package maintainer enough confidence to include the patch and release it for Debian.

@hashar I built patched debs for buster at https://people.debian.org/~legoktm/doxygen/, could you test those out before I build fixed versions for stretch (assuming the defacto Debian maintainer is OK with my plan)?

stretch was easier than I expected, patched packages are at https://people.debian.org/~legoktm/doxygen-stretch/ - please test :)

Bump. This came up again at the Hackathon and made documentation significantly harder to navigate for developers.

Basically, you have to be an experienced contributor and an experienced Doxygen user to 1) Know that search is broken - the blank results page isn't just a fluke, 2) Know the exact name of the class you are looking for, search by description or method is not possible, 3) Know that Doxygen has an "all class list" somewhere and where that somewhere is and use that.

Failing that, MediaWiki PHP documentation is currently unavailable to users.

I poked a different upstream meanwhile.

@MoritzMuehlenhoff we would need to fork the Stretch Debian package for Doxygen. What would be the best course of action for that please? Kunal already rebuild it with a patch at https://people.debian.org/~legoktm/doxygen-stretch/ , but I am not sure what are SRE requirements. Should it be first forked to our Gerrit repo then rebuild by SRE and uploaded to apt.wikimedia.org ?

There's two angles to address:

1. Getting this fixed internally

@hashar Did you confirm that Kunal's test package fixes the problem? If so we can rebuild on boron as 1.8.13-4~wmf1 and import to stretch-wikimedia/main.

2. Getting this fixed in unstable/buster (which is also a requirement that the patch can land in a Stretch point release which obsoletes our internal update):

doxygen is orphaned and while an "adoption" is in progress every DD can do a "QA upload", i.e. apply https://github.com/doxygen/doxygen/commit/22b67836d678cea695b977ec648c0aa013339c55 and upload as 1.8.13-5. The patch is small enough and fixes a genuine issue, so I assume it would still be eligible to enter buster in the freeze phase (while a new 1.8.15 release would not). I don't have the time currently to commit do that myself, though.

The example given works now! https://doc.wikimedia.org/mediawiki-core/master/php/search.php?query=ObjectCache

Still have to regenerate the documentation for all repositories, all branches and all tags.

To regenerate the doc for mediawiki/core tags, on contint1001 I am issuing:

TAG=1.23.17 bash -c 'zuul enqueue-ref --trigger gerrit --pipeline publish --project mediawiki/core --ref $TAG --newrev $TAG'

:)

Mentioned in SAL (#wikimedia-releng) [2019-12-13T20:19:44Z] <hashar> Regenerating MediaWiki core doxygen (and jsduck doc) # T218233

mediawiki/core tags are being regenerated.

For branches, the job https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/ uses the Jenkins git plugin which polls the repository. I am not sure how to reset it, the console log shows:
getCandidateRevisions(false,null,,,hudson.plugins.git.util.BuildData@a762a759[scmName=<null>,remoteUrls=[https://gerrit.wikimedia.org/r/p/mediawiki/core.git],buildsByBranchName={origin/master=Build #12103 of Revision 980666d5c7eb8fc43f5b0770be579bfaf8091c05 (origin/master), origin/REL1_28=Build #11 of Revision e9f315ccc13104597c681cf4f7da0d0dab5450f1 (origin/REL1_28), origin/REL1_29=Build #1540 of Revision 0e704369adf97f85d71006060bda1c0f9315e75a (origin/REL1_29), origin/REL1_26=Build #9 of Revision 2e3e7395f1f290fff646510233bf6386fcf01a5d (origin/REL1_26), origin/REL1_27=Build #7460 of Revision 9acd49ea5e13e1a7150f4de07d87e98e85a08509 (origin/REL1_27), origin/REL1_25=Build #8 of Revision 01b4001e707ea9535810af364fec2b0cba0f5735 (origin/REL1_25), origin/REL1_33=Build #12046 of Revision a8d6ea4d438122c8d35bd3a02350a104965f587d (origin/REL1_33), origin/REL1_23=Build #10 of Revision 57062b01fdeafa5d63387b45b92bd2350d220a88 (origin/REL1_23), origin/REL1_34=Build #12030 of Revision f0492a17150e915516a7be07fee2a3115df80ce6 (origin/REL1_34), origin/REL1_31=Build #12031 of Revision e1ceccda7242006d507f5d74ad2e29f94fb62e09 (origin/REL1_31), origin/REL1_32=Build #12065 of Revision 591523d7773d8c81a3fdea0740fd1e2855ed4702 (origin/REL1_32), origin/REL1_30=Build #7452 of Revision 41405a1239bda1e725510c6791a8d8b32392b277 (origin/REL1_30)},lastBuild=Build #12103 of Revision 980666d5c7eb8fc43f5b0770be579bfaf8091c05 (origin/master)]) considering branches to build

So I guess it just look at the previous builds.

I have nuked the build history on contint1001 in /srv/jenkins/builds/mediawiki-core-doxygen-docker

The git plugin no more has any history about what it has build:

getCandidateRevisions(false,null,,,hudson.plugins.git.util.BuildData@32fde4b3[scmName=<null>,remoteUrls=[https://gerrit.wikimedia.org/r/p/mediawiki/core.git],buildsByBranchName={},lastBuild=null])

And would build the REL1_25 branch:

Seen branch in repository origin/REL1_25
Starting with all the branches: [
  ...
  Revision 01b4001e707ea9535810af364fec2b0cba0f5735 (origin/REL1_25),
  ...
  ]
After branch filtering: Revision 01b4001e707ea9535810af364fec2b0cba0f5735 (origin/REL1_25) ...
After non-tip filtering: Revision 01b4001e707ea9535810af364fec2b0cba0f5735 (origin/REL1_25) ...
Removing what's already been built: {}
After filtering out what's already been built: Revision 01b4001e707ea9535810af364fec2b0cba0f5735 (origin/REL1_25) ...

Multiple candidate revisions
Scheduling another build to catch up with mediawiki-core-doxygen-docker
Checking out Revision 01b4001e707ea9535810af364fec2b0cba0f5735 (origin/REL1_25)

It fails later due to mediawiki/vendor missing that branch:

+ git clone --depth 1 --reference /srv/git/mediawiki/vendor.git --branch REL1_25 -- https://gerrit.wikimedia.org/r/p/mediawiki/vendor.git src/vendor
Cloning into 'src/vendor'...
warning: Could not find remote branch REL1_25 to clone.
fatal: Remote branch REL1_25 not found in upstream origin

Not a big deal, we should remove that branch from mediawiki/core anyway.

The job is currently regenerating builds for all branches.

For other repositories, we can find obsolete documentations on contint1001 with:

(find /srv/docroot/org/wikimedia/doc -name index.html -exec grep -H 'Generated by Doxygen' {} \;)|grep -v '1.8.16 '

I have retriggered all builds. For tags using:

TAG=1.27.4 bash -c 'zuul enqueue-ref --trigger gerrit --pipeline publish --project mediawiki/core --ref $TAG --newrev $TAG'

For branches, by looking in Gerrit for the latest change and enqueuing it manually in the postmerge pipeline. Example:

zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/extensions/Kartographer --change 440856,2

What is left are 1.27 / 1.28 tags for MediaWiki core. They fail due to some missing dependencies when running maintenance/mwdocgen.php . But given they are long EOL, I guess we can just get rid of them.