Our fork is marked 1.103-wmf.1. Patches are listed at https://gerrit.wikimedia.org/r/#/q/project:+integration/jenkinsci/xunit-plugin and we just have @Legoktm patch https://gerrit.wikimedia.org/r/#/c/integration/jenkinsci/xunit-plugin/+/425965/ Added skipped tag for phpunit xml.

Latest upstream is 2.3.5 - https://plugins.jenkins.io/xunit

The breaking change seems to be the introduction of a schema validation. The default is to just support Ant Junit and Maven Surefire output format. We can probably assume that PHPUnit and others use the same format. Anyway 2.0.0 had the patch included:

• JENKINS-42715 xUnit does not support PHPUnit 5.x skipped attribute (nfalco79 to ray)

So I guess it is safe to say PHPUnit produced output will work fine. I guess we should just upgrade the plugin then monitor.

Mentioned in SAL (#wikimedia-releng) [2020-05-28T13:31:24Z] <hashar> Rolling back Xunit Jenkins plugin # T194096

I should have mentioned it. 2.0.0 is a breaking change, the plugin now passes the xml through schema validation https://plugins.jenkins.io/xunit/

Turns out the script we use to convert lintian output to junit is not recognized / invalid :/

The Jenkins xunit plugin embeds a bunch of XSD files https://github.com/jenkinsci/xunit-plugin/tree/master/src/main/resources/org/jenkinsci/plugins/xunit/types

With Xunit 2.0.0, they started validating the given XML files against the XSD, the default being junit. For other set of tools they are validating them as well then use an XSL to normalize the input to be Junit compatible (as I understand it).

The default XSD can be retrieved at https://github.com/jenkinsci/xunit-plugin/blob/master/src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd

So:

git clone https://github.com/jenkinsci/xunit-plugin
xmllint \
  --noout \
  --schema ./xunit-plugin/src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd \
  lintian.xml

lintian.xml:2: element testsuite: Schemas validity error : Element 'testsuite', attribute 'assertions': The attribute 'assertions' is not allowed.
lintian.xml:2: element testsuite: Schemas validity error : Element 'testsuite': The attribute 'name' is required but missing.
lintian.xml:3: element testcase: Schemas validity error : Element 'testcase', attribute 'assertions': The attribute 'assertions' is not allowed.
lintian.xml fails to validate

Or with the PHUnit one:

lintian-orig.xml:2: element testsuite: Schemas validity error : Element 'testsuite': The attribute 'name' is required but missing.
lintian-orig.xml fails to validate

If add name='Name of suite' to the <testsuite> element, it validates just fine.

The lintian-junit-report script we use comes from jenkins-debian-glue. At the top it refers to http://windyroad.com.au/dl/Open%20Source/JUnit.xsd which states JUnit test result schema for the Apache Ant JUnit and JUnitReport tasks but that does not even validate against it:

$ xmllint --noout --schema JUnit.xsd lintian.xml 
lintian-orig.xml:2: element testsuite: Schemas validity error : Element 'testsuite', attribute 'skipped': The attribute 'skipped' is not allowed.
lintian-orig.xml:2: element testsuite: Schemas validity error : Element 'testsuite', attribute 'assertions': The attribute 'assertions' is not allowed.
lintian-orig.xml:2: element testsuite: Schemas validity error : Element 'testsuite': The attribute 'name' is required but missing.
lintian-orig.xml:2: element testsuite: Schemas validity error : Element 'testsuite': The attribute 'timestamp' is required but missing.
lintian-orig.xml:2: element testsuite: Schemas validity error : Element 'testsuite': The attribute 'hostname' is required but missing.
lintian-orig.xml:3: element testcase: Schemas validity error : Element 'testcase': This element is not expected. Expected is ( properties ).
lintian-orig.xml fails to validate

Does that mean we need to get Debian upstream to fix lintian and release it?

A quick and dirty fix is https://github.com/mika/jenkins-debian-glue/pull/233 :]

https://github.com/mika/jenkins-debian-glue/pull/233 got merged by upstream with congratulations \o/ I am so happy.