When the mediawiki-core-php70-phan-docker job fails due to the patch adding phan "error" level violations, the link reported to Gerrit by jenkins-bot takes me to https://integration.wikimedia.org/ci/job/mediawiki-core-php70-phan-docker/1271/checkstyleResult/, which says there were "0 warnings" in total, including all priorities.
Clicking on the Console Output shows that there were, in fact, some errors output. The relevant bit, since I know these eventually expire:
19:29:04 [mediawiki-core-php70-phan-docker] $ /bin/bash -xe /tmp/jenkins3158697743129440311.sh 19:29:04 ++ pwd 19:29:04 + exec docker run --rm --volume /srv/jenkins-workspace/workspace/mediawiki-core-php70-phan-docker/src:/mediawiki wmfreleng/mediawiki-phan:v2017.10.16.20.38 -m checkstyle 19:29:04 + umask 002 19:29:04 + /mediawiki/tests/phan/bin/phan -m checkstyle 19:30:00 <?xml version="1.0" encoding="ISO-8859-15"?> 19:30:00 <checkstyle version="6.5"> 19:30:00 <file name="includes/user/User.php"> 19:30:00 <error line="2372" severity="warning" message="Parameter of undeclared type \IDatabase" source="PhanUndeclaredTypeParameter"/> 19:30:00 <error line="2404" severity="error" message="Call to method insert from undeclared class \IDatabase" source="PhanUndeclaredClassMethod"/>.. 19:30:00 <error line="4410" severity="warning" message="Parameter of undeclared type \IDatabase" source="PhanUndeclaredTypeParameter"/> 19:30:00 <error line="4414" severity="error" message="Call to method insert from undeclared class \IDatabase" source="PhanUndeclaredClassMethod"/>.. 19:30:00 </file> 19:30:00 <file name="maintenance/reassignEdits.php"> 19:30:00 <error line="190" severity="warning" message="Parameter of undeclared type \IDatabase" source="PhanUndeclaredTypeParameter"/> 19:30:00 </file> 19:30:00 </checkstyle> 19:30:00 Build step 'Execute shell' marked build as failure 19:30:00 [PostBuildScript] - Execution post build scripts. 19:30:00 [PostBuildScript] Build is not success : do not execute script 19:30:00 [CHECKSTYLE] Collecting checkstyle analysis files... 19:30:00 [CHECKSTYLE] Searching for all files in /srv/jenkins-workspace/workspace/mediawiki-core-php70-phan-docker that match the pattern src/tests/phan/issues/latest 19:30:00 [CHECKSTYLE] No files found. Configuration error? 19:30:00 [CHECKSTYLE] Computing warning deltas based on reference build #1270 19:30:00 [CHECKSTYLE] Ignore new warnings since this is the first valid build 19:30:00 [CHECKSTYLE] Plug-in Result: Success - no threshold has been exceeded 19:30:00 Finished: FAILURE
Is it writing the test result data to standard output instead of to the expected file, perhaps?