Seems sporadic or already fixed: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/OAuth/+/582768

Happened while doing https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/582857, so it's not "already fixed".

The relevant command is

SECCHECK_MODE=mwext-fast-config.php
cd /mediawiki/extensions/OAuth
PHAN=/opt/phan/vendor/phan/phan/phan
CONFIG=/opt/phan/vendor/mediawiki/phan-taint-check-plugin/scripts/mwext-fast-config.php
php -dextension=ast_1.0.1.so /opt/phan/vendor/phan/phan/phan -d . -k /opt/phan/vendor/mediawiki/phan-taint-check-plugin/scripts/mwext-fast-config.php --output php://stdout -m checkstyle
tee /mediawiki/seccheck-issues

so this is microsoft/tolerant-php-parser in OAuth's own vendor dir, not the MediaWiki vendor. (The extension depends on mediawiki/mediawiki-phan-config which depends on phan which depends on tolerant-php-parser.) Indeed, it's in the composer log for the extension (at 09:16:43).

The file that results in the error has been in tolerant-php-parser since forever, so this must be a change in what directories are parsed by phan. Given the apparent stochastic behavior, maybe a race condition with the creation of some phan config file?

This also happened on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CheckUser/+/583339 . @Reedy investigated this briefly, but was confused because there's a specific exclusion for this directory in the phan config

Why are we getting the test files when there's https://github.com/microsoft/tolerant-php-parser/blob/master/.gitattributes and it's been there for ages (unless we're using some ancient branch)?

And also https://github.com/wikimedia/mediawiki-tools-phan/blob/master/src/config.php#L125 shouldn't be looking there...

11:15:36 [265.2MB/29.03s]   - Installing microsoft/tolerant-php-parser (v0.0.18): [265.2MB/29.03s] Downloading[265.2MB/29.10s]     Failed to download microsoft/tolerant-php-parser from dist: Could not authenticate against github.com
11:15:36 [265.2MB/29.10s]     Now trying to download from source
11:15:36 [265.2MB/29.10s]   - Installing microsoft/tolerant-php-parser (v0.0.18): [265.2MB/29.75s] Cloning e255aa978b from cache

e255aa978b is https://github.com/microsoft/tolerant-php-parser/releases/tag/v0.0.18 which has the tests export-ignore in .gitattributes

But if it's doing it like this.. Is it git cloning the whole repo, bringing it all in?

Which doesn't explain why https://github.com/wikimedia/mediawiki-tools-phan/blob/master/src/config.php#L125 isn't taking effect, but is possibly a side effect of T248387: Could not authenticate against github.com ?

For anyone following along:

[11:40:33] <Daimona> Reedy: I know what's wrong with the tolerant parser thingy. I'll write down an explanation later today, as soon as I have a bit of free time

So, things to consider here:

1. The issue comes from taint-check, not phan itself. Hence, the mediawiki-tools-phan repo isn't playing any role here.
   1. Specifically, https://github.com/wikimedia/mediawiki-tools-phan/blob/master/src/config.php is NOT the file used by taint-check
2. Taint-check runs in its own CI job, separated from the main phan job. It also uses a different version of phan (latest stable uses phan 1.3.2
3. Phan 1.3.2 does have tests marked as export-ignore, but somehow that isn't working (?)
4. Taint-check also uses its own config file, which does NOT exclude test files
5. Additionally, PHP 7.2 is bugged, in that it throws a fatal ("Cannot use the final modifier on an abstract class") when tokenizing invalid code, even if that code isn't going to be executed. See phan issue #3407 and linked PHP bug report.

As for the solution: in theory, it's possible to udate the config files of taint-check, tag a release, and upgrade. In practice, I'd like to have version 3.0.1 released (T240069). That includes bumping phan to the latest version available. After doing that, we can merge taint-check into the phan job, so that they also use the same config, and we don't have to apply config changes twice (T235390).

Change 589294 had a related patch set uploaded (by Daimona Eaytoy; owner: Daimona Eaytoy):
[mediawiki/tools/phan/SecurityCheckPlugin@master] Exclude invalid PHP files from analysis

https://gerrit.wikimedia.org/r/589294

Change 589294 merged by jenkins-bot:
[mediawiki/tools/phan/SecurityCheckPlugin@master] Exclude invalid PHP files from analysis

https://gerrit.wikimedia.org/r/589294

Change 602499 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/tools/phan@master] Require taint-check 3.03, up from 3.0.2

https://gerrit.wikimedia.org/r/602499

Change 602499 merged by jenkins-bot:
[mediawiki/tools/phan@master] Require taint-check 3.0.3, up from 3.0.2

https://gerrit.wikimedia.org/r/602499

Change 629060 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/tools/phan@master] Release 0.10.3

https://gerrit.wikimedia.org/r/629060