1) TitlePermissionTest::testQuickPermissions Failed asserting that two arrays are equal. --- Expected +++ Actual @@ @@ Array ( 0 => Array ( 0 => 'badaccess-groups' - 1 => '*, [[Traviswiki:Users|Users]]' + 1 => '*, [[MyWiki:Users|Users]]' /home/travis/build/wikimedia/mediawiki/tests/phpunit/includes/TitlePermissionTest.php:324 /home/travis/build/wikimedia/mediawiki/tests/phpunit/MediaWikiTestCase.php:475 /home/travis/build/wikimedia/mediawiki/maintenance/doMaintenance.php:94 2) TitlePermissionTest::testPageRestrictions Failed asserting that two arrays are equal. --- Expected +++ Actual @@ @@ Array ( 0 => Array ( 0 => 'badaccess-groups' - 1 => '*, [[Traviswiki:Users|Users]]' + 1 => '*, [[MyWiki:Users|Users]]' /home/travis/build/wikimedia/mediawiki/tests/phpunit/includes/TitlePermissionTest.php:687 /home/travis/build/wikimedia/mediawiki/tests/phpunit/MediaWikiTestCase.php:475 /home/travis/build/wikimedia/mediawiki/maintenance/doMaintenance.php:94
- Mentioned In
- T209056: Use Travis CI cache and reduce git clone depth to speed up build setup time
T202050: Several travis-ci builds failing for mediawiki
- Mentioned Here
- T206130: Non logged in users are able to review and patrol pages on enwiki using the API
T201900: PHP Notice: Trying to get property 'num_rows' of non-object in /home/travis/build/wikimedia/mediawiki/includes/libs/rdbms/database/DatabaseMysqli.php on line 233 on PHP 7.2 travis builds
This looks related to the ContentLanguage service change, not the ParserFactory one. I can't reproduce the failure locally, but probably adding an overrideMwServices() to setUp() will fix the problem.
Well, one solution would be to force the sitename to MyWiki for these tests, right? Or we could just leave them disabled. (Assuming nobody wants to figure out what the actual problem is, which isn't so easy if it can't be reproduced locally.)
Yes, tests that produce asserted values that vary by configuration should mock or set those configuration key explicitly. That's done in most other tests as well. I imagine it may've been avoided here if it's difficult to do so, but until we find away, the test should't be breaking CI for several days, so I've re-reverted it for now.
See also https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/464386/ / T206130, which triggered a similar build failure.
I can reproduce by editing suite.xml to run only HtmlTest.php, TitleMethodsTest.php, and TitlePermissionTest.php (by adding <exclude> for everything else in includes/). Now working on narrowing down more.
Yep, but doing it for everything has larger consequences and may break compat with existing tests as we found. Let's first unbreak the immediate test and improve the other tests that aren't broken, later :)
I've submitted https://gerrit.wikimedia.org/r/465537 which resets the site name for that test specifically. That's our convention elsewhere and is what should've been done in the first place.
Change 465537 abandoned by Krinkle:
title: Fix flaky TitlePermission test cases
I give up. Something somewhere is messing with the global state in the shared job. Presumably in such a way that TestCase is unable to reset.
That probably means our resets are too brave by stashing and restoring, instead of actually clearing/resetting.
Anyhow, given we've spent 7 days and over a dozen commits trying to do this, I give up. Kosta has a patch that reduces the test to not needlessly assert what the wgSitename value is, which makes a ton of sense, given the test isn't about that at all. I'm merging that now.
We can continue pondering about why we can't have nice resets after that, including how to make them work with our existing problems in SimpleCaptcha, Flow, and Wikibase.