Page MenuHomePhabricator

UpdateEmergencyDefaultsTest::testUpdateOK: Failed asserting that true is an instance of class "stdClass".
Closed, ResolvedPublic

Description

The following failure was seen in r1248425 for CirrusSearch (job https://integration.wikimedia.org/ci/job/quibble-with-gated-extensions-vendor-mysql-php83/17794/console), which is in turn trying to fix shared failure T419113:

1) MediaWiki\Extension\CommunityConfiguration\Tests\Integration\UpdateEmergencyDefaultsTest::testUpdateOK
Failed asserting that true is an instance of class "stdClass".

/workspace/src/extensions/CommunityConfiguration/tests/phpunit/integration/maintenance/UpdateEmergencyDefaultsTest.php:61
Logs generated by test
=== Logs generated by test case
[Wikibase] [debug] {method}: setting {settingName} was given as a closure, resolve it to {logValue} {"method":"Wikibase\\Lib\\SettingsArray::getSetting","settingName":"entitySources","logValue":"array (\n  'local' => \n  array (\n    'entityNamespaces' => \n    array (\n      'item' => 120,\n      'property' => 122,\n      'mediainfo' => '6\/mediainfo',\n    ),\n    'repoDatabase' => false,\n    'baseUri' => 'http:\/\/127.0.0.1:9413\/entity\/',\n    'rdfNodeNamespacePrefix' => 'wd',\n    'rdfPredicateNamespacePrefix' => '',\n    'interwikiPrefix' => '',\n  ),\n)"}
[objectcache] [debug] MainWANObjectCache using store {class} {"class":"Wikimedia\\ObjectCache\\HashBagOStuff"}
[localisation] [debug] LocalisationCache using store MediaWiki\Language\LCStoreNull []
[wfDebug] [debug] MediaWiki\Parser\ParserFactory: using default preprocessor {"private":false}
[session] [debug] SessionManager using store Wikimedia\ObjectCache\HashBagOStuff []
===

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I can't reproduce the failure locally, but that might be due to my setup. If not, someone could try replicating the quibble run, but I don't have time for that right now.

No idea what is going on, but I'm now looking into it.

No idea what is going on, but I'm now looking into it.

In case we're not quickly able to figure out what is going on and fix it, that test is one that should be safe to skip.

fwiw, I tried running that locally (this tool makes it pretty quick to reproduce runs that don't depend on multiple patches) and also didn't get any error from two separate runs on my machine.

Change #1248467 had a related patch set uploaded (by Michael Große; author: Michael Große):

[mediawiki/extensions/CommunityConfiguration@master] Skip strangely broken test

https://gerrit.wikimedia.org/r/1248467

Change #1248468 had a related patch set uploaded (by Arendpieter; author: Arendpieter):

[mediawiki/extensions/CommunityConfiguration@master] tests: Fix require_once returning true in UpdateEmergencyDefaultsTest

https://gerrit.wikimedia.org/r/1248468

Change #1248467 merged by Mszwarc:

[mediawiki/extensions/CommunityConfiguration@master] Skip strangely broken test

https://gerrit.wikimedia.org/r/1248467

mszwarc assigned this task to Michael.

Issue fixed by skipping test. The patch has been force-merged, so that we're able to unblock the CI with the patch for CirrusSearch.

@Arendpieter's change should fix it. The most likely hypothesis is for me that through the test-parallelization, a different test requiring the same file (\MediaWiki\Extension\CommunityConfiguration\Tests\SchemaProviderTestCase::testSchemaDefaultValues) happened to run earlier, causing the require_once in this test to return true instead of the value of the required file.

fwiw, I tried running that locally (this tool makes it pretty quick to reproduce runs that don't depend on multiple patches) and also didn't get any error from two separate runs on my machine.

First: amazing tool, thanks for building it! Second: That is VERY weird... I didn't expect any differences, but I too tried that and it passed.

@Arendpieter's change should fix it. The most likely hypothesis is for me that through the test-parallelization, a different test requiring the same file (\MediaWiki\Extension\CommunityConfiguration\Tests\SchemaProviderTestCase::testSchemaDefaultValues) happened to run earlier, causing the require_once in this test to return true instead of the value of the required file.

Hmmmm I suppose in principle it's possible that the changes for T345481 changed the test ordering, especially in combination with the custom test splitter.

At any rate, I imagine we don't really have to investigate this now that it works.

Change #1248468 merged by jenkins-bot:

[mediawiki/extensions/CommunityConfiguration@master] tests: Fix require_once returning true in UpdateEmergencyDefaultsTest

https://gerrit.wikimedia.org/r/1248468

If it the issue was caused by test grouping / ordering (and based on the fix it probably was), it's worth noting that the test grouping is only stable for a given results cache. We're running jobs all the time that are updating a global cache of test timings, and that it what determines how the tests are split into groups. Although that's all written so that the ordering of the tests within buckets is stable, it could be that new timing information came it that pushed this test from a bucked where require had already been run, to a bucket where require had not.