Page MenuHomePhabricator

Travis CI failures in ApiQueryLanguageinfoTest
Closed, ResolvedPublic

Description

Ping T220415

https://travis-ci.org/wikimedia/mediawiki/jobs/542392942

There were 2 failures:
1) ApiQueryLanguageinfoTest::testAllPropsForSingleLanguage
=== Logs generated by test case
[wfDebug] [debug] IP: 127.0.0.1 {"private":false}
[wfDebug] [debug] LocalisationCache::isExpired(sh): cache missing, need to make one {"private":false}
[wfDebug] [debug] LocalisationCache::recache: got localisation for sh from source {"private":false}
[wfDebug] [debug] ApiMain::setCacheMode: setting cache mode public {"private":false}
[wfDebug] [debug] IP: 127.0.0.1 {"private":false}
===
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => Array (
         'code' => 'sh'
         'bcp47' => 'sh'
         'autonym' => 'srpskohrvatski / српскохрватски'
-        'name' => 'Serbo-Croatian'
+        'name' => 'srpskohrvatski / српскохрватски'
/home/travis/build/wikimedia/mediawiki/tests/phpunit/MediaWikiTestCase.php:2106
/home/travis/build/wikimedia/mediawiki/tests/phpunit/includes/api/ApiQueryLanguageinfoTest.php:82
/home/travis/build/wikimedia/mediawiki/tests/phpunit/MediaWikiTestCase.php:427
/home/travis/build/wikimedia/mediawiki/maintenance/doMaintenance.php:99
2) ApiQueryLanguageinfoTest::testAllPropsForSingleCustomLanguage
=== Logs generated by test case
[wfDebug] [debug] IP: 127.0.0.1 {"private":false}
[wfDebug] [debug] ApiMain::setCacheMode: setting cache mode public {"private":false}
[wfDebug] [debug] IP: 127.0.0.1 {"private":false}
===
Failed asserting that an array does not have the key 'warnings'.
/home/travis/build/wikimedia/mediawiki/tests/phpunit/includes/api/ApiQueryLanguageinfoTest.php:61
/home/travis/build/wikimedia/mediawiki/tests/phpunit/includes/api/ApiQueryLanguageinfoTest.php:87
/home/travis/build/wikimedia/mediawiki/tests/phpunit/MediaWikiTestCase.php:427
/home/travis/build/wikimedia/mediawiki/maintenance/doMaintenance.php:99

Details

Related Gerrit Patches:

Event Timeline

Reedy created this task.Jun 6 2019, 6:52 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 6 2019, 6:52 PM

Failed asserting that an array does not have the key 'warnings'.

Would be useful if PHPUnit dumped the contents of the warnings, of course…

Adding Language::getLocalisationCache()->unloadAll(); in setUp() might help – is there any way to test it on Travis CI without merging such a change to master? Perhaps push it to a branch on GitHub?

Reedy added a comment.Jun 7 2019, 1:11 PM

Adding Language::getLocalisationCache()->unloadAll(); in setUp() might help – is there any way to test it on Travis CI without merging such a change to master? Perhaps push it to a branch on GitHub?

If you do a github PR it should do so...

I opened #101, but the Travis pull requests page says “no pull request builds for this repository”, and the “requests” list explains:

Build type disabled via repository settings (pull_request).

Reedy added a comment.Jun 7 2019, 1:39 PM

I opened #101, but the Travis pull requests page says “no pull request builds for this repository”, and the “requests” list explains:

Build type disabled via repository settings (pull_request).

I've just turned it on again... Want to try doing something to trigger it to try again? Push another commit ontop of it, or force push a rebase or something

repo admin can configure whether to test PR. Looks like it is disable in wikimedia/mediawiki.

Okay, the Language::getLocalisationCache()->unloadAll(); doesn’t seem to make a difference. But I can keep trying more things (and adding debug code) on GitHub now that pull request building is enabled… though I don’t know how much time I’ll have for this :/

What’s the priority of this task, by the way? Do lots of people depend on the Travis CI builds?

The key is that the Travis CI builds includes a full test of phpunit, and the error also appeared in my test environment.

Reedy added a comment.Jun 7 2019, 6:19 PM

The key is that the Travis CI builds includes a full test of phpunit, and the error also appeared in my test environment.

What is that supposed to mean? That we don't run all the phpunit tests in WMF CI?

Okay, the Language::getLocalisationCache()->unloadAll(); doesn’t seem to make a difference. But I can keep trying more things (and adding debug code) on GitHub now that pull request building is enabled… though I don’t know how much time I’ll have for this :/
What’s the priority of this task, by the way? Do lots of people depend on the Travis CI builds?

I don't actually know what the priority is.

We don't depend on them, but we try and avoid them being broken (and therefore fixing them when we know they are broken). They're usually a good source of edge case failures (like this one), so obviously should be fixed

The key is that the Travis CI builds includes a full test of phpunit, and the error also appeared in my test environment.

What is that supposed to mean? That we don't run all the phpunit tests in WMF CI?

Looks WMF CI just test Database group

php tests/phpunit/phpunit.php --debug-tests --group Database --exclude-group Broken,ParserFuzz,Stub --log-junit /workspace/log/junit-db.xml

and the Travis CI:

php tests/phpunit/phpunit.php
RazeSoldier added a comment.EditedJun 7 2019, 6:41 PM

A full test should contain 20,000+ tests.

What is that supposed to mean? That we don't run all the phpunit tests in WMF CI?

Looks WMF CI just test Database group

php tests/phpunit/phpunit.php --debug-tests --group Database --exclude-group Broken,ParserFuzz,Stub --log-junit /workspace/log/junit-db.xml

It looks that way, but that's because in WMF CI we run the PHPUnit tests in two parts (to give faster responses). The last part, which you might see at the end of the Jenkins output, is for "group Database". The first part, is for the other groups.

Krinkle triaged this task as High priority.Jun 7 2019, 11:21 PM

Change 516280 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] api: Temporarily disable ApiQueryLanguageinfoTest

https://gerrit.wikimedia.org/r/516280

Change 516280 merged by jenkins-bot:
[mediawiki/core@master] api: Temporarily disable ApiQueryLanguageinfoTest

https://gerrit.wikimedia.org/r/516280

Regarding the actual issue, my guess is that it is a difference between whether the cldr extension is available or not when the tests are run.

Okay, the Language::getLocalisationCache()->unloadAll(); doesn’t seem to make a difference. But I can keep trying more things (and adding debug code) on GitHub now that pull request building is enabled… though I don’t know how much time I’ll have for this :/

Poking at things locally a little bit, I can reproduce the errors when I run all API tests rather than just the one file. It looks like you were on the right track, but clearing the wrong cache. It seems what actually needs clearing is Language::$languageNameCache, which can be cleared with Language::clearCaches(). If you want to submit that to Github for verification, please do.

It seems all the tests that run before this one and trigger that cache population (e.g. ApiParseTest) are marked as "Database", so Wikimedia's CI that runs Database and non-Database separately doesn't run into it.

Thanks! I updated #101 accordingly – so far the Travis job is stuck being queued, but hopefully it’ll start soonish and tell us whether that fully fixes the problem.

Anomie moved this task from Unsorted to In Dev on the MediaWiki-API board.Jun 11 2019, 5:07 PM

Change 516590 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/core@master] Fix and re-enable ApiQueryLanguageinfoTest

https://gerrit.wikimedia.org/r/516590

Travis build succeeded except for the PHP 7.3 version, but that seems to be failing for unrelated reasons.

Reedy added a comment.Jun 12 2019, 8:29 AM

Travis build succeeded except for the PHP 7.3 version, but that seems to be failing for unrelated reasons.

7.3 is “allowed to fail”, for now...

Change 516590 merged by jenkins-bot:
[mediawiki/core@master] Fix and re-enable ApiQueryLanguageinfoTest

https://gerrit.wikimedia.org/r/516590

Lucas_Werkmeister_WMDE closed this task as Resolved.Jun 12 2019, 5:14 PM
Lucas_Werkmeister_WMDE claimed this task.

Travis is green again :)

I guess building of pull requests should now be turned off again.

I guess building of pull requests should now be turned off again.

Done.

For future reference, you can also trigger these builds from your GitHub fork directly without needing to create a pull request first. To do so, from travis-ci.org enable builds for your repository. Then the builds will work as long as they are in your master branch or the branch is named "travis-ci/something". Alternatively, as part of your commit remove the brach filter in travis.yml to have it work for any branch in your personal repo.

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:06 PM