Page MenuHomePhabricator

[Lingo] tests are failling when run with MediaWiki structure tests
Open, Needs TriagePublic

Description

On Lingo I came accross a little challenge. A couple tests are passing when invoked specifically, but fails as part of the main run.

Failure:

  • 1) Lingo\Tests\Integration\ArticleAnnotationTest::testArticleAnnotation
$ php tests/phpunit/phpunit.php --testsuite extensions
There were 2 failures:

1) Lingo\Tests\Integration\ArticleAnnotationTest::testArticleAnnotation with data set #0 ('/latin.xml', 'Lorem ipsum dolor sit amet, p...oluta.', array(array('ipsum', 'ipsum-def', null, null), array('vel', 'vel-def', null, null), array('vel', 'vel-def', null, null), array('mandamus', 'mandamus-def', null, null), array('ius', 'ius-def', null, null), array('modus', 'modus-def', null, null), array('omnes', 'omnes-def', null, null), array('has', 'has-def', null, null)), '<div class="mw-parser-output"...div>\n')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'<div class="mw-parser-output"><p>Lorem <span class="mw-lingo-term" data-lingo-term-id="e78f5438b48b39bcbdea61b73679449d">ipsum</span> dolor sit amet, pri admodum alienum at, decore facete ei cum,\n
-discere admodum dissentias quo et. Ex iusto antiopam sed, <span class="mw-lingo-term" data-lingo-term-id="689c4f6017e042e16a6315622d4fc11d">ius</span> eu stet periculis\n
-complectitur. Quas apeirian complectitur nec no. Nam <span class="mw-lingo-term" data-lingo-term-id="5ebcdbfc8779f1ac8898f70672713e85">omnes</span> nullam ex. Ne sed\n
+'<div class="mw-parser-output"><p>Lorem ipsum dolor sit amet, pri admodum alienum at, decore facete ei cum,\n
+discere admodum dissentias quo et. Ex iusto antiopam sed, ius eu stet periculis\n
+complectitur. Quas apeirian complectitur nec no. Nam omnes nullam ex. Ne sed\n
 eros homero, utroque pertinacia eam eu, in est ubique latine minimum. Vel\n
...


extensions/Lingo/tests/phpunit/Integration/ArticleAnnotationTest.php:86
maintenance/doMaintenance.php:94
  • 2) Lingo\Tests\Unit\ElementTest::testGetFormattedTerm_4
2) Lingo\Tests\Unit\ElementTest::testGetFormattedTerm_4
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'Main Page'
+'Main Page (page does not exist)'

extensions/Lingo/tests/phpunit/Unit/ElementTest.php:498
extensions/Lingo/tests/phpunit/Unit/ElementTest.php:189
maintenance/doMaintenance.php:94

If I run them directly by using --filter:

$ php tests/phpunit/phpunit.php --testsuite extensions --filter ArticleAnnotation
...
OK (3 tests, 3 assertions)
$ php tests/phpunit/phpunit.php --testsuite extensions --filter testGetFormattedTerm_4
.
OK (1 test, 5 assertions)

Ditto using the file that holds the tests:

$ php tests/phpunit/phpunit.php --testsuite extensions ../extensions/Lingo/tests/phpunit/Unit/ElementTest.php 
............                                                      12 / 12 (100%)
OK (12 tests, 58 assertions)

Details

Related Gerrit Patches:
mediawiki/extensions/Lingo : masterSkip testArticleAnnotation on CI
integration/config : masterMigrate Lingo to Quibble
mediawiki/extensions/Lingo : masterMigrate to PHPUnit 6: getMock() > createMock()
mediawiki/extensions/Lingo : masterUnit test against the first page

Event Timeline

hashar created this task.Jun 5 2018, 12:44 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 5 2018, 12:44 PM

If I delete mediawiki/core directory tests/phpunit/structure the tests pass just fine. So for there is some bad interaction going on between MediaWiki core structure tests and Lingo tests :(

hashar renamed this task from [Lingo] tests are failling when using MediaWiki test case database setup/teardown to [Lingo] tests are failling when run with MediaWiki structure tests.Jun 5 2018, 12:52 PM

CI tests for Lingo are run on Travis. The build script for Travis is at https://github.com/wikimedia/mediawiki-extensions-Lingo/blob/master/build/travis/script.sh

It might well be that the tests environments are not compatible, e.g. existence of the Main Page.

Change 437471 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/Lingo@master] Unit test against the first page

https://gerrit.wikimedia.org/r/437471

Change 437471 merged by Foxtrott:
[mediawiki/extensions/Lingo@master] Unit test against the first page

https://gerrit.wikimedia.org/r/437471

Change 437459 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/Lingo@master] Migrate to PHPUnit 6: getMock() > createMock()

https://gerrit.wikimedia.org/r/437459

hashar added a comment.Jun 5 2018, 2:52 PM

I noticed the Travis script . I am trying to get the tests to pass on the Wikimedia CI now :]

I managed to get Lingo\Tests\Integration\ArticleAnnotationTest::testArticleAnnotation to pass. MediaWikiTestCase creates a dummy page:

// Make 1 page with 1 revision
$page = WikiPage::factory( Title::newFromText( 'UTPage' ) );
if ( $page->getId() == 0 ) { 
    $page->doEditContent(
        new WikitextContent( 'UTContent' ),
        'UTPageSummary',
        EDIT_NEW | EDIT_SUPPRESS_RC,
        false,
        $user
    );

If I drop that doEditContent() call, the test pass just fine. So I guess it depends on Main_Page to exist.

Change 437459 merged by Foxtrott:
[mediawiki/extensions/Lingo@master] Migrate to PHPUnit 6: getMock() > createMock()

https://gerrit.wikimedia.org/r/437459

hashar updated the task description. (Show Details)Jun 5 2018, 5:49 PM

Lingo\Tests\Integration\ArticleAnnotationTest::testArticleAnnotation with data set #0 is the last one failing.

Vvjjkkii renamed this task from [Lingo] tests are failling when run with MediaWiki structure tests to 7lbaaaaaaa.Jul 1 2018, 1:05 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
CommunityTechBot renamed this task from 7lbaaaaaaa to [Lingo] tests are failling when run with MediaWiki structure tests.Jul 1 2018, 9:04 PM
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added subscribers: gerritbot, Aklapper.

Change 454305 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/Lingo@master] Skip testArticleAnnotation on CI

https://gerrit.wikimedia.org/r/454305

Change 454307 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Migrate Lingo to Quibble

https://gerrit.wikimedia.org/r/454307

Change 454307 merged by jenkins-bot:
[integration/config@master] Migrate Lingo to Quibble

https://gerrit.wikimedia.org/r/454307

Change 454305 merged by jenkins-bot:
[mediawiki/extensions/Lingo@master] Skip testArticleAnnotation on CI

https://gerrit.wikimedia.org/r/454305