Page MenuHomePhabricator

Potentially fragile test in GrowthExperiments
Closed, ResolvedPublic

Description

We don't know if it's still failing, but there was no clear fix committed recently, so it's likely that the test is too fragile:

Patch affected: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/FileImporter/+/514438/
Log: https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php70-docker/22340/consoleFull

14:25:30 Failed asserting that two strings are equal.
14:25:30 --- Expected
14:25:30 +++ Actual
14:25:30 @@ @@
14:25:30 -'(3)'
14:25:30 +'19)'
14:25:30 
14:25:30 /workspace/src/extensions/GrowthExperiments/tests/phpunit/HelpModuleQuestionPosterTest.php:117
14:25:30 /workspace/src/tests/phpunit/MediaWikiTestCase.php:427
14:25:30 /workspace/src/maintenance/doMaintenance.php:99

Event Timeline

awight created this task.Jun 5 2019, 10:18 PM
Restricted Application added a project: Growth-Team. · View Herald TranscriptJun 5 2019, 10:18 PM
SBisson added a subscriber: SBisson.

(Not a browser test)

We've seen it failing for the first time today and then it passed on recheck. There's indeed something fragile about it.

kostajh added a subscriber: kostajh.Jun 6 2019, 1:21 AM

Change 514639 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Disable flaky test

https://gerrit.wikimedia.org/r/514639

Change 514639 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Disable flaky test

https://gerrit.wikimedia.org/r/514639

JTannerWMF added a subscriber: JTannerWMF.

We will take a look at this as a low-priority item for this sprint

Change 526784 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Make section numbering test unit instead of integration

https://gerrit.wikimedia.org/r/526784

Change 526784 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Make section numbering test unit instead of integration

https://gerrit.wikimedia.org/r/526784

kostajh closed this task as Resolved.Aug 12 2019, 1:33 PM
mmodell changed the subtype of this task from "Task" to "Production Error".Wed, Aug 28, 11:06 PM