Page MenuHomePhabricator

ResourceLoaderModuleTest::testGetVersionHash is flaky
Closed, ResolvedPublic

Description

https://integration.wikimedia.org/ci/job/mediawiki-phpunit-hhvm/13036/console

23:41:24 1) ResourceLoaderModuleTest::testGetVersionHash
23:41:24 Instance is insignificant
23:41:24 Failed asserting that two strings are equal.
23:41:24 --- Expected
23:41:24 +++ Actual
23:41:24 @@ @@
23:41:24 -'"LbL7EHCM"'
23:41:24 +'"Dm06PwWa"'
23:41:24

I've seen similar failures multiple times today already.

Event Timeline

Legoktm created this task.Aug 17 2015, 11:48 PM
Legoktm updated the task description. (Show Details)
Legoktm raised the priority of this task from to Needs Triage.
Legoktm added a subscriber: Legoktm.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 17 2015, 11:48 PM
Krinkle added a subscriber: Krinkle.EditedAug 20 2015, 7:25 PM

When removing the makeHash() call from ResourceLoaderModule::getVersionHash one can get a hold of the underlying difference:

 '"{\"_class\":\"ResourceLoaderFileModule\",\"_cacheEpoch\":\"20140101000000\",\"0\":{
  \"options\":{\"scripts\":[\"foo.js\",\"bar.js\"],\"debugScripts\":[],\"loaderScripts\":[],\"styles\":[],\"languageScripts\":[],\"skinScripts\":[],\"skinStyles\":[],\"messages\":[\"hello\",\"world\"],\"templates\":[],\"skipFunction\":null,\"debugRaw\":true,\"raw\":false},
  \"fileHashes\":[\"\",\"\"],
- \"msgBlobMtime\":\"1442350245\"}}"'
 '"{\"_class\":\"ResourceLoaderFileModule\",\"_cacheEpoch\":\"20140101000000\",\"0\":{
  \"options\":{\"scripts\":[\"foo.js\",\"bar.js\"],\"debugScripts\":[],\"loaderScripts\":[],\"styles\":[],\"languageScripts\":[],\"skinScripts\":[],\"skinStyles\":[],\"messages\":[\"hello\",\"world\"],\"templates\":[],\"skipFunction\":null,\"debugRaw\":true,\"raw\":false},
  \"fileHashes\":[\"\",\"\"],
+ \"msgBlobMtime\":\"1442350246\"}}"'

This isn't new and was also previously happening with testDefinitionSummary for getModifiedTime. It's due to lack of in-process caching and the clock ticking during the test execution. It happens about 1 every 50 runs when I reproduce it locally (by running php phpunit.php includes/resourceloader/ResourceLoaderModuleTest.php many times in a row).

See also:

Krinkle triaged this task as Low priority.Aug 20 2015, 7:25 PM
Krinkle set Security to None.
Krinkle added a comment.EditedSep 15 2015, 8:55 PM

-

It happens about 1 every 50 runs

Maybe I just have bad luck, but I keep running into it when merging patches. What's the plan to fix this? We (well, you really) would have disabled any other flaky test like this long ago.

ori added a subscriber: ori.Sep 26 2015, 4:15 AM

It happens about 1 every 50 runs

Maybe I just have bad luck, but I keep running into it when merging patches. What's the plan to fix this? We (well, you really) would have disabled any other flaky test like this long ago.

It'll be fixed very soon. Both @Krinkle and I are looking at how module versioning works.

Change 247503 had a related patch set uploaded (by Krinkle):
resourceloader: Disable flaky test ResourceLoaderModule::getVersionHash

https://gerrit.wikimedia.org/r/247503

Change 247503 merged by jenkins-bot:
resourceloader: Disable flaky test ResourceLoaderModule::getVersionHash

https://gerrit.wikimedia.org/r/247503

Change 258021 had a related patch set uploaded (by Krinkle):
resourceloader: Re-enable getVersionHash test

https://gerrit.wikimedia.org/r/258021

Change 258021 merged by jenkins-bot:
resourceloader: Re-enable getVersionHash test

https://gerrit.wikimedia.org/r/258021

Krinkle closed this task as Resolved.Dec 10 2015, 8:07 PM