Page MenuHomePhabricator
Create Task
Maniphest T153121

Apparent random failing of RevisionSlider qunit tests
Closed, ResolvedPublic
Actions

Description

The last successfull run can be found at https://integration.wikimedia.org/ci/job/mwext-qunit-jessie/6407/console from the 30th November.
The first failure run can be found at https://integration.wikimedia.org/ci/job/mwext-qunit-jessie/6711 from the 10 Dec

The failure is confirmed by the job validation patch at https://gerrit.wikimedia.org/r/#/c/326533/ https://integration.wikimedia.org/ci/job/mwext-qunit-jessie/6775/console
Output can be seen below:

20:16:40 Chrome 53.0.2785 (Linux 0.0.0) ext.RevisionSlider.RevisionListView tooltip is composed correctly with en lang FAILED
20:16:40 	Test the page size.
20:16:40 	Expected: true
20:16:40 	Actual: null
20:16:40 	    at Object.<anonymous> (http://localhost:9876/jenkins-mwext-qunit-jessie-6775/load.php?debug=false&lang=en&modules=ext.RevisionSlider.DiffPage%2CHelpDialog%2CPointer%2CPointerLine%2CPointerView%2CRevision%2CRevisionList%2CRevisionListView%2CSlider%2CSliderView%2CdialogImages%2Ctests%7Cjquery.accessKeyLabel%2CautoEllipsis%2CbyteLength%2CbyteLimit%2CcheckboxShiftClick%2Cclient%2Ccolor%2CcolorUtil%2Ccookie%2CgetAttrs%2Chidpi%2ChighlightText%2Clocalize%2CmakeCollapsible%2Cmw-jump%2CmwExtension%2Cplaceholder%2CtabIndex%2Ctablesorter%2CtextSelection%7Cmediawiki.ForeignApi%2CRegExp%2CTitle%2CUri%2Capi%2Ccldr%2Ccookie%2Cexperiments%2CjqueryMsg%2Clanguage%2CmessagePoster%2Cnotify%2Cstorage%2Ctemplate%2Ctoc%2Cuser%2Cutil%2Cviewport%7Cmediawiki.ForeignApi.core%7Cmediawiki.api.category%2Cedit%2Cmessages%2Coptions%2Cparse%2Cupload%2Cuser%2Cwatch%7Cmediawiki.language.data%2Cinit%2Cmonths%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.ready%2Cstartup%7Cmediawiki.special.recentchanges%7Cmediawiki.template.mustache%2Cregexp%7Cmoment%2Coojs%2Coojs-ui%2Coojs-ui-core%2Coojs-ui-toolbars%2Coojs-ui-widgets%2Coojs-ui-windows%7Coojs-ui-core.styles%7Coojs-ui.styles.icons%2Cindicators%2Ctextures%7Ctest.mediawiki.qunit.suites%2Ctestrunner%7Ctest.sinonjs%7Cuser.defaults&skin=fallback&version=1be0466:61:373)
20:16:40 	    at runTest (node_modules/qunitjs/qunit/qunit.js:905:28)
20:16:40 	    at Object.run (node_modules/qunitjs/qunit/qunit.js:890:4)
20:16:40 	    at node_modules/qunitjs/qunit/qunit.js:1032:11
20:16:40 	    at process (node_modules/qunitjs/qunit/qunit.js:692:24)
20:16:40 	    at begin (node_modules/qunitjs/qunit/qunit.js:674:2)
20:16:40 	
20:16:40 	Test the change size.
20:16:40 	Expected: true
20:16:40 	Actual: null
20:16:40 	    at Object.<anonymous> (http://localhost:9876/jenkins-mwext-qunit-jessie-6775/load.php?debug=false&lang=en&modules=ext.RevisionSlider.DiffPage%2CHelpDialog%2CPointer%2CPointerLine%2CPointerView%2CRevision%2CRevisionList%2CRevisionListView%2CSlider%2CSliderView%2CdialogImages%2Ctests%7Cjquery.accessKeyLabel%2CautoEllipsis%2CbyteLength%2CbyteLimit%2CcheckboxShiftClick%2Cclient%2Ccolor%2CcolorUtil%2Ccookie%2CgetAttrs%2Chidpi%2ChighlightText%2Clocalize%2CmakeCollapsible%2Cmw-jump%2CmwExtension%2Cplaceholder%2CtabIndex%2Ctablesorter%2CtextSelection%7Cmediawiki.ForeignApi%2CRegExp%2CTitle%2CUri%2Capi%2Ccldr%2Ccookie%2Cexperiments%2CjqueryMsg%2Clanguage%2CmessagePoster%2Cnotify%2Cstorage%2Ctemplate%2Ctoc%2Cuser%2Cutil%2Cviewport%7Cmediawiki.ForeignApi.core%7Cmediawiki.api.category%2Cedit%2Cmessages%2Coptions%2Cparse%2Cupload%2Cuser%2Cwatch%7Cmediawiki.language.data%2Cinit%2Cmonths%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.ready%2Cstartup%7Cmediawiki.special.recentchanges%7Cmediawiki.template.mustache%2Cregexp%7Cmoment%2Coojs%2Coojs-ui%2Coojs-ui-core%2Coojs-ui-toolbars%2Coojs-ui-widgets%2Coojs-ui-windows%7Coojs-ui-core.styles%7Coojs-ui.styles.icons%2Cindicators%2Ctextures%7Ctest.mediawiki.qunit.suites%2Ctestrunner%7Ctest.sinonjs%7Cuser.defaults&skin=fallback&version=1be0466:61:433)
20:16:40 	    at runTest (node_modules/qunitjs/qunit/qunit.js:905:28)
20:16:40 	    at Object.run (node_modules/qunitjs/qunit/qunit.js:890:4)
20:16:40 	    at node_modules/qunitjs/qunit/qunit.js:1032:11
20:16:40 	    at process (node_modules/qunitjs/qunit/qunit.js:692:24)
20:16:40 	    at begin (node_modules/qunitjs/qunit/qunit.js:674:2)
20:16:40 	
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 365 of 378 (1 FAILED) (0 secs / 6.08 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 366 of 378 (1 FAILED) (0 secs / 6.081 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 367 of 378 (1 FAILED) (0 secs / 6.083 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 368 of 378 (1 FAILED) (0 secs / 6.085 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 369 of 378 (1 FAILED) (0 secs / 6.086 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 370 of 378 (1 FAILED) (0 secs / 6.087 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 371 of 378 (1 FAILED) (0 secs / 6.089 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 372 of 378 (1 FAILED) (0 secs / 6.091 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 373 of 378 (1 FAILED) (0 secs / 6.094 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0) ext.RevisionSlider.RevisionListView big change number is formatted correctly FAILED
20:16:40 	Expected: "+1,000"
20:16:40 	Actual: "+1.000"
20:16:40 	    at Object.<anonymous> (http://localhost:9876/jenkins-mwext-qunit-jessie-6775/load.php?debug=false&lang=en&modules=ext.RevisionSlider.DiffPage%2CHelpDialog%2CPointer%2CPointerLine%2CPointerView%2CRevision%2CRevisionList%2CRevisionListView%2CSlider%2CSliderView%2CdialogImages%2Ctests%7Cjquery.accessKeyLabel%2CautoEllipsis%2CbyteLength%2CbyteLimit%2CcheckboxShiftClick%2Cclient%2Ccolor%2CcolorUtil%2Ccookie%2CgetAttrs%2Chidpi%2ChighlightText%2Clocalize%2CmakeCollapsible%2Cmw-jump%2CmwExtension%2Cplaceholder%2CtabIndex%2Ctablesorter%2CtextSelection%7Cmediawiki.ForeignApi%2CRegExp%2CTitle%2CUri%2Capi%2Ccldr%2Ccookie%2Cexperiments%2CjqueryMsg%2Clanguage%2CmessagePoster%2Cnotify%2Cstorage%2Ctemplate%2Ctoc%2Cuser%2Cutil%2Cviewport%7Cmediawiki.ForeignApi.core%7Cmediawiki.api.category%2Cedit%2Cmessages%2Coptions%2Cparse%2Cupload%2Cuser%2Cwatch%7Cmediawiki.language.data%2Cinit%2Cmonths%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.ready%2Cstartup%7Cmediawiki.special.recentchanges%7Cmediawiki.template.mustache%2Cregexp%7Cmoment%2Coojs%2Coojs-ui%2Coojs-ui-core%2Coojs-ui-toolbars%2Coojs-ui-widgets%2Coojs-ui-windows%7Coojs-ui-core.styles%7Coojs-ui.styles.icons%2Cindicators%2Ctextures%7Ctest.mediawiki.qunit.suites%2Ctestrunner%7Ctest.sinonjs%7Cuser.defaults&skin=fallback&version=1be0466:64:480)
20:16:40 	    at runTest (node_modules/qunitjs/qunit/qunit.js:905:28)
20:16:40 	    at Object.run (node_modules/qunitjs/qunit/qunit.js:890:4)
20:16:40 	    at node_modules/qunitjs/qunit/qunit.js:1032:11
20:16:40 	    at process (node_modules/qunitjs/qunit/qunit.js:692:24)
20:16:40 	    at begin (node_modules/qunitjs/qunit/qunit.js:674:2)
20:16:40 	
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 374 of 378 (2 FAILED) (0 secs / 6.097 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0) ext.RevisionSlider.RevisionListView page size is formatted correctly FAILED
20:16:40 	failed, expected argument to be truthy, was: null
20:16:40 	Expected: true
20:16:40 	Actual: null
20:16:40 	    at Object.<anonymous> (http://localhost:9876/jenkins-mwext-qunit-jessie-6775/load.php?debug=false&lang=en&modules=ext.RevisionSlider.DiffPage%2CHelpDialog%2CPointer%2CPointerLine%2CPointerView%2CRevision%2CRevisionList%2CRevisionListView%2CSlider%2CSliderView%2CdialogImages%2Ctests%7Cjquery.accessKeyLabel%2CautoEllipsis%2CbyteLength%2CbyteLimit%2CcheckboxShiftClick%2Cclient%2Ccolor%2CcolorUtil%2Ccookie%2CgetAttrs%2Chidpi%2ChighlightText%2Clocalize%2CmakeCollapsible%2Cmw-jump%2CmwExtension%2Cplaceholder%2CtabIndex%2Ctablesorter%2CtextSelection%7Cmediawiki.ForeignApi%2CRegExp%2CTitle%2CUri%2Capi%2Ccldr%2Ccookie%2Cexperiments%2CjqueryMsg%2Clanguage%2CmessagePoster%2Cnotify%2Cstorage%2Ctemplate%2Ctoc%2Cuser%2Cutil%2Cviewport%7Cmediawiki.ForeignApi.core%7Cmediawiki.api.category%2Cedit%2Cmessages%2Coptions%2Cparse%2Cupload%2Cuser%2Cwatch%7Cmediawiki.language.data%2Cinit%2Cmonths%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.ready%2Cstartup%7Cmediawiki.special.recentchanges%7Cmediawiki.template.mustache%2Cregexp%7Cmoment%2Coojs%2Coojs-ui%2Coojs-ui-core%2Coojs-ui-toolbars%2Coojs-ui-widgets%2Coojs-ui-windows%7Coojs-ui-core.styles%7Coojs-ui.styles.icons%2Cindicators%2Ctextures%7Ctest.mediawiki.qunit.suites%2Ctestrunner%7Ctest.sinonjs%7Cuser.defaults&skin=fallback&version=1be0466:64:903)
20:16:40 	    at runTest (node_modules/qunitjs/qunit/qunit.js:905:28)
20:16:40 	    at Object.run (node_modules/qunitjs/qunit/qunit.js:890:4)
20:16:40 	    at node_modules/qunitjs/qunit/qunit.js:1032:11
20:16:40 	    at process (node_modules/qunitjs/qunit/qunit.js:692:24)
20:16:40 	    at begin (node_modules/qunitjs/qunit/qunit.js:674:2)
20:16:40 	
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 375 of 378 (3 FAILED) (0 secs / 6.099 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 376 of 378 (3 FAILED) (0 secs / 6.099 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 377 of 378 (3 FAILED) (0 secs / 6.124 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 378 of 378 (3 FAILED) (0 secs / 6.143 secs)
20:16:40 Chrome 53.0.2785 (Linux 0.0.0): Executed 378 of 378 (3 FAILED) (6.386 secs / 6.143 secs)
20:16:40 12 12 2016 20:16:39.337:DEBUG [karma]: Run complete, exiting.
20:16:40 12 12 2016 20:16:39.339:DEBUG [launcher]: Disconnecting all browsers
20:16:40 12 12 2016 20:16:39.863:DEBUG [launcher]: Process Chrome exited with code 0
20:16:40 12 12 2016 20:16:39.864:DEBUG [temp-dir]: Cleaning temp dir /home/jenkins/tmpfs/jenkins-0/karma-26044742
20:16:40 12 12 2016 20:16:39.921:DEBUG [launcher]: Finished all browsers
20:16:40 Warning: Task "karma:main" failed. Use --force to continue.
20:16:40 
20:16:40 Aborted due to warnings.

It looks like these can be reproduced locally

pasted_file (302×1 px, 98 KB)

Details

SubjectRepoBranchLines +/-
Reset custom separator transform tables in number formatting testsmediawiki/extensions/RevisionSlidermaster+12 -1
Reset custom separator transform tables in number formatting testsmediawiki/extensions/RevisionSliderwmf/1.29.0-wmf.6+12 -1
Customize query in gerrit

Related Objects

Event Timeline

Addshore created this task.Dec 13 2016, 8:34 PM
Restricted Application added a project: TCB-Team (now WMDE-TechWish). · View Herald TranscriptDec 13 2016, 8:34 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Addshore added a project: WMDE-TechWish.Dec 13 2016, 8:35 PM
Addshore moved this task from Unsorted 💣 to Watching 👀 on the User-Addshore board.
Addshore moved this task from Proposed to Watching/Blocked/External on the WMDE-TechWish board.
Addshore updated the task description. (Show Details)Dec 13 2016, 8:42 PM
Tobi_WMDE_SW added a comment.Dec 14 2016, 8:26 AM

@Addshore can you describe a bit more under which circumstances this was reproducible locally?

hashar mentioned this in T153038: mwext-qunit-jessie test fails on unrelated change.Dec 14 2016, 8:57 AM
WMDE-leszek added a comment.Dec 14 2016, 9:55 AM

FWIW I also finally managed to reproduce the failures, somehow. So I have a setup with only core and RevisionSlider extension enabled. When I run all Qunit tests, those 3 RS tests fail. When I rerun all tests in most cases those tests pass (although I didn't verify this scientifically enough to be able to say they only fail on the first test run). Also, when I only run test for the module in which failing test are, they have been passing in all my attempts.
So there might either be some race condition involved, or maybe some of core tests dealing with language setting is affecting RS test in a way that they fail. Will be trying to figure out more what's going on there.
Maybe we should try to make RS tests dealing with user language more reliable. What's done over there seems a bit hacky, after all.

gerritbot subscribed.Dec 14 2016, 10:26 AM

Change 327166 had a related patch set uploaded (by WMDE-leszek):
Reset custom separator transform tables in number formatting tests

https://gerrit.wikimedia.org/r/327166

gerritbot added a project: Patch-For-Review.Dec 14 2016, 10:26 AM
WMDE-leszek mentioned this in rERSLb46c1052024d: Reset custom separator transform tables in number formatting tests.Dec 14 2016, 10:27 AM
gerritbot added a comment.Dec 14 2016, 10:40 AM

Change 327166 merged by jenkins-bot:
Reset custom separator transform tables in number formatting tests

https://gerrit.wikimedia.org/r/327166

ReleaseTaggerBot added a project: MW-1.29-release (WMF-deploy-2017-01-03_(1.29.0-wmf.7)).Dec 14 2016, 11:01 AM
Tobi_WMDE_SW closed this task as Resolved.Dec 14 2016, 12:33 PM
Tobi_WMDE_SW claimed this task.
Tobi_WMDE_SW moved this task from Watching/Blocked/External to Demoed on the WMDE-TechWish board.
Tobi_WMDE_SW moved this task from Incoming to Revision Slider on the TCB-Team (now WMDE-TechWish) board.Dec 14 2016, 12:37 PM
hashar added a comment.Dec 14 2016, 4:19 PM

Due to mediawiki/core change https://gerrit.wikimedia.org/r/#/c/324387/

Congratulations to have reproduced and fixed it !

WMDE-leszek mentioned this in T152476: mw.language unittest cleanup failing.Dec 15 2016, 8:15 AM
Addshore moved this task from Watching 👀 to Closing ✔️ on the User-Addshore board.Dec 15 2016, 11:46 AM
gerritbot added a comment.Dec 15 2016, 1:41 PM

Change 327499 had a related patch set uploaded (by Addshore):
Reset custom separator transform tables in number formatting tests

https://gerrit.wikimedia.org/r/327499

Addshore mentioned this in rERSL2afab9732fde: Reset custom separator transform tables in number formatting tests.Dec 15 2016, 1:42 PM
gerritbot added a comment.Dec 15 2016, 2:18 PM

Change 327499 merged by jenkins-bot:
Reset custom separator transform tables in number formatting tests

https://gerrit.wikimedia.org/r/327499

ReleaseTaggerBot added a project: MW-1.29-release (WMF-deploy-2016-12-13_(1.29.0-wmf.6)).Dec 15 2016, 3:00 PM
Tobi_WMDE_SW moved this task from Incoming to Done on the Revision-Slider board.Dec 16 2016, 10:34 AM
Tobi_WMDE_SW moved this task from Demoed to Done on the WMDE-TechWish board.Dec 16 2016, 2:52 PM
Tobi_WMDE_SW moved this task from Done to Demoed on the WMDE-TechWish board.Dec 20 2016, 12:16 PM
Tobi_WMDE_SW removed projects: MW-1.29-release (WMF-deploy-2016-12-13_(1.29.0-wmf.6)), WMDE-TechWish.Jun 7 2017, 12:01 PM
thiemowmde removed a project: TCB-Team (now WMDE-TechWish).Jan 7 2022, 3:07 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL