Various tests from VisualEditor and MobileFrontend fail when run in plain mode (check out @Krinkle patch I69f735eb56c1362189298d9859d3add576faaadb https://gerrit.wikimedia.org/r/#/c/216886/ to test)
Run > grunt qunit in the mediawiki core directory
Tests should be amended so this works.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Krinkle | T103027 QUnit should bypass target mode | |||
| Resolved | Jdlrobson | T104362 Certain tests failing in bypassed target mode | |||
| Resolved | Jdlrobson | T106622 test_SearchOverlay.js should not use mw.loader.using or M.on | |||
| Resolved | Jdlrobson | T106623 Problems with Browser memoisation tests | |||
| Resolved | Sn1per | T106628 InfiniteScroll and api qunit tests are being skipped. | |||
| Resolved | Jdlrobson | T107122 VisualEditor tests fail in target agnostic mode | |||
| Resolved | phuedx | T106759 22:37:23 ERROR: '[GatherClicks] Missing or empty schema' in qunit tests | |||
| Resolved | Jdlrobson | T107357 ULS tests fail in bypassed target mode |
Change 226435 had a related patch set uploaded (by Jdlrobson):
Package message up with the module that uses it.
Change 226437 had a related patch set uploaded (by Jdlrobson):
Update Overlay test to be skin agnostic
Change 226448 had a related patch set uploaded (by Jdlrobson):
Toggling tests should be able to run outside alpha mode
With all the above patches merged if I blank out these three files it all works:
modified: tests/qunit/mobile.browser/test_browser.js
modified: tests/qunit/mobile.infiniteScroll/test_InfiniteScroll.js
modified: tests/qunit/mobile.search/test_SearchOverlay.js
I've raised subtasks for fixing these three tests. @Krinkle if fixes take too long we could always skip them to help you clean up the qa architecture they need.
For completeness the following patches are needed to be reviewed:
https://gerrit.wikimedia.org/r/226435
https://gerrit.wikimedia.org/r/226448
https://gerrit.wikimedia.org/r/226437
https://gerrit.wikimedia.org/r/226942
https://gerrit.wikimedia.org/r/226940
Change 226435 merged by jenkins-bot:
Package message up with the module that uses it.
Change 226448 merged by jenkins-bot:
Stop running toggling tests in alpha environment
Thanks @bmansurov ! I've rebased the patch from @Krinkle - hopefully it passes and I can +2
Removing reading web since there's nothing actionable from our side now and we have T103027
On Tuesday August 4th our mobile specific qunit jenkins job broke due to the fix for T107399
You'll notice that running grunt qunit in MobileFrontend repository works fine on the commit before https://gerrit.wikimedia.org/r/227627 but times out after it.
As a result, I disabled the job in https://gerrit.wikimedia.org/r/229287
Problematically this now means we rely on mwext-MobileFrontend-qunit which does not run all our tests as it runs with target=desktop
There are actually 428 tests in total in the MobileFrontend repository. This patch [1] shows the job running all of them - only 271 [2] are currently running - but obviously should not be merged.
To press forward we need to get T104362 fixed to get these their coverage again.
@bmansurov @rmoen @phuedx @Florian @Jhernandez please be sure to run qunit tests manually in the browser in the mean time.
Note: the grunt qunit job is now broken and also needs fixing.
[1] https://gerrit.wikimedia.org/r/#/c/229289/
[2] https://integration.wikimedia.org/ci/job/mwext-MobileFrontend-qunit/15352/console
As can be seen the missing 157 tests get run when we revert https://gerrit.wikimedia.org/r/#/c/229268/:
528 => 685
Tests failing all claim to be in VisualEditor but it's likely there is some compatibility between modules. @Krinkle any ideas on how to debug?
Change 229600 had a related patch set uploaded (by Jdlrobson):
DONOTMERGE: Temporarily skip some tests that are choking the test suite
Change 229268 had a related patch set uploaded (by Jdlrobson):
Revert "Don't register unloadable test modules"