tests/pagegenerators_tests.py::EventStreamsPageGeneratorTestCase::test_RC_pagegenerator_result SKIPPED
The reason is that we have no test with EXTRAS=1 and NO_RC=0
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| [tests] Allow more verbose output for pytest | pywikibot/core | master | +2 -2 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T152961 Improve tests suite | |||
| Resolved | Xqt | T223481 Fix tests that never run | |||
| Declined | None | T222898 test_RC_pagegenerator_result is skipped on Travis due to unknown reason |
Event Timeline
Comment Actions
Perhaps we should turn on unittest --verbose temporarily (for a week) to get reasons for all skipped tests. We already experienced such thing multiple times, that tests were skipped for no reason, perhaps this could help
Comment Actions
I run a verbose pytest and here are the skip reasons:
SKIPPED [1] tests/family_tests.py:237: Family.from_url() does not work for wowwiki (T215077) SKIPPED [1] tests/site_tests.py:552: Main page is not in any categories. SKIPPED [1] tests/tools_tests.py:305: lzma not importable SKIPPED [3] ../../../../../opt/python/2.7.14/lib/python2.7/unittest/case.py:60: browseragents not installed SKIPPED [2] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: TestInternetArchive: hostname https://archive.org/wayback/available?url=invalid failed: HTTP status: 500 SKIPPED [6] tests/utils.py:84: Test is allowed to fail. SKIPPED [1] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: TestSiteLoadRevisionsSysop: No sysop username for wikipedia:en SKIPPED [1] tests/api_tests.py:1177: wikipedia:en is not running on a replicated database cluster. SKIPPED [1] ../../../../../opt/python/2.7.14/lib/python2.7/unittest/case.py:60: bs4 not installed SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "casechecker" SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "revertbot" SKIPPED [1] tests/pagegenerators_tests.py:1614: Site is using a Wikibase repository SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "cfd" SKIPPED [6] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/unittest.py:93: memento_client not installed SKIPPED [1] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: Tkinter tests are disabled on Travis-CI SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "standardize_interwiki" SKIPPED [1] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: HttpServerProblemTestCase: hostname http://httpbin.org/status/502 failed: HTTP status: 502 SKIPPED [1] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: Tkdialog tests are disabled on Travis-CI SKIPPED [4] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/unittest.py:93: requires Windows console SKIPPED [1] tests/oauth_tests.py:75: OAuth tokens not set SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "unusedfiles" SKIPPED [4] ../../../../../opt/python/2.7.14/lib/python2.7/unittest/case.py:60: mwparserfromhell not installed SKIPPED [15] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/unittest.py:93: T193637 and T114318 SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "shell" SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "noreferences" SKIPPED [1] tests/pagegenerators_tests.py:1008: Pages in namespaces [] not found. SKIPPED [3] ../../../../../opt/python/2.7.14/lib/python2.7/unittest/case.py:60: fake_useragent not installed SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "clean_sandbox" SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "welcome" SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "followlive" SKIPPED [2] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/unittest.py:93: mwparserfromhell not installed SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "blockpageschecker" SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "checkimages" SKIPPED [1] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: sseclient is not available SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "login" SKIPPED [589] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: L10N tests disabled. SKIPPED [2] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: SiteSysopTestCase: No sysop username for wikipedia:en SKIPPED [204] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: Site code "test" and PYWIKIBOT_TEST_PROD_ONLY is set. SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "category_redirect" SKIPPED [2] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/unittest.py:93: pydot not installed SKIPPED [2] tests/tools_tests.py:157: 7za not installed SKIPPED [1] ../../../../../opt/python/2.7.14/lib/python2.7/unittest/case.py:60: Python 3+ specific test SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "nowcommons" SKIPPED [1] tests/tools_tests.py:296: lzma not importable SKIPPED [1] tests/oauth_tests.py:68: OAuth tokens not set SKIPPED [1] tests/site_tests.py:867: specialpage-cantexecute: You don't have permission to view the results of this special page. [help:See https://en.wikipedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes.] SKIPPED [21] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/unittest.py:93: bs4 not installed SKIPPED [1] tests/script_tests.py:221: Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable) "imagerecat" SKIPPED [2] tests/tools_tests.py:171: lzma not importable SKIPPED [11] /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/_pytest/nose.py:30: Site code "mediawiki" and PYWIKIBOT_TEST_PROD_ONLY is set. SKIPPED [1] tests/pagegenerators_tests.py:1016: Pages in namespaces [] not found. 1648 passed, 912 skipped, 26 xfailed, 4 xpassed, 27 warnings in 934.04 seconds
(py2.7, enwiki, PYWIKIBOT_TEST_PROD_ONLY=1, https://travis-ci.org/wikimedia/pywikibot/jobs/530360684)
This comment was removed by Dvorapa.
Comment Actions
I think I've found the reason!
The reason here is that sseclient is missing. It is missing because PYSETUP_TEST_EXTRAS=1 is not set. So I looked on all the other, where PYSETUP_TEST_EXTRAS=1 is set. But all of these have PYWIKIBOT_TEST_NO_RC=1 set. So the test never runs!
Comment Actions
I run another test on testwiki trying to avoid any skipping set by environment variables and we'll see
Comment Actions
But it seems no rc test skipped by this issue is failing.
I can think of these solutions:
- move sseclient from extras to test dependencies
- have one test NO_RC=0 EXTRAS=1
- install extras to all tests (what is the reason to not install them? I don't think it is such a time consuming thing)
Comment Actions
btw here is my try to run L10N tests and install extras in all tests and also try to test everything possible on testwiki (tests 3,6,9 and 12):
https://travis-ci.org/wikimedia/pywikibot/builds/530514228
It is quite interesting. There are some issues that could be fixed and also it reveals just how many tests are skipped for no reason (usually 100-800, here 77).
For example there are some failures of scripts in script_tests (cfd, checkimages, clean_sandbox, revertbot, unusedfiles) on testwiki, missing L10N of some scripts for testwiki, etc.
Comment Actions
Very interesting. btw L10N tests should be made in i18n repository only if the sources comes from twn:
https://travis-ci.org/wikimedia/pywikibot-i18n/builds
There are two skips and they are notified.
Comment Actions
After looking on the log I propose to always use the verbose option with tests. I don't see any reason to deactivate it.
Comment Actions
-r chars show extra test summary info as specified by chars
(f)ailed, (E)error, (s)skipped, (x)failed, (X)passed,
(p)passed, (P)passed with output, (a)all except pP.
Warnings are displayed at all times except when
--disable-warnings is setWhich ones do we want to show the reasons for? I think fail and error are already detailed, so maybe -rsxX? (see also https://docs.pytest.org/en/latest/skipping.html)
Comment Actions
Which ones do we want to show the reasons for? I think fail and error are already detailed, so maybe -rsxX? (see also https://docs.pytest.org/en/latest/skipping.html)
+1, let's try.
Comment Actions
Change 509380 had a related patch set uploaded (by Dvorapa; owner: Dvorapa):
[pywikibot/core@master] [tests] Allow more verbose output for pytest
Comment Actions
Change 509380 merged by jenkins-bot:
[pywikibot/core@master] [tests] Allow more verbose output for pytest