pytest doesnt support load-tests protocol.
https://github.com/pytest-dev/pytest/issues/992
A conftest.py hack adding it was build in https://github.com/grpc/grpc/pull/3548 , which is under the 3-clause BSD license, but it appears to have been removed.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Declined | None | T115313 Find an advanced test runner with logging | |||
| Declined | None | T135912 pytest doesnt support load_tests protocol |
I add that conftest.py, and see the following errors (appears to be identical problem for our two load_tests functions:
______________________________________________________________________ ERROR collecting tests/__init__.py ______________________________________________________________________
/usr/lib/python3.4/site-packages/_pytest/runner.py:150: in __init__
self.result = func()
/usr/lib/python3.4/site-packages/_pytest/main.py:435: in _memocollect
return self._memoizedcall('_collected', lambda: list(self.collect()))
/usr/lib/python3.4/site-packages/_pytest/main.py:315: in _memoizedcall
res = function()
/usr/lib/python3.4/site-packages/_pytest/main.py:435: in <lambda>
return self._memoizedcall('_collected', lambda: list(self.collect()))
/usr/lib/python3.4/site-packages/_pytest/python.py:605: in collect
return super(Module, self).collect()
/usr/lib/python3.4/site-packages/_pytest/python.py:465: in collect
l.sort(key=lambda item: item.reportinfo()[:2])
E TypeError: unorderable types: str() < tuple()
....
____________________________________________________________________ ERROR collecting tests/script_tests.py ____________________________________________________________________
/usr/lib/python3.4/site-packages/_pytest/runner.py:150: in __init__
self.result = func()
/usr/lib/python3.4/site-packages/_pytest/main.py:435: in _memocollect
return self._memoizedcall('_collected', lambda: list(self.collect()))
/usr/lib/python3.4/site-packages/_pytest/main.py:315: in _memoizedcall
res = function()
/usr/lib/python3.4/site-packages/_pytest/main.py:435: in <lambda>
return self._memoizedcall('_collected', lambda: list(self.collect()))
/usr/lib/python3.4/site-packages/_pytest/python.py:605: in collect
return super(Module, self).collect()
/usr/lib/python3.4/site-packages/_pytest/python.py:465: in collect
l.sort(key=lambda item: item.reportinfo()[:2])
E TypeError: unorderable types: tuple() < str()The simple fix is to change the LoadTestsSuiteCollector.reportinfo to
def reportinfo(self): return (str(self.suite), )
The result is the load_tests suite is duplicates of the normally collected suite.
That can be fixed by changing tox.ini python_files=tests/__init__.py.
Then, the unit tests are named like
__init__.py::load_tests::suite_child_of_mine::suite_child_of_mine::tests.ui_tests.TestTerminalOutput.test_stdout <- tests/conftest.py PASSED
eww, but very likely to be fixable.
Result will occur at https://travis-ci.org/jayvdb/pywikibot-core/builds/131933388