Rather than our current setup in which all tests are bundled in a single directory split the unit and integration tests into 2 separate directories.
This would enable us to:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T84948 MediaWiki core unit tests should be true unit tests (tracking) | |||
| Resolved | kostajh | T89432 Make PHPUnit tests run without installing MediaWiki | |||
| Open | None | T87781 Split mediawiki tests into unit and integration tests | |||
| Duplicate | None | T223968 Split Wikibase tests into unit and integration tests | |||
| Resolved | kostajh | T226911 Support running unit tests and extending MediaWikiUnitTestCase in skins and extensions | |||
| Declined | BUG REPORT | None | T229220 Unit tests are not being run for extensions under PHPUnit 4.x (HHVM) | ||
| Declined | None | T234721 Provide an alternative way to use MediaWikiUnitTestCase for tests not in unit/ |
@Ladsgroup @WMDE-leszek @TK-999 (and anyone else interested), how about we meet at 11 AM on Friday to make a poster, in case others want to help? We can make a plan for dividing up the work at that time too.
Change 510225 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Use phpunit TestCase instead of MediawikiTestCase class for unittests
Change 510226 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/core@master] Use phpunit TestCase instead of MediawikiTestCase class for unittests
Change 511035 had a related patch set uploaded (by Michael Große; owner: Michael Große):
[mediawiki/extensions/Wikibase@master] DNM! Bootstrap and move unittests to their own dir
Hey, this is the script to find unittests:
import os import sys import re import subprocess from subprocess import PIPE def find_test_files(path): files = [] for r, d, f in os.walk(path): for file_name in f: if file_name.endswith('Test.php'): files.append(os.path.join(r, file_name)) return files def check_file(path): with open(path, 'r') as f: content = f.read() old_content = content search_result = re.findall(r'(\nclass (\S+) extends \\?MediaWikiTestCase)', content) if not search_result: return with open(path, 'w') as f: f.write( content.replace( search_result[0][0], '\nclass {} extends \MediaWikiUnitTestCase'.format( search_result[0][1])).replace('\nuse MediaWikiTestCase;', '') .replace('\n\n\n', '\n\n').replace('\n\n\n', '\n\n')) res = subprocess.run(["php", sys.argv[2], path], stdout=PIPE, stderr=PIPE) if res.returncode != 0: print('It did not work :((') with open(path, 'w') as f: f.write(old_content) else: print('IT WORKED!!!') for f in find_test_files(sys.argv[1]): #print('Checking ' + f) check_file(f)
You can run the code to change unittests like this:
python unittest_checker.py /var/lib/mediawiki/extensions/Wikibase/tests/phpunit/ /var/lib/mediawiki/tests/phpunit/phpunit.php
The first argument is the directory of tests you want to do, the second one is the phpunit entry point (but be careful, it run integration tests as unit test, it might drop things in your localhost database if you don't protect it properly).
Change 513106 had a related patch set uploaded (by TK-999; owner: TK-999):
[mediawiki/core@master] [WIP/PoC]: Separate MediaWiki unit and integration tests
Change 510226 abandoned by Ladsgroup:
Introduce and use MediawikiUnitTestCase instead of MediawikiTestCase class for unittests
Reason:
Suppressed by Iad01033a0548afd4d2a6f2c1ef6fcc9debf72c0d
Change 513106 merged by Jforrester:
[mediawiki/core@master] Separate MediaWiki unit and integration tests
Change 517550 had a related patch set uploaded (by TK-999; owner: TK-999):
[mediawiki/core@master] Introduce separate unit tests PHPUnit configuration
Change 517550 merged by jenkins-bot:
[mediawiki/core@master] Introduce separate unit tests PHPUnit configuration
Change 517701 had a related patch set uploaded (by TK-999; owner: TK-999):
[mediawiki/core@master] Move trivially compatible tests to the unit tests suite
Change 519165 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] Unit tests: prevent access to global functions and variables
Change 519165 merged by jenkins-bot:
[mediawiki/core@master] Define unit and integration test suites
Change 517701 merged by jenkins-bot:
[mediawiki/core@master] Move trivially compatible tests to the unit tests suite
Change 519166 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/quibble@master] Support running PHPUnit unit tests
Change 520193 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Move trivial unit tests to the new structure
Change 520212 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/MobileFrontend@master] Migrate unit tests to the new structure
Change 520905 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] Reorganize testsuites
Change 519166 merged by jenkins-bot:
[integration/quibble@master] Support running PHPUnit unit tests
Change 521046 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/quibble@master] Fix composer phpunit:unit command
Change 521186 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/core@master] Move unit tests, round III
Change 521182 had a related patch set uploaded (by Kosta Harlan; owner: Ladsgroup):
[mediawiki/core@master] Move unit tests FormatJsonTest.php to a dedicated file in unit tests
Change 521046 merged by jenkins-bot:
[integration/quibble@master] Fix composer phpunit:unit command
Change 521182 merged by jenkins-bot:
[mediawiki/core@master] Move unit tests FormatJsonTest.php to a dedicated file in unit tests
Change 521264 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] Unset all globals unneeded for unit tests
I was wondering whether we should also disable DB access via MediaWikiServices::disableStorageBackend. Although maybe unit tests for the DB class and related could need it?
Change 521264 abandoned by Kosta Harlan:
Unset all globals unneeded for unit tests
Reason:
in favor of I16691fc8ac063705ba0c2bc63b96c4534ca8660b
Change 521268 had a related patch set uploaded (by Kosta Harlan; owner: Ladsgroup):
[mediawiki/core@master] Unset all globals unneeded for unit tests
Change 520212 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Migrate unit tests to the new structure
Change 510225 abandoned by Ladsgroup:
Use phpunit TestCase instead of MediawikiTestCase class for unittests
Reason:
Suppressed by Ib4411ca2e12ceecaa258fd0d9f962df6d3316806
Change 521477 had a related patch set uploaded (by Daimona Eaytoy; owner: Daimona Eaytoy):
[mediawiki/core@master] [POC] UnitTests: disable storage backend
Change 521515 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/quibble@master] Don't run phpunit-unit stage if the composer script doesn't exist
Change 521477 abandoned by Daimona Eaytoy:
[POC] UnitTests: disable storage backend
Reason:
Seems unnecessary
Change 521268 merged by jenkins-bot:
[mediawiki/core@master] Unset all globals unneeded for unit tests, assert correct directory
Change 521186 merged by jenkins-bot:
[mediawiki/core@master] Move unit tests, round III
Change 521908 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] PHPUnit bootstrap: less aggressive unsetting of globals
Change 521909 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] Tests: Assert that integration tests are not in unit test directory
Change 521909 merged by jenkins-bot:
[mediawiki/core@master] Tests: Assert that integration tests are not in unit test directory
Change 521908 merged by jenkins-bot:
[mediawiki/core@master] PHPUnit bootstrap: less aggressive unsetting of globals
Testing https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/521955 I found that the extension dependencies are not loaded by the php unit boostrap script.
I'm perhaps running this incorrectly (vendor/bin/phpunit extensions/CirrusSearch/tests/phpunit/unit/ or composer phpunit:unit extensions/CirrusSearch/tests/phpunit/unit/).
Loading extensions/*/vendor/autoload.php makes the tests pass for cirrus.
In general, it's better to combine composer dependencies of extension into core using composer merge plugin: https://www.mediawiki.org/wiki/Composer#Using_composer-merge-plugin
That would fix lots of headaches (in here and lots of other places)
Thanks, should this be done by default or will we have instructions to tell developers to tweak their composer.local.json file to run the unit tests?
Change 522506 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/quibble@master] Use vendor/bin/phpunit instead of composer phpunit:unit
Change 521515 abandoned by Kosta Harlan:
Don't run phpunit-unit stage if the composer script doesn't exist
Reason:
abandoning in favor of Ib8c5b7679689df089f183eea181b3fcf9eed3d0d
Change 522166 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/core@master] Split HttpTest and SessionTest to unit and integration
I think it's tricky to provide defaults in this case since the composer.local.json file is intended to be ignored by VCS by design. So the next best thing would be to document this caveat so people working on extensions with composer dependencies know how to get started.
Change 522618 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/core@master] Load GlobalFunctions.php to tests/phpunit/bootstrap.php
Change 522166 merged by jenkins-bot:
[mediawiki/core@master] Split HttpTest and SessionTest to unit and integration
Change 522618 merged by jenkins-bot:
[mediawiki/core@master] Load GlobalFunctions.php to tests/phpunit/bootstrap.php
Change 521515 restored by Kosta Harlan:
Don't run phpunit-unit stage if the composer script doesn't exist
Change 522506 abandoned by Kosta Harlan:
Use vendor/bin/phpunit instead of composer phpunit:unit
Reason:
let's go with https://gerrit.wikimedia.org/r/c/integration/quibble/ /521515 instead
Change 521515 merged by jenkins-bot:
[integration/quibble@master] Don't run phpunit-unit stage if the composer script doesn't exist
Mentioned in SAL (#wikimedia-releng) [2019-07-25T14:52:18Z] <hashar> Tagged Quibble 0.0.33 b2f9e36 | T193824 T87781 T199116
Change 519448 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] (wip) Integration tests: Always assume database usage
Change 530084 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] Tests: Allow vendor/bin/phpunit from within extension directory
Change 530084 merged by jenkins-bot:
[mediawiki/core@master] Tests: Allow vendor/bin/phpunit from within extension directory
Change 534774 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] Unit tests: Whitelist global so LoggerFactory doesn't explode
Change 534774 merged by jenkins-bot:
[mediawiki/core@master] Unit tests: Whitelist global so LoggerFactory doesn't explode
Change 534845 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Start converting Cirrus PHP test to MediaWikiUnitTestCase
Change 519448 abandoned by Kosta Harlan:
(wip) Integration tests: Always assume database usage
Change 520193 abandoned by Ladsgroup:
[mediawiki/extensions/Wikibase@master] Move trivial unit tests to the new structure
Reason:
This is done mostly through other patches.