PS1 of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Scribunto/+/853713 is totally broken (wrong class name used). The broken function is covered by tests. However jenkins-bot did not flag it as failing tests.
This suggests that tests are not being run on backports to this extension.
Additionally when i tried to run the tests locally i got 515 failures which is not reassuring.
A change send to Scribunto REL branch triggers three jobs:
| mwgate-node14-docker | npm test (linters, style checkers) |
| quibble-composer-mysql-php74-selenium-docker | npm selenium-test (webdriver.io) |
| quibble-composer-mysql-php74-noselenium-docker | Tests |
That last jobs invoke Quibble with:
docker run \ ... \ docker-registry.wikimedia.org/releng/quibble-buster-php74:1.4.7 \ ... --skip selenium,npm-test,phpunit-standalone,api-testing
It skips the phpunit-standalone stage which would have run phpunit --group Standalone. Those are tests only run when a patchset affects the extension which got optimized out cause there is no need to run Scribunto integration tests on other extensions which depend on it. That was done via T225068
What I don't get is Scribunto is configured with a job template to run the standalone jobs:
zuul/layout.yaml
- name: mediawiki/extensions/Scribunto
template:
- name: in-wikimedia-production
- name: extension-quibble
- name: extension-quibble-only-standalone # <------------
- name: extension-coverage
- name: extension-gateAnd here is the job template:
- name: extension-quibble-only-standalone test: - quibble-vendor-mysql-php74-phpunit-standalone-docker php: - quibble-vendor-mysql-php80-phpunit-standalone-docker gate-and-submit: - quibble-vendor-mysql-php74-phpunit-standalone-docker - quibble-vendor-mysql-php80-phpunit-standalone-docker experimental: - quibble-vendor-mysql-php81-phpunit-standalone-docker
The job is defined for test and gate-and-submit which filter out REL branches. Hence the standalone job is never triggered. We need to add the REL based pipelines to the template.
Change 853929 had a related patch set uploaded (by Hashar; author: Hashar):
[integration/config@master] zuul: run PHPUnit standalone tests on REL branches
Change 853929 merged by jenkins-bot:
[integration/config@master] zuul: run PHPUnit standalone tests on REL branches
Change 854024 had a related patch set uploaded (by Hashar; author: Hashar):
[mediawiki/extensions/Scribunto@REL1_39] Dummy change to run Standalone tests
Change 854025 had a related patch set uploaded (by Hashar; author: Hashar):
[mediawiki/extensions/CirrusSearch@REL1_39] Dummy change to run Standalone tests
Change 854027 had a related patch set uploaded (by Hashar; author: Hashar):
[mediawiki/extensions/WikiLambda@REL1_39] Dummy change to run Standalone tests
PHPUnit Standalone groups not running in CI for release branches also affects CirrusSearch and WikiLambda.
I have deployed the configuration change and have send a dummy change against REL1_39 for each of the extensions:
CirrusSearch fails on installation apparently due to Wikibase:
00:01:00.077 Creating wbt_text table...done.
00:01:00.077 [eada4869f6bba518752b5520] [no req] Error: Class 'Onoi\MessageReporter\ObservableMessageReporter' not found
00:01:00.077 Backtrace:
00:01:00.077 from /workspace/src/extensions/Wikibase/repo/includes/Store/Sql/DatabaseSchemaUpdater.php(186)
00:01:00.077 #0 /workspace/src/includes/installer/DatabaseUpdater.php(547): Wikibase\Repo\Store\Sql\DatabaseSchemaUpdater::rebuildPropertyTerms()
00:01:00.077 #1 /workspace/src/includes/installer/DatabaseUpdater.php(515): DatabaseUpdater->runUpdates()
00:01:00.077 #2 /workspace/src/maintenance/update.php(202): DatabaseUpdater->doUpdates()
00:01:00.077 #3 /workspace/src/maintenance/includes/MaintenanceRunner.php(309): UpdateMediaWiki->execute()
00:01:00.077 #4 /workspace/src/maintenance/doMaintenance.php(85): MediaWiki\Maintenance\MaintenanceRunner->run()
00:01:00.077 #5 /workspace/src/maintenance/update.php(312): require_once(string)
00:01:00.077 #6 {main}Wikilambda has various failures, might be cause it is not stable in REL1_39.
Lot of Scribunto tests fail with Exception: Failed to create Module:CommonTests-data.json: Lua error at line 1: unexpected symbol near '{'.
Change 854027 abandoned by Jforrester:
[mediawiki/extensions/WikiLambda@REL1_39] Dummy change to run Standalone tests
Reason:
Fixed now! Thanks.
Change 854025 abandoned by Hashar:
[mediawiki/extensions/CirrusSearch@REL1_39] Dummy change to run Standalone tests
Reason:
Change 854024 abandoned by Hashar:
[mediawiki/extensions/Scribunto@REL1_39] Dummy change to run Standalone tests
Reason:
Standalone tests are running again ;)
Change 854024 restored by Hashar:
[mediawiki/extensions/Scribunto@REL1_39] Dummy change to run Standalone tests
Change 854074 had a related patch set uploaded (by Hashar; author: Hashar):
[mediawiki/extensions/Scribunto@master] Dummy change to run Standalone tests
Change 854074 abandoned by Hashar:
[mediawiki/extensions/Scribunto@master] Dummy change to run Standalone tests
Reason:
Change to master does run PHPUnit Standalone tests and they pass. Hurrah.
Change 854024 abandoned by Hashar:
[mediawiki/extensions/Scribunto@REL1_39] Dummy change to run Standalone tests
Reason:
REL1_39 fixed by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Scribunto/+/854075