On noop patch https://gerrit.wikimedia.org/r/#/c/mediawiki/skins/chameleon/+/597812/ there are quite a lot of test failures...
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-docker/67285/consoleFull
On noop patch https://gerrit.wikimedia.org/r/#/c/mediawiki/skins/chameleon/+/597812/ there are quite a lot of test failures...
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-docker/67285/consoleFull
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | • eprodromou | T235270 Wikimedia API Gateway | |||
Resolved | apaskulin | T247837 Create API Portal MVP | |||
Declined | CCicalese_WMF | T253321 Chameleon test failures on WMF CI |
There is a notable difference between running phpunit locally (which works) and in Wikimedia CI. https://github.com/ProfessionalWiki/chameleon/blob/master/docs/testing.md require adding -c phpunit.xml.dist, which may have inconsistent options for phpunit. Removing that run the local command results in multiple instances of the error:
Use of undefined constant USE_EXTERNAL_HTML_VALIDATOR - assumed 'USE_EXTERNAL_HTML_VALIDATOR' (this will throw an Error in a future version of PHP)
This is different from the error seen multiple times in Wikimedia CI:
InvalidArgumentException: Key "CanonicalNamespaceNames" not found in input sources /workspace/src/includes/config/ServiceOptions.php:52
But, in both cases, it is a single error that is repeated in multiple tests.
USE_EXTERNAL_HTML_VALIDATOR is only defined in phpunit.xml.dist. But as that's not failing in CI, I can presume we're including/using the phpunit config file.
Then they use this file to run phpunit via composer.
I don't think you need to worry about that; if CI was failing with that error, we'd have something wrong with the CI config etc.
Looking at the first error..
16:58:28 1) Skins\Chameleon\Tests\Unit\ChameleonTemplateTest::testCanConstruct 16:58:28 Undefined index: egChameleonLayoutFile 16:58:28 16:58:28 /workspace/src/skins/chameleon/tests/phpunit/Unit/ChameleonTemplateTest.php:55
Something is seemingly going amiss with the config here, as it's defined by default in skin.json
"config_prefix": "eg", "config": { "ChameleonLayoutFile": { "value": "layouts/standard.xml" }, "ChameleonEnableVisualEditor": { "value": true } },
Also, the tests don't pass locally for me...
reedy@ubuntu64-web-esxi:/var/www/wiki/mediawiki/core/skins/chameleon$ php /var/www/wiki/mediawiki/core/tests/phpunit/phpunit.php --group skins-chameleon -c phpunit.xml.dist #!/usr/bin/env php Using PHP 7.3.11-0ubuntu0.19.10.4 Using the MediaWiki vendor autoloader ... PHPUnit 8.5.5 by Sebastian Bergmann and contributors. Runtime: PHP 7.3.11-0ubuntu0.19.10.4 with Xdebug 2.7.2 Configuration: /var/www/wiki/mediawiki/skins/chameleon/phpunit.xml.dist Warning - The configuration file did not pass validation! The following problems have been detected: Line 40: - Element 'phpunit', attribute 'checkForUnintentionallyCoveredCode': The attribute 'checkForUnintentionallyCoveredCode' is not allowed. Test results may not be as expected. S.............................................................. 63 / 285 ( 22%) ............................................................... 126 / 285 ( 44%) .............................................F................. 189 / 285 ( 66%) ............................................................... 252 / 285 ( 88%) ................................. 285 / 285 (100%) Time: 1.47 seconds, Memory: 54.50 MB There was 1 failure: 1) Skins\Chameleon\Tests\Unit\Components\NavbarHorizontal\PersonalToolsTest::testGetHtml_LoggedInUserHasNewMessages with data set #0 (DOMElement Object ()) Failed asserting that the given fragment contained the described node. Failed asserting that false is true. /var/www/wiki/mediawiki/skins/chameleon/tests/phpunit/Unit/Components/GenericComponentTestCase.php:284 /var/www/wiki/mediawiki/skins/chameleon/tests/phpunit/Unit/Components/NavbarHorizontal/PersonalToolsTest.php:62 /var/www/wiki/mediawiki/core/maintenance/doMaintenance.php:105 -- There was 1 skipped test: 1) Skins\Chameleon\Tests\Integration\StylesCompileTest::testStylesCompile Test does not work on recent MW+PHP /var/www/wiki/mediawiki/skins/chameleon/tests/phpunit/Integration/StylesCompileTest.php:52 /var/www/wiki/mediawiki/core/maintenance/doMaintenance.php:105 FAILURES! Tests: 285, Assertions: 343, Failures: 1, Skipped: 1.
This is what I'm getting testing locally:
root@b4e039095739:/var/www/mediawiki/skins/Chameleon# php /var/www/mediawiki/tests/phpunit/phpunit.php --group skins-chameleon -c phpunit.xml.dist Using PHP 7.3.17 Using the MediaWiki vendor autoloader ... PHPUnit 8.5.5 by Sebastian Bergmann and contributors. Runtime: PHP 7.3.17 Configuration: /var/www/wikifarm/skins/Chameleon/phpunit.xml.dist Warning - The configuration file did not pass validation! The following problems have been detected: Line 40: - Element 'phpunit', attribute 'checkForUnintentionallyCoveredCode': The attribute 'checkForUnintentionallyCoveredCode' is not allowed. Test results may not be as expected. S.............................................................. 63 / 285 ( 22%) ............................................................... 126 / 285 ( 44%) ............................................................... 189 / 285 ( 66%) ............................................................... 252 / 285 ( 88%) ................................. 285 / 285 (100%) Time: 2.92 seconds, Memory: 30.00 MB There was 1 skipped test: 1) Skins\Chameleon\Tests\Integration\StylesCompileTest::testStylesCompile Test does not work on recent MW+PHP /var/www/wikifarm/skins/Chameleon/tests/phpunit/Integration/StylesCompileTest.php:52 /var/www/mediawiki/maintenance/doMaintenance.php:105 OK, but incomplete, skipped, or risky tests! Tests: 285, Assertions: 343, Skipped: 1.
I removed the line causing the 'checkForUnintentionallyCoveredCode', and the warning went away, but everything else stayed the same.
I'm slightly irked by this... Why not check it's defined at all?
public function assertValidHTML( $actual, $message = 'HTML text is not valid. ' ) { if ( !USE_EXTERNAL_HTML_VALIDATOR ) { $doc = $this->loadXML( $actual, true ); $this->assertNotFalse( $doc, $message ); return; }
Gonna change this to
public function assertValidHTML( $actual, $message = 'HTML text is not valid. ' ) { if ( !defined( 'USE_EXTERNAL_HTML_VALIDATOR' ) || !USE_EXTERNAL_HTML_VALIDATOR ) { $doc = $this->loadXML( $actual, true ); $this->assertNotFalse( $doc, $message ); return; }
I pushed a fix for that upstream. It's been deprecated since PHPUnit 5.2, removed in 6.0. No value lost
https://github.com/ProfessionalWiki/chameleon/issues/160 filed for the test failure I see on my version with PHP
Ah. Just did some digging.
T243006: Key "CanonicalNamespaceNames" not found in input sources in includes/config/ServiceOptions.php:52 and T237032: Key "CanonicalNamespaceNames" not found in input sources in includes/config/ServiceOptions.php:52
Not exactly the same, but it's similar....
This is getting ridiculous.
I half wonder if it's because it's not using the proper MW test classes/bootstrapping
01:39:31 2) Skins\Chameleon\Tests\Unit\Components\LogoTest::testGetHtmlwithEmptyElement 01:39:31 Error: Call to a member function getIP() on null 01:39:31 01:39:31 /workspace/src/includes/user/User.php:2149 01:39:31 /workspace/src/includes/session/SessionBackend.php:734 01:39:31 /workspace/src/includes/session/SessionBackend.php:207 01:39:31 /workspace/src/includes/session/Session.php:73 01:39:31 /workspace/src/includes/session/SessionManager.php:865 01:39:31 /workspace/src/includes/session/SessionManager.php:310 01:39:31 /workspace/src/includes/session/SessionManager.php:244 01:39:31 /workspace/src/includes/session/SessionManager.php:194 01:39:31 /workspace/src/includes/WebRequest.php:830 01:39:31 /workspace/src/includes/user/User.php:1165 01:39:31 /workspace/src/includes/user/User.php:340 01:39:31 /workspace/src/includes/user/User.php:2121 01:39:31 /workspace/src/includes/user/User.php:3227 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:547 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:455 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:124 01:39:31 /workspace/src/includes/user/User.php:2756 01:39:31 /workspace/src/includes/context/RequestContext.php:351 01:39:31 /workspace/src/includes/language/Message.php:402 01:39:31 /workspace/src/includes/language/Message.php:1305 01:39:31 /workspace/src/includes/language/Message.php:1000 01:39:31 /workspace/src/includes/Linker.php:2114 01:39:31 /workspace/src/includes/Linker.php:2305 01:39:31 /workspace/src/skins/chameleon/src/Components/Logo.php:85 01:39:31 /workspace/src/skins/chameleon/src/Components/Logo.php:72 01:39:31 /workspace/src/skins/chameleon/src/Components/Logo.php:50 01:39:31 /workspace/src/skins/chameleon/tests/phpunit/Unit/Components/GenericComponentTestCase.php:85 01:39:31 01:39:31 Caused by 01:39:31 Error: Call to a member function getIP() on null 01:39:31 01:39:31 /workspace/src/includes/user/User.php:2149 01:39:31 /workspace/src/includes/session/SessionBackend.php:734 01:39:31 /workspace/src/includes/session/SessionBackend.php:618 01:39:31 /workspace/src/vendor/wikimedia/scoped-callback/src/ScopedCallback.php:96 01:39:31 /workspace/src/vendor/wikimedia/scoped-callback/src/ScopedCallback.php:56 01:39:31 /workspace/src/includes/session/SessionManager.php:865 01:39:31 /workspace/src/includes/session/SessionManager.php:310 01:39:31 /workspace/src/includes/session/SessionManager.php:244 01:39:31 /workspace/src/includes/session/SessionManager.php:194 01:39:31 /workspace/src/includes/WebRequest.php:830 01:39:31 /workspace/src/includes/user/User.php:1165 01:39:31 /workspace/src/includes/user/User.php:340 01:39:31 /workspace/src/includes/user/User.php:2121 01:39:31 /workspace/src/includes/user/User.php:3227 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:547 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:455 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:124 01:39:31 /workspace/src/includes/user/User.php:2756 01:39:31 /workspace/src/includes/context/RequestContext.php:351 01:39:31 /workspace/src/includes/language/Message.php:402 01:39:31 /workspace/src/includes/language/Message.php:1305 01:39:31 /workspace/src/includes/language/Message.php:1000 01:39:31 /workspace/src/includes/Linker.php:2114 01:39:31 /workspace/src/includes/Linker.php:2305 01:39:31 /workspace/src/skins/chameleon/src/Components/Logo.php:85 01:39:31 /workspace/src/skins/chameleon/src/Components/Logo.php:72 01:39:31 /workspace/src/skins/chameleon/src/Components/Logo.php:50 01:39:31 /workspace/src/skins/chameleon/tests/phpunit/Unit/Components/GenericComponentTestCase.php:85 01:39:31 01:39:31 3) Skins\Chameleon\Tests\Unit\Components\NavMenuTest::testGetHtmlwithEmptyElement 01:39:31 MWException: Unable to determine IP. 01:39:31 01:39:31 /workspace/src/includes/WebRequest.php:1263 01:39:31 /workspace/src/includes/user/User.php:2149 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:547 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:455 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:124 01:39:31 /workspace/src/includes/user/User.php:2756 01:39:31 /workspace/src/includes/context/RequestContext.php:351 01:39:31 /workspace/src/includes/language/Message.php:402 01:39:31 /workspace/src/includes/language/Message.php:1305 01:39:31 /workspace/src/includes/language/Message.php:1000 01:39:31 /workspace/src/skins/chameleon/src/Components/NavMenu.php:201 01:39:31 /workspace/src/skins/chameleon/src/Components/NavMenu.php:58 01:39:31 /workspace/src/skins/chameleon/tests/phpunit/Unit/Components/GenericComponentTestCase.php:85 01:39:31 01:39:31 4) Skins\Chameleon\Tests\Unit\Components\SearchBarTest::testGetHtmlwithEmptyElement 01:39:31 MWException: Unable to determine IP. 01:39:31 01:39:31 /workspace/src/includes/WebRequest.php:1263 01:39:31 /workspace/src/includes/user/User.php:2149 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:547 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:455 01:39:31 /workspace/src/includes/user/UserOptionsManager.php:124 01:39:31 /workspace/src/includes/user/User.php:2756 01:39:31 /workspace/src/includes/context/RequestContext.php:351 01:39:31 /workspace/src/includes/language/Message.php:402 01:39:31 /workspace/src/includes/language/Message.php:1305 01:39:31 /workspace/src/includes/language/Message.php:1000 01:39:31 /workspace/src/includes/Linker.php:2114 01:39:31 /workspace/src/includes/Linker.php:2325 01:39:31 /workspace/src/skins/chameleon/src/Components/SearchBar.php:58 01:39:31 /workspace/src/skins/chameleon/tests/phpunit/Unit/Components/GenericComponentTestCase.php:85 01:39:31 01:39:31 5) Skins\Chameleon\Tests\Unit\Menu\MenuFromLinesTest::testBuildEmptyMenu 01:39:31 Error: Call to a member function getVal() on null 01:39:31 01:39:31 /workspace/src/includes/Title.php:2247 01:39:31 /workspace/src/skins/chameleon/src/Menu/MenuFromLines.php:211 01:39:31 /workspace/src/skins/chameleon/src/Menu/MenuFromLines.php:198 01:39:31 /workspace/src/skins/chameleon/src/Menu/MenuFromLines.php:165 01:39:31 /workspace/src/skins/chameleon/src/Menu/MenuFromLines.php:130 01:39:31 /workspace/src/skins/chameleon/src/Menu/MenuFromLines.php:88 01:39:31 /workspace/src/skins/chameleon/src/Menu/MenuFromLines.php:68 01:39:31 /workspace/src/skins/chameleon/tests/phpunit/Unit/Menu/MenuFromLinesTest.php:121
I created a pull request for both Bootstrap and Chameleon to convert to using MediaWiki test classes and bootstrapping: https://github.com/ProfessionalWiki/Bootstrap/pull/36 and https://github.com/ProfessionalWiki/chameleon/pull/163. The tests pass on travis. It would be good to see if https://gerrit.wikimedia.org/r/c/mediawiki/skins/chameleon/+/597812 would pass CI with the changes.
Fwiw, you can push them to gerrit too if you add a remote for gerrit
git push gerrit HEAD:refs/for/master
And then see the patches running in our CI
Obviously we won’t merge them, but helps test it here too
Thanks. I will try that next time. Unfortunately, this did not fix the errors in CI. Are you still getting errors testing locally? I am not getting any errors locally.
When I try that I get the following error message:
remote: error: commit d30920b: email address noreply@github.com is not registered in your account, and you lack 'forge committer' permission.
Am I missing some permission on gerrit that I need in order to do this?
Maybe it is as simple as that I am not in the skin-chameleon and extension-Bootstrap gerrit groups.
Yeah, probably. But I can't add you unfortunately, needs a gerrit admin
I'm not sure why it's seemingly trying to push a noreply@github.com commit though...
Right, the "noreply@github.com" bit has me confused as well. I'm obviously not pulling the changes from the github repo correctly into my local clone of the gerrit repo. What steps do you use to pull from github?
$ git remote -v gerrit ssh://reedy@gerrit.wikimedia.org:29418/mediawiki/skins/chameleon (fetch) gerrit ssh://reedy@gerrit.wikimedia.org:29418/mediawiki/skins/chameleon (push) origin https://github.com/ProfessionalWiki/chameleon.git (fetch) origin https://github.com/ProfessionalWiki/chameleon.git (push) reedy git@github.com:reedy/chameleon.git (fetch) reedy git@github.com:reedy/chameleon.git (push)
master tracks origin/master, so I've just been doing git push gerrit master for mirroring purposes, not trying to push a changeset to gerrit though
Have you done git fetch --all after adding the remote?
But if I try and push to gerrit for normal... (I generated the Change-Id on another repo rather than installing the hook)
https://gerrit.wikimedia.org/r/#/c/mediawiki/skins/chameleon/+/600952/
reedy@ubuntu64-web-esxi:~/chameleon$ nano README.md reedy@ubuntu64-web-esxi:~/chameleon$ git commit -a -m "Test" [master 919c9ba] Test 1 file changed, 2 insertions(+) reedy@ubuntu64-web-esxi:~/chameleon$ git push noreply@github.com^C reedy@ubuntu64-web-esxi:~/chameleon$ git push gerrit HEAD:refs/for/master Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Delta compression using up to 2 threads Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 289 bytes | 289.00 KiB/s, done. Total 3 (delta 2), reused 0 (delta 0) remote: Resolving deltas: 100% (2/2) remote: Processing changes: refs: 1, done remote: commit 919c9ba: ERROR: missing Change-Id in message footer remote: remote: Hint: to automatically insert a Change-Id, install the hook: remote: gitdir=$(git rev-parse --git-dir); scp -p -P 29418 reedy@gerrit.wikimedia.org:hooks/commit-msg ${gitdir}/hooks/ remote: and then amend the commit: remote: git commit --amend remote: To ssh://gerrit.wikimedia.org:29418/mediawiki/skins/chameleon ! [remote rejected] HEAD -> refs/for/master (commit 919c9ba: missing Change-Id in message footer) error: failed to push some refs to 'ssh://reedy@gerrit.wikimedia.org:29418/mediawiki/skins/chameleon' reedy@ubuntu64-web-esxi:~/chameleon$ git rev-parse --git-dir .git reedy@ubuntu64-web-esxi:~/chameleon$ git commit --amend -m "Test > > Change-Id: Id0fccf6e13c702a1a0aa5225ccbec065226f752a" [master 787b530] Test Date: Sun May 31 18:34:07 2020 +0100 1 file changed, 2 insertions(+) reedy@ubuntu64-web-esxi:~/chameleon$ git log -n 1 commit 787b53013f684bd2eb533943c9b013a0254dd721 (HEAD -> master) Author: Reedy <reedy@wikimedia.org> Date: Sun May 31 18:34:07 2020 +0100 Test Change-Id: Id0fccf6e13c702a1a0aa5225ccbec065226f752a reedy@ubuntu64-web-esxi:~/chameleon$ git push gerrit HEAD:refs/for/master Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Delta compression using up to 2 threads Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 333 bytes | 333.00 KiB/s, done. Total 3 (delta 2), reused 0 (delta 0) remote: Resolving deltas: 100% (2/2) remote: Processing changes: new: 1, done remote: remote: New Changes: remote: https://gerrit.wikimedia.org/r/#/c/mediawiki/skins/chameleon/+/600952 Test remote: To ssh://gerrit.wikimedia.org:29418/mediawiki/skins/chameleon * [new branch] HEAD -> refs/for/master reedy@ubuntu64-web-esxi:~/chameleon$
I'm wondering if the "noreply@github.com" is just a red herring since I don't have appropriate access to the repo for pushing a change set using HEAD:refs/for/master. I don't see anything significantly different in what I'm am doing from above. I don't get the Change-Id error, but that's probably because I get the other error before it gets that far.
I had not tried pushing directly for mirroring. Since Bootstrap on gerrit is behind github, I tried git push gerrit master and got the expected access denied correctly listing my username.
I updated the code to use MW test classes/bootstrappping (see https://gerrit.wikimedia.org/r/c/mediawiki/skins/chameleon/+/600986), but it does not fix the problem.
Given that T243006 and T237032 are exhibiting the same issue, this does not appear to be a Chameleon issue. Interesting that phpunit has no issues in my local environment, but fails in yours and in Wikimedia CI.