mediawiki/extensions/Math/+/634038 failed to merge with error message HTTP request blocked. Use MockHttpTrait.
Example:
+'<strong class="error texerror">Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "HTTP request blocked. Use MockHttpTrait."): {\displaystyle e^{i \pi} + 1 = 0\,\!}</strong>\nCaused by T262443: Do not hit actual urls during phpunit tests
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T265628 Identify which extensions' tests make HTTP requests and so are now broken post-T262443 | |||
| Resolved | Physikerwelt | T258369 MathSearch CI Jobs fail |
Also affects UploadWizard: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/UploadWizard/+/634290
Change 634293 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/UploadWizard@master] ApiFlickrBlacklistTest: Don't try to access HTTP in integration tests
Change 634294 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/extensions/UploadWizard@master] Skip ApiFlickrBlacklistTest tests
Change 634294 abandoned by Jforrester:
[mediawiki/extensions/UploadWizard@master] Skip ApiFlickrBlacklistTest tests
Reason:
Let's go with mine?
Change 634305 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/Math@master] MathCoverageTest: Skip all tests for now as HTTP requests are banned
Change 634293 merged by jenkins-bot:
[mediawiki/extensions/UploadWizard@master] ApiFlickrBlacklistTest: Don't try to access HTTP in integration tests
Change 634259 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/UploadWizard@wmf/1.36.0-wmf.13] ApiFlickrBlacklistTest: Don't try to access HTTP in integration tests
Change 634259 merged by jenkins-bot:
[mediawiki/extensions/UploadWizard@wmf/1.36.0-wmf.13] ApiFlickrBlacklistTest: Don't try to access HTTP in integration tests
wow ... unbreak now? For math this will deactive regression tests. The fact that those are included in the ordinary php unit testing cyle is not optimal. Thus I think it is ok to deactive them for now.
Looking at @Jdforrester-WMF patch effect this also affects
one way to move forwared is to set the default rendering to source for testing.
Do the tests currently do real requests to a real service? That should not be the case. Phpunit tests need to be able to run without any connection to the outside world.
In any case, another possibility is to use MockHttpTrait to emulate the response.
@daniel yes, I abused PHPunit for integration tests. I know this is not state of the art. And the problem with external connection is also the reason that Wikibase switched off the math tests. Eventually we ended up with T266496 (math does not work with wikibase anymore). To me the top priority is to get math rendering back to wikibase. Whatever, it takes to get the tests deactivated, we need to be able to perform changes to get math rendering back. In the future (after we have established a model to review math related patches after @mobrovac left the foundation) math will be a service, and testing will be much more straigt forward.
@Physikerwelt just a sitenote: the daily i18n patches from translatewiki.net are blocked currently, i.e. https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Math/+/636584
Change 634305 merged by jenkins-bot:
[mediawiki/extensions/Math@master] MathCoverageTest: Skip all tests for now as HTTP requests are banned
HTTP request blocked: https://onionoo.torproject.org/details?type=relay&running=true&flag=Exit by TorExitNodes::fetchExitNodesFromOnionooServer. Use MockHttpTrait.
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/45155/console
does not show helpful information:
PHPUnit extensions suite (without database or standalone) 15:46:53 INFO:quibble.commands:>>> Start: PHPUnit extensions suite (without database or standalone) 15:46:53 INFO:quibble.commands:PHPUnit extensions suite (without database or standalone) 15:46:53 INFO:quibble.commands:php tests/phpunit/phpunit.php --testsuite extensions --exclude-group Broken,ParserFuzz,Stub,Database,Standalone --log-junit /workspace/log/junit-dbless.xml 15:46:53 Using PHP 7.2.31-1+0~20200514.41+debian9~1.gbpe2a56b+wmf1 15:46:53 PHPUnit 8.5.8 by Sebastian Bergmann and contributors. 15:46:53 15:46:56 HTTP request blocked: https://onionoo.torproject.org/details?type=relay&running=true&flag=Exit by TorExitNodes::fetchExitNodesFromOnionooServer. Use MockHttpTrait. 15:46:56 INFO:quibble.commands:<<< Finish: PHPUnit extensions suite (without database or standalone), in 3.209 s 15:46:56 INFO:backend.MySQL:Terminating MySQL
Structure tests or test setup functions, not sure.
Change 641833 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/extensions/TorBlock@master] Use fake exit node IPs in unit tests
This is a complete shot in the dark. I don't even know where or how to trigegr the failure reported above. TorBlock itself doesn't appear to have any tests at all.
Anyway, this code should prevent the offending HTTP requests.
Change 641833 merged by jenkins-bot:
[mediawiki/extensions/TorBlock@master] Use fake exit node IPs in unit tests
Next one is MultiMaps
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/50496/console
HTTP request blocked: http://open.mapquestapi.com/nominatim/v1/search.php?format=json&addressdetails=0&limit=1&q=2%22%2C+10%C2%B010%2712%22 by Http::get. Use MockHttpTrait.
And MathSearch
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/50472/console
HTTP requests to https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex blocked. Use MockHttpTrait.
Change 648957 had a related patch set uploaded (by Physikerwelt; owner: Physikerwelt):
[mediawiki/extensions/MathSearch@master] Update HTTP interface
Change 648957 merged by Physikerwelt:
[mediawiki/extensions/MathSearch@master] Update HTTP interface
@Umherirrender it seems that for mathsearch there is an interference with other extensions. See https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/53013/console
00:05:05.816 22) PreferenceHandlingTest::testHandlingOfPreferences with data set #1 ('Totally valid preference shou...rately', array('soup-label', 'something-something-desc-side', 'http://example.org/features', 'http://example.org/feedback'), array('soup-label', 'something-something-desc-side', 'http://example.org/features', 'http://example.org/feedback', 'HTMLFeatureField', 'betafeatures'))
00:05:05.816 HTTP requests to https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex blocked. Use MockHttpTrait.
00:05:05.816PageViewsTest::testLog in WIkimediaEvents is broken, since EventLogging tries to pull a schema defition from meta via HTTP:
HTTP request blocked: https://meta.wikimedia.org/w/api.php?action=jsonschema&revid=18910134&formatversion=2&format=json by RemoteSchema::httpGet. Use MockHttpTrait. /var/www/html/w/tests/phpunit/mocks/NullHttpRequestFactory.php:43 /var/www/html/w/includes/http/HttpRequestFactory.php:189 /var/www/html/w/includes/http/HttpRequestFactory.php:212 /var/www/html/w/extensions/EventLogging/includes/RemoteSchema.php:108 /var/www/html/w/extensions/EventLogging/includes/RemoteSchema.php:57 /var/www/html/w/extensions/EventLogging/includes/EventLogging.php:191 /var/www/html/w/extensions/EventLogging/includes/EventLogging.php:84 /var/www/html/w/extensions/WikimediaEvents/includes/PageViews.php:229 /var/www/html/w/extensions/WikimediaEvents/tests/phpunit/PageViewsTest.php:489 /var/www/html/w/tests/phpunit/MediaWikiIntegrationTestCase.php:437
Change 649873 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/extensions/WikimediaEvents@master] Use MockHttpTrait in PageViewsTest
Change 649873 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Use MockHttpTrait in PageViewsTest
I did some debugging for MathSearch https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MathSearch/+/650798 and it seems that the problem is caused by special pages and hooks. I think the best way to deal with those issues on a case by case basis following the idea that special pages must not throw fatals even if required services are down.
Why should they not throw fatal exceptions in such a case? It seems the like correct thing to do...
I'd rather thing that an extension that is designed to make HTTP calls, and makes such calls from a hook that is triggered during normal operation, needs to ensure that HTTP calls are disabled during testing.
A more general approach would be to disable hooks in most unit tests, and create an easy way for extensions to run tests with only their own hook handlers set.
Special pages are user-facing, so should display an end-user grokable error state rather than a 503. :-) But yes, it's an error state.
Oh no. The logs are gone:-( However, I did look at all the special pages and could not find an obvious problem. Also, the error message does not give any pointers to the MathSearch extension:
Regression
MediaWiki.Skins.Vector.Tests.Integration.HTMLLegacySkinVersionFieldTest.testGetInput with data set #0 (from MediaWiki_Skins_Vector_Tests_Integration_HTMLLegacySkinVersionFieldTest__testGetInput)
Failing for the past 1 build (Since Failed#55230 )
Took 10 ms.
Stacktrace
MediaWiki\Skins\Vector\Tests\Integration\HTMLLegacySkinVersionFieldTest::testGetInput with data set #0 ('1', true)
HTTP requests to https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex blocked. Use MockHttpTrait.
/workspace/src/tests/phpunit/mocks/NullMultiHttpClient.php:29
/workspace/src/includes/libs/virtualrest/VirtualRESTServiceClient.php:254
/workspace/src/extensions/Math/src/MathRestbaseInterface.php:167
/workspace/src/extensions/Math/src/MathMathML.php:99
/workspace/src/extensions/Math/src/MathHooks.php:351
/workspace/src/includes/HookContainer/HookContainer.php:333
/workspace/src/includes/HookContainer/HookContainer.php:140
/workspace/src/includes/HookContainer/HookRunner.php:2892
/workspace/src/includes/parser/Parser.php:1678
/workspace/src/includes/parser/Parser.php:649
/workspace/src/includes/cache/MessageCache.php:1306
/workspace/src/includes/language/Message.php:1248
/workspace/src/includes/language/Message.php:878
/workspace/src/includes/language/Message.php:931
/workspace/src/includes/htmlform/HTMLFormField.php:416
/workspace/src/skins/Vector/includes/HTMLForm/Fields/HTMLLegacySkinVersionField.php:42
/workspace/src/skins/Vector/tests/phpunit/integration/HTMLForm/Fields/HTMLLegacySkinVersionFieldTest.php:70
/workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:437
/workspace/src/maintenance/doMaintenance.php:106Moreover, locally I can not reproduce the problem when running the test manually.
In production, sure. But we are talking about errors caused by preventing HTTP requests in phpunit tests... Or did I miss the context of your suggestion?
The context is that I have no idea why those tests fail for mathsearch but not for math. If there is a test that for some reason loads a special page that might explain it... but I am still not really convinced that the special page is the key to the solution. In the stack trace there is no hint why a math tag ended up in the Parser output. It would be interesting if one can crate a special localSettings for the mathsearch ci job which puts the rendering mode to source so that math rags would not cause a http request.
Can you make the rendering mode default to source if the MW_PHPUNIT_TEST constant is set? I'm no fan of "Volkswagen code", but if we have a default that should not be used in testing, it seems appropriate.
I'm also wondering if we could allow HTTP requests to reserved domain names like example.com in tests. The response would of course be fake (perhaps empty, or "dummy" or something), but it would not make the test fail immediately. The same could be done for IPs in the TEST-NET range.
You could then change the default URL for rendering to an example domain, rather than changing the mode to source. Not sure that would make much of a difference, though.
Change 657960 had a related patch set uploaded (by Physikerwelt; owner: Physikerwelt):
[mediawiki/extensions/MathSearch@master] Switch to source rendering mode for WMF CI
Change 657960 abandoned by Physikerwelt:
[mediawiki/extensions/MathSearch@master] Switch to source rendering mode for WMF CI
Reason:
Somehow this does not really overwrite the rendering mode used by other extension. I think I will need to download some of them locally and try to debug why they are now trying to contact math which they would not do otherwise. Moreover, the problem might go away when we implement automatic fallback to source rendering mode on error.
Change 679960 had a related patch set uploaded (by Umherirrender; author: Umherirrender):
[mediawiki/extensions/FileImporter@master] Set empty hook handlers on ExtensionRegistry for testing
Change 679960 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Set empty hook handlers on ExtensionRegistry for testing
It seems I'm unable to run MediaWiki core's unit tests locally without ExtraParserTest, MovePageTest and others failing due to:
HTTP request blocked: https://commons.wikimedia.org/w/api.php?titles=File%3AExistent.jpg&iiprop=timestamp%7Cuser%7Ccomment%7Curl%7Csize%7Csha1%7Cmetadata%7Cmime%7Cmediatype%7Cextmetadata&prop=imageinfo&iimetadataversion=2&iiextmetadatamultilang=1&format=json&action=query&redirects=true&uselang=en by ForeignAPIRepo::httpGet. Use MockHttpTrait.
/var/www/mediawiki/tests/phpunit/mocks/NullHttpRequestFactory.php:43 /var/www/mediawiki/includes/http/MWHttpRequest.php:199 /var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:529 /var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:575 /var/www/mediawiki/includes/libs/objectcache/wancache/WANObjectCache.php:1714 /var/www/mediawiki/includes/libs/objectcache/wancache/WANObjectCache.php:1542 /var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:585 /var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:199 /var/www/mediawiki/includes/filerepo/file/ForeignAPIFile.php:65 /var/www/mediawiki/includes/filerepo/FileRepo.php:431 /var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:115 /var/www/mediawiki/includes/filerepo/FileRepo.php:471 /var/www/mediawiki/includes/filerepo/RepoGroup.php:161 /var/www/mediawiki/includes/page/WikiFilePage.php:65 /var/www/mediawiki/includes/page/WikiFilePage.php:112 /var/www/mediawiki/includes/Storage/DerivedPageDataUpdater.php:569 /var/www/mediawiki/includes/Storage/PageUpdater.php:404 /var/www/mediawiki/includes/Storage/PageUpdater.php:755 /var/www/mediawiki/includes/page/WikiPage.php:2222 /var/www/mediawiki/includes/page/WikiPage.php:2076 /var/www/mediawiki/tests/phpunit/MediaWikiIntegrationTestCase.php:268 /var/www/mediawiki/tests/phpunit/includes/MovePageTest.php:104
Change 711504 had a related patch set uploaded (by Ppchelko; author: Ppchelko):
[mediawiki/extensions/Math@master] Enable most tests which were previously disabled
Change 711504 merged by jenkins-bot:
[mediawiki/extensions/Math@master] Enable most tests which were previously disabled
Change 852785 had a related patch set uploaded (by Fomafix; author: Fomafix):
[mediawiki/extensions/MultiMaps@master] Skip tests for now as HTTP requests are banned
Change 852785 merged by Fomafix:
[mediawiki/extensions/MultiMaps@master] Skip tests for now as HTTP requests are banned
Change 863360 had a related patch set uploaded (by Umherirrender; author: Fomafix):
[mediawiki/extensions/MultiMaps@REL1_39] Skip tests for now as HTTP requests are banned
Change 863360 merged by jenkins-bot:
[mediawiki/extensions/MultiMaps@REL1_39] Skip tests for now as HTTP requests are banned
Change 868361 had a related patch set uploaded (by Umherirrender; author: Fomafix):
[mediawiki/extensions/MultiMaps@REL1_38] Skip tests for now as HTTP requests are banned
Change 868361 merged by jenkins-bot:
[mediawiki/extensions/MultiMaps@REL1_38] Skip tests for now as HTTP requests are banned