Page MenuHomePhabricator

Identify which extensions' tests make HTTP requests and so are now broken post-T262443
Open, HighPublic

Description

mediawiki/extensions/Math/+/634038 failed to merge with error message HTTP request blocked. Use MockHttpTrait.

Example:

+'<strong class="error texerror">Failed to parse (Conversion error. Server (&quot;https://wikimedia.org/api/rest_&quot;) reported: &quot;HTTP request blocked. Use MockHttpTrait.&quot;): {\displaystyle e^{i \pi} + 1 = 0\,\!}</strong>\n

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/41299/consoleFull

Caused by T262443: Do not hit actual urls during phpunit tests

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 634293 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/UploadWizard@master] ApiFlickrBlacklistTest: Don't try to access HTTP in integration tests

https://gerrit.wikimedia.org/r/634293

Change 634294 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/extensions/UploadWizard@master] Skip ApiFlickrBlacklistTest tests

https://gerrit.wikimedia.org/r/634294

Change 634294 abandoned by Jforrester:
[mediawiki/extensions/UploadWizard@master] Skip ApiFlickrBlacklistTest tests

Reason:
Let's go with mine?

https://gerrit.wikimedia.org/r/634294

Change 634305 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/Math@master] MathCoverageTest: Skip all tests for now as HTTP requests are banned

https://gerrit.wikimedia.org/r/634305

Change 634293 merged by jenkins-bot:
[mediawiki/extensions/UploadWizard@master] ApiFlickrBlacklistTest: Don't try to access HTTP in integration tests

https://gerrit.wikimedia.org/r/634293

Change 634259 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/UploadWizard@wmf/1.36.0-wmf.13] ApiFlickrBlacklistTest: Don't try to access HTTP in integration tests

https://gerrit.wikimedia.org/r/634259

Jdforrester-WMF renamed this task from HTTP request blocked. Use MockHttpTrait. to Identify which extensions' tests make HTTP requests and so are now broken post-T262443.Oct 15 2020, 6:04 PM
Jdforrester-WMF raised the priority of this task from High to Unbreak Now!.

Change 634259 merged by jenkins-bot:
[mediawiki/extensions/UploadWizard@wmf/1.36.0-wmf.13] ApiFlickrBlacklistTest: Don't try to access HTTP in integration tests

https://gerrit.wikimedia.org/r/634259

Jdforrester-WMF added a subscriber: Jdforrester-WMF.

Tagging Platform for visibility as it's their change that is at the centre of this.

wow ... unbreak now? For math this will deactive regression tests. The fact that those are included in the ordinary php unit testing cyle is not optimal. Thus I think it is ok to deactive them for now.

Looking at @Jdforrester-WMF patch effect this also affects

  • Wikibase
  • ApiMobileViewTest
  • ApiParseExtenderTest

one way to move forwared is to set the default rendering to source for testing.

one way to move forwared is to set the default rendering to source for testing.

Do the tests currently do real requests to a real service? That should not be the case. Phpunit tests need to be able to run without any connection to the outside world.

In any case, another possibility is to use MockHttpTrait to emulate the response.

@daniel yes, I abused PHPunit for integration tests. I know this is not state of the art. And the problem with external connection is also the reason that Wikibase switched off the math tests. Eventually we ended up with T266496 (math does not work with wikibase anymore). To me the top priority is to get math rendering back to wikibase. Whatever, it takes to get the tests deactivated, we need to be able to perform changes to get math rendering back. In the future (after we have established a model to review math related patches after @mobrovac left the foundation) math will be a service, and testing will be much more straigt forward.

@Physikerwelt just a sitenote: the daily i18n patches from translatewiki.net are blocked currently, i.e. https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Math/+/636584

Change 634305 merged by jenkins-bot:
[mediawiki/extensions/Math@master] MathCoverageTest: Skip all tests for now as HTTP requests are banned

https://gerrit.wikimedia.org/r/634305

AMooney lowered the priority of this task from Unbreak Now! to High.Nov 2 2020, 6:40 PM

Removing Math as the remaining work for math is tracked in T265646

HTTP request blocked: https://onionoo.torproject.org/details?type=relay&running=true&flag=Exit by TorExitNodes::fetchExitNodesFromOnionooServer. Use MockHttpTrait.

What test triggers this error? TorBlock itself doesn't appear to have any tests.

HTTP request blocked: https://onionoo.torproject.org/details?type=relay&running=true&flag=Exit by TorExitNodes::fetchExitNodesFromOnionooServer. Use MockHttpTrait.

What test triggers this error? TorBlock itself doesn't appear to have any tests.

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/45155/console
does not show helpful information:

PHPUnit extensions suite (without database or standalone)

15:46:53 INFO:quibble.commands:>>> Start: PHPUnit extensions suite (without database or standalone)
15:46:53 INFO:quibble.commands:PHPUnit extensions suite (without database or standalone)
15:46:53 INFO:quibble.commands:php tests/phpunit/phpunit.php --testsuite extensions --exclude-group Broken,ParserFuzz,Stub,Database,Standalone --log-junit /workspace/log/junit-dbless.xml
15:46:53 Using PHP 7.2.31-1+0~20200514.41+debian9~1.gbpe2a56b+wmf1
15:46:53 PHPUnit 8.5.8 by Sebastian Bergmann and contributors.
15:46:53 
15:46:56 HTTP request blocked: https://onionoo.torproject.org/details?type=relay&running=true&flag=Exit by TorExitNodes::fetchExitNodesFromOnionooServer. Use MockHttpTrait.
15:46:56 INFO:quibble.commands:<<< Finish: PHPUnit extensions suite (without database or standalone), in 3.209 s
15:46:56 INFO:backend.MySQL:Terminating MySQL

Structure tests or test setup functions, not sure.

Change 641833 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/extensions/TorBlock@master] Use fake exit node IPs in unit tests

https://gerrit.wikimedia.org/r/641833

Change 641833 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/extensions/TorBlock@master] Use fake exit node IPs in unit tests

https://gerrit.wikimedia.org/r/641833

This is a complete shot in the dark. I don't even know where or how to trigegr the failure reported above. TorBlock itself doesn't appear to have any tests at all.

Anyway, this code should prevent the offending HTTP requests.

Change 641833 merged by jenkins-bot:
[mediawiki/extensions/TorBlock@master] Use fake exit node IPs in unit tests

https://gerrit.wikimedia.org/r/641833

Next one is MultiMaps

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/50496/console

HTTP request blocked: http://open.mapquestapi.com/nominatim/v1/search.php?format=json&addressdetails=0&limit=1&q=2%22%2C+10%C2%B010%2712%22 by Http::get. Use MockHttpTrait.

And MathSearch

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/50472/console

HTTP requests to https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex blocked. Use MockHttpTrait.

Change 648957 had a related patch set uploaded (by Physikerwelt; owner: Physikerwelt):
[mediawiki/extensions/MathSearch@master] Update HTTP interface

https://gerrit.wikimedia.org/r/648957

Change 648957 merged by Physikerwelt:
[mediawiki/extensions/MathSearch@master] Update HTTP interface

https://gerrit.wikimedia.org/r/648957

@Umherirrender it seems that for mathsearch there is an interference with other extensions. See https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/53013/console

 00:05:05.816 22) PreferenceHandlingTest::testHandlingOfPreferences with data set #1 ('Totally valid preference shou...rately', array('soup-label', 'something-something-desc-side', 'http://example.org/features', 'http://example.org/feedback'), array('soup-label', 'something-something-desc-side', 'http://example.org/features', 'http://example.org/feedback', 'HTMLFeatureField', 'betafeatures'))
00:05:05.816 HTTP requests to https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex blocked. Use MockHttpTrait.
00:05:05.816

PageViewsTest::testLog in WIkimediaEvents is broken, since EventLogging tries to pull a schema defition from meta via HTTP:

HTTP request blocked: https://meta.wikimedia.org/w/api.php?action=jsonschema&revid=18910134&formatversion=2&format=json by RemoteSchema::httpGet. Use MockHttpTrait.
 /var/www/html/w/tests/phpunit/mocks/NullHttpRequestFactory.php:43
 /var/www/html/w/includes/http/HttpRequestFactory.php:189
 /var/www/html/w/includes/http/HttpRequestFactory.php:212
 /var/www/html/w/extensions/EventLogging/includes/RemoteSchema.php:108
 /var/www/html/w/extensions/EventLogging/includes/RemoteSchema.php:57
 /var/www/html/w/extensions/EventLogging/includes/EventLogging.php:191
 /var/www/html/w/extensions/EventLogging/includes/EventLogging.php:84
 /var/www/html/w/extensions/WikimediaEvents/includes/PageViews.php:229
 /var/www/html/w/extensions/WikimediaEvents/tests/phpunit/PageViewsTest.php:489
 /var/www/html/w/tests/phpunit/MediaWikiIntegrationTestCase.php:437

Change 649873 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/extensions/WikimediaEvents@master] Use MockHttpTrait in PageViewsTest

https://gerrit.wikimedia.org/r/649873

Change 649873 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Use MockHttpTrait in PageViewsTest

https://gerrit.wikimedia.org/r/649873

I did some debugging for MathSearch https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MathSearch/+/650798 and it seems that the problem is caused by special pages and hooks. I think the best way to deal with those issues on a case by case basis following the idea that special pages must not throw fatals even if required services are down.

I think the best way to deal with those issues on a case by case basis following the idea that special pages must not throw fatals even if required services are down.

Why should they not throw fatal exceptions in such a case? It seems the like correct thing to do...

I'd rather thing that an extension that is designed to make HTTP calls, and makes such calls from a hook that is triggered during normal operation, needs to ensure that HTTP calls are disabled during testing.

A more general approach would be to disable hooks in most unit tests, and create an easy way for extensions to run tests with only their own hook handlers set.

I think the best way to deal with those issues on a case by case basis following the idea that special pages must not throw fatals even if required services are down.

Why should they not throw fatal exceptions in such a case? It seems the like correct thing to do...

Special pages are user-facing, so should display an end-user grokable error state rather than a 503. :-) But yes, it's an error state.

Oh no. The logs are gone:-( However, I did look at all the special pages and could not find an obvious problem. Also, the error message does not give any pointers to the MathSearch extension:

Regression

MediaWiki.Skins.Vector.Tests.Integration.HTMLLegacySkinVersionFieldTest.testGetInput with data set #0 (from MediaWiki_Skins_Vector_Tests_Integration_HTMLLegacySkinVersionFieldTest__testGetInput)
Failing for the past 1 build (Since Failed#55230 )
Took 10 ms.
Stacktrace

MediaWiki\Skins\Vector\Tests\Integration\HTMLLegacySkinVersionFieldTest::testGetInput with data set #0 ('1', true)
HTTP requests to https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex, https://wikimedia.org/api/rest_v1/media/math/check/tex blocked. Use MockHttpTrait.

/workspace/src/tests/phpunit/mocks/NullMultiHttpClient.php:29
/workspace/src/includes/libs/virtualrest/VirtualRESTServiceClient.php:254
/workspace/src/extensions/Math/src/MathRestbaseInterface.php:167
/workspace/src/extensions/Math/src/MathMathML.php:99
/workspace/src/extensions/Math/src/MathHooks.php:351
/workspace/src/includes/HookContainer/HookContainer.php:333
/workspace/src/includes/HookContainer/HookContainer.php:140
/workspace/src/includes/HookContainer/HookRunner.php:2892
/workspace/src/includes/parser/Parser.php:1678
/workspace/src/includes/parser/Parser.php:649
/workspace/src/includes/cache/MessageCache.php:1306
/workspace/src/includes/language/Message.php:1248
/workspace/src/includes/language/Message.php:878
/workspace/src/includes/language/Message.php:931
/workspace/src/includes/htmlform/HTMLFormField.php:416
/workspace/src/skins/Vector/includes/HTMLForm/Fields/HTMLLegacySkinVersionField.php:42
/workspace/src/skins/Vector/tests/phpunit/integration/HTMLForm/Fields/HTMLLegacySkinVersionFieldTest.php:70
/workspace/src/tests/phpunit/MediaWikiIntegrationTestCase.php:437
/workspace/src/maintenance/doMaintenance.php:106

Moreover, locally I can not reproduce the problem when running the test manually.

Special pages are user-facing, so should display an end-user grokable error state rather than a 503. :-) But yes, it's an error state.

In production, sure. But we are talking about errors caused by preventing HTTP requests in phpunit tests... Or did I miss the context of your suggestion?

The context is that I have no idea why those tests fail for mathsearch but not for math. If there is a test that for some reason loads a special page that might explain it... but I am still not really convinced that the special page is the key to the solution. In the stack trace there is no hint why a math tag ended up in the Parser output. It would be interesting if one can crate a special localSettings for the mathsearch ci job which puts the rendering mode to source so that math rags would not cause a http request.

The context is that I have no idea why those tests fail for mathsearch but not for math. If there is a test that for some reason loads a special page that might explain it... but I am still not really convinced that the special page is the key to the solution. In the stack trace there is no hint why a math tag ended up in the Parser output. It would be interesting if one can crate a special localSettings for the mathsearch ci job which puts the rendering mode to source so that math rags would not cause a http request.

Can you make the rendering mode default to source if the MW_PHPUNIT_TEST constant is set? I'm no fan of "Volkswagen code", but if we have a default that should not be used in testing, it seems appropriate.

I'm also wondering if we could allow HTTP requests to reserved domain names like example.com in tests. The response would of course be fake (perhaps empty, or "dummy" or something), but it would not make the test fail immediately. The same could be done for IPs in the TEST-NET range.

You could then change the default URL for rendering to an example domain, rather than changing the mode to source. Not sure that would make much of a difference, though.

Change 657960 had a related patch set uploaded (by Physikerwelt; owner: Physikerwelt):
[mediawiki/extensions/MathSearch@master] Switch to source rendering mode for WMF CI

https://gerrit.wikimedia.org/r/657960

Change 657960 abandoned by Physikerwelt:
[mediawiki/extensions/MathSearch@master] Switch to source rendering mode for WMF CI

Reason:
Somehow this does not really overwrite the rendering mode used by other extension. I think I will need to download some of them locally and try to debug why they are now trying to contact math which they would not do otherwise. Moreover, the problem might go away when we implement automatic fallback to source rendering mode on error.

https://gerrit.wikimedia.org/r/657960

Change 679960 had a related patch set uploaded (by Umherirrender; author: Umherirrender):

[mediawiki/extensions/FileImporter@master] Set empty hook handlers on ExtensionRegistry for testing

https://gerrit.wikimedia.org/r/679960

Change 679960 merged by jenkins-bot:

[mediawiki/extensions/FileImporter@master] Set empty hook handlers on ExtensionRegistry for testing

https://gerrit.wikimedia.org/r/679960

It seems I'm unable to run MediaWiki core's unit tests locally without ExtraParserTest, MovePageTest and others failing due to:

HTTP request blocked: https://commons.wikimedia.org/w/api.php?titles=File%3AExistent.jpg&iiprop=timestamp%7Cuser%7Ccomment%7Curl%7Csize%7Csha1%7Cmetadata%7Cmime%7Cmediatype%7Cextmetadata&prop=imageinfo&iimetadataversion=2&iiextmetadatamultilang=1&format=json&action=query&redirects=true&uselang=en by ForeignAPIRepo::httpGet. Use MockHttpTrait.
/var/www/mediawiki/tests/phpunit/mocks/NullHttpRequestFactory.php:43
/var/www/mediawiki/includes/http/MWHttpRequest.php:199
/var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:529
/var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:575
/var/www/mediawiki/includes/libs/objectcache/wancache/WANObjectCache.php:1714
/var/www/mediawiki/includes/libs/objectcache/wancache/WANObjectCache.php:1542
/var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:585
/var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:199
/var/www/mediawiki/includes/filerepo/file/ForeignAPIFile.php:65
/var/www/mediawiki/includes/filerepo/FileRepo.php:431
/var/www/mediawiki/includes/filerepo/ForeignAPIRepo.php:115
/var/www/mediawiki/includes/filerepo/FileRepo.php:471
/var/www/mediawiki/includes/filerepo/RepoGroup.php:161
/var/www/mediawiki/includes/page/WikiFilePage.php:65
/var/www/mediawiki/includes/page/WikiFilePage.php:112
/var/www/mediawiki/includes/Storage/DerivedPageDataUpdater.php:569
/var/www/mediawiki/includes/Storage/PageUpdater.php:404
/var/www/mediawiki/includes/Storage/PageUpdater.php:755
/var/www/mediawiki/includes/page/WikiPage.php:2222
/var/www/mediawiki/includes/page/WikiPage.php:2076
/var/www/mediawiki/tests/phpunit/MediaWikiIntegrationTestCase.php:268
/var/www/mediawiki/tests/phpunit/includes/MovePageTest.php:104