Page MenuHomePhabricator

Intermittent mediawiki-extensions-hhvm failures due to Flow unit test fetching live SpamBlacklist configuration
Closed, ResolvedPublic

Description

Since Flow got added to the mediawiki-extensions job (T88206) in https://gerrit.wikimedia.org/r/#/c/187902/, the job intermittently fail:

https://integration.wikimedia.org/ci/job/mediawiki-extensions-hhvm/2653/consoleFull

There was 1 error:

1) Flow\Tests\Api\ApiFlowReplyTest::testTopLevelReply
Failed to open https://meta.wikimedia.org/w/index.php?title=Spam_blacklist&action=raw&sb_ver=1 (Operation timed out after 4159 milliseconds ..)

/src/extensions/SpamBlacklist/BaseBlacklist.php:273
/src/extensions/SpamBlacklist/BaseBlacklist.php:247
/src/extensions/SpamBlacklist/BaseBlacklist.php:196
/src/extensions/SpamBlacklist/SpamBlacklist_body.php:47
/src/extensions/Flow/includes/SpamFilter/SpamBlacklist.php:24
/src/extensions/Flow/includes/SpamFilter/Controller.php:45
/src/extensions/Flow/includes/Block/Block.php:301
/src/extensions/Flow/includes/Block/TopicList.php:92
/src/extensions/Flow/includes/Block/Block.php:194
/src/extensions/Flow/includes/SubmissionHandler.php:104
/src/extensions/Flow/includes/WorkflowLoader.php:63
/src/extensions/Flow/includes/api/ApiFlowBasePost.php:20
/src/extensions/Flow/includes/api/ApiFlow.php:81
/src/includes/api/ApiMain.php:1024
/src/includes/api/ApiMain.php:362
/src/tests/phpunit/includes/api/ApiTestCase.php:115
/src/extensions/Flow/tests/phpunit/api/ApiTestCase.php:53
/src/extensions/Flow/tests/phpunit/api/ApiTestCase.php:67
/src/extensions/Flow/tests/phpunit/api/ApiFlowReplyTest.php:18
/src/tests/phpunit/MediaWikiTestCase.php:141

Please address the unit tests and/or the underlying configuration to not fetch meta.wikimedia.org configuration (or any other url for that matter). Perhaps a mock object can do here. Or perhaps ensuring wg-variables are set correctly instead of relying on Jenkins or MediaWiki default settings (which is bad practice).

See MediaWiki core phpunit tests for examples of where relevant wg-variables are set to explicit values.

Event Timeline

Krinkle raised the priority of this task from to Unbreak Now!.
Krinkle updated the task description. (Show Details)
Krinkle added a subscriber: Krinkle.

@DannyH can someone please fix this ASAP? It's breaking other people's builds. Fix is great, otherwise we'll have to remove Flow from that group test job (before end of the day).

cc: @EBernhardson

gerritbot added a subscriber: gerritbot.

Change 189614 had a related patch set uploaded (by Mattflaschen):
Disable shared SpamBlacklist during tests.

https://gerrit.wikimedia.org/r/189614

Patch-For-Review

The underlying cause is that the SpamBlacklist extension defaults to using a page from Meta.

We already have some code that's supposed to suppress this.

But it's obviously not working (anymore?). Anyway, I'm working on it.

Ran flow's test suite through strace with the '-e trace=connect' parameters to record all outgoing connections, SpamBlacklist contacting meta is the only external network request made. Depending on configuration parsoid is contacted, but that is disabled when run inside jenkins.

Checked a few core tests with strace as well, EditPageTest has the same problem of triggering SpamBlacklist via the default configuration. There may be others I didn't run the full core test suite.

@hashar or @zeljkofilipin or @Cmcmahon: can you make a call on this during your morning so that work isn't delayed tomorrow?

Change 189614 merged by jenkins-bot:
Disable shared SpamBlacklist during tests.

https://gerrit.wikimedia.org/r/189614

This should be fixed. However, it doesn't look like this is actually running on Flow commits, so it's hard to be sure.

If we did fix it, the job could easily re-break if it's not changed to vote on all the projects it runs, including Flow.

@Mattflaschen: I think @hashar knows more about how we'll rollout the new mediawiki-extensions-* jobs (to replace mwext-{name}-testextension-*), however I see it has been enabled in Flow a few weeks ago in the experimental pipeline. Leave a comment saying "check experimental" on a Flow patch to trigger it.

zuul#mwext-Flow and before that via the zuul#extensions-template.

Leave a comment saying "check experimental" on a Flow patch to trigger it.

Thanks

Re-open if it happens again.

Thanks @Mattflaschen !

Last week I have noticed the debug.log have a lot of messages related to SpamBlacklist regex content and did notice it was fetching from meta. Should have filled a bug for it.

We will need a way to drop the default when running tests:

$wgBlacklistSettings = array(
    'spam' => array(
        'files' => array( "https://meta.wikimedia.org/w/index.php?title=Spam_blacklist&action=raw&sb_ver=1" )
    )
);

I have filled tasks: