CommandTest failing from time to time
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Reedy
	Jan 4 2018, 11:27 AM

Description

I think this might've been caused by https://github.com/wikimedia/mediawiki/commit/0e211c4f29d2f52a0962db69c026e4e76482cf3f

Test failure on Travis PHP 5.5 (reproduceable on PHP 7) https://travis-ci.org/wikimedia/mediawiki/builds/324919993 https://travis-ci.org/wikimedia/mediawiki/jobs/324919994

1) CommandTest::testInput
Failed asserting that 81920 is identical to 1000000.
/home/travis/build/wikimedia/mediawiki/tests/phpunit/includes/shell/CommandTest.php:172
/home/travis/build/wikimedia/mediawiki/maintenance/doMaintenance.php:94
FAILURES!
Tests: 17117, Assertions: 97888, Failures: 1, Skipped: 23, Risky: 5.

Details

	Subject	Repo	Branch	Lines +/-
	Shell: Set pipes to non-blocking	mediawiki/core	master	+19 -7

Customize query in gerrit

Event Timeline

Reedy created this task.Jan 4 2018, 11:27 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 4 2018, 11:27 AM

Reedy added a project: MediaWiki-Core-Tests.Jan 4 2018, 11:28 AM

Reedy updated the task description. (Show Details)

That commit added the test so that's a pretty safe assumption :)
Command reads the input in blocks of 65535, so it seems like the input (or output?) is truncated after 1.25 blocks. No clue why that would be happening :(

This test also failed in the experimental PHP 7 coverage job (https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-php7/1/console) - but then it didn't fail in a later run (https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-php7/2/console)

FWIW I tested the patch in vagrant (PHP 5.6) and that worked. 3v4l does not allow shell execution so can't test there :(

This is easily reproduceable in PHP7 too if you put that test's code inside a loop. Observations:

It doesn't happen if the executable is not outputting anything at the same time.
When this happen, the script waits for a long time.

MaxSem renamed this task from CommandTest failing on Travis PHP 5.5 to CommandTest failing from time to time.Jan 19 2018, 5:33 AM

MaxSem updated the task description. (Show Details)

Can we disable the test in the meanwhile until we've figured this out? Or at least reduce the size to under 8000 for the sake of not having our builds broken or less-than-useful.

Nikerabbit subscribed.Jan 23 2018, 3:23 PM

Krinkle triaged this task as High priority.Jan 23 2018, 5:18 PM

Krinkle moved this task from Inbox to PHPUnit on the MediaWiki-Core-Tests board.

I ran into a similar bug today when running tests locally: this test was seeming to hang and I killed it. Probably it would eventually have timed out and given the failure here.

It turns out that the select system call only guarantees non-blocking for "sufficiently small writes", and when it was hanging it was blocked in a write system call (as was the cat process). Running all the pipes through stream_set_blocking() (and then fixing the read/write loop) seems to have fixed it.

Change 407483 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@master] Shell: Set pipes to non-blocking

https://gerrit.wikimedia.org/r/407483

gerritbot added a project: Patch-For-Review.Feb 1 2018, 7:22 PM

Also affecting our own CI jobs, seen at https://integration.wikimedia.org/ci/job/mediawiki-phpunit-php55-jessie/1856/console for https://gerrit.wikimedia.org/r/#/c/406788/

1) CommandTest::testInput
Failed asserting that 139264 is identical to 1000000.

I'm landing the proposed patch by @Anomie now. But if it shows up again after that, I assume it will be uncontroversial to disable CommandTest until resolved.

Change 407483 merged by jenkins-bot:
[mediawiki/core@master] Shell: Set pipes to non-blocking

https://gerrit.wikimedia.org/r/407483

ReleaseTaggerBot added a project: MW-1.31-release-notes (WMF-deploy-2018-02-06 (1.31.0-wmf.20)).Feb 2 2018, 5:00 AM

Krinkle closed this task as Resolved.Feb 2 2018, 10:45 PM

Krinkle claimed this task.

Aklapper removed a subscriber: Anomie.Oct 16 2020, 5:40 PM

Maintenance_bot removed a project: Patch-For-Review.Oct 16 2020, 6:36 PM

CommandTest failing from time to timeClosed, ResolvedPublicActions

Description

Details

Event Timeline

CommandTest failing from time to time
Closed, ResolvedPublic
Actions