CirrusSearch\SearcherTest::testSearchText PHPUnit tests take a while and runs for everyone
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	hashar
	Jun 6 2019, 8:41 AM

Description

CirrusSearch is an important extension and it is part of the set of repositories tested together which is nice. However, one of its PHPUnit test suite takes a while (1 min 30 - 2 mins) and since it is run for any repository that depends on CirrusSearch, that slightly slow down changes processing in CI.

From T221434 the identified PHPUnit suite is CirrusSearch\SearcherTest::testSearchText

It should show up on the test report https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-hhvm-docker/lastCompletedBuild/testReport/(root)/ and sorting by duration.

Details

	Subject	Repo	Branch	Lines +/-
	Only run certain big, isolated tests in this repo, not in gate	mediawiki/extensions/CirrusSearch	master	+3 -0
	Randomize searcherText cases and limit them to 200 by default	mediawiki/extensions/CirrusSearch	master	+82 -21

Customize query in gerrit

Related Objects

Mentioned In: T285950: Investigate moving some Wikibase tests to @group Standalone
T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min
T225248: Consider moving browser based tests (Selenium and QUnit) to a non-voting pipeline
T221434: Ensure we're testing appropriately and not over-testing across Wikimedia-deployed code
Mentioned Here: T225068: Add a PHPUnit group to skip test on gated CI runs
T221434: Ensure we're testing appropriately and not over-testing across Wikimedia-deployed code

Event Timeline

hashar created this task.Jun 6 2019, 8:41 AM

Restricted Application added a project: Discovery-Search. · View Herald TranscriptJun 6 2019, 8:41 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

If there are ways to easily speed it up, that would be very appreciated.

Eventually we might flag that specific test suite to only run it when a change is for CirrusSearch. We have a similar issue with a Scribunto test suite. The idea is being discussed on T225068

we blindly copied all browser tests fixtures and run them on 3 different setup, I'm sure we can prune down the list of tested cases.

Ladsgroup awarded a token.Jun 6 2019, 8:57 AM

Ladsgroup subscribed.

Change 514677 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Randomize searcherText cases and limit them to 200 by default

https://gerrit.wikimedia.org/r/514677

gerritbot added a project: Patch-For-Review.Jun 6 2019, 10:12 AM

I finally opted for a randomized approach where I limit the number of fixture at 200 by default per run. The fixtures to test are selected randomly. If merged the full suite of unit tests for Cirrus would now take 25sec instead of 1m30 on my laptop.

dcausse moved this task from Incoming to Needs review on the Discovery-Search (Current work) board.Jun 6 2019, 11:44 AM

Change 514677 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Randomize searcherText cases and limit them to 200 by default

https://gerrit.wikimedia.org/r/514677

Maintenance_bot removed a project: Patch-For-Review.Jun 7 2019, 10:10 AM

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.10; 2019-06-18).Jun 7 2019, 11:00 AM

Ladsgroup mentioned this in T225248: Consider moving browser based tests (Selenium and QUnit) to a non-voting pipeline.Jun 8 2019, 4:32 PM

EBernhardson moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Jun 10 2019, 11:32 PM

Krinkle mentioned this in T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min.Jun 13 2019, 3:28 PM

Thank you @dcausse :] Discovery covered its part. The second part is to only run this test when CI is triggered by CirrusSearch, the latest idea is to flag it with @group standalone and we instrument that in CI to skip the test ( T225068#5247536 ).

debt closed this task as Resolved.Jun 21 2019, 2:19 PM

Change 587883 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/CirrusSearch@master] [DNM] Only run the SearcherTests for this repo, not the gate

https://gerrit.wikimedia.org/r/587883

gerritbot added a project: Patch-For-Review.Apr 9 2020, 10:54 PM

Change 587883 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Only run certain big, isolated tests in this repo, not in gate

https://gerrit.wikimedia.org/r/587883

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.28; 2020-04-14).Apr 13 2020, 11:00 PM

Maintenance_bot removed a project: Patch-For-Review.Apr 13 2020, 11:10 PM

Lucas_Werkmeister_WMDE mentioned this in T285950: Investigate moving some Wikibase tests to @group Standalone.Jul 1 2021, 1:48 PM