HashRingTest::testHashRingKetamaMode takes 3+ seconds
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	hashar
	Jun 13 2019, 12:08 PM

Description

I eventually found a unit test which takes more than 3 seconds on my machine: HashRingTest::testHashRingKetamaMode. It has been introduced in f2e6543c46fff5ce43839982ecd56ebcc10f4d62 by @aaron

It does compute a hashring for 100 nodes as far as I can understand. Would be nice to make it faster, we probably do not need to compute the ring for 100 nodes?

Details

Project	Branch	Lines +/-	Subject
mediawiki/core	REL1_33	+2 -2	Reduce HashRing test load to avoid several seconds of CPU
mediawiki/core	REL1_32	+2 -2	Reduce HashRing test load to avoid several seconds of CPU
mediawiki/core	master	+2 -2	Reduce HashRing test load to avoid several seconds of CPU

Customize query in gerrit

Related Objects

Mentioned In: T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min
Mentioned Here: rMWf2e6543c46ff: Rewrite HashRing to use consistent hashing

Event Timeline

hashar created this task.Jun 13 2019, 12:08 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 13 2019, 12:08 PM

I assume the slow part would be $ketama_test( 1e5 ). It could probably use 1e4 if there is a speed problem.

Mainframe98 mentioned this in T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min.Jun 13 2019, 5:49 PM

Krinkle moved this task from Inbox to PHPUnit on the MediaWiki-Core-Tests board.Jun 13 2019, 5:58 PM

@aaron indeed the $ketama_test( 1e5 ) is the culprit and lowering it down to 10000 get it down to half a second. Is there any need to verify it against a larger number? Given the test already ran it for 100 entry, I would guess it could result that output, do the md5sum and compare against the expected md5sum from .c code?

Change 517058 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] Reduce HashRing test load to avoid several seconds of CPU

https://gerrit.wikimedia.org/r/517058

gerritbot added a project: Patch-For-Review.Jun 14 2019, 11:57 AM

Change 517065 had a related patch set uploaded (by Hashar; owner: Aaron Schulz):
[mediawiki/core@REL1_33] Reduce HashRing test load to avoid several seconds of CPU

https://gerrit.wikimedia.org/r/517065

Change 517066 had a related patch set uploaded (by Hashar; owner: Aaron Schulz):
[mediawiki/core@REL1_32] Reduce HashRing test load to avoid several seconds of CPU

https://gerrit.wikimedia.org/r/517066

Change 517058 merged by jenkins-bot:
[mediawiki/core@master] Reduce HashRing test load to avoid several seconds of CPU

https://gerrit.wikimedia.org/r/517058

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.10; 2019-06-18).Jun 16 2019, 8:00 AM

Before:

Test Result : HashRingTest

Took 0.93 sec.


* testHashRingKetamaMode		0.88 s	Passed	 <=
* testHashRingRatios with data set #0	28 ms	Passed
* testHashRingRatios2 with data set #0	5 ms	Passed

After:

Test Result : HashRingTest

Took 61 ms.

* testHashRingRatios with data set #0	27 ms	Passed
* testHashRingKetamaMode		11 ms	Passed	<=
* testHashRingRatios2 with data set #0	5 ms	Passed

Krinkle closed this task as Resolved.Jun 17 2019, 12:19 AM

Krinkle assigned this task to aaron.

Krinkle edited projects, added Performance-Team; removed MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Patch-For-Review.

Change 517066 merged by jenkins-bot:
[mediawiki/core@REL1_32] Reduce HashRing test load to avoid several seconds of CPU