load test #2
ActivePublic
Actions

Authored by achou on Mar 28 2024, 4:07 PM.

Tags

None

Referenced Files

	F43676638: load test #2
	Mar 28 2024, 4:07 PM

Subscribers

None

	[locust]
	headless = true
	users = 50
	spawn-rate = 10
	run-time = 60s
	host = "https://inference-staging.svc.codfw.wmnet:30443"
	# host = "https://api.wikimedia.org/service/lw/inference"
	only-summary = true

	(liftwing-test) aikochou@stat1005:~/inference-services/test/locust$ MODEL=revertrisk locust --csv results/revertrisk
	[2024-03-28 15:36:48,089] stat1005/INFO/locust.main: Run time limit set to 60 seconds
	[2024-03-28 15:36:48,089] stat1005/INFO/locust.main: Starting Locust 2.24.1
	[2024-03-28 15:36:48,090] stat1005/INFO/locust.runners: Ramping to 50 users at a rate of 10.00 per second
	[2024-03-28 15:36:52,097] stat1005/INFO/locust.runners: All users spawned: {"RevertriskLanguageAgnostic": 25, "RevertriskMultilingual": 25} (50 total users)
	[2024-03-28 15:37:47,513] stat1005/INFO/locust.main: --run-time limit reached, shutting down
	Load test results are within the threshold
	[2024-03-28 15:37:47,678] stat1005/INFO/locust.main: Shutting down (exit code 1)
	Type Name # reqs # fails \| Avg Min Max Med \| req/s failures/s
	--------\|---------------------------------------------------------------------\|-------\|-------------\|-------\|-------\|-------\|-------\|--------\|-----------
	POST /v1/models/revertrisk-language-agnostic:predict 422 16(3.79%) \| 467 32 7713 130 \| 7.11 0.27
	POST /v1/models/revertrisk-multilingual:predict 88 3(3.41%) \| 11349 42 24806 10000 \| 1.48 0.05
	--------\|---------------------------------------------------------------------\|-------\|-------------\|-------\|-------\|-------\|-------\|--------\|-----------
	Aggregated 510 19(3.73%) \| 2345 32 24806 140 \| 8.59 0.32

	Response time percentiles (approximated)
	Type Name 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100% # reqs
	--------\|-------------------------------------------------------------------------\|--------\|------\|------\|------\|------\|------\|------\|------\|------\|------\|------\|------
	POST /v1/models/revertrisk-language-agnostic:predict 130 140 160 180 310 3200 7600 7700 7700 7700 7700 422
	POST /v1/models/revertrisk-multilingual:predict 11000 12000 14000 17000 21000 22000 23000 25000 25000 25000 25000 88
	--------\|-------------------------------------------------------------------------\|--------\|------\|------\|------\|------\|------\|------\|------\|------\|------\|------\|------
	Aggregated 140 180 310 3700 9600 14000 19000 22000 25000 25000 25000 510

	Error report
	# occurrences Error
	------------------\|--------------------------------------------------------------------------------------------------------------------------------------
	10 POST /v1/models/revertrisk-language-agnostic:predict: ConnectionRefusedError(111, 'Connection refused')
	2 POST /v1/models/revertrisk-multilingual:predict: ConnectionRefusedError(111, 'Connection refused')
	6 POST /v1/models/revertrisk-language-agnostic:predict: BadStatusCode('https://inference-staging.svc.codfw.wmnet:30443/v1/models/revertrisk-language-agnostic:predict', code=503)
	1 POST /v1/models/revertrisk-multilingual:predict: BadStatusCode('https://inference-staging.svc.codfw.wmnet:30443/v1/models/revertrisk-multilingual:predict', code=503)
	------------------\|--------------------------------------------------------------------------------------------------------------------------------------

Event Timeline

achou created this paste.Mar 28 2024, 4:07 PM

achou mentioned this in T361234: Fix locust load testing for Revert Risk models.Mar 28 2024, 4:57 PM

achou mentioned this in T361881: Investigate the inconsistent load test results (locust) for revertrisk.Apr 4 2024, 8:08 PM

load test #2ActivePublicActions

Event Timeline

load test #2
ActivePublic
Actions