Paste P1966

initial completion suggester bench
ActivePublic
Actions

Authored by dcausse on Sep 2 2015, 1:23 PM.

Tags

Referenced Files

	F2539890: initial completion suggester bench
	Sep 4 2015, 12:50 PM

	F2535459: initial completion suggester bench
	Sep 3 2015, 6:56 AM

	F2532429: initial completion suggester bench
	Sep 2 2015, 1:24 PM

	F2532427: initial completion suggester bench
	Sep 2 2015, 1:23 PM

Subscribers

None

	======================================================
	normal case (2 exacts + 2 fuzzy, pref len=0, internal size 20, no second pass query):
	======================================================
	ab -k -n 100000 -c 520 'https://en.wikipedia.org/w/api.php?action=cirrus-suggest&text=maxweber&limit=10&format=json'
	Document Path: /w/api.php?action=cirrus-suggest&text=maxweber&limit=10&format=json
	Document Length: 729 bytes

	Concurrency Level: 520
	Time taken for tests: 33.403 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 99408
	Total transferred: 145696997 bytes
	HTML transferred: 72900000 bytes
	Requests per second: 2993.73 [#/sec] (mean)
	Time per request: 173.697 [ms] (mean)
	Time per request: 0.334 [ms] (mean, across all concurrent requests)
	Transfer rate: 4259.54 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 3 47.6 0 1000
	Processing: 31 166 83.7 152 1467
	Waiting: 31 166 83.7 152 1467
	Total: 31 170 95.2 152 1467

	Percentage of the requests served within a certain time (ms)
	50% 152
	66% 188
	75% 212
	80% 228
	90% 271
	95% 316
	98% 403
	99% 463
	100% 1467 (longest request)

	======================================================
	best case (2 exacts, internal size 20, no second pass)
	======================================================
	ab -k -n 100000 -c 520 'https://en.wikipedia.org/w/api.php?action=cirrus-suggest&text=m&limit=10&format=json'
	Document Path: /w/api.php?action=cirrus-suggest&text=m&limit=10&format=json
	Document Length: 597 bytes

	Concurrency Level: 520
	Time taken for tests: 22.073 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 99402
	Total transferred: 132497010 bytes
	HTML transferred: 59700000 bytes
	Requests per second: 4530.47 [#/sec] (mean)
	Time per request: 114.778 [ms] (mean)
	Time per request: 0.221 [ms] (mean, across all concurrent requests)
	Transfer rate: 5862.05 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 3 43.0 0 616
	Processing: 26 108 59.3 98 1247
	Waiting: 25 108 59.3 98 1247
	Total: 26 111 72.6 99 1247

	Percentage of the requests served within a certain time (ms)
	50% 99
	66% 122
	75% 138
	80% 149
	90% 180
	95% 210
	98% 251
	99% 287
	100% 1247 (longest request)

	======================================================
	best case 2 (2 exacts, internal size 10, no second pass)
	======================================================
	ab -k -n 100000 -c 520 'https://en.wikipedia.org/w/api.php?action=cirrus-suggest&text=m&limit=5&format=json'
	Document Path: /w/api.php?action=cirrus-suggest&text=m&limit=5&format=json
	Document Length: 321 bytes

	Concurrency Level: 520
	Time taken for tests: 15.818 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 99391
	Total transferred: 104796912 bytes
	HTML transferred: 32100000 bytes
	Requests per second: 6322.10 [#/sec] (mean)
	Time per request: 82.251 [ms] (mean)
	Time per request: 0.158 [ms] (mean, across all concurrent requests)
	Transfer rate: 6470.08 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 3 46.8 0 1005
	Processing: 24 75 39.6 68 1160
	Waiting: 24 75 39.6 68 1160
	Total: 24 78 60.9 68 1160

	Percentage of the requests served within a certain time (ms)
	50% 68
	66% 84
	75% 95
	80% 102
	90% 121
	95% 140
	98% 165
	99% 192
	100% 1160 (longest request)


	======================================================
	worst case (2 exacts + 2 fuzzy, pref len=0, internal size 20, with second pass query):
	======================================================
	dcausse@terbium:~$ ab -k -n 100000 -c 100 'https://en.wikipedia.org/w/api.php?action=cirrus-suggest&text=einsteinal&limit=10&format=json'
	Document Path: /w/api.php?action=cirrus-suggest&text=einsteinal&limit=10&format=json
	Document Length: 658 bytes

	Concurrency Level: 100
	Time taken for tests: 130.881 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 99042
	Total transferred: 138595123 bytes
	HTML transferred: 65800000 bytes
	Requests per second: 764.05 [#/sec] (mean)
	Time per request: 130.881 [ms] (mean)
	Time per request: 1.309 [ms] (mean, across all concurrent requests)
	Transfer rate: 1034.12 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 0 4.9 0 1003
	Processing: 34 130 94.5 91 1416
	Waiting: 34 130 94.5 91 1416
	Total: 34 130 94.7 91 1416

	Percentage of the requests served within a certain time (ms)
	50% 91
	66% 126
	75% 189
	80% 218
	90% 274
	95% 312
	98% 348
	99% 370
	100% 1416 (longest request)


	======================================================
	worst case #second attempt-unfinished (2 exacts + 2 fuzzy, pref len=0, internal size 20, with second pass query):
	======================================================
	dcausse@terbium:~$ ab -k -n 100000 -c 520 'https://en.wikipedia.org/w/api.php?action=cirrus-suggest&text=einsteinal&limit=10&format=json'
	Document Path: /w/api.php?action=cirrus-suggest&text=einsteinal&limit=10&format=json
	Document Length: 658 bytes

	Concurrency Level: 520
	Time taken for tests: 24.392 seconds
	Complete requests: 58495
	Failed requests: 12
	(Connect: 0, Receive: 0, Length: 12, Exceptions: 0)
	Write errors: 0
	Keep-Alive requests: 57979
	Total transferred: 81064042 bytes
	HTML transferred: 38482318 bytes
	Requests per second: 2398.14 [#/sec] (mean)
	Time per request: 216.835 [ms] (mean)
	Time per request: 0.417 [ms] (mean, across all concurrent requests)
	Transfer rate: 3245.52 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 6 62.3 0 692
	Processing: 21 209 117.5 170 1436
	Waiting: 21 209 117.5 170 1436
	Total: 21 215 136.1 171 2097

	Percentage of the requests served within a certain time (ms)
	50% 171
	66% 235
	75% 286
	80% 316
	90% 385
	95% 442
	98% 532
	99% 719
	100% 2097 (longest request)



	==========================================
	pref worstcase
	==========================================

	dcausse@terbium:~$ ab -H "cookie:XXX" -k -n 100000 -c 50 'https://en.wikipedia.org/w/api.php?action=opensearch&format=json&search=m&namespace=0&limit=10&suggest=&format=json'
	Document Path: /w/api.php?action=opensearch&format=json&search=m&namespace=0&limit=10&suggest=&format=json
	Document Length: 2197 bytes

	Concurrency Level: 50
	Time taken for tests: 496.618 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 98947
	Total transferred: 282885099 bytes
	HTML transferred: 219700000 bytes
	Requests per second: 201.36 [#/sec] (mean)
	Time per request: 248.309 [ms] (mean)
	Time per request: 4.966 [ms] (mean, across all concurrent requests)
	Transfer rate: 556.27 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 0 1.5 0 70
	Processing: 120 248 52.1 243 1430
	Waiting: 120 248 52.1 243 1430
	Total: 120 248 52.1 243 1430

	Percentage of the requests served within a certain time (ms)
	50% 243
	66% 260
	75% 272
	80% 279
	90% 300
	95% 320
	98% 353
	99% 397
	100% 1430 (longest request)


	==============================
	comp worstcase
	==============================
	dcausse@terbium:~$ ab -k -n 100000 -c 50 'https://en.wikipedia.org/w/api.php?action=cirrus-suggest&text=einsteinal&limit=10&format=json'
	Server Software: nginx/1.9.3
	Server Hostname: en.wikipedia.org
	Server Port: 443
	SSL/TLS Protocol: TLSv1/SSLv3,ECDHE-ECDSA-AES128-GCM-SHA256,256,128

	Document Path: /w/api.php?action=cirrus-suggest&text=einsteinal&limit=10&format=json
	Document Length: 659 bytes

	Concurrency Level: 50
	Time taken for tests: 139.667 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 99024
	Total transferred: 138695033 bytes
	HTML transferred: 65900000 bytes
	Requests per second: 715.99 [#/sec] (mean)
	Time per request: 69.834 [ms] (mean)
	Time per request: 1.397 [ms] (mean, across all concurrent requests)
	Transfer rate: 969.76 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 0 1.6 0 74
	Processing: 33 70 29.4 63 1112
	Waiting: 33 70 29.4 63 1112
	Total: 33 70 29.6 63 1112

	Percentage of the requests served within a certain time (ms)
	50% 63
	66% 75
	75% 84
	80% 89
	90% 103
	95% 116
	98% 131
	99% 142
	100% 1112 (longest request)

	====================================
	pref normal case
	====================================
	dcausse@terbium:~$ ab -H "XXX" -k -n 100000 -c 50 'https://en.wikipedia.org/w/api.php?action=opensearch&format=json&search=max+weber&namespace=0&limit=10&suggest=&format=json'
	Server Software: nginx/1.9.3
	Server Hostname: en.wikipedia.org
	Server Port: 443
	SSL/TLS Protocol: TLSv1/SSLv3,ECDHE-ECDSA-AES128-GCM-SHA256,256,128

	Document Path: /w/api.php?action=opensearch&format=json&search=max+weber&namespace=0&limit=10&suggest=&format=json
	Document Length: 2198 bytes

	Concurrency Level: 50
	Time taken for tests: 215.268 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 98958
	Total transferred: 283090962 bytes
	HTML transferred: 219800000 bytes
	Requests per second: 464.54 [#/sec] (mean)
	Time per request: 107.634 [ms] (mean)
	Time per request: 2.153 [ms] (mean, across all concurrent requests)
	Transfer rate: 1284.24 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 0 1.3 0 66
	Processing: 62 108 46.7 101 1478
	Waiting: 62 107 46.7 101 1478
	Total: 62 108 46.7 101 1478

	Percentage of the requests served within a certain time (ms)
	50% 101
	66% 111
	75% 119
	80% 125
	90% 143
	95% 157
	98% 177
	99% 197
	100% 1478 (longest request)

	===================================
	sugg normal case
	===================================
	Server Software: nginx/1.9.3
	Server Hostname: en.wikipedia.org
	Server Port: 443
	SSL/TLS Protocol: TLSv1/SSLv3,ECDHE-ECDSA-AES128-GCM-SHA256,256,128

	Document Path: /w/api.php?action=cirrus-suggest&text=maxweber&limit=10&format=json
	Document Length: 718 bytes

	Concurrency Level: 50
	Time taken for tests: 107.006 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 99016
	Total transferred: 144595037 bytes
	HTML transferred: 71800000 bytes
	Requests per second: 934.53 [#/sec] (mean)
	Time per request: 53.503 [ms] (mean)
	Time per request: 1.070 [ms] (mean, across all concurrent requests)
	Transfer rate: 1319.61 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 0 1.4 0 70
	Processing: 27 53 27.6 47 1083
	Waiting: 27 53 27.6 47 1083
	Total: 27 53 27.6 47 1083

	Percentage of the requests served within a certain time (ms)
	50% 47
	66% 54
	75% 61
	80% 67
	90% 81
	95% 92
	98% 103
	99% 113
	100% 1083 (longest request)


	================================
	pref best case
	================================
	dcausse@terbium:~$ ab -H "XXX" -k -n 100000 -c 50 'https://en.wikipedia.org/w/api.php?action=opensearch&format=json&search=doesnotexists&namespace=0&limit=10&suggest=&format=json'
	Document Path: /w/api.php?action=opensearch&format=json&search=doesnotexists&namespace=0&limit=10&suggest=&format=json
	Document Length: 26 bytes

	Concurrency Level: 50
	Time taken for tests: 111.747 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 98973
	Total transferred: 65592765 bytes
	HTML transferred: 2600000 bytes
	Requests per second: 894.88 [#/sec] (mean)
	Time per request: 55.874 [ms] (mean)
	Time per request: 1.117 [ms] (mean, across all concurrent requests)
	Transfer rate: 573.22 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 0 1.4 0 68
	Processing: 29 56 27.0 50 1480
	Waiting: 29 56 27.0 50 1480
	Total: 29 56 27.0 50 1480

	Percentage of the requests served within a certain time (ms)
	50% 50
	66% 59
	75% 66
	80% 71
	90% 84
	95% 94
	98% 105
	99% 115
	100% 1480 (longest request)

	======================================
	sugg best case
	======================================

	dcausse@terbium:~$ ab -k -n 100000 -c 50 'https://en.wikipedia.org/w/api.php?action=cirrus-suggest&text=m&limit=10&format=json'
	Document Path: /w/api.php?action=cirrus-suggest&text=m&limit=10&format=json
	Document Length: 597 bytes

	Concurrency Level: 50
	Time taken for tests: 98.714 seconds
	Complete requests: 100000
	Failed requests: 0
	Write errors: 0
	Keep-Alive requests: 99022
	Total transferred: 132495066 bytes
	HTML transferred: 59700000 bytes
	Requests per second: 1013.03 [#/sec] (mean)
	Time per request: 49.357 [ms] (mean)
	Time per request: 0.987 [ms] (mean, across all concurrent requests)
	Transfer rate: 1310.75 [Kbytes/sec] received

	Connection Times (ms)
	min mean[+/-sd] median max
	Connect: 0 0 1.5 0 75
	Processing: 26 49 24.3 45 1078
	Waiting: 26 49 24.3 44 1078
	Total: 26 49 24.4 45 1078

	Percentage of the requests served within a certain time (ms)
	50% 45
	66% 50
	75% 56
	80% 60
	90% 75
	95% 86
	98% 97
	99% 106
	100% 1078 (longest request)

Event Timeline

dcausse edited the content of this paste. (Show Details)Sep 2 2015, 1:23 PM

dcausse changed the title of this paste from untitled to initial completion suggester bench.

dcausse updated the paste's language from autodetect to autodetect.

dcausse added a project: CirrusSearch.

dcausse edited the content of this paste. (Show Details)

dcausse mentioned this in T111120: Run load tests with the completion suggester in prod.Sep 2 2015, 1:32 PM

dcausse edited the content of this paste. (Show Details)Sep 3 2015, 6:56 AM

dcausse edited the content of this paste. (Show Details)Sep 4 2015, 12:50 PM

initial completion suggester benchActivePublicActions

Event Timeline

initial completion suggester bench
ActivePublic
Actions