Page MenuHomePhabricator
Paste P13662

Speedtest cloudceph2001-dev.codfw.wmnet write cache disable
ActivePublic

Authored by dcaro on Thu, Jan 7, 3:01 PM.
##### Linear read:
test: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
fio-3.12
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=27953: Thu Jan 7 14:50:59 2021
read: IOPS=123, BW=492MiB/s (516MB/s)(28.9GiB/60113msec)
slat (usec): min=23, max=21962, avg=8100.78, stdev=7983.79
clat (msec): min=106, max=333, avg=251.61, stdev=14.00
lat (msec): min=121, max=348, avg=259.71, stdev=13.86
clat percentiles (msec):
| 1.00th=[ 226], 5.00th=[ 228], 10.00th=[ 234], 20.00th=[ 241],
| 30.00th=[ 245], 40.00th=[ 247], 50.00th=[ 251], 60.00th=[ 255],
| 70.00th=[ 262], 80.00th=[ 266], 90.00th=[ 271], 95.00th=[ 271],
| 99.00th=[ 279], 99.50th=[ 279], 99.90th=[ 292], 99.95th=[ 321],
| 99.99th=[ 334]
bw ( KiB/s): min=327680, max=548864, per=99.75%, avg=503023.80, stdev=26692.51, samples=120
iops : min= 80, max= 134, avg=122.78, stdev= 6.52, samples=120
lat (msec) : 250=46.53%, 500=53.47%
cpu : usr=0.06%, sys=1.03%, ctx=3779, majf=0, minf=584
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=99.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=7401,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=492MiB/s (516MB/s), 492MiB/s-492MiB/s (516MB/s-516MB/s), io=28.9GiB (31.0GB), run=60113-60113msec
Disk stats (read/write):
sdc: ios=29949/0, merge=0/0, ticks=3619333/0, in_queue=1916624, util=99.90%
##### Linear write:
test: (g=0): rw=write, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=32
fio-3.12
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=28028: Thu Jan 7 14:51:59 2021
write: IOPS=127, BW=511MiB/s (536MB/s)(30.0GiB/60118msec); 0 zone resets
slat (usec): min=80, max=32732, avg=7805.60, stdev=7491.63
clat (msec): min=112, max=342, avg=242.49, stdev=10.61
lat (msec): min=127, max=358, avg=250.30, stdev= 9.38
clat percentiles (msec):
| 1.00th=[ 230], 5.00th=[ 232], 10.00th=[ 232], 20.00th=[ 232],
| 30.00th=[ 234], 40.00th=[ 247], 50.00th=[ 247], 60.00th=[ 247],
| 70.00th=[ 247], 80.00th=[ 249], 90.00th=[ 249], 95.00th=[ 253],
| 99.00th=[ 275], 99.50th=[ 284], 99.90th=[ 296], 99.95th=[ 326],
| 99.99th=[ 342]
bw ( KiB/s): min=393216, max=540672, per=99.77%, avg=522145.56, stdev=16772.36, samples=120
iops : min= 96, max= 132, avg=127.46, stdev= 4.10, samples=120
lat (msec) : 250=93.52%, 500=6.48%
cpu : usr=2.97%, sys=0.88%, ctx=3896, majf=0, minf=11
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=99.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=0,7681,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=511MiB/s (536MB/s), 511MiB/s-511MiB/s (536MB/s-536MB/s), io=30.0GiB (32.2GB), run=60118-60118msec
Disk stats (read/write):
sdc: ios=16/30960, merge=0/0, ticks=1/3622883, in_queue=1922928, util=99.85%
#### Peak parallel random read:
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.12
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=28123: Thu Jan 7 14:53:00 2021
read: IOPS=96.5k, BW=377MiB/s (395MB/s)(22.1GiB/60001msec)
slat (nsec): min=948, max=1910.6k, avg=9654.97, stdev=23371.44
clat (usec): min=613, max=4981, avg=1316.19, stdev=221.88
lat (usec): min=627, max=4982, avg=1325.92, stdev=223.17
clat percentiles (usec):
| 1.00th=[ 1020], 5.00th=[ 1045], 10.00th=[ 1057], 20.00th=[ 1139],
| 30.00th=[ 1172], 40.00th=[ 1237], 50.00th=[ 1287], 60.00th=[ 1352],
| 70.00th=[ 1385], 80.00th=[ 1450], 90.00th=[ 1549], 95.00th=[ 1614],
| 99.00th=[ 2180], 99.50th=[ 2442], 99.90th=[ 2933], 99.95th=[ 3097],
| 99.99th=[ 3458]
bw ( KiB/s): min=384608, max=387112, per=99.99%, avg=385987.22, stdev=507.54, samples=120
iops : min=96152, max=96778, avg=96496.79, stdev=126.88, samples=120
lat (usec) : 750=0.01%, 1000=0.65%
lat (msec) : 2=97.83%, 4=1.52%, 10=0.01%
cpu : usr=6.29%, sys=21.50%, ctx=626981, majf=0, minf=136
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued rwts: total=5790708,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=128
Run status group 0 (all jobs):
READ: bw=377MiB/s (395MB/s), 377MiB/s-377MiB/s (395MB/s-395MB/s), io=22.1GiB (23.7GB), run=60001-60001msec
Disk stats (read/write):
sdc: ios=5779926/0, merge=1/0, ticks=3599475/0, in_queue=1914072, util=99.88%
#### Single-threaded read latency:
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.12
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=28211: Thu Jan 7 14:54:00 2021
read: IOPS=20.6k, BW=80.4MiB/s (84.3MB/s)(4824MiB/60001msec)
slat (nsec): min=1665, max=40544, avg=4722.99, stdev=2451.93
clat (usec): min=12, max=1679, avg=43.03, stdev=28.34
lat (usec): min=25, max=1682, avg=47.92, stdev=28.65
clat percentiles (usec):
| 1.00th=[ 28], 5.00th=[ 29], 10.00th=[ 30], 20.00th=[ 30],
| 30.00th=[ 31], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 36],
| 70.00th=[ 37], 80.00th=[ 37], 90.00th=[ 96], 95.00th=[ 114],
| 99.00th=[ 145], 99.50th=[ 145], 99.90th=[ 147], 99.95th=[ 151],
| 99.99th=[ 157]
bw ( KiB/s): min=79352, max=91312, per=100.00%, avg=82339.18, stdev=2991.17, samples=119
iops : min=19838, max=22828, avg=20584.77, stdev=747.78, samples=119
lat (usec) : 20=0.04%, 50=87.50%, 100=4.21%, 250=8.25%, 500=0.01%
lat (usec) : 750=0.01%
lat (msec) : 2=0.01%
cpu : usr=4.34%, sys=15.26%, ctx=1234843, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1234834,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=80.4MiB/s (84.3MB/s), 80.4MiB/s-80.4MiB/s (84.3MB/s-84.3MB/s), io=4824MiB (5058MB), run=60001-60001msec
Disk stats (read/write):
sdc: ios=1232638/0, merge=0/0, ticks=53341/0, in_queue=51100, util=85.22%
#### Peak parallel random write:
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.12
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=28326: Thu Jan 7 14:55:01 2021
write: IOPS=84.2k, BW=329MiB/s (345MB/s)(19.3GiB/60001msec); 0 zone resets
slat (nsec): min=1013, max=5687.1k, avg=11119.97, stdev=28311.50
clat (usec): min=670, max=11032, avg=1509.34, stdev=269.48
lat (usec): min=764, max=11036, avg=1520.54, stdev=271.16
clat percentiles (usec):
| 1.00th=[ 1172], 5.00th=[ 1188], 10.00th=[ 1237], 20.00th=[ 1303],
| 30.00th=[ 1385], 40.00th=[ 1418], 50.00th=[ 1500], 60.00th=[ 1549],
| 70.00th=[ 1631], 80.00th=[ 1663], 90.00th=[ 1762], 95.00th=[ 1811],
| 99.00th=[ 2474], 99.50th=[ 2802], 99.90th=[ 3523], 99.95th=[ 4015],
| 99.99th=[ 7767]
bw ( KiB/s): min=317032, max=338600, per=100.00%, avg=336619.59, stdev=2754.24, samples=120
iops : min=79258, max=84650, avg=84154.92, stdev=688.47, samples=120
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=97.11%, 4=2.84%, 10=0.05%, 20=0.01%
cpu : usr=5.59%, sys=19.23%, ctx=610846, majf=0, minf=10
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued rwts: total=0,5049595,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=128
Run status group 0 (all jobs):
WRITE: bw=329MiB/s (345MB/s), 329MiB/s-329MiB/s (345MB/s-345MB/s), io=19.3GiB (20.7GB), run=60001-60001msec
Disk stats (read/write):
sdc: ios=43/5040033, merge=0/1, ticks=9/3609103, in_queue=1915432, util=99.89%
#### Journal write latency (with sync):
test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.12
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=28422: Thu Jan 7 14:56:01 2021
write: IOPS=37.8k, BW=148MiB/s (155MB/s)(8855MiB/60001msec); 0 zone resets
slat (nsec): min=1729, max=63823, avg=2533.58, stdev=674.96
clat (usec): min=9, max=434, avg=23.50, stdev= 2.50
lat (usec): min=22, max=441, avg=26.15, stdev= 2.76
clat percentiles (nsec):
| 1.00th=[21120], 5.00th=[21376], 10.00th=[21632], 20.00th=[22144],
| 30.00th=[22656], 40.00th=[22912], 50.00th=[22912], 60.00th=[23424],
| 70.00th=[23936], 80.00th=[24192], 90.00th=[25216], 95.00th=[27520],
| 99.00th=[30848], 99.50th=[34560], 99.90th=[39168], 99.95th=[40704],
| 99.99th=[47360]
bw ( KiB/s): min=144872, max=160776, per=99.94%, avg=151038.59, stdev=5603.94, samples=119
iops : min=36218, max=40194, avg=37759.63, stdev=1400.99, samples=119
lat (usec) : 10=0.01%, 20=0.03%, 50=99.97%, 100=0.01%, 250=0.01%
lat (usec) : 500=0.01%
cpu : usr=4.07%, sys=15.51%, ctx=2266932, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,2266910,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=148MiB/s (155MB/s), 148MiB/s-148MiB/s (155MB/s-155MB/s), io=8855MiB (9285MB), run=60001-60001msec
Disk stats (read/write):
sdc: ios=43/2262624, merge=0/0, ticks=9/53184, in_queue=43648, util=72.78%
#### Journal write latency (with fsync):
test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.12
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=28524: Thu Jan 7 14:57:02 2021
write: IOPS=36.0k, BW=141MiB/s (148MB/s)(8445MiB/60001msec); 0 zone resets
slat (nsec): min=1833, max=28038, avg=2601.19, stdev=632.90
clat (usec): min=2, max=1374, avg=24.03, stdev= 4.92
lat (usec): min=22, max=1378, avg=26.75, stdev= 5.04
clat percentiles (usec):
| 1.00th=[ 22], 5.00th=[ 23], 10.00th=[ 23], 20.00th=[ 23],
| 30.00th=[ 24], 40.00th=[ 24], 50.00th=[ 24], 60.00th=[ 24],
| 70.00th=[ 24], 80.00th=[ 25], 90.00th=[ 26], 95.00th=[ 28],
| 99.00th=[ 32], 99.50th=[ 36], 99.90th=[ 41], 99.95th=[ 42],
| 99.99th=[ 192]
bw ( KiB/s): min=130392, max=153560, per=99.94%, avg=144037.34, stdev=3783.04, samples=119
iops : min=32598, max=38390, avg=36009.31, stdev=945.75, samples=119
lat (usec) : 4=0.01%, 10=0.01%, 20=0.02%, 50=99.96%, 100=0.01%
lat (usec) : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%
fsync/fdatasync/sync_file_range:
sync (nsec): min=16, max=16312, avg=28.00, stdev=41.17
sync percentiles (nsec):
| 1.00th=[ 21], 5.00th=[ 23], 10.00th=[ 25], 20.00th=[ 26],
| 30.00th=[ 27], 40.00th=[ 27], 50.00th=[ 28], 60.00th=[ 28],
| 70.00th=[ 29], 80.00th=[ 29], 90.00th=[ 30], 95.00th=[ 32],
| 99.00th=[ 34], 99.50th=[ 38], 99.90th=[ 88], 99.95th=[ 91],
| 99.99th=[ 100]
cpu : usr=4.96%, sys=16.88%, ctx=2161813, majf=0, minf=14
IO depths : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,2161803,0,2161802 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=141MiB/s (148MB/s), 141MiB/s-141MiB/s (148MB/s-148MB/s), io=8445MiB (8855MB), run=60001-60001msec
Disk stats (read/write):
sdc: ios=43/2157667, merge=0/0, ticks=10/51899, in_queue=43456, util=72.47%
#### Single-threaded random write latency:
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.12
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=28635: Thu Jan 7 14:58:03 2021
write: IOPS=37.9k, BW=148MiB/s (155MB/s)(8882MiB/60001msec); 0 zone resets
slat (nsec): min=1656, max=43941, avg=2445.47, stdev=628.45
clat (nsec): min=391, max=1886.6k, avg=23405.39, stdev=6353.82
lat (usec): min=22, max=1911, avg=25.96, stdev= 6.51
clat percentiles (usec):
| 1.00th=[ 22], 5.00th=[ 22], 10.00th=[ 22], 20.00th=[ 22],
| 30.00th=[ 23], 40.00th=[ 23], 50.00th=[ 23], 60.00th=[ 24],
| 70.00th=[ 24], 80.00th=[ 25], 90.00th=[ 25], 95.00th=[ 28],
| 99.00th=[ 31], 99.50th=[ 34], 99.90th=[ 40], 99.95th=[ 41],
| 99.99th=[ 151]
bw ( KiB/s): min=142552, max=160240, per=100.00%, avg=151598.35, stdev=5952.37, samples=119
iops : min=35638, max=40060, avg=37899.52, stdev=1488.08, samples=119
lat (nsec) : 500=0.01%
lat (usec) : 10=0.01%, 20=0.02%, 50=99.97%, 100=0.01%, 250=0.01%
lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%
cpu : usr=4.52%, sys=14.90%, ctx=2273682, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,2273666,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=148MiB/s (155MB/s), 148MiB/s-148MiB/s (155MB/s-155MB/s), io=8882MiB (9313MB), run=60001-60001msec
Disk stats (read/write):
sdc: ios=43/2269595, merge=0/0, ticks=10/53114, in_queue=45684, util=76.18%

Event Timeline

dcaro created this paste.Thu, Jan 7, 3:01 PM
dcaro changed the title of this paste from Command-Line Input to Speedtest cloudceph2001-dev.codfw.wmnet write cache disable.Thu, Jan 7, 3:01 PM
dcaro updated the paste's language from autodetect to shell.