Тесты производительности EBS

от автора

Привет! image

Очень много в интернете позитива/негатива об EBS. Так же не мало тестов на их производительность. Я решил сам провести несколько тестов и сделать небольшое исследование по этому поводу.

Итак, в тесте участвовала машинка m1.large к которой были примаунчены диски:

  • EBS Standard, 100 GB
  • EBS IO-1 500 IOPS, 100 GB
  • EBS IO-1 1000 IOPS, 100 GB
  • EBS IO-1 2000 IOPS, 200 GB
  • 8xEBS Standard, 30 GB, RAID 10
  • Ephemeral, 450 GB

Были проведены несколько тестов:

# hdparm -tT /dev/xcdX 

# dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc # echo 3 > /proc/sys/vm/drop_caches # dd if=tempfile of=/dev/null bs=5M count=1024 

# sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw prepare # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run 


Результаты вывода в консоль:

EBS Standard, 100 GB

# hdparm -tT /dev/xvdj  /dev/xvdj:  Timing cached reads:   4866 MB in  2.00 seconds = 2428.53 MB/sec  Timing buffered disk reads:  242 MB in  3.00 seconds =  80.54 MB/sec   Timing cached reads:   5146 MB in  2.00 seconds = 2579.25 MB/sec  Timing buffered disk reads:  294 MB in  3.01 seconds =  97.59 MB/sec   Timing cached reads:   4870 MB in  2.00 seconds = 2440.55 MB/sec  Timing buffered disk reads:  306 MB in  3.00 seconds = 101.89 MB/sec  # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 161.222 s, 33.3 MB/s  # echo 3 > /proc/sys/vm/drop_caches  # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 86.4683 s, 62.1 MB/s   #  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12:  multi-threaded system evaluation benchmark  Running the test with following options: Number of threads: 16  Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done.  Operations performed:  6000 Read, 4000 Write, 12800 Other = 22800 Total Read 93.75Mb  Written 62.5Mb  Total transferred 156.25Mb  (69.816Mb/sec)  4468.25 Requests/sec executed  Test execution summary:     total time:                          2.2380s     total number of events:              10000     total time taken by event execution: 1.5942     per-request statistics:          min:                                  0.01ms          avg:                                  0.16ms          max:                                 65.00ms          approx.  95 percentile:               0.02ms  Threads fairness:     events (avg/stddev):           625.0000/249.48     execution time (avg/stddev):   0.0996/0.03 

EBS IO-1 500 IOPS, 100 GB

# hdparm -tT /dev/xvdh  /dev/xvdh:  Timing cached reads:   4314 MB in  2.00 seconds = 2161.08 MB/sec  Timing buffered disk reads:   72 MB in  3.05 seconds =  23.57 MB/sec   Timing cached reads:   3646 MB in  2.00 seconds = 1826.09 MB/sec  Timing buffered disk reads:   76 MB in  3.04 seconds =  25.01 MB/sec   Timing cached reads:   4346 MB in  2.00 seconds = 2175.61 MB/sec  Timing buffered disk reads:   76 MB in  3.03 seconds =  25.12 MB/sec  # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 271.993 s, 19.7 MB/s  # echo 3 > /proc/sys/vm/drop_caches  # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 182.106 s, 29.5 MB/s   #  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12:  multi-threaded system evaluation benchmark  Running the test with following options: Number of threads: 16  Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done.  Operations performed:  6000 Read, 4000 Write, 12800 Other = 22800 Total Read 93.75Mb  Written 62.5Mb  Total transferred 156.25Mb  (16.794Mb/sec)  1074.78 Requests/sec executed  Test execution summary:     total time:                          9.3042s     total number of events:              10000     total time taken by event execution: 0.2975     per-request statistics:          min:                                  0.01ms          avg:                                  0.03ms          max:                                 30.70ms          approx.  95 percentile:               0.02ms  Threads fairness:     events (avg/stddev):           625.0000/553.34     execution time (avg/stddev):   0.0186/0.02 

EBS IO-1 1000 IOPS, 100 GB

# hdparm -tT /dev/xvdf  /dev/xvdf:  Timing cached reads:   5090 MB in  2.00 seconds = 2550.81 MB/sec  Timing buffered disk reads:  104 MB in  3.03 seconds =  34.30 MB/sec   Timing cached reads:   5000 MB in  2.00 seconds = 2505.62 MB/sec  Timing buffered disk reads:   98 MB in  3.10 seconds =  31.64 MB/sec   Timing cached reads:   5046 MB in  2.01 seconds = 2507.34 MB/sec  Timing buffered disk reads:   98 MB in  3.04 seconds =  32.19 MB/sec   # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 167.252 s, 32.1 MB/s  # echo 3 > /proc/sys/vm/drop_caches  # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 126.793 s, 42.3 MB/s  #  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12:  multi-threaded system evaluation benchmark  Running the test with following options: Number of threads: 16  Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done.  Operations performed:  6001 Read, 4000 Write, 12800 Other = 22801 Total Read 93.766Mb  Written 62.5Mb  Total transferred 156.27Mb  (37.871Mb/sec)  2423.73 Requests/sec executed  Test execution summary:     total time:                          4.1263s     total number of events:              10001     total time taken by event execution: 0.8466     per-request statistics:          min:                                  0.01ms          avg:                                  0.08ms          max:                                 22.31ms          approx.  95 percentile:               0.02ms  Threads fairness:     events (avg/stddev):           625.0625/318.25     execution time (avg/stddev):   0.0529/0.02 

EBS IO-1 2000 IOPS, 200 GB

# hdparm -tT /dev/xvdi  /dev/xvdi:  Timing cached reads:   4846 MB in  2.00 seconds = 2428.51 MB/sec  Timing buffered disk reads:   90 MB in  3.02 seconds =  29.80 MB/sec   Timing cached reads:   5122 MB in  2.05 seconds = 2503.64 MB/sec  Timing buffered disk reads:  100 MB in  3.07 seconds =  32.56 MB/sec   Timing cached reads:   4330 MB in  2.04 seconds = 2123.52 MB/sec  Timing buffered disk reads:  102 MB in  3.05 seconds =  33.41 MB/sec    # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 161.549 s, 33.2 MB/s  # echo 3 > /proc/sys/vm/drop_caches  # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 155.51 s, 34.5 MB/s   #  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw prepare sysbench 0.4.12:  multi-threaded system evaluation benchmark  128 files, 40960Kb each, 5120Mb total Creating files for the test... [root@ip-10-98-91-92 4]#  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12:  multi-threaded system evaluation benchmark  Running the test with following options: Number of threads: 16  Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done.  Operations performed:  6000 Read, 4000 Write, 12801 Other = 22801 Total Read 93.75Mb  Written 62.5Mb  Total transferred 156.25Mb  (74.645Mb/sec)  4777.28 Requests/sec executed  Test execution summary:     total time:                          2.0932s     total number of events:              10000     total time taken by event execution: 1.0015     per-request statistics:          min:                                  0.01ms          avg:                                  0.10ms          max:                                 10.10ms          approx.  95 percentile:               0.02ms  Threads fairness:     events (avg/stddev):           625.0000/177.29     execution time (avg/stddev):   0.0626/0.02 

8xEBS Standard, 30 GB, RAID 10

# hdparm -tT /dev/md0  /dev/md0:  Timing cached reads:   3964 MB in  1.93 seconds = 2048.84 MB/sec  Timing buffered disk reads:  230 MB in  3.53 seconds =  65.13 MB/sec   Timing cached reads:   3994 MB in  1.99 seconds = 2002.16 MB/sec  Timing buffered disk reads:  398 MB in  3.00 seconds = 132.64 MB/sec   Timing cached reads:   4334 MB in  2.03 seconds = 2138.00 MB/sec  Timing buffered disk reads:  302 MB in  3.02 seconds =  99.84 MB/sec   # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 113.234 s, 47.4 MB/s  # echo 3 > /proc/sys/vm/drop_caches  # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 97.9346 s, 54.8 MB/s  # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.5:  multi-threaded system evaluation benchmark  Running the test with following options: Number of threads: 16 Random number generator seed is 0 and will be ignored   Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of IO requests: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started!  Operations performed:  5998 reads, 4002 writes, 12800 Other = 22800 Total Read 93.719Mb  Written 62.531Mb  Total transferred 156.25Mb  (87.287Mb/sec)  5586.40 Requests/sec executed  General statistics:     total time:                          1.7901s     total number of events:              10000     total time taken by event execution: 1.1625s     response time:          min:                                  0.01ms          avg:                                  0.12ms          max:                                  8.99ms          approx.  95 percentile:               0.03ms  Threads fairness:     events (avg/stddev):           625.0000/171.83     execution time (avg/stddev):   0.0727/0.01 

Ephemeral, 450 GB

# hdparm -tT /dev/xvdb   /dev/xvdb:  Timing cached reads:   4048 MB in  2.00 seconds = 2027.97 MB/sec  Timing buffered disk reads:  1794 MB in  3.12 seconds = 575.84 MB/sec   Timing cached reads:   4854 MB in  2.00 seconds = 2432.18 MB/sec  Timing buffered disk reads:  1830 MB in  3.00 seconds = 609.94 MB/sec   Timing cached reads:   3434 MB in  2.00 seconds = 1719.73 MB/sec  Timing buffered disk reads:  770 MB in  3.13 seconds = 245.97 MB/sec  # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 95.9093 s, 56.0 MB/s  # echo 3 > /proc/sys/vm/drop_caches # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 55.5027 s, 96.7 MB/s   # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12:  multi-threaded system evaluation benchmark  Running the test with following options: Number of threads: 16  Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done.  Operations performed:  6000 Read, 4000 Write, 12800 Other = 22800 Total Read 93.75Mb  Written 62.5Mb  Total transferred 156.25Mb  (11.263Mb/sec)   720.82 Requests/sec executed  Test execution summary:     total time:                          13.8731s     total number of events:              10000     total time taken by event execution: 0.1603     per-request statistics:          min:                                  0.01ms          avg:                                  0.02ms          max:                                  2.34ms          approx.  95 percentile:               0.02ms  Threads fairness:     events (avg/stddev):           625.0000/117.83     execution time (avg/stddev):   0.0100/0.00 

Обработка результатов

1. Сравнения скорости чтения/записи через dd, Mb/s:
image

2. Количество запросов в секунду через sysbench, Requests/s:
image

3. Время потраченное на 10к запросов через sysbench, s:
image

Итоги

Итак, похоже быстрее всех работает именно Ephemeral, как и ожидалось, но IOPS к нему самый низкий.

Скорость соединения со стандартными EBS выше, чем к оптимизированным, даже с 2к IOPS. Но количество IOPS больше у оптимизированных EBS, хотя RAID по IOPS всех победил.

В итоге, как и следовало ожидать, по количеству операций в секунду и простоте настроек оптимизированные EBS лучше, хотя использование рейдов из них будет ещё быстродейственнее.

Если вам важна скорость обращения к файлу — используйте ephemeral или стандартные EBS.

ссылка на оригинал статьи http://habrahabr.ru/company/epam_systems/blog/163777/


Комментарии

Добавить комментарий

Ваш адрес email не будет опубликован. Обязательные поля помечены *