Очень много в интернете позитива/негатива об EBS. Так же не мало тестов на их производительность. Я решил сам провести несколько тестов и сделать небольшое исследование по этому поводу.
Итак, в тесте участвовала машинка m1.large к которой были примаунчены диски:
- EBS Standard, 100 GB
- EBS IO-1 500 IOPS, 100 GB
- EBS IO-1 1000 IOPS, 100 GB
- EBS IO-1 2000 IOPS, 200 GB
- 8xEBS Standard, 30 GB, RAID 10
- Ephemeral, 450 GB
Были проведены несколько тестов:
# hdparm -tT /dev/xcdX
# dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc # echo 3 > /proc/sys/vm/drop_caches # dd if=tempfile of=/dev/null bs=5M count=1024
# sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw prepare # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run
# hdparm -tT /dev/xvdj /dev/xvdj: Timing cached reads: 4866 MB in 2.00 seconds = 2428.53 MB/sec Timing buffered disk reads: 242 MB in 3.00 seconds = 80.54 MB/sec Timing cached reads: 5146 MB in 2.00 seconds = 2579.25 MB/sec Timing buffered disk reads: 294 MB in 3.01 seconds = 97.59 MB/sec Timing cached reads: 4870 MB in 2.00 seconds = 2440.55 MB/sec Timing buffered disk reads: 306 MB in 3.00 seconds = 101.89 MB/sec # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 161.222 s, 33.3 MB/s # echo 3 > /proc/sys/vm/drop_caches # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 86.4683 s, 62.1 MB/s # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 16 Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done. Operations performed: 6000 Read, 4000 Write, 12800 Other = 22800 Total Read 93.75Mb Written 62.5Mb Total transferred 156.25Mb (69.816Mb/sec) 4468.25 Requests/sec executed Test execution summary: total time: 2.2380s total number of events: 10000 total time taken by event execution: 1.5942 per-request statistics: min: 0.01ms avg: 0.16ms max: 65.00ms approx. 95 percentile: 0.02ms Threads fairness: events (avg/stddev): 625.0000/249.48 execution time (avg/stddev): 0.0996/0.03
# hdparm -tT /dev/xvdh /dev/xvdh: Timing cached reads: 4314 MB in 2.00 seconds = 2161.08 MB/sec Timing buffered disk reads: 72 MB in 3.05 seconds = 23.57 MB/sec Timing cached reads: 3646 MB in 2.00 seconds = 1826.09 MB/sec Timing buffered disk reads: 76 MB in 3.04 seconds = 25.01 MB/sec Timing cached reads: 4346 MB in 2.00 seconds = 2175.61 MB/sec Timing buffered disk reads: 76 MB in 3.03 seconds = 25.12 MB/sec # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 271.993 s, 19.7 MB/s # echo 3 > /proc/sys/vm/drop_caches # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 182.106 s, 29.5 MB/s # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 16 Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done. Operations performed: 6000 Read, 4000 Write, 12800 Other = 22800 Total Read 93.75Mb Written 62.5Mb Total transferred 156.25Mb (16.794Mb/sec) 1074.78 Requests/sec executed Test execution summary: total time: 9.3042s total number of events: 10000 total time taken by event execution: 0.2975 per-request statistics: min: 0.01ms avg: 0.03ms max: 30.70ms approx. 95 percentile: 0.02ms Threads fairness: events (avg/stddev): 625.0000/553.34 execution time (avg/stddev): 0.0186/0.02
# hdparm -tT /dev/xvdf /dev/xvdf: Timing cached reads: 5090 MB in 2.00 seconds = 2550.81 MB/sec Timing buffered disk reads: 104 MB in 3.03 seconds = 34.30 MB/sec Timing cached reads: 5000 MB in 2.00 seconds = 2505.62 MB/sec Timing buffered disk reads: 98 MB in 3.10 seconds = 31.64 MB/sec Timing cached reads: 5046 MB in 2.01 seconds = 2507.34 MB/sec Timing buffered disk reads: 98 MB in 3.04 seconds = 32.19 MB/sec # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 167.252 s, 32.1 MB/s # echo 3 > /proc/sys/vm/drop_caches # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 126.793 s, 42.3 MB/s # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 16 Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done. Operations performed: 6001 Read, 4000 Write, 12800 Other = 22801 Total Read 93.766Mb Written 62.5Mb Total transferred 156.27Mb (37.871Mb/sec) 2423.73 Requests/sec executed Test execution summary: total time: 4.1263s total number of events: 10001 total time taken by event execution: 0.8466 per-request statistics: min: 0.01ms avg: 0.08ms max: 22.31ms approx. 95 percentile: 0.02ms Threads fairness: events (avg/stddev): 625.0625/318.25 execution time (avg/stddev): 0.0529/0.02
# hdparm -tT /dev/xvdi /dev/xvdi: Timing cached reads: 4846 MB in 2.00 seconds = 2428.51 MB/sec Timing buffered disk reads: 90 MB in 3.02 seconds = 29.80 MB/sec Timing cached reads: 5122 MB in 2.05 seconds = 2503.64 MB/sec Timing buffered disk reads: 100 MB in 3.07 seconds = 32.56 MB/sec Timing cached reads: 4330 MB in 2.04 seconds = 2123.52 MB/sec Timing buffered disk reads: 102 MB in 3.05 seconds = 33.41 MB/sec # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 161.549 s, 33.2 MB/s # echo 3 > /proc/sys/vm/drop_caches # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 155.51 s, 34.5 MB/s # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw prepare sysbench 0.4.12: multi-threaded system evaluation benchmark 128 files, 40960Kb each, 5120Mb total Creating files for the test... [root@ip-10-98-91-92 4]# sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 16 Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done. Operations performed: 6000 Read, 4000 Write, 12801 Other = 22801 Total Read 93.75Mb Written 62.5Mb Total transferred 156.25Mb (74.645Mb/sec) 4777.28 Requests/sec executed Test execution summary: total time: 2.0932s total number of events: 10000 total time taken by event execution: 1.0015 per-request statistics: min: 0.01ms avg: 0.10ms max: 10.10ms approx. 95 percentile: 0.02ms Threads fairness: events (avg/stddev): 625.0000/177.29 execution time (avg/stddev): 0.0626/0.02
# hdparm -tT /dev/md0 /dev/md0: Timing cached reads: 3964 MB in 1.93 seconds = 2048.84 MB/sec Timing buffered disk reads: 230 MB in 3.53 seconds = 65.13 MB/sec Timing cached reads: 3994 MB in 1.99 seconds = 2002.16 MB/sec Timing buffered disk reads: 398 MB in 3.00 seconds = 132.64 MB/sec Timing cached reads: 4334 MB in 2.03 seconds = 2138.00 MB/sec Timing buffered disk reads: 302 MB in 3.02 seconds = 99.84 MB/sec # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 113.234 s, 47.4 MB/s # echo 3 > /proc/sys/vm/drop_caches # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 97.9346 s, 54.8 MB/s # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.5: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 16 Random number generator seed is 0 and will be ignored Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of IO requests: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Operations performed: 5998 reads, 4002 writes, 12800 Other = 22800 Total Read 93.719Mb Written 62.531Mb Total transferred 156.25Mb (87.287Mb/sec) 5586.40 Requests/sec executed General statistics: total time: 1.7901s total number of events: 10000 total time taken by event execution: 1.1625s response time: min: 0.01ms avg: 0.12ms max: 8.99ms approx. 95 percentile: 0.03ms Threads fairness: events (avg/stddev): 625.0000/171.83 execution time (avg/stddev): 0.0727/0.01
# hdparm -tT /dev/xvdb /dev/xvdb: Timing cached reads: 4048 MB in 2.00 seconds = 2027.97 MB/sec Timing buffered disk reads: 1794 MB in 3.12 seconds = 575.84 MB/sec Timing cached reads: 4854 MB in 2.00 seconds = 2432.18 MB/sec Timing buffered disk reads: 1830 MB in 3.00 seconds = 609.94 MB/sec Timing cached reads: 3434 MB in 2.00 seconds = 1719.73 MB/sec Timing buffered disk reads: 770 MB in 3.13 seconds = 245.97 MB/sec # dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 95.9093 s, 56.0 MB/s # echo 3 > /proc/sys/vm/drop_caches # dd if=tempfile of=/dev/null bs=5M count=1024 1024+0 records in 1024+0 records out 5368709120 bytes (5.4 GB) copied, 55.5027 s, 96.7 MB/s # sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 16 Extra file open flags: 0 128 files, 40Mb each 5Gb total file size Block size 16Kb Number of random requests for random IO: 10000 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Done. Operations performed: 6000 Read, 4000 Write, 12800 Other = 22800 Total Read 93.75Mb Written 62.5Mb Total transferred 156.25Mb (11.263Mb/sec) 720.82 Requests/sec executed Test execution summary: total time: 13.8731s total number of events: 10000 total time taken by event execution: 0.1603 per-request statistics: min: 0.01ms avg: 0.02ms max: 2.34ms approx. 95 percentile: 0.02ms Threads fairness: events (avg/stddev): 625.0000/117.83 execution time (avg/stddev): 0.0100/0.00
Обработка результатов
1. Сравнения скорости чтения/записи через dd, Mb/s:
2. Количество запросов в секунду через sysbench, Requests/s:
3. Время потраченное на 10к запросов через sysbench, s:
Итоги
Итак, похоже быстрее всех работает именно Ephemeral, как и ожидалось, но IOPS к нему самый низкий.
Скорость соединения со стандартными EBS выше, чем к оптимизированным, даже с 2к IOPS. Но количество IOPS больше у оптимизированных EBS, хотя RAID по IOPS всех победил.
В итоге, как и следовало ожидать, по количеству операций в секунду и простоте настроек оптимизированные EBS лучше, хотя использование рейдов из них будет ещё быстродейственнее.
Если вам важна скорость обращения к файлу — используйте ephemeral или стандартные EBS.
ссылка на оригинал статьи http://habrahabr.ru/company/epam_systems/blog/163777/
Добавить комментарий