|
|
# Example RAID setups
|
|
|
|
|
|
Our pilot 24-disk system could for example be organized as follows:
|
|
|
|
|
|
* Optimized for storage capacity:
|
|
|
* 1 x (21 +3) (meaning '''1''' pool consisting of '''21''' data disks and '''3''' disks for redundancy), abbreviated as 1x21r3
|
|
|
* 4 x (5 + 1) - 4x5r1
|
|
|
* Optimized for stream write:
|
|
|
* 8 x (2 + 1) - 8x2r1
|
|
|
* 4 x (4 + 2) - 4x4r2
|
|
|
* 12 x (1 + 1) - 12x1r1
|
|
|
* Optimized for stream read:
|
|
|
* 12 x (1 + 1) - 12x1r1
|
|
|
* 4 x (4 + 2) - 4x4r2
|
|
|
* 4 x (5 + 1) - 4x5r1
|
|
|
* Optimized for random read:
|
|
|
* 12 x (1 + 1) - 12x1r1
|
|
|
* 8 x (2 + 1) - 8x2r1
|
|
|
|
|
|
To calculate the usable space simply multiply the number of data disks by the capacity of a single HDD, e.g. for a 4 x (5 + 1) setup (4x5r1), one has 4*5=20 data disks and thus 240TByte usable capacity - assuming 12TByte disks. In reality, due to differences of calculating/displaying values, base1000 versus base1024 the real net capacity may be different by a few percent.
|
|
|
|
|
|
# Benchmarks
|
|
|
* [fio](https://github.com/axboe/fio)
|
|
|
* [iozone](http://iozone.org/)
|
|
|
* [bonnie++][(https://www.coker.com.au/bonnie++/)
|
|
|
* [io500](https://www.vi4io.org/io500/)
|
|
|
|
|
|
## fio results on ZFS 24 HDD
|
|
|
### Stream write
|
|
|
|
|
|
Results are in GByte/s
|
|
|
|
|
|
| Nr of open files | 12 | 24 | 48 | 96 | 192 | 384 | 768 | 1536 | 3072 | 6144 |
|
|
|
|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
|
|
|
|
|
|
||12x1p1 ||1.50+-0.04 ||1.48+-0.04 ||1.50+-0.03 ||1.50+-0.04 ||1.49+-0.04 ||1.49+-0.03 ||1.50+-0.05 ||1.49+-0.04 ||1.49+-0.06 ||1.25+-0.08 ||
|
|
|
||1x21p3 ||0.79+-0.04 ||0.79+-0.05 ||0.78+-0.05 ||0.79+-0.05 ||0.80+-0.07 ||0.79+-0.06 ||0.80+-0.05 ||0.80+-0.07 ||0.79+-0.06 ||0.65+-0.05 ||
|
|
|
||2x10p2 ||1.13+-0.07 ||1.09+-0.06 ||1.11+-0.09 ||1.09+-0.10 ||1.10+-0.09 ||1.09+-0.10 ||1.11+-0.08 ||1.11+-0.09 ||1.08+-0.10 ||0.89+-0.07 ||
|
|
|
||3x6p2 ||1.33+-0.08 ||1.28+-0.11 ||1.30+-0.08 ||1.30+-0.07 ||1.30+-0.07 ||1.31+-0.08 ||1.31+-0.08 ||1.29+-0.08 ||1.30+-0.11 ||1.05+-0.10 ||
|
|
|
||4x4p2 ||1.54+-0.09 ||1.52+-0.11 ||1.54+-0.10 ||1.51+-0.12 ||1.53+-0.11 ||1.54+-0.10 ||1.53+-0.09 ||1.53+-0.10 ||1.52+-0.13 ||1.24+-0.12 ||
|
|
|
||4x5p1 ||1.42+-0.10 ||1.38+-0.08 ||1.37+-0.09 ||1.39+-0.08 ||1.38+-0.08 ||1.39+-0.10 ||1.41+-0.10 ||1.36+-0.09 ||1.40+-0.14 ||1.13+-0.12 ||
|
|
|
||6x3p1 ||1.47+-0.11 ||1.44+-0.10 ||1.44+-0.10 ||1.45+-0.10 ||1.41+-0.10 ||1.44+-0.09 ||1.42+-0.10 ||1.44+-0.10 ||1.41+-0.18 ||1.19+-0.11 ||
|
|
|
||8x2p1 ||1.85+-0.05 ||1.82+-0.07 ||1.82+-0.06 ||1.84+-0.04 ||1.83+-0.06 ||1.83+-0.05 ||1.85+-0.05 ||1.85+-0.05 ||1.82+-0.09 ||1.39+-0.11 ||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
=== Stream read ===
|
|
|
Results are in GByte/s
|
|
|
||Nr of open files ||'''12''' ||'''24''' ||'''48''' ||'''96''' ||'''192''' ||'''384''' ||'''768''' ||
|
|
|
||12x1p1 ||2.67+-0.18 ||2.66+-0.17 ||2.68+-0.28 ||2.64+-0.36 ||2.45+-0.50 ||1.75+-1.86 ||1.52+-0.48 ||
|
|
|
||1x21p3 ||1.92+-0.17 ||1.91+-0.16 ||1.87+-0.18 ||1.90+-0.18 ||1.88+-0.24 ||1.65+-0.61 ||0.33+-0.46 ||
|
|
|
||2x10p2 ||1.93+-0.20 ||1.95+-0.18 ||1.96+-0.18 ||1.95+-0.20 ||1.93+-0.25 ||1.91+-0.64 ||0.16+-0.22 ||
|
|
|
||3x6p2 ||1.90+-0.25 ||1.90+-0.24 ||1.91+-0.22 ||1.91+-0.22 ||1.88+-0.27 ||1.86+-0.60 ||2.09+-1.11 ||
|
|
|
||4x4p2 ||2.44+-0.44 ||2.45+-0.39 ||2.48+-0.42 ||2.46+-0.50 ||2.58+-0.86 ||2.75+-0.97 ||3.09+-1.54 ||
|
|
|
||4x5p1 ||2.07+-0.22 ||2.07+-0.23 ||2.07+-0.23 ||2.07+-0.25 ||2.03+-0.31 ||2.01+-0.66 ||0.87+-0.75 ||
|
|
|
||6x3p1 ||1.63+-0.24 ||1.63+-0.25 ||1.62+-0.24 ||1.64+-0.23 ||1.63+-0.24 ||1.58+-0.42 ||1.72+-0.86 ||
|
|
|
||8x2p1 ||1.70+-0.17 ||1.70+-0.17 ||1.70+-0.17 ||1.69+-0.18 ||1.68+-0.18 ||1.66+-0.43 ||1.76+-0.95 ||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
=== Random IO ===
|
|
|
Results are I/O per second
|
|
|
||Nr of open files ||'''12''' ||'''24''' ||'''48''' ||'''96''' ||'''192''' ||'''384''' ||'''768''' ||
|
|
|
||12x1p1 ||2016.00+-2580.00 ||1596.00+-36.00 ||1656.00+-108.00 ||1680.00+-168.00 ||1596.00+-36.00 ||1572.00+-72.00 ||1584.00+-24.00 ||
|
|
|
||1x21p3 ||876.00+-6288.00 ||252.00+-24.00 ||264.00+-12.00 ||252.00+-24.00 ||264.00+-12.00 ||264.00+-12.00 ||264.00+-12.00 ||
|
|
|
||2x10p2 ||528.00+-804.00 ||444.00+-36.00 ||516.00+-72.00 ||456.00+-12.00 ||456.00+-24.00 ||456.00+-12.00 ||456.00+-12.00 ||
|
|
|
||3x6p2 ||732.00+-1176.00 ||612.00+-24.00 ||672.00+-96.00 ||624.00+-24.00 ||612.00+-24.00 ||612.00+-24.00 ||612.00+-24.00 ||
|
|
|
||4x4p2 ||1008.00+-1428.00 ||744.00+-48.00 ||852.00+-96.00 ||768.00+-24.00 ||756.00+-36.00 ||768.00+-24.00 ||768.00+-24.00 ||
|
|
|
||4x5p1 ||768.00+-252.00 ||684.00+-24.00 ||768.00+-48.00 ||696.00+-24.00 ||684.00+-48.00 ||684.00+-24.00 ||696.00+-24.00 ||
|
|
|
||6x3p1 ||1188.00+-2124.00 ||900.00+-24.00 ||996.00+-132.00 ||864.00+-72.00 ||900.00+-24.00 ||900.00+-24.00 ||888.00+-24.00 ||
|
|
|
||8x2p1 ||1308.00+-1908.00 ||1080.00+-24.00 ||1152.00+-108.00 ||1104.00+-96.00 ||1080.00+-36.00 ||1056.00+-72.00 ||1080.00+-24.00 ||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
=== Capacity ===
|
|
|
||'''RAID''' ||'''available capacity [TB] ''' ||
|
|
|
||12x1p1 ||127 ||
|
|
|
||1x21p3 ||203 ||
|
|
|
||2x10p2 ||193 ||
|
|
|
||3x6p2 ||180 ||
|
|
|
||4x4p2 ||169 ||
|
|
|
||4x5p1 ||203 ||
|
|
|
||6x3p1 ||184 ||
|
|
|
||8x2p1 ||169 || |