Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
P
Public Spacebox
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 0
    • Merge Requests 0
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • atlas-admins
  • Public Spacebox
  • Wiki
  • setup benchmark

Last edited by Carsten Aulbert Mar 28, 2020
Page history

setup benchmark

  • Example RAID setups
  • Benchmarks
    • fio results on ZFS 24 HDD
      • Stream write
      • Stream read
      • Random IO
      • Capacity

Example RAID setups

Our pilot 24-disk system could for example be organized as follows:

(here M x (N + R) also written as MxNrR means: M stripes of N data disks plus R reserve disks (mirror or parity))

setup abbreviation optimized for
1 x (21 + 3) 1x21r3 capacity
4 x (5 + 1) 4x5r1 capacity, stream read
8 x (2 + 1) 8x2r1 stream write, random read
4 x (4 + 2) 4x4r2 stream write, stream read
12 x (1 + 1) 12x1r1 stream write, stream read, random read

To calculate the usable space simply multiply the number of data disks by the capacity of a single HDD, e.g. for a 4 x (5 + 1) setup (4x5r1), one has 4*5=20 data disks and thus 240TByte usable capacity - assuming 12TByte disks. In reality, due to differences of calculating/displaying values, base1000 versus base1024 the real net capacity may be different by a few percent.

Benchmarks

The usual candidates for testing:

  • fio
  • iozone
  • bonnie++
  • io500

fio results on ZFS 24 HDD

Stream write

Results are in GByte/s

↓setup ; files(parallelism)→ 12 24 48 96 192 384 768 1536 3072 6144
12x1r1 1.50 (±0.04) 1.48 (±0.04) 1.50 (±0.03) 1.50 (±0.04) 1.49 (±0.04) 1.49 (±0.03) 1.50 (±0.05) 1.49 (±0.04) 1.49 (±0.06) 1.25 (±0.08)
1x21r3 0.79 (±0.04) 0.79 (±0.05) 0.78 (±0.05) 0.79 (±0.05) 0.80 (±0.07) 0.79 (±0.06) 0.80 (±0.05) 0.80 (±0.07) 0.79 (±0.06) 0.65 (±0.05)
2x10r2 1.13 (±0.07) 1.09 (±0.06) 1.11 (±0.09) 1.09 (±0.10) 1.10 (±0.09) 1.09 (±0.10) 1.11 (±0.08) 1.11 (±0.09) 1.08 (±0.10) 0.89 (±0.07)
3x6r2 1.33 (±0.08) 1.28 (±0.11) 1.30 (±0.08) 1.30 (±0.07) 1.30 (±0.07) 1.31 (±0.08) 1.31 (±0.08) 1.29 (±0.08) 1.30 (±0.11) 1.05 (±0.10)
4x4r2 1.54 (±0.09) 1.52 (±0.11) 1.54 (±0.10) 1.51 (±0.12) 1.53 (±0.11) 1.54 (±0.10) 1.53 (±0.09) 1.53 (±0.10) 1.52 (±0.13) 1.24 (±0.12)
4x5r1 1.42 (±0.10) 1.38 (±0.08) 1.37 (±0.09) 1.39 (±0.08) 1.38 (±0.08) 1.39 (±0.10) 1.41 (±0.10) 1.36 (±0.09) 1.40 (±0.14) 1.13 (±0.12)
6x3r1 1.47 (±0.11) 1.44 (±0.10) 1.44 (±0.10) 1.45 (±0.10) 1.41 (±0.10) 1.44 (±0.09) 1.42 (±0.10) 1.44 (±0.10) 1.41 (±0.18) 1.19 (±0.11)
8x2r1 1.85 (±0.05) 1.82 (±0.07) 1.82 (±0.06) 1.84 (±0.04) 1.83 (±0.06) 1.83 (±0.05) 1.85 (±0.05) 1.85 (±0.05) 1.82 (±0.09) 1.39 (±0.11)

Stream read

Results are in GByte/s

↓setup ; files(parallelism)→ 12 24 48 96 192 384 768
12x1r1 2.67 (±0.18) 2.66 (±0.17) 2.68 (±0.28) 2.64 (±0.36) 2.45 (±0.50) 1.75 (±1.86) 1.52 (±0.48)
1x21r3 1.92 (±0.17) 1.91 (±0.16) 1.87 (±0.18) 1.90 (±0.18) 1.88 (±0.24) 1.65 (±0.61) 0.33 (±0.46)
2x10r2 1.93 (±0.20) 1.95 (±0.18) 1.96 (±0.18) 1.95 (±0.20) 1.93 (±0.25) 1.91 (±0.64) 0.16 (±0.22)
3x6r2 1.90 (±0.25) 1.90 (±0.24) 1.91 (±0.22) 1.91 (±0.22) 1.88 (±0.27) 1.86 (±0.60) 2.09 (±1.11)
4x4r2 2.44 (±0.44) 2.45 (±0.39) 2.48 (±0.42) 2.46 (±0.50) 2.58 (±0.86) 2.75 (±0.97) 3.09 (±1.54)
4x5r1 2.07 (±0.22) 2.07 (±0.23) 2.07 (±0.23) 2.07 (±0.25) 2.03 (±0.31) 2.01 (±0.66) 0.87 (±0.75)
6x3r1 1.63 (±0.24) 1.63 (±0.25) 1.62 (±0.24) 1.64 (±0.23) 1.63 (±0.24) 1.58 (±0.42) 1.72 (±0.86)
8x2r1 1.70 (±0.17) 1.70 (±0.17) 1.70 (±0.17) 1.69 (±0.18) 1.68 (±0.18) 1.66 (±0.43) 1.76 (±0.95)

Random IO

Results are I/O per second

↓setup ; files(parallelism)→ 12 24 48 96 192 384 768
12x1r1 2016 (±2580) 1596 (±36) 1656 (±108) 1680 (±168) 1596 (±36) 1572 (±72) 1584 (±24)
1x21r3 876 (±6288) 252 (±24) 264 (±12) 252 (±24) 264 (±12) 264 (±12) 264 (±12)
2x10r2 528 (±804) 444 (±36) 516 (±72) 456 (±12) 456 (±24) 456 (±12) 456 (±12)
3x6r2 732 (±1176) 612 (±24) 672 (±96) 624 (±24) 612 (±24) 612 (±24) 612 (±24)
4x4r2 1008 (±1428) 744 (±48) 852 (±96) 768 (±24) 756 (±36) 768 (±24) 768 (±24)
4x5r1 768 (±252) 684 (±24) 768 (±48) 696 (±24) 684 (±48) 684 (±24) 696 (±24)
6x3r1 1188 (±2124) 900 (±24) 996 (±132) 864 (±72) 900 (±24) 900 (±24) 888 (±24)
8x2r1 1308 (±1908) 1080 (±24) 1152 (±108) 1104 (±96) 1080 (±36) 1056 (±72) 1080 (±24)

Capacity

setup available capacity [TB]
12x1r1 127
1x21r3 203
2x10r2 193
3x6r2 180
4x4r2 169
4x5r1 203
6x3r1 184
8x2r1 169
Clone repository
  • Hardware Details
  • Home
  • setup benchmark