* Disk sector size is 4KB (Advanced Format)

* If you want to test the disk/raid, the output should be a block device (/dev/sda2), not a file

* Understanding Record Size (reclen): IOzone benchmarks a filesystem by breaking up a file of a given size into records, for example if you test a 100MB file with a 1MB record size, it means that 100 operations will be done to achieve the read/write tests.

* Be careful about kernel / filesystem caching, use IOzone "-I" if you want to test the disk and not the filesystem (or "-U" to unmount and remount the mount point before beginning each test)

dd
[cc lang='bash']
# flush kernel cache
\$ sync; echo 3 > /proc/sys/vm/drop_caches

# one big block, Direct IO
\$ dd if=/dev/zero of=/tmp/test.dat oflag=direct bs=1G count=1
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 17.719 s, 60.6 MB/s

# many small blocks, Direct IO
\$ dd if=/dev/zero of=/tmp/test.dat oflag=direct bs=8k count=131072
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 44.0714 s, 24.4 MB/s

# many small blocks, Direct IO and Sync after each block
\$ dd if=/dev/zero of=/tmp/test.dat oflag=direct,sync bs=8k count=131072
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 205.451 s, 5.2 MB/s
[/cc]

iozone
[cc lang='bash']
# add the "non-free" component to /etc/apt/sources.list
\$ apt-get update
\$ apt-get install iozone3 -y

# auto mode - all tests, file size from 64k to 512MB with a 4k to 16M record size
\$ iozone -I -a
# output in IOzone OPS
\$ iozone -I -a -O

# run only sequential and random read/write tests
\$ iozone -I -a -O -i0 -i1 -i2

# set file size
\$ iozone -I -a -O -i0 -i1 -i2 -s10m
# set record size
\$ iozone -I -a -O -i0 -i1 -i2 -s10m -r16k

# set file path (to test different mount points)
\$ iozone -I -a -O -i0 -i1 -i2 -s10m -r16k -f /nfs/test

# include additional timers
\$ iozone -I -a -O -i0 -i1 -i2 -s10m -r16k -f /nfs/test -c -e
[/cc]

Some examples:
[cc lang='bash']
# local fs
\$ iozone -I -a -O -i0 -i1 -i2 -s10m -r16k -f /root/test.iozone -c -e
random random
KB reclen write rewrite read reread read write
10240 16 6150 4710 5904 6473 1427 842

# drdb Protocol C over Internet
# (local write operations on the primary node are considered completed
# only after both the local and the remote disk write have been confirmed)
\$ iozone -I -a -O -i0 -i1 -i2 -s10m -r16k -f /root/drbd/test.iozone -c -e
random random
KB reclen write rewrite read reread read write
10240 16 26 26 3419 6140 1675 26
[/cc]