The purpose of this how-to article is to hash out an open standard benchmarking system for storage devices. Storage devices are complicated, and this makes it hard to compare one device to another. Unlike systems and servers (which are benchmarked in a fairly straightforward way), storage devices have multiple components that contribute performance (and performance limitations) to different types of applications at different points.
To start any discourse on benchmarking, a common pool of definitions is needed.
storage device definition
host device definition (operating system, applications)
Anatomy of I/O
Different applications perform input and output differently.
List of protocols
iSCSI file protocols
- note: there is a convergence between direct attached and networked protocols
SCSI ESCON SAS
Benchmarking storage is complicated. In a more linear system with less complexity, benchmarks are
Storage Benchmarking History
history, background, controversy
only benchmark available currently, allows companies to see if/when the device they're considering will no longer be able to saturate the back end disks
Cons and Common Criticisms
not reflective of real life, does not reflect anything but back end device performance
Storage Benchmarking Future
Proposed new open benchmarks
Should provide as close to real life comparison as possible, if necessary with many different workload simulators for different types of environments
Should be tested by multiple parties, the more the better. At least one should be either neutral or a competitor to the vendor (client or competitor)
Should allow anyone to recreate the test with the same gear to check the reproducibility of the test, should also allow anyone to create a variant or fork