Originating Author: Peter Burris
Performance benchmarking has been a first step on almost all paths to infrastructure technology standardization; storage should be no different. In networking, server, and most other infrastructure domains, performance benchmarking has shined a strong analytic light on product performance, capacity, and functional claims, highlighting specific, common attributes of value in a manner that encourages reasonable comparisons. Even when performance benchmarks are imperfect or incomplete, benchmarking nonetheless has fostered emergence of interoperability protocols, configuration and implementation conventions, and consistent approaches to assessing functional capabilities. The SPC (the Storage Performance Council) no doubt is imperfect, and will be attacked as not representing real workloads. However, it will provide a springboard for comparison now, and will improve rapidly, becoming more representative and gaining credibility, as users demand this and vendors invest in making it better.
Action Item: Users should welcome the SPC benchmark as a basis for product comparison. Benchmarks can be expensive to run, which can put smaller storage suppliers at a disadvantage. But only smaller suppliers should be given a benchmarking pass by users. Larger suppliers should be held accountable for investing in efforts to discredit benchmarks instead of investing in reasonable benchmarking.
Footnotes: