Savior of the Universe
[It] save everyone of us
[It]’s a miracle
King of the impossible
-slightly modified version of the Queen song “Flash”
The drumbeat of the storage industry has reminded me a lot of the classic Queen song. Every single existing vendor and start-up is talking about how they are innovating with flash technology. It is a very fast moving market, when EMC launched solid state drives (SSD) on its Symmetrix platform in 2008, competitors all said that the technology was too expensive for enterprise IT. By 2010, EMC shipper over 10PB of flash for the year (only outdone by server-based Fusion-io solutions), and all of its traditional competitors were adding various flash offerings into the portfolio. There are also a slew of new start-ups launching with various flash-based technologies. Not every solution is truly innovative, but there are lots of different ways that flash can be used in a storage environment.
Most people know about flash technology from its consumer use in iPods, smartphones, tablets and laptops; it is the growth of mass adoption that drove the price down, making it viable for enterprise deployments. What differentiates flash from disk is that it has higher performance, lower latency and lower power requirements. Different applications have different needs for these characteristics and therefore, different architectures can be designed to optimize for the required criteria. Wikibon’s CTO David Floyer is fond of the idiom “horses for courses”, defined as: what is suitable for one person or situation might be unsuitable for another. Playing with that phrase, David posted the graphic below to denote the various architectural ways to use flash technology:
In the accompanying article, David talks about a number of the vendors from the 2011 Flash Memory Summit and which bucket they fit into. It is such a fast moving space that since David’s updates yesterday, another company, Pure Storage, has entered the mix. Pure Storage fits in the “Flash-only arrays” category; the primary differentiation is that this new solution is designed for Tier 1, scalable, HA environments (such as VMware and databases) at a price point that competes with high performance disk or hybrid flash/disk arrays. Startups like Pure Storage have the opportunity to design architectures that don’t just use, but are optimized for flash technology (SSD supplier Samsung is one the company’s investors). Through the use of native in-line compression and deduplication, Pure Storage claims at least 5x data reduction, which helps alleviate some of the capacity limitations that are inherent to flash drives (when compared to disk). While vendor claims of high reduction rates are often inflated, Pure Storage has a “Purity Reduction Evaluator” (PRE) tool that customers can run to set proper expectations for the real data and says that beta customers have been seeing even better than expected data rates. The Wall Street Journal did a good job listing the high profile pedigree of both the founders (including Michael Cornwell who put flash on the iPod and iPhone) and investors (including VMware founders Diane Greene and Mendel Rosenblum). Pure and the other startups in the space will still need to compete against highly entrenched legacy vendors, sales teams and the storage industry is well known for being slow to adopt change (understandably because data loss or data unavailability can put a company out of business). It’s a good time to be a customer in this space as there is strong competition and a rapidly changing marketplace.
Flash, Disk and Tape
“Every technology is dead” is a nice meme that gets people engaged in a dialogue. We have all watched the changing formats of music delivery, but as with other areas, moves in IT technology move much slower than the consumer world. Tape has been declared dead for decades, but even Google Gmail used tape to recover from data loss earlier this year. The tape market has been on the decline for many years, but when it comes to long-term retention, the price and stability of tape still has a place. Now flash technologies are looking to kill disk as the king of information storage in the datacenter. Flash is disrupting disk from the top based on performance and dropping costs (very similar to what disk did to tape). Even as flash rides down the price curve, it is expected that high capacity disks (SATA) will continue to dominate for storage capacity (85%), but dip below 50% of enterprise spend, as shown in this forecast by David Floyer:
Flash is disruptive to disk and tape. A couple of years ago, there was criticism that there wasn’t enough innovation in the storage industry. I put forth that the waves of flash, cloud and big data have a focus on finding new uses for data and related technologies like never before. While the other 80’s B-movie with a soundtrack by Queen (Highlander) proclaimed “There can only be one”, this flash story is complex and rapidly evolving. Users should watch this space closely and leverage the strong competitive nature to negotiate with vendors. CIOs should be sure that vendors can clearly match the functionality of the new generation of flash solutions to application requirements.
Additional Wikibon articles on Flash: