Contents |
Introduction
Flash technologies have made a fundamental difference to the design of storage and applications. Magnetic media always has a mechanical component, with as tape transport or disk magnetic heads. Flash has no mechanical component, and can write data and access data much much faster - IO volume and IO speed are no longer scarce resources. Capacity storage will still use magnetic capacity technologies, both disk and tape.
Designing flash optimized storage arrays is evolving rapidly. What were not bottlenecks in the magnetic era become bottlenecks in a flash era. For new applications, some of the features provided in arrays will be built into software - e.g., Exchange 2012 & 2013. One of the areas of development is that NAND flash technology itself is developing fast. SLC flash (Single-Level Cell) was the original flash used for storage. MLC flash (Multi-level Cell) is improving is reliability and writability, and in density with additional layers. Tiering within flash is now an important element as the cost differential between different types of flash grows. Magnetic disk drive technology is also improving is density and cost/TB. This Wikibon Professional Alert focuses on the Dell architecture as a reference example of the application of flash tiering within a storage array, together with integration of magnetic drives.
Different Flash Technologies
SLC (Single-Level Cell) NAND flash is very fast for writes and reads, expensive, and handles being overwritten 30 times/day. MLC (Multi-level Cell) flash is fast for reads, 1/5 the price per gigabyte because of higher density, but only handles being overwritten 3 times/day (1/10 SLC). eMLC (Enterprise MLC) is an intermediate MLC technology with an increased number of write cycles. Figure 1 illustrates the differences among the different types of NAND flash technology.
Source: © Wikibon 2013
Based on information provided in a Dell Presentation to Wikibon in October 2013
The challenge for storage designers is to ensure that the technologies are mixed in a way that provides predictable performance at the lowest cost.
Designing for Flash
The challenge for storage designers is to ensure that the technologies are mixed in a way that provides predictable performance at the lowest cost. Figure 1 shows that SLC flash is less than half the cost for intensive write IO, but MLC flash is by far the lowest cost technology for other IO environments. Storage systems such as Dell’s Compellent storage array and Oracle’s ZFS Storage Appliance have found imaginative ways of separating out the reads and writes and applying the right flash technology at the right time. The result is faster computing at significantly lower cost by making greater use of MLC. This Wikibon Professional Alert focuses on the Dell architecture.
Compellent was built around storage tiering from the beginning, under the original stewardship of CTO Larry Aszmann. The Data Progression technology supports three levels of tiering. The design principle is that data starts being written at the highest level (higher performance and highest cost) of the storage tiers and moves down over time to the lower levels. For reads, the data can migrate up from Tier 3, but if Tier 2 is implemented in MLC flash, it will not migrate up further to Tier 1, leaving this free for write intensive IO. After the initial write, data will migrate down according to how frequently it is accessed.
Wikibon has talked in detail to many Compellent practitioners who have universally found that after the initial installation, additional storage could be added at just the lowest tier. The amount of flash storage for Tiers 1 & 2 usually remains constant over the life of the array. Wikibon has found from other studies that data not accessed for 90 days is very unlikely to be accessed again.
The Data Progression software was originally run in batch model overnight, and the data movement done at a time with lower IO performance levels. The latest level of Dell Storage Center software (SC 6.4) now supports real-time running of the Data Progression feature and the much faster flash SSD devices (about 0.2 milliseconds for an IO), and supports that much greater difference in disk performance vs. what one usually gets from SATA 7.2K RPM drives (about 20 milliseconds, 100 times slower). Figure 2 shows the Dell three-level tiering topology with the amount of Tier 1 SLC SSD and Tier 2 MLC SSD varied to meet the IOPS and read/write characteristics of the workload.
Source: © Wikibon 2013
Based on information provided in a Dell Presentation to Wikibon in October 2013
Server PCIe Flash Storage
Dell introduced some neat PCIe NAND flash technology into its servers early in 2013. PCIe cards cannot themselves be hot-pluggable,so Dell introduced hot-pluggable and hot swappable PCIe drives with a smart interface, removing one of the major concerns for enterprise customers, the ability to upgrade or to swap-out failed SSD drives.
When used with a SAN, the PCIe flash cards are primarily used for read-only caching at the server level. This allows for very fast caching under the control of the server software without storage network delays and can also lead to some (small) reduction in load on network-attached storage arrays.
The PCIe cards can also be used for holding the master copy of data for applications (particularly database) where the complete database can be held in flash. The remaining challenge for Dell for traditional ISV software running on PCIe cards is providing access to the data on the PCIe flash on a failing server (by say dual-porting on PCIe cards) or technology such as RDMA replication (e.g. from Virident). Modern cloud software will take responsibility within the application for replication over multiple servers in a cluster (e.g., FaceBook, Microsoft’s Exchange 2012 and 2013), and traditional software will be migrated slowly to this model with high availability add-ons.
Conclusions
Wikibon has written extensively about the impact of the migration to the mobile-cloud application model in a post-PC world. Dell was caught in this maelstrom, with a significant reduction in revenue in 2010. Dell is moving quickly to stabilize revenue, treat consumer PCs as a cash cow, and grow its enterprise solutions business to support mobile and cloud. Figure 3 in the footnotes shows Dell’s financial progress as it goes private.
Dell storage also went through a downturn in revenue, as shown in Figure 4 in the footnotes. Dell has stabilized storage results in 2013, and has continued to invest and introduce innovative upgrades to the Dell Compellent storage array, as discussed above. Wikibon is confident about Dell's future strategy and Dell's position as an enterprise storage vendor.
The innovative use of SLC flash for writes and MLC for reads together with low-cost high-capacity traditional disk drives means that 50,000-100,000 IOPS at an average of about 1 millisecond can be achieved at an overall cost of less than $1 per gigabyte for most enterprise applications found in small and intermediate enterprise data centers.
However, the most important reason for improving the performance of IO is the business benefit. The ability to run query transactions at the same time as transactional workloads, the improved response time and lower elapsed time for IO intensive workloads lead to greater end-user productivity and supports new business models.
Action Item: The Dell Compellent three-tiered storage array with small amounts of SLC flash SSDs for Tier 1 and larger amounts of low-cost MLC flash SSDs for Tier 2 respectively is a well-engineered and very cost effective way of providing for higher levels of performance. With the use of high capacity traditional disk drives in Tier 3, Dell provides a very low cost high performance hybrid storage platform. For intermediate storage arrays requiring additional performance, Wikibon would unreservedly recommend that Dell Compellent Storage be in included in any RFP.
Footnotes:
Source: © Wikibon 2013, Dell SEC Filings, Downloaded October 2013
Source: © Wikibon 2013 from Dell downloaded October 2013
