EMC announced a clustered storage implementation (V-Max and Virtual Matrix Architecture) and software (FAST) that would exploit it Q4 2009. EMC received extensive press coverage, praise from the blogs and is generally touted as the industry leader.
Hitachi announced a clustered storage solution that would work with its existing USP V hardware and Hitachi High Availability Manager software (AM) that would exploit it in Q4 2009. HDS got some press coverage, brick-bats from the blogs and there is confusion about AM functionality.
The best way of dealing with confusion in storage is to take a data center view on the business value of two functions, tiered storage and non-disruptive migration "forever." There is significant overlap between the two functions.
The business value of tiered storage is the ability to move data to the optimum cost and performance storage location. The main value is reduction in cost. The business driver is the increasing difference in cost between the highest performance disks (SLC solid state disk [SSDs]) and the lowest cost disks (1 TB SATA). The problem that has to be solved is that manual migration of data is expensive as measured in storage administrative time, disruptive to the applications, and error prone. The result is that data is put on higher function disk (just in case) and is never moved.
The solution offered by Hitachi uses Tiered Storage Manager, Universal Volume Manager and Virtual Partition Manager software to allow the configuration, management, and migration of data across an entire storage pool, including volumes on other vendors' storage arrays. The migration is non-disruptive to the applications on a volume basis and works well. It uses the virtualized architecture of the USP V to simplify migration. It is not an automated solution. The solution has been available for several years, and works well in practice but still requires significant storage administrative effort to manage the process, as well as trust from the end-users that the performance of their data is not being affected.
EMC supports manual non-disruptive migration of data within the Symmetrix V-Max hardware using virtual LUNs (VLUNs) within a single array. EMC has announced fully automated storage tiering (FAST) for delivery in 4Q 2009. It is reasonable to assume that this would include migration from one V-Max to another across the virtual matrix at some stage, but this has not been explicitly promised for the first release. There are no details about how it will work, the level of automation, the granularity of the objects managed, and how inter-tier migration policy will be set.
Automated tiered storage is an important function to users. It is likely to take several years for vendors to roll-out and for users to implement. Buy-in from end-users will be a critical component. The overall potential is to reduce the cost of storage by up to 40%. Clustered storage helps the implementation but is not essential.
Non-disruptive migration "forever"
Migration of data from a block-based storage array at the end of life is a complex and time-consuming operation, and disruptive to applications. Wikibon has studied this in detail, and fully loaded costs are typically about $50,000/array with an elapsed time of several months.
Hitachi has made the migration of data from arrays that are attached to a USP V as external storage much simpler and non-disruptive. However, at the end of the life of the USP V users face a major migration of the data on the USP V and the external data attached, which is disruptive.
Hitachi has announced a solution to that problem with Hitachi High Availability Manager on existing USP V arrays. Two USP V arrays can be connected together and to an external quorum disk, and clustered. The USP Vs can be attached or detached dynamically. This means that users will be able to migrate data held on USP V and externally attached storage to another USP V and external storage non-disruptively; the data can be migrated “forever” without a system interruption. it also supports mirroring copies of data between the arrays for very high availability solutions.
EMC has announced the V-Max and the virtual matrix architecture. EMC has not explicitly announced a similar functionality for its clustered storage, although it is reasonable to suggest that it is in the EMC product plans.
The ability to migrate data non-disruptively is of significant importance to users and will reduce the cost of migration from $50,000 to potentially $5,000 and time from months to days. Other than EMC's implied capability and Hitachi's promise of an announcement, this benefit cannot be achieved by another solution today, including IBM's SVC, LSI's SVM and EMC's Invista. Clustered storage arrays are essential for effective implementation of non-disruptive migration "forever."
Action Item: Both Hitachi and EMC have announced inter-array clustered storage products ahead of their normal schedule. This is useful if it can guide users on strategic future functionality. However, when it is used as a marketing cudgel or marketing shield it is just confusing. Now that clustered storage system have been announced, users should expect and demand from EMC and Hitachi much more detail on what these systems can deliver on what timescale at what price. And Hitachi, marketing is kinda important.