Storage Peer Incite: Notes from Wikibon’s August 7, 2007 Research Meeting
This week Wikibon presents Storage consolidation 2.0. The first round of storage consolidation was based on storage area networks (SANs) and resulted in huge savings from physical consolidations and any-to-any connectivity. Nine years later, the storage marketplace is standing on the verge of the second major storage consolidation, this time based on a new generation of storage controllers and promising storage virtualization. This new controller generation supports movement of data volumes between classes of storage, a step toward the long-promised goal of automated, rules-based tiered storage. In fact, some advanced users are already moving to an automated two-tier system. It also promises further storage consolidation and much faster and less expensive provisioning of new and replacement storage devices, akin to the huge improvements in provisioning provided by server virtualization. However, this new generation will not provide the 100% improvements in storage space efficiencies that SAN yielded. Instead of moving from 40%-50% utilization on the present SAN generation to 80%+, the best users can hope for with the new technology is about 60-70% utilization. And while two-tier architectures and consolidation of many SAN islands will be achievable, a full, automated three-tier storage architecture still depends on further technological advances, particularly in integrating automated tape libraries to the overall structure. And while the increased utilization and partial tier automation does promise savings, it will require investment in expensive controller software and hardware and will increase underlying infrastructure complexity. So while this new generation will be an advance, particularly for large corporate infrastructures, it does not offer as clear cut a financial advantage as the original SAN migration provided. Bert Latamore
After nine years of experience with installing, managing and exploiting SAN infrastructure, the IT industry is entering a new phase of storage and server consolidation. Unlike previous phases, which emphasized better physical allocation of resources (e.g., EMC big-box consolidations of the 1990s) and physical any-to-any connectivity, (e.g., fabric-based SANs of the 2000s) this next era will focus on consolidation of virtual server and storage resources and automated tiered storage strategies.
This means that provisioning new storage controller capacity and moving data across arrays will occur without disrupting application availability, creating the infrastructure to support automated, policy-based movement of data volumes across storage tiers. This is a major focus for users, for whom the inability to move data volumes across arrays without interrupting service levels of high-performance, often transactional applications, is a major block to the adoption of the much-promised tiered-storage architecture.
In addition to providing improved efficiencies, which include reducing costs of up to $50,000 for migrating to new arrays and compliance-driven consolidation of business processes that currently reside on different systems, three main factors are driving new consolidations of block-based SANs:
- Time to provision new servers and storage capacity remains onerous particularly for users with substantial capacity coming off leases needing to migrate to new arrays. Customers report provisioning windows extending from weeks to several months.
- Consolidation promises opportunity to improve customer service levels by automating the provisioning of capacity, reallocation of resources and responses to failures.
- High-quality server and storage virtualization technologies – including server virtualization from VMware, IBM and many others for servers and, on the storage side, IBM’s SAN Volume Controller (SVC), Hitachi’s USP, EMC’s Invista and a broad range of other new storage products – have emerged
This new era has many challenges, starting with financial considerations. In pre-SAN days when DAS ruled, storage utilization was routinely in the 20% range. SANs have at least doubled this metric and provided substantial management benefits, easily offsetting the incremental costs of switches and more expensive software. Consolidation 2.0 promises better utilization but not nearly the 100% improvement seen with the SAN introduction. Users should expect utilization to increase from its current 40%-50% to 60%-70% but not much higher, even with thin provisioning.
Further compounding the challenge of Consolidation 2.0 is the requirement for storage virtualization and great desirability of server virtualization to support much of the automation that provides the basic financial value. This brings a conundrum that while organizations are under extreme pressure to cut costs and improve service levels, virtualization will probably bring with it substantial overhead, increased complexity and require new skill sets. Users must be careful to apply virtualization in small chunks, testing performance, documenting procedural changes, managing software complexity, and getting rid of legacy assets.
Wikibon users consolidating SAN islands suggest the following best practices:
- Virtualize server and storage sides in parallel and consider technology alternatives. VMware for Windows has the early momentum on the server as does IBM’s SVC in storage, but alternatives from several other server and storage suppliers – including EMC, Hitachi, IBM, Sun and upstarts like 3Par – are also available.
- Keep consolidation simple to minimize tiers: Set service-level requirements for applications to include on Tier 1 and allocate everything else to Tier 2 to reduce costs and avoid unnecessary software license expenses. Use Tier 3 for archiving as necessary and as technologies like virtual tape evolve.
- Virtualize everything on Tier 2, but test and retest to be sure application behavior is predictable.
- Create a single SAN environment so there are no constrains to migrating storage to the best array for the job.
The goal of consolidation should be simplification, which will concentrate both vendor and organizational power in fewer hands. This presents pricing and negotiation challenges for users that can be managed over time. Getting this consolidated infrastructure to work properly should be the near-term priority. To avoid long-term dependencies on a single storage supplier, users should establish independent rules, roles, responsibilities and data structures that can accommodate continuous negotiations and multiple storage virtualization strategies over time.
Action item: IT in general and storage administrations in particular must prepare for lots of changes in the data center as virtual server and storage technologies align and integrate. To prepare for the next phase of storage consolidation, users should simplify block-based SAN strategies by choosing a single storage virtualization approach, minimizing storage tiers and standardizing on a single software management suite for their initial implementations.
For the past decade or so, increasing productivity -- the measure of work performed divided by resources consumed to perform that work (P=O/C, where P is productivity, O is output, and C is cost) -- has been a driving force of CIO performance. Offshoring, use of open source software, and storage consolidation efforts that focused mainly on reducing the amount of hardware as a means of increasing gigabyte utilization are examples of very successful IT programs to increase productivity by reducing the amount of resources required to perform a given amount of work (P goes up when O is constant but C is reduced).
In the case of storage consolidation, most of the low-hanging, cost-based, "storage consolidation 1.0" opportunities to increase productivity have been harvested. This next era of "storage consolidation 2.0" will emphasize the amount of business work that can be supported by steady-state (or decreasing) levels of storage-related investment (P goes up when O increases but C is constant). Storage consolidation efforts that are predicated on virtualization, tiered storage, or thin provisioning technologies, capable of dramatically enhancing infrastructure automation, flexibility, responsiveness, and optimization, can help sustain storage-related contributions to IT productivity improvements.
However, IT organizations and businesses have to recognize that these productivity gains will be more difficult to measure and quantify. Moreover, the storage consolidation practices that enjoyed so much success when the focus was on reducing DAS will not be broadly transferable to software-orientated storage consolidation 2.0 efforts. Ultimately, organizations that seek productivity improvements from storage consolidation 2.0 must recognize that this will be a long, committed process that will be highly dependent on the success of storage administration personnel to, effectively, automate their jobs -- perhaps out of existence.
Action Item: Automation of the storage administration function is at the core of storage consolidation 2.0. Successful implementation of advanced storage software, not hardware, will be the key. To achieve storage consolidation 2.0 objectives, storage administration personnel will have to adopt complex, and often foreign, practices usually associated with application development and operations. This will be a very difficult transition for most storage operations groups.
In today's IT world, lots of folks talk about solutions but very few vendors actually do a world class job of delivering them. In new SAN consolidation projects, the sale starts after the initial product is shipped. That's because these types of initiatives often are a journey, requiring complex and sophisticated migrations over periods as long as two to three years. This means 50-70% of the negotiations for a consolidation project will actually occur after the first round of hardware and software is delivered.
While there are still some holes, most suppliers are getting their consolidation product acts together. But just as buyers must update skills to exploit new technologies, suppliers must aggressively sustain an approach that can support the delivery of complex services at a level that many storage companies aren't delivering today. Specifically, users need help to support the exploitation of and testing for server- and storage-side virtualization, services to define and classify data, support for developing automated tiering strategies and migration services to support automating processes and procedures in general.
Consolidation 2.0 will be hard work and successful vendors will demonstrate that they have deep relationships that can sustain knowledge transfer over long periods of time. As switching costs decline, storage vendors must realize that the sale starts after the box is delivered. SAN consolidation 2.0 is not a good business opportunity for startups looking for a quick exit.
Action Item: While storage suppliers must continue to fill gaps in consolidation product offerings, vendors wanting a piece of the next wave of SAN consolidations must invest in services capabilities that can facilitate the adoption of new virtualized SAN infrastructure and demonstrate best practice knowledge across a variety of industry domains.
One of the profound changes that storage consolidation 2.0 will bring is that storage volumes will become mobile, able to be migrated to the optimum price/performance spot. The only group that has the overall view of storage and can best judge the best use of storage resources will be the storage administrative professionals. They will have to earn the right to be trusted that they can deliver. To earn that right they will need to develop not just the skills required to allocate storage, but the even more complex skills of setting criteria for automating the procedures round automatic provisioning, allocation, and re-allocation of different tiers of storage. They will have to institute solid testing procedures for existing and future application types.
Action item: Storage administration should be given the responsibility to manage and automate storage allocation from soup to nuts, given the tools and training to do the job, and given the rewards or consequences for success or failure.
Our recent interviews with storage executives have shown a consistent pattern in strategy. Many installations are committing themselves to a strong server consolidation strategy, and committing themselves to virtualization of storage. The business driver behind this move is to create a more flexible, reliable and cost effective server & storage infrastructure, particularly in the rack-mounted and/or blade Windows, Linux and Unix areas.
Server and storage virtualization projects can be separate and distinct initiatives, each with its own set of business drivers and criteria for success. In this case, the sum of the project benefits when implemented together can be greater than the sum of the parts. Both of them enhance to agility to apply resources where they are most needed in real time and to automate such processes.
Action item: CIO and CTOs should clearly articulate the vision of a virtualized server and storage infrastructure, and initiate test and pilot project to develop the internal skills and integrate the technology implementations.
After nearly a decade of experience installing and exploiting SAN infrastructure, it is not uncommon for a 4,000 person organization with eight to ten geographic locations to have anywhere from five to seven distinct SANs serving hundreds of applications and 1,500 - 2,000 clients. Unique tier 1 demands, pet projects, mergers and acquisitions and the like have created a situation where installed storage consists of EMC DMX’s and CLARiiON’s, HP EVA’s, IBM arrays and subsystems from a variety of server vendors including plenty of direct attached storage (DAS)—all adding up to a petabyte of capacity under management and sets of defacto tiers established along platform lines.
Each of these SANs has generally distinct hardware, storage resource management software and processes. The good news is that storage administrators can manage between 55-75 TBs per person, up from well under 10TB per admin pre-SAN. The bad news is the management edict is: “What have you done for me lately?”
Best practice for rationalizing SAN sprawl suggests:
- Clearly define recovery point objective (RPO) and recovery time objective (RTO) and let this be the driver of data placement;
- Ensure designs allow access to all tiers of storage from each server;
- Implement virtualization, creating a services layer external to the storage controller enabling the attachment of inexpensive, unintelligent arrays (rather than being held hostage to individual array software licensing practices).
Action Item: IT should collapse multiple SANs into fewer physical installations, add virtualization and data migration tools and create two or three tiers of storage so provisioning and movement can be automated. Define, agree and publish clear standards for RPO and RTO so that the storage department defines where to place data based on business needs rather than business lines defining where they want data placed.