Originating Author: Dave Vellante
This article addresses what organizations consolidating infrastructure via storage area networks (SAN) and network attached storage (NAS) should expect in terms of key metrics such as utilization and staff productivity. Organizational goals and implementation advice related to storage consolidation are also provided.
Many organizations have deployed dozens or even hundreds of applications by dedicating storage and server resources to individual applications. Very often, this infrastructure is physically decentralized making the sharing of storage resources virtually impossible. This type of distributed infrastructure, while simple to deploy, leads to inefficiencies that include poor storage utilization, administrative challenges, less than optimal availability/security and inflexibility especially with regard to application deployment. Two classic examples of waste and inefficiency in a non-consolidated infrastructure include: 1) frequently running out of disk space and 2) managing many disparate processes and procedures.
Contents |
Consolidated storage capability
Storage consolidation works by provisioning storage that can be shared across servers and applications and managed with common processes and procedures. This most typically involves deploying storage area networks (SAN) or network attached storage (NAS). With SANs, storage and servers are connected to a centralized switch fabric that allows an 'any-to-any' type of connection between storage resources and servers. NAS typically consolidates storage using a device with a file system, referred to as an appliance that servers access via network protocols like NFS.
SAN requires more expensive switch fabric but offers high performance and is suitable for more transaction oriented workloads. NAS is simpler and cheaper and is best suited for files and workloads that are not as transaction intensive. This gross generalization needs to be further researched before the decision to use SAN or NAS is made. In either case however, dramatic simplification in infrastructure management, improved availability and faster provisioning can be achieved by consolidating storage.
Specific operational goals of consolidated storage
A successful consolidated storage infrastructure will: 1) simplify infrastructure management by creating a single point of control; 2) improve efficiencies in terms of disk utilization, backup process and day-to-day storage management; 3) improve application service levels and the ease of providing security; 4) speed provisioning and change management; 5) focus operational tasks like performance tuning, backup, recovery and provisioning to streamline job descriptions and minimize duplication and 6) improve the overall quality of infrastructure as measured by cost, availability and speed to provision additional capacity and function.
Storage consolidation project costs will range from a few hundred thousand dollars to upwards of several million dollars, depending on the number of servers/applications and amount of storage being consolidated. Projects will typically span three to nine months.
The following specific measurements can be used as benchmarks of success (see Figure 1):
[Note: Figures assume an organization with $1B in revenue; 4,000 employees; IT budget of $40M/annum. Consolidation scenario analyzed includes 50 distributed Windows servers with 10TB of protected storage attached-- see Standard WikiBon business model. Consolidated configuration is 15 servers with 10TB of protected storage attached.]
- Over a two to four year period, storage total cost of ownership (TCO) is reduced by 10-30%
- Capacity managed per admin is increased 2-4X
- Disk utilization increases from less than 40% to 60%+
- Ongoing management costs (people) decline from 66% to less than 33% of the overall total cost of ownership (TCO) of managing storage. A target of 20% is achievable over time.
- The incremental capital costs of deploying a consolidated infrastructure (switches, more advanced hardware/software and implementation costs) are offset within a six to eighteen month timeframe by lower labor costs.
- The IRR on storage consolidation projects should exceed 25% and will often approach triple digits. [Note: this should be measured relative to the costs of not consolidating – i.e. the ‘delta’].
- Time to provision storage is cut in half.
- Training for backup and recovery and other key processes is consolidated to a single set of courseware with 100% training participation by existing and new storage administrators.
Exit strategies regarding storage consolidation involve: 1) evolving infrastructure to tiered storage and 2) consolidating both homogenous and heterogeneous applications. These moves will often necessitate retiring servers and storage devices. As a general rule of thumb, between 20 – 33% of the installed infrastructure will 'churn' annually.
Risks of implementing consolidated storage
The main risks associated with consolidating storage relate to: 1) ensuring the goals are clearly understood and the organization will support the goals; 2) the candidates for consolidation will deliver the types of benefits expected and 3) the technical choices are sound given the application profiles, skill sets of the organization and maturity of hardware, software and processes.
Often, the politics of storage consolidation involve taking resources and spending authority away from decentralized business or IT groups and consolidating that authority under a centralized management structure. If decentralized groups don’t ‘play by the rules’ the objectives of consolidation will not be met.
Importantly, not all applications and workloads will benefit from consolidation. Many workloads that are less transaction/database oriented will thrive in decentralized environments. As a general rule, the more complex the workload to manage, the greater degree of cost savings realized.
Technical risks relate to choosing NAS versus a switched SAN fabric , the degree of software maturity implemented and the ability of staff to manage added complexity. Of important note is network considerations as often consolidating storage will require redesigning the network to alleviate potential bottlenecks caused by centralizing storage.
The consolidated storage initiative
Expectations (out-of-scope)
In general, these conditions will be in place for a successful storage consolidation project:
- A large enough installation or expectation of high enough growth to warrant the effort. Usually a successful consolidation will involve more than five servers / applications and more often than not, dozens or even hundreds of servers and applications.
- A clear existing or foreseen problem in the form of current or expected inefficiencies, difficulties achieving application service levels and difficulty provisioning new storage to the extent that it is limiting or expected to limit business success.
- The workloads targeted for consolidation are 'consolidation friendly.'
- Finance and business lines have agreed that storage acquisition and management will be centralized.
Analyze phase
For individuals performing a storage consolidation analysis, a fully detailed blueprint and step-by-step project outline with example business cases can be found at Analyzing storage consolidation requirements.
Acceptance test considerations
A successful storage consolidation analysis will meet at a minimum the four following conditions with agreement/signoff from stakeholders:
- A survey of existing infrastructure is completed with a clear understanding of the number of servers, types of applications and amount of storage that will be consolidated.
- An estimate exists of the replacement costs of the existing infrastructure and the ongoing costs of managing the infrastructure.
- An estimate of the one-time and ongoing costs of the consolidated environment is complete.
- A financial and business benefits analysis has been performed and agreed specifically citing costs saved and other benefits derived.
See Figure 2 for a sample business case.
Key analysis milestones
Data collected and analysis complete
The following activities support the analysis of an as-is compared to a to-be business case:
- Survey of current infrastructure and replacement costs along with the major types of applications.
- Current and future storage utilization estimated – typically distributed storage will exhibit utilization rates of less than 50% and quite often less than 40%, while consolidated storage utilization will often exceed 60%.
- Fully loaded labor costs (measured as full time equivalents X fully loaded costs) to manage the infrastructure on a day-to-day basis are calculated. Typically administrators can manage between 1-3TB in distributed environments, depending on workload type. Consolidation will improve this efficiency by approximately 2-4X.
- Growth rates in server and storage resources estimated. Growth must be factored to include forecasts of servers, capacity managed, software costs and labor required.
[Note: Fully loaded admin costs in North America and Western Europe will range from $75,000 - $150,000 annually. Less expensive regions will see fully loaded costs of between $20,000 - $50,000 annually.]
A consolidation generally occurs at an installation that has a mix of the following three topologies: 1) physically distributed locations with server resident direct attached storage (DAS); 2) physically collocated storage, residing within a data center using DAS or 3) a basic consolidation has taken place using either SAN with switched fabric or shared NAS and is a candidate for more advanced hardware and software infrastructure investment. As well, frequently a disaster recovery (DR) plan is under consideration and consolidation enables cost-effective DR.
Typically, the more distributed an infrastructure the greater potential benefit of consolidation.
Design phase
Acceptance test considerations
A successful design will meet the following three criteria:
- A schematic detailing the consolidated to-be environment is presented to operations and agreement is reached that the proposed design can be implemented.
- A detailed study of as-is and to-be service levels for resources including servers, storage and software functionality has been performed and documented.
- A progression plan is included in the design to document the phases of consolidation that will take place in which order.
Key design milestones
Design constructed and tested against requirements
The following activities impacts should be considered in the design phase:
- Design ensures service level requirements are held constant or improved as a result of the consolidation.
- Design specifies parts of business to be consolidated when.
- Capacity and functionality for each phase are clearly documented to include infrastructure, data protection, backup choice, replication strategy, switching design and optimization approaches.
- Design specifies degree of application consolidation - i.e. which applications share server and storage resources-- generally, homogeneous workloads can be successfully consolidated whereas consolidating heterogeneous workloads is more challenging.
- Design considers additional tradeoffs including balancing price/performance and price/ functionality.
Deploy phase
Acceptance test considerations
A successful deployment will meet the following four criteria:
- Consolidated storage is provisioned and delivers comparable service levels.
- Functionality has been tested and performs to design specification
- Operations people have been trained and are self-sufficient in new processes
- The consolidated infrastructure has ‘gone live’ and is supporting in production applications
Key deployment milestones
Consolidated storage provisioned and tested
The key skills required for deployment are experienced operations professionals familiar with the current environment who have sufficient training and technical skills to migrate from the existing environment to the consolidated infrastructure. In addition:
- Infrastructure built,function tested and service levels confirmed.
- Compatibility ensured (e.g. array to switch to server connectivity); software functionality across server, array and third party providers tested.
Training and knowledge transfer completed
- Develop training with sufficient operational knowledge on specific processes and procedures including backup, restore, copy services, data migration and provisioning.
- Reallocated or rationalize labor to realize business benefits.
Initiative summary
A storage consolidation can yield excellent business benefits for the right workloads. The following factors should be given substantial consideration:
- Consolidating storage will typically increase capital costs due to new switching infrastructure (in the case of SAN) and new software function
- However these costs will frequently be offset by improved hardware utilization and dramatically improved labor efficiencies
- Redeployment of labor is a key consideration. Labor resources must usually be reallocated or rationalized to demonstrate cost savings.
- The choice of SAN vs. NAS should be workload based. NAS workloads are typically less expensive to manage than SAN workloads.
- However trying to service SAN workloads with NAS and vice versa is not advised and will often result in poor performance and increased expense.
- Meaningful benefits to availability and speed to provision storage should be seen by the business within months of consolidation.
In addition to proper management support, the success of a storage consolidation project will ultimately boil down to three factors: 1) the appropriateness of the workloads for consolidation; 2) the skill sets of the individuals performing the consolidation and 3) the maturity and robustness of the technology deployed.
Passionate about writing? Share your ideas!