VMware server virtualization technology has not only transformed the x86 server market, it has created a ripple throughout data center infrastructure. VMware has built a $45B ecosystem around server virtualization, but the abstraction of resource caused problems in other parts of the stack. Specifically, while server virtualization increases server utilization, it had the unintended effect of breaking storage; specifically it caused performance degradation and broke data protection software that worked at the LUN level rather VM-level.
VMware and storage vendors know that remedying these issues is crucial to increasing the adoption of virtualization, especially as customers want to move along the spectrum of usage from test/dev environments into mission critical applications. One remedy is to create integrations between VMware and the storage infrastructure. Wikibon researched the state of storage integration with VMware. Ultimately, the better-integrated solutions improve performance and allow administrators to be more efficient and effective; through this tuning of infrastructure, organizations will be able to virtualize more applications.
Volume and Value of Integration
All of the storage vendors have initiatives to work with VMware and claim to have great solutions. In order to analyze the various offerings, which often use disparate terminology, Wikibon first created a master list of integration points spanning storage, backup, network, security, and management. The creation of the list was done through research and discussions with the top six storage vendors (Dell, EMC, HDS, HP, IBM, and NetApp). The list is only for vSphere 4.x integrations and was limited to items that include some specific integration or coding specifically for VMware. Functionality that was isolated to the storage array that did not include an integration (but could still be useful to a virtual environment) was not included; internal storage tiering or optimization features that work across any OS also fall into this bucket. Eleven storage array families from the six vendors were reviewed against a list of 32 integrations (see Table 1 in the footnotes). The number of integration points was done on the array-family basis to eliminate inflating the number of integrations for large companies with broad portfolios of products.
Beyond simply creating a list of integrations that can be compared across vendors, Wikibon wanted to dig deeper into the utility of the features for end users. Wikibon did a survey of its community of IT practitioners to determine which advanced VMware storage features were being used, and the value of the various features. Users rated a consolidated list of integration points on ease-of-use (both on the deployment and management of the functionality) and the value that the feature added to their environment. It is worth noting that more than half of the advanced storage integration features are in use by less that half of the community. This data helped the Wikibon analyst team to rank the features listed in Table 1 in order of importance.
A score for the total value of integration was then calculated for each storage family using integration points, the quality of the implementations, and the relative value that users get from the functionality. Wikibon’s CTO David Floyer writes that the purpose of the scoring is so that “practitioners can use the methodology in this report to help position the importance of integration features for their own VMware storage decisions and to help decide which vendors to include in RFPs. This analysis only looks at storage integration with VMware; other factors such as price, reliability, quality of reseller, quality of support, and other storage functionality are not taken into account at all and would have to be separately assessed.”
Overall, the “pure play” storage vendors EMC and NetApp came out on top of the study. EMC’s has shown a tenacious drive to deliver more integrations than its competitors. A VMware specialist can be found in every EMC product team, and the company also has a vSpecialist field force dedicated to working on EMC/VMware solutions. VMware deployments are also a sweet spot for NetApp, where the company leverages its simple and integrated solutions to gain a greater market share than it has in the general marketplace. The results of the customer survey showed users pick EMC and NetApp as being the best vendors for VMware storage (see Figure 2 below).
Wikibon consolidated recent acquisitions under the acquiring company; there were write-in entries for 3PAR, now part of HP, and Compellent, now part of Dell. Since the survey was a discussion of storage, it is likely that users discount or are not aware of the server and networking solutions geared for VMware from HP, IBM, and Dell.
vSphere 5 and Beyond
After the research project was completed, VMware announced vSphere 5 (see Wikibon’s coverage). Several integrations are available in vSphere 5 (such as automated SRM failback and Thin Provisioning Suspend) that when supported by all vendors will likely shrink the integration gap among vendors. As always, new releases of technology will take time to roll out and become adopted, and at that point Wikibon can assess the full impact of the new solution. From a technical standpoint, vSphere 5 eliminates some of the remaining hurdles that would prevent mission-critical applications from being deployed in a VM; now storage management and organizational challenges lead the objection list.
The bottom line on feedback from the user community is that despite significant progress, storage on VMware still sucks. Hypervisors introduce another layer into the stack, causing the storage provisioning, management, and problem determination to become more complex. Currently VM Admins, DBAs, server and storage administration staff need to communicate about storage. In the event of a problem, storage will be assumed to be guilty until proved otherwise.
The trend of storage for VMware is toward making storage “invisible” – allowing it to be automatically configured and tuned so that virtualization administrators can do their jobs without interacting with storage. Storage specialists don’t go away, but the expectation is that there will be a lot fewer of them, and some companies may choose to outsource that expertise to the vendor or service provider. While “silo” experts are not going away, the population of virtualization experts and cloud architects will be much larger than that of people focused solely on storage.
As VMware develops, differentiation among storage products for VMware environments will decrease and integrations will become table stakes. EMC is likely to continue to be a front runner, and NetApp is likely to keep pace. The server vendors have responded by acquiring best-of-breed storage pioneers and working more closely with VMware. The quality of the services and management software offered by storage vendors to support VMware environments is likely to become a key differentiation.
Looking at the integration journey milestones that Wikibon put forth in 2010, the industry is now delivering on many of the mid-term deliverables. Wikibon expects significantly more innovation in the area of VMware storage, and that the easiest solution will win in the long term. Vendors should focus on ease-of-use ahead of other advanced features that have a long adoption ramp. While existing vendors are adding “VM-aware” functionality, several start-ups are looking to be the “VMware of Storage”. CIOs are under constant pressure to deliver more applications on flat budgets, and the virtual data center must keep up. VMware solutions are continuing to spread into mission-critical applications and users must make sure that storage does not negate the benefits gained by server virtualization. Even at this mid-journey milestone, the VMware ecosystem has created huge value for users, and it will continue to grow for those companies that embrace the opportunities of virtualization. The big picture impact of storage integration goes beyond the TCO of the solution by allowing businesses to extract value through infrastructure that enables change.
The Services Angle
The downstream issues involved with server virtualization are non-trivial. Specifically, server virtualization stresses storage and backup. While integration is a key solution to this problem, professional services (either internal or outsourced) will be required to fully exploit the changes coming from VMware and its ecosystem. Identifying key skills such as process design, virtualization expertise, data movement, disaster recovery, VDI, etc are crucial to project success and often outsourcing key components of these tasks, even to simply bring in an outside perspective, is prudent.
Action Item: VMware puts significant demands on storage infrastructure. Senior IT staff with responsibilities for storage should specifically include VMware storage integration in storage RFPs and rank it as important in assessing storage alternatives for VMware environments.
Primary researchers on this project were David Floyer, Nick Allen and Stuart Miniman
- Integration Points and Scoring Research
- Definitions of vSphere 4 Integration Points
- Wikibon User Survey Results
- VMware Storage Leaderboard: Who Has the Best Integration?
- The Value of the VMware Integration Journey (2010)