Storage Road Map

Before Storage Area Networks (SANs) were introduced in the 1990s, block storage was directly attached to servers. At the same time, network-attached storage (NAS) was applied only for very low performance workloads. As a result, it was very difficult to utilize storage well, and moving information around to meet availability or capacity needs was next to impossible. The introduction of SANs and high function arrays, along with the emergence of high-performance NAS and Ethernet networks separated storage management from server management, and enabled much higher levels of performance, availability and utilization to be achieved. Sophisticated tools have been developed at the storage level to monitor performance and provide additional storage services such as copy and replication services.

Figure 1: Description of Virtual Infrastructure Storage Roadmap
Source: Wikibon 2010

The introduction of virtualization on servers has meant significant changes to the storage subsystem. The roadmap for integrating storage into the virtualized infrastructure journey has to focus on greater flexibility to meet rapidly changing demands. Storage itself needs to be virtualized to make storage invisible. The days of carving out LUNs, masking, and presenting storage to hosts, or configuring NAS and mounting filesystems for use in virtualized environments are coming to an end as storage and virtualization become increasingly coupled. VMware has announced the vStorage APIs for Array Integration (VAAI) which will offload storage services to the array, and paves the way for further VMware/storage integration.

The roadmap should include:

Resource Virtualization

• • Hiding of LUN and filesytem management. This abstraction is not relevant for virtual machines, and needs to be hidden from all storage operations.
  • The physical and logical mapping of storage has to be available at the vApp level rather than the “container” (datastores in VMware environments). This needs to be applied and enforced at the physical storage level, so that problems in IO can be analyzed automatically by storage administration tools but applied to the pertinent object – the vApp, not the storage object..
  • Ability of offload storage services to the storage array, using the vStorage APIs for array integration. These offloads can dramatically accelerate common tasks like VM provisioning, cloning, snapshots, replication. They can also improve VMFS scaling by an order of magnitude by hardware-handling of consistency locking mechanisms.
  • Transparent support for thin provisioning and automated performance tools. The virtual machine and OS need to understand when additional resources need to be allocated and allow “graceful” waiting while additional resources are added. For example, thin provisioning or tiered storage movements should result in a timeout at the application level; avoiding application crashes.
  • Integrated storage visibility and provisioning directly in the vCenter context, and element management that integrates with the vCenter APIs

Application Encapsulation

• • Integration of storage metadata at the vApp level, including volatile capacity (RAM/cache), persistent flash and disk capacity, temporary storage and storage latency.
  • Integration of system recovery metadata (RPO & RTO) at the vApp level, and integration with array-based remote replication services.
  • Provisioning of multi-pathing capabilities seamlessly between VMs, guest OS and vApp and the storage infrastructure (This will need close integration at the Hypervisor level).
  • Internal Location Independence
  • Ability to move applications and data across the infrastructure non-disruptively.
  • Ability to automatically provide the level of service required as defined at the vApp level, and dynamically allocate resources across the storage infrastructure to meet those service levels across many virtualized systems.
  • Ability to implement and exploit an active/active topology for remote and local high-availability storage services.
  • Integration of distributed lock monitoring into vApp metadata.

External Location Independence

• • Integration of data security (e.g., encryption) services for data in flight and data at rest.
  • Integration of storage security into VM system security that can be monitored at the vApp level.
  • Infrastructure capabilities to support geographic datacenter dispersion and hybrid cloud import/export will become increasingly important over time.

Action Item:

Footnotes: This research is an expansion of a section of research looking into The Value of the VMware Integration Journey