Research done in collaboration with David Floyer
See the community feedback from the CrowdChat on Server SAN (#serversan) that was held on February 5, 2014
Introducing the Server SAN
The largest Web-based companies in the world have proven a new model for managing and scaling a combined architecture of compute and storage. The hyperscale companies, such as Facebook, Google and Amazon, use a design of local distributed storage. The key differentiation of this new architecture is the ability to be extremely scalable and simple to manage. The advantages of this architecture can also benefit enterprise and service provider customers. Wikibon believes that this new category, which shall be called Server SAN, is an important component of software-led infrastructure. This paper will start the discussion for defining this new category and explaining how it fits into the IT marketplace.
Server SAN architecture sits at the intersection of the hyperscale, convergence and flash trends. The need to share and manage large volumes of data spawned the creation of storage arrays as separate devices and networked storage (including DAS and SAN). With the emergence of flash for enterprise solutions, storage intelligence has been moving back closer to compute. Converged infrastructure and hyperscale architectures have simplified operations by creating repeatable rack level deployments. The advent of high speed, low latency networking, flash, continued explosion of data and new application designs bring forth an opportunity to redesign the IT infrastructure.
Defining Server SAN
Server SAN is defined as a combined compute and pooled storage resource comprising more than one storage device directly attached to separate multiple servers (more than one). Communications between the direct attached storage (DAS) occurs via a high speed interconnect (such as InfiniBand or Low Latency Ethernet), where coherency is managed by the software of the solution. Server SAN multi-protocol storage can utilize both spinning disk and flash storage. Ideally, Server SAN is configured in enterprise applications to ensure high availability.
|Architecture||Diagram||Advantage||Disadvantage||# of Apps||Network Overhead|
|DAS||Performance||Storage not shared, limited scalability||Some||Low|
|Networked Storage (SAN/NAS)||Pooled resources, lots of apps, lots of data||Cost and complexity||Many||High|
|Cloud Storage||Infinite volumes||Each application is separate||Each separate||Very High|
|Hyperscale||Highly scalable, operational simplicity||Requires significant effort to design||Single||Low|
|Server SAN||See benefit section below||See challenges section below||Many||Low|
Figure 1 lays out how each of the existing architectures deliver performance and functionality. Figure 2 shows how Server SAN fits into the discussion; the technology and available solutions will go through multiple phases of maturity over time (phase 1 today, phase 2 in 1-2 years and phase 3 in 3-5 years).
Characteristics and Benefits of Server SAN
Many architectural variations will fit under the category of Server SAN. The management of storage should be simply as “storage”, in other words the administrator should not need to worry about LUNs, ports or switch configurations. The system will understand performance characteristics such that there do not need to be knobs or an understanding of RAID types. The solution must support multiple applications and volumes; software should be able to optimize the total throughput of the system or of a single tenant or application. Some architectures will deliver the ability to dynamically migrate data to applications or applications to data; this ability is impossible to do with traditional networked storage that separates the compute and storage. Performance can be lower than a traditional SAN when that data is near processors with minimal network latency. Server SAN moves beyond the performance and capacity limitations of DAS. Support of both disk and flash broadens the applicable application scenarios. The potential for highest performance is with PCIe Flash and atomic writes.
The litmus test for whether a solution fits into the Server SAN category comes down to the software, hardware support envelope, ability to scale-out and simplicity of the solution. Software is the core of the solution, and from a storage standpoint it will be measured by the advanced services that can be offered. While hyperscale providers can spend significant time to design or even hyper-specialize an architecture, enterprise users need solutions that deliver functionality with a simplicity that does not require a team of PhDs. A differentiation from other "software-defined storage" is that compute must be part of the configuration, rather than as separate nodes. A true scale-out architecture provides the ability to seamlessly add capacity and performance as needed without any manual configuration or management intervention for either adding or removing drives or nodes. As for hardware, Server SAN can leverage commodity compute-based architecture; this means that there is an elimination of an external or separate storage array. A convergence of compute and storage allows for lower latency and simplicity of deployment and management. Many ISVs, including Microsoft, have a strong preference for DAS; Server SAN can meet this requirement without losing the benefits of being networked.
Server SAN architectures can deliver on cost advantages based on the simplicity of the solutions. Convergence allows for the collapsing of silos and automating storage placement and performance, which leads to operational savings. The elastic nature of the solution also allows for reduction or elimination of migration costs between generations of arrays, which according to Wikibon research is more than 20% of the lifetime cost of the array. By having the environment be a pool of resources, the useful life of storage assets can also be extended. Additionally, software licenses are typically perpetual compared to the fixed life of a hardware array. Overall infrastructure costs of the combined server and storage should be lower than solutions utilizing a traditional storage array. Costs and elasticity of the environment should compare favorably to hyperscale solutions.
Challenges for Server SAN
The first metric that users will look for from a Server SAN solution is that it delivers on the simplicity of usability and scalability. In order to cross the chasm beyond early adopters, the solution will need to rapidly add functionality (as shown in Figure 2) while delivering on cost benefits and find customer adoption in a broad spectrum of use cases. In order for the new market segment to gain footholds against multiple existing alternatives, it will need multiple strong competitors to drive awareness and adoption. The vendors will need to work closely with server partners such as Intel, Dell, Cisco and Quanta. Ideally, the software will develop to understand the sophisticated knowledge of physical and latency topologies via advanced metadata.
Figure 3 shows the starting market landscape for Server SAN. While the solutions have been segmented on the diagram, the line between appliances and software-only solutions are blurring greatly. Nutanix, SimpliVity and Scale Computing have been called “hyperconvergence” vendors (dig deeper into these solutions in this Wikibon research note), and these solutions deliver the convergence of compute, storage, networking and management in a single device rather than multiple products that are packaged together. The IP for all of these solutions is in the software rather than hardware and some of the solutions can be deployed in ways other than an appliance shipped by the vendor. While some of the marketing from the hyperconvergence vendors has been to “kill the SAN”, these new solutions bring add simplicity and lower cost on top of the value proposition of functionality and networking that SANs delivered.
While the appliance solutions can sometimes ship as software, the software-only solutions do not simply ship as only code to customers, rather the software is part of the overall deployment. Nexenta and Sanbolic work with channel partners to deliver fully tested solutions. EMC ScaleIO can be expected to be added into the VSPEX program and Wikibon would look for EMC to deliver additional offerings through its partner and channel programs. VMware VSAN is still in beta, so any go to market details are not available. To deliver simplicity, VSAN and other software-based solutions should have prescriptive guidelines of hardware options so that interoperability and performance testing by users can be minimized. The last items listed are the data systems which are a combination of protocols and architectures that fit into the Server SAN market landscape.
Requirements and Differentiation of Server SAN Solutions
As users evaluate Server SAN solutions, here are a few of the characteristics that should be compared and contrasted:
- Environment support
- Virtualization (VMware, HyperV, KVM)
- Physical (non-virtualized environments)
- Media support (Flash only/Mix flash+disk)
- Cloud integration
- Management/API support for the storage layer
Action Item: The investment that companies have made in people, processes and technology of traditional storage infrastructure is slowly going to die. Customers do not need to rip and replace what they have, but they do need to learn about these new architectures and consider how new applications and new infrastructure models like Server SAN can fit into future deployments. Wikibon believes that a large percentage of applications will leverage Server SAN architectures by the end of the decade. Wikibon welcomes the input of the community on this emerging space and will provide additional analysis on Server SAN in the coming months.
Footnotes: This market definition was originally published on January 8, 2014, with subsequent edits from the Wikibon community. Comments and edits are welcome.