Wikibon Blog

Current network topologies are inadequate to meet the flexibility and scalability demands of burgeoning virtualized data center environments.  New switches and new network architectures are emerging that transform the data center to Infrastructure 2.0 (comment or edit a vendor-independent definition of Infrastructure 2.0 on the wiki).  Users should be aware that moving to this new environment is a disruptive, rip and replace initiative that requires substantial planning.  Despite this caveat, a modernization process provides the opportunity to streamline current siloed infrastructure spanning network and servers in a virtualized context.

Participate in the upcoming Peer Incite call Beyond Spanning Tree Protocol on July 27, 2010, Noon EST.  It is a chance to learn more about one aspect of Infrastructure 2.0 technology, hear what others in the community think and get your questions answered.

Traditional Networks

While not identical, the changes required for network architectures in many ways parallel what we saw in the server space which led to the adoption of server virtualization.  Let’s look at two of the current network architectural practices that need to be changed for customers that want a data center designed for virtualization and cloud adoption:

1. Oversubscription
2. 3-tiered architecture

Oversubscription: Server virtualization helped take resources which were under-utilized and consolidate them into fewer devices that had a much higher-utilization.  Similarly, network architectures are under-utilized (oversubscribed) due to Spanning Tree Protocol (STP) which ensures a loop-free topology by disabling those links that are not part of the spanning tree, leaving a single active path between any two network nodes.  Since only a single link is active between nodes, the total bandwidth of the network can be significantly oversubscribed (see diagram on the right provided by Cisco for a “typical” environment).  Switch architectures were designed with limited bandwidth to support these oversubscribed configurations.

3-tiered architecture: Pre-virtualized server environments were siloed by application, similarly, traditional network architectures allow for physical and logical isolation of applications.  The predominate network architecture is to have an access layer, aggregation/distribution layer and core layer.  This architecture is designed for “north-south” traffic which means that data and services are mostly going from the access layer to the core, rather than between access layer devices which would be “east-west” traffic.  Three-tiered architectures initially became popular as a means of balancing network utilization, performance and flexibility. However, as was the case with servers, virtualization provides an opportunity to streamline the infrastructure.

Virtualized Networks

As organizations pursue server virtualization, the requirements for the network have changed since higher-utilized servers translate into higher network bandwidth needs.  Rather than the 3-tiered, oversubscribed solution, the new generation of switches that are being built for virtualized environments are non-blocking (i.e. bandwidth to support fully utilized ports) and being deployed into a flatter network architecture (technically 1-2 tiers depending on vendor).  Mobility of applications between servers (such as with VMotion) is an activity which requires high bandwidth “east-west”, which is not easily handled by traditional 3-tier solutions.  Spanning Tree Protocol is being supplemented or completely replaced with technologies that allow for multipathing and redundancy at layer 2.  The flatter architecture allows for a more flexible flow of traffic both “north-south” and “east-west” (see diagram on the right provided by Cisco for a 2-tiered, non-blocking environment).  The switching environment becomes virtualized both through managing multiple switches as a single pool and by blurring the boundaries between the physical switching infrastructure and the virtual switching environment that is part of hypervisors.  Unlike virtualization in servers and storage, the transition to network virtualization is a replacement rather than an extension of existing infrastructure.

Last week, Cisco unveiled the products and vision for a 2-tiered architecture.  Their solution is FabricPath, which is software and the first instantiation of the product is on the new F-series module for the Nexus 7000 series switch.   From a standards perspective, Cisco is calling FabricPath a “superset” of TRILL.  TRILL (Transparent Interconnection of Lots of Links) is an IETF standard which when ratified (expected soon), provides an option to replace STP.  Leading the TRILL standard is Radia Perlman who created STP and brings increased credibility to TRILL.  While Cisco is participating in the standards effort and says that the Nexus and F-series will support TRILL, in typical Cisco fashion, the company is blazing its own trail, trying to create a de facto advantage in the marketplace.  In fairness, other leading vendors are not on the TRILL bandwagon, however Cisco has wrapped itself in the standards flag as a marketing tactic. The reality is that Cisco is trying to catch up with Juniper and HP (3Com) which already have innovative flatter architectures designed for virtual environments.

Transitioning Your Network

CIOs today are looking to get more out of their existing resources, but find themselves with 3-tiered architectures built with older switch technologies (like Cisco’s flagship Catalyst switches) that can not meet the flexibility and scale that they want in a virtualized data center.  Updating the network is an expensive and disruptive process with many transitions including:

1. New hardware – not just adding 10GbE (typically requiring new cabling also), but moving to non-blocking switches
2. New architecture – from 3-tier to 2-tier
3. New processes – not only all of the new management, but organizations should look at reporting structures and team interactions. Specifically, a non-siloed virtual network is going to be better served by a less siloed network, server and storage teams (especially when considering network convergence such as with FCoE).

Cisco’s newer Nexus family are non-blocking switches, but even if companies have made the investment in Nexus, they will need to spend more money and re-architect their environments if they add FabricPath when it is available in Q3 ’10.  Both Juniper – with its Project Stratus and “3-2-1″ data center architecture – and HP (3Com) – with its Intelligent Resilient Framework (IRF) – have alternatives that companies should evaluate when making the move to a 10GbE next generation of switches and architecture. Cisco, HP and Juniper have dramatically different approaches to architecting nextgen networks. In short, Cisco wants to maintain its substantial lock-in advantage, HP wants to bomb pricing and Juniper wants to disrupt everything so it can steal share. Customers should understand that no matter which path they choose for virtualizing networks, they must plan for disruption and look toward developer-friendly, multi-vendor, best-of-breed solutions to minimize lock-in.

Customers considering updating their systems need to balance their asset management cycles (installed base), technology cycles (adopting innovations such as Nexus or non-Cisco) and business cycles (especially capital budgets).  As such, it is recommended that customers pilot the new configurations to determine the impact of new architectures on their stack and on change control and management practices.

Conclusions

All vendors are racing to bring virtualized networks to market and migration to the new paradigm is inevitable.  Cisco, HP (3com) and Juniper all have solutions in various stages of readiness. Cisco is the big dog and has the most to lose in a transition phase. Network managers should avoid expensive extensions to existing networks and implement trials of the new paradigm.  They should make the planning assumption that the business case for migration to a 2-tier architecture will become overwhelming in the next two to three years. Rip and replace decisions are not easy choices for customers and expose Cisco in particular to transitional risks.  These are three key developments that observers should watch for indicators of success in the coming transition:

1. Cisco’s ability to affect rapid adoption of converged network, compute and storage architectures with UCS and FCoE
2. HP’s skill at delivering a truly converged offering of networks, servers and storage to become the clear number two player
3. Juniper’s capacity to leverage an ecosystem of cloud service providers to reach enterprise customers