On March 20th, I had the privilege of joining the Wikibon Peer Incite: "The Rise of 10Gb Ethernet and the Impact of Intels Xeon E5 Family of Processors". This was an important discussion that drilled down on many aspects of 10Gb Ethernet, it’s speed advantages over the existing 1Gb Ethernet environment, and HP’s Flex-LOM architecture that makes it easier to choose the right time for upgrading to faster speeds.
Another aspect of the 10Gb Ethernet upgrade cycle is the impact it has on how data centers are now being built. The traditional 3-tier data center (presentation tier, application tier, and database tier) leaves server resources in islands that are optimized for north-south networking traffic only. This means that each tier talked to its adjacent tier, but not to servers in the same tier (this is called east-west traffic). Reconfiguration to adapt to changing workloads in this kind of data center is a physical exercise which is both labor and time intensive. Read this as limited flexibility with a lot of operating expense (OpEx) when a change is needed.
This traditional three-tier data center is rapidly giving way to the construction of “virtual” data centers, where all the servers are fully connected via 10Gb Ethernet in a “flat” network, that is one with fewer network tiers. This not only simplifies network construction but allows ANY set of servers to be configured in any ‘logical’ tier of the data center as the need arises. Ultimately, this eliminates compute islands, increases flexibility and lowers OpEx. This is one of the biggest lessons learned from utility/public cloud computing architectural practices: Fast, fat, and flat networks save time and money.
Action Item: One of the revolutions of the rise of 10Gb Ethernet in the data center is to simplify data center design by avoiding inflexible and expensive reconfiguration in order to adapt to today’s changing work loads. Fully connected 10Gb Ethernet networks deployed all the way to the edge of the network (inside modern servers) accomplishes long-term cost savings by eliminating un-needed “physical” network tiers and compute islands whose resources can’t be easily shared or reconfigured.
Footnotes: Greg Sherer is the Vice President of Server and Storage Strategy for Broadcom Corporation
