System and networking specs rate computer performance according to bandwidth and clock speed and ignore latency. Latency is the time that elapses between a request for data and its delivery. It is the sum of the delays each component adds in processing a request. Since it applies to every byte or packet that travels through a system, latency is at least as important as bandwidth, a much-quoted spec whose importance is overrated. High bandwidth just means having a wide, smooth road instead of a bumpy country lane. Moreover, latency is actually a much thornier problem in a world where applications are broken into pieces and often distributed around the world.

Advocates of the Cloud Computing hoopla tend to reduce relationships between software components and their associated data as taking place within some idealized network abstraction. Such an abstraction is a useful concept, but it does tend to de-emphasize how parts interact in the real, physical world.

Even a lot of staunch Cloud Computing advocates who liken using processor cycles out of some network grid to a computing version of the electric utility generally concede that storage is a trickier problem. Whereas computing is something you just consume, data has state. And if you lose that state, the data is gone. This is a fundamentally more serious problem than losing access to a compute utility for a few minutes or even an hour. For this reason, internal storage clouds will likely become way more popular than external storage clouds.

We tend to run applications close to the data they operate on for a reason: Application performance is often largely a function of how quickly it can read and write the data that it’s working on. And data stored on a local hard disk can almost always be accessed faster than that same data sitting at the other end of a network pipe hundreds or even thousands of miles away.

Thus, if storage stays inside organizations, a lot of the processing of that data will stay inside as well. And the general trend towards more data-intensive modeling and mining only strengthens this relationship, because ""latency matters more than ever"" in a world where the pipes are distributed networks

Local disk subsystems these days typically deliver data response times of five milliseconds or less. What does the Internet yield? To research this we conducted a few simple tests. To sample typical Internet delays, we pinged 15 of the most popular sites as listed by Alexa once a second for a period of one minute with the following results:

• Average Latency 72 ms
• Maximum Latency 142 ms
• Minimum Latency 25 ms

Interestingly, more than half of the larger latencies were encountered en route in the Internet and not by the targeted sites, usually by some one or two routers that seemed to be having a bad day. This is an all-too-common occurrence.

While ping times are practically immeasurable on small LANs (i.e., less than one ms.), bandwidth and protocol overhead dominate. To explore, we copied a one gigabyte file first locally and then over our LAN. All desktops were running Windows XP Pro, had CPUs in the 1 GHz range and 7200 RPM disk drives. The LAN was operating at 100 MHz. The results were:

• File copy disk to same disk 215 seconds
• File copy disk to a LAN attached desktop 420 seconds

So, just to copy this file over a local network to a system only five feet away took almost twice as long. From experience, we know that the time would be significantly better, but still not great, over a 1-gbit network. This is often the reason users shun iSCSI for higher performing applications, turning to Fibre Channel SANs or direct attach disk instead.

These simple tests illustrate that latency on the Internet is 100-1000 times higher than local networks. Most disconcerting is that latency in the Internet is highly variable and unpredictable. For the most part, users will tolerate slow, consistent response times and not want to deal with the unpredictability of inconsistent ones. What’s more, getting tech support for a slow Internet is like going down a black hole.

Action Item: Don’t forget that the best I/O is no I/O. Cloud storage players need to reveal how they deal with high and extremely variable latencies. Users should approach cloud storage with informed skepticism.

Footnotes: