Moving Big Data at Maximum Speed Over Wide Area Networks

In this digital world, fast and reliable movement of digital data, including massive sizes over global distances, is becoming vital to business success across virtually every industry. The Transmission Control Protocol (TCP) that has traditionally been the engine of this data movement, however, has inherent bottlenecks in performance (Figure 1), especially for networks with high round-trip time (RTT) and packet loss, and most pronounced on high-bandwidth networks. It is well understood that these inherent “soft” bottlenecks are caused by TCP’s Additive-Increase-Multiplicative-Decrease (AIMD) congestion avoidance algorithm, which slowly probes the available bandwidth of the network, increasing the transmission rate until packet loss is detected and then exponentially reducing the transmission rate. However, it is less understood that other sources of packet loss, such as losses due to the physical network media, not associated with network congestion equally reduce the transmission rate. In fact, TCP AIMD itself creates losses, and equally contributes to the bottleneck. In ramping up the transmission rate until loss occurs, AIMD inherently overdrives the available bandwidth. In some cases, this self-induced loss actually surpasses loss from other causes (e.g. physical media or bursts of cross traffic) and turns a loss-free communication “channel” to an unreliable “channel” with an unpredictable loss ratio.