Slashdot Mirror

← Back to Stories (view on slashdot.org)

Fast TCP To Increase Speed Of File Transfers?

Posted by simoniker on from the turbocharge-your-interweb dept.

Wrighter writes "There's a new story at Yahoo about a new version of TCP called Fast TCP that might help increase the speed of file transfers. Sounds like it basically estimates the maximum efficient speed of your network, and then goes for it, dumping a lot of time-consuming error checking." There's also an article at the New Scientist with some additional information.

15 of 401 comments (clear)

  1. Re:Isn't this called UDP? by Ark42 · · Score: 4, Interesting

    No, UDP has error checking per packet via a checksum. What they are talking about is probably something to do with TCP "slow-start" where TCP connections speed increases slowly so as not to flood the network at first. I think the speed starts out exponentially with each packet then backing down some when packets are dropped.

    --
    Morphing Software
  2. zmodem??? by case_igl · · Score: 5, Interesting

    I remember back in my BBS days what a big deal zmodem was when it started getting used all over the place. As I recall, it changed the block size that you would receive dynamically based on line quality.

    So when you sent a block of 2k and got no errors, the frame size increased to 4k...8k... etc etc... Sounds like a similar approach.

    Case

    P.S. That was a long time ago in a FidoNet far far away, so my terms may be off.

    1. Re:zmodem??? by joshuac · · Score: 4, Interesting

      Actually, the Zmodem that was widely used (real zmodem) maxed out at 1k blocks, but it would steadily scale down to as small as 16 byte blocks (if I recall correctly).

      There were variants that did 8k blocks (and often referred to themselves as Zmodem8k), but none of these were true zmodem protocol.

      Still, nothing can be quite as fast as ymodem-g :)

      A little more on topic; what they are describing does not dynamically scale the packet size, only dynamically adjust the transmission speed up to the point that ack's start slowing down, but (hopefully) before any packets actually get dropped. I suspect disney and such will be quite disappointed if they think they are going to get a 6000x speedup in practical use as hinted at in the articles. Perhaps a 10% speedup for joe blow on a dialup modem, _maybe_. Take a look at your connection some time when downloading a file; you will probably find you can already peg your bandwidth quite nicely.

    2. Re:zmodem??? by G27+Radio · · Score: 4, Interesting

      I used to run an Apple II BBS/AE in the mid to late 80's (201). X-modem was king when I started. But Y-modem and then Z-modem surpassed it.

      X-modem transmitted files as 256 byte blocks of data along with an 8 bit checksum (IIRC.) The receiver would respond with an ACK (Acknowledgement) or a NAK (Negative Acknowledgement) after each block. If it was a NAK the sender would re-send the block. If it was an ACK it would send the next block.

      Y-modem increased the block size to 1k which was helpful since the turnaround time between packet and acknowledgement was wasting a lot of time. It also used a 16-bit CRC (Cyclic Redundancy Check) instead of an 8-bit checksum. Apparently the CRC was much more reliable.

      Around the time that error correcting modems started becoming popular (USR Courier 9600 HST) a variation of Ymodem popped up called Ymodem-G. Ymodem-G would send 1k-blocks with CRC's non-stop without waiting for an ACK. If the receiver got a bad block it would simply abort the transfer and you'd have to start it over.

      Zmodem would also send blocks and CRC's non-stop unless it got a NAK back. It would resume sending at the block that caused the NAK. The variably sized blocks were pretty cool too.

      Feel free to correct any errors. It's been a long time.

  3. Re:Isn't this called UDP? by Ark42 · · Score: 4, Interesting

    The article seems kinda stupid to me, it describes a basic "stop-and-wait" protocol where only 1 packet can be in transit at a given time, and if it gets lost, it is retransmitted. I am pretty sure normal TCP has a window where it can send up to X packets at once and retransmit any particular missing one. I am sure there is room for improvement, but TCP is a fairly complex protocol already and the article seems to forget about all that.

    --
    Morphing Software
  4. smells like... by wotevah · · Score: 4, Interesting
    When the researchers tested 10 Fast TCP systems together it boosted the speed to more than 6,000 times the capacity of the ordinary broadband links.

    Does that mean TCP has 99.99% (humor me) overhead ?

    But seriously, you can probably use large windows to send streams of packets such that a single ack is required for a bunch of them, but it's impossible to achieve 6000x more throughput just by "optimizing" the TCP protocol. Even over Internet (I'm not even talking LANs since there is obviously not that much room for improvement due to the low latency).

    1. Re:smells like... by wotevah · · Score: 4, Interesting
      It's lame to respond to my own post, but the other article points out that they actually used a different architecture where TCP achieved 266Mbps and their optimized version got 925Mbps, which the author chose to compare with broadband speeds (6000x the capacity of broadband).

      Still, those numbers don't look right. AFAIK TCP has 5-15% overhead, so they must have been using a high-bandwidth, really-high-latency line to get that much improvement. Really high.

      Under these conditions (that obviously are unfavorable to TCP) I would be curious to see how "fast TCP" compares to any real streaming protocol (UDP-based with client feedback control). I have a feeling that the UDP stream is faster.

  5. Nothing to see here, move along..... by trinity93 · · Score: 4, Interesting

    Looks like this this

    SCTP is a reliable transport protocol operating on top of a connectionless packet network such as IP. It offers the following services to its users:

    -- acknowledged error-free non-duplicated transfer of user data,
    -- data fragmentation to conform to discovered path MTU size,
    -- sequenced delivery of user messages within multiple streams,
    with an option for order-of-arrival delivery of individual user
    messages,
    -- optional bundling of multiple user messages into a single SCTP
    packet, and
    -- network-level fault tolerance through supporting of multi-
    homing at either or both ends of an association.

    The design of SCTP includes appropriate congestion avoidance behavior
    and resistance to flooding and masquerade attacks.

    --
    We substituted the coffee Slashdot normally drinks with "Sandoz Crystals", Lets see if they notice the difference
  6. Re:Isn't this called UDP? by subreality · · Score: 4, Interesting

    #1. No. UDP has error checking. The difference between UDP and TCP is that TCP is a connection-based, sequence-enforcing protocol, where UDP is basically raw connectionless datagrams that arrive in any order and you have to handle packet loss and reordering in your application.

    #2. RTFA.

    #3. They're not getting rid of error checking. It sounds like they're reworking the windows for ACKs in TCP to allow better streaming over high speed, but realistic (IE, slightly lossy) networks. Current TCP aggressively backs off when packet loss is detected, to prevent flooding the weak link in a network connection. It works really well for consumer network speeds, but on very high speed networks (EG, 45 Mbps), even very light packet loss will drop your speed dramatically down. TCP just wasn't meant to scale to these kinds of speeds, and some reengineering needs to be done to make it work smoothly. Many of the current extensions to TCP have made matters a lot better, but it's still going to have trouble scaling to gigabit, high latency networks, and it's best to start dealing with these issues early.

  7. Caltech Site by mib · · Score: 4, Interesting

    This is part of a whole bunch of TCP and networking related work at CalTech.

    I hate to do this to them, but the Caltech Networking Lab site has more info.

    From what I see, the improvement here is to use packet delay instead of packet loss for congestion control. They claim this has a bunch of advantages for both speed and quality.

    Here is a Google cached copy of their paper from March 2003.

  8. Fast TCP is TCP + congestion control by po8 · · Score: 4, Interesting

    As near as I can tell from the popular articles, and the web page referenced in the New Scientist article, "Fast TCP" is not a new protocol, but rather better congestion control for standard TCP. I'm not a network guru by any means, so please take the comments below with a grain of salt.

    Currently, TCP implementations use a standard trick to play nice with small router queues. Using precise timing would be better. I hassled Mike Karels over it about 10-15 years ago, but the consensus at the time was that the hardware wasn't up to it. Now it is. Also, modern routers have gotten clever about queue management, which screws up the trick.

    The new proposal is to take advantage of modern HW to measure latencies. Existing TCP could thus be used more efficiently, by allowing larger amounts of data to be outstanding on the network without trashing routers.

    It is not widely understood that in 1988 the Internet DOSed itself because of a protocol design issue, and Van Jacobsen got everybody to fix it by a consensus change to the reference implementation of TCP. These articles appear to report (badly) ongoing research into that issue.

  9. Article is inaccurate and misleading by zeds · · Score: 5, Interesting
    The New scientist writer clearly has no understanding of how TCP/IP or the Internet work in general and how Caltech's FAST could improve data transfer efficiency. His sensationalist claims that this could enable downloading a dvd in seconds are so much ignorant crap. 6000x faster than broadband? That has more to do with the fact they used an INCREDIBLY FAT PIPE (a 10gigabit connection), probably in a laboratory setting, than any of FAST's optimizations. It's true TCP/IP's efficiency maxes out at a certain rate, but that doesn't really matter in the real world, because nobody is actually downloading movies from dedicated 10gigabit links to the backbone. Not to mention that you won't see anyone serving anything at these speeds for the next decade or so. I wonder what this suggests about the accuracy of articles on subjects I know nothing about. It's an academic curiosity folks.

    See caltech's press release on FAST for an article that actually makes sense.

    Also, could someone please explain to me why boringly predictable stereotypical slashdot feedback is being modded up?

    "Whoa! Faster pr0n!"

    "Imagine a beowolf cluster of these!"

    -Insert completely unrelated Microsoft bashing post here-

    -Insert completely unrelated technobabble from some geek posting out of their ass (without reading the article first)-

    News for nerds. Stuff that matters. Discussion that doesn't.

  10. Nothing new here by Vipester · · Score: 4, Interesting
    Whoever wrote these articles is not the brightest crayon in the box. Their explanations of how "regular" TCP works and how FAST works are both exceedingly wrong. Read the FAST group's overview for an explanation of what they're doing. It's semi-heavy with technical networking terms but you'll learn that this has nothing to do with error checking.

    Congestion control based on roundtrip times is old news but is uncommon AFAIK. What really happens is direct feedback from routers along a transmission's path. This is done in TCP Vegas, which was first proposed in 1994 and I think is fairly common now. The problem with scaling this or any of the other common TCP implementations to high speed/high delay links is the reaction to detected congestion. "Normal" TCP aggressively scales back its send window (send rate) when it detects congestion, usually chopping it in half. The window/rate then grows linearly until something goes wrong again. This results in alot of lost throughput in high-speed networks especially if the amount of "real" congestion is low. The FAST group is working on a new TCP implementation that doesn't react so aggressively to congestion. This is great for those high-speed/low-congestion networks we all wish really existed but is not something you want to use on the always-backed-up Internet. Would probably make things worse.

  11. Eliminating 'burstiness' of TCP/IP by Daniel_ · · Score: 5, Interesting
    If I'm reading the article right, they're using the same technique that a doctoral candidate did his Phd thesis on at OSU about 3 years ago.

    TCP is extremely bursty - it pumps all the packets it can as fast as it can over the network as soon as the window opens. Then it waits for replies to all the packets. What typically happens is the burst from the NIC overloads the local router causing numerous dropped packets. This gives the imporession to the sending machine that the network is overloaded and results in a ~90% reduction in bandwidth utilization.

    The change is to include a timer that allows the NIC to space the initial burst over the entire window. This prevents the overloading at the router and permits the NIC to reach near its theoretical maximum bandwidth.

    In tests involving one router, the results were an order of magnitude increase in bandwith utilization. I'd be interested in seeing their test setup to see how they got such dramatic improvements. Normally TCP/IP is not that ineffecient - even with its extreme burstiness.

    --
    The number you have dialed is imaginary, please rotate your phone 90 degrees and try again.
  12. Re:6000 TIMES !!! by elem · · Score: 5, Interesting

    Actually if you read the New Scientist artictle you can see that that's a lie. What they actually did was bundle 10 FastTcp connetions (one must assume on fast lines) togeather and, fairly unsuprisingly, got speeds 6,000 times fast than a broadband connection... wow... 10 high speed lines are faster than broadband??

    This would be more interesting had they actually tested it on a standard 512kbs connection and seen if there was a speed increase. IMO it most likely would not make a huge a difference anyway since alot of the slowdown on a consumer broadband connecting is the connection buffers at your ISP. For a better explanation read the Traffic Shaping HOWTO.