Fast TCP To Increase Speed Of File Transfers?

Wrighter writes "There's a new story at Yahoo about a new version of TCP called Fast TCP that might help increase the speed of file transfers. Sounds like it basically estimates the maximum efficient speed of your network, and then goes for it, dumping a lot of time-consuming error checking." There's also an article at the New Scientist with some additional information.

Comments

Re:stupid non network guy question

[Poofat]

Thats what TCP does, just in a passive fashion. The sender queues all sent packets for resend, (on a delay) and packets are deleted from the queue when an acknowledgement for a range of packets is recived from the remote computer.

If you're curious: RFC 793

New Scientist didn't put it very well...

[nweaver]

Looking at the information on their web page at caltech, the FAST network project is working with alternate TCP window sizing schemes.

Namely, instead of reducing window size in the case of packet loss, window size is changed based on round trip latency. The problem being that reducing the window size in response to loss works well on most networks, but has a serious problem when dealing with very high-bandwidth links.

In such a case, the conventional TCP windowing will shrink greatly in response to even one or two lost packets, which when you are sending a LOT of data, will occur.

The real work (and it seems to be somewhat covered in their web pages) is how to use latency for congestion detection/control, but I haven't read it in enough detail to quite understand this, NOR how this scheme will interact with conventional TCP streams.

Re:New Scientist didn't put it very well...

[stj]

Well, as far as I remember, there were more problems than that.

The problems with very high bandwidth links, TCP and RTT estimation start from the fact that TCP can do that estimation just every ACK received and with very high bandwidth links it changes much faster and in greater degree. So, TCP can't efectively estimate the available capacity, since it cannot probe the channel frequently enough.

Caltech's Vegas looks great on pictures, however, there were papers pointing out that it's not exactly fair, especially with multiple bottlenecks of real-world topology. Then there were papers fixing that, and papers critisizing those solutions, and as the result I don't see Vegas anywhere around (except for some Cisco routers maybe) - the best I see is NewReno+SACK+FACK+ECN. I can imagine that more aggressive scheme will have an advantage over TCP, although NewReno is pretty aggressive if compared to most RT rate control schemes, so it's difficult to imagine anything more aggressive than that, that would yield in times of congestion.

The best description of what they really propose seems to be in their Infocom's paper from April this year. That looks pretty good, too. But again, as it was with original Vegas, it will probably come out that it has some flaws, they will be fixed, the fixes will have some flaws, and so on. And for the time being everybody will continue to use NewReno. *snicker*

Fact is that there is enormous (partly bad) experience with using TCP Reno, and with current abundance of capacity in the backbones, it doesn't seem that there is much on an interest in precise traffic control. I've got to have my first problem watching some movie trailer, yet. ;-)

One thing worth mentioning - no reasonable application uses TCP for multimedia (why Disney then?). RTP/UDP with a reasonable model-based rate control can easily at least match Vegas, and often outperform it because of the kind and amount of feedback used to adapt to the network conditions for particular application. Caltech's scheme was constructed for ultra highspeed networking and tested for processing of vast data volumes produced by LHC to overcome deficiences of traditional TCP in that case. They have a real nice article on experiments with that with good results. But that's not quite the same as typical situation.

Fixes problems with slow-start and resends....

[malfunct]

It looks like this protocol is more proactive in monitoring the line. It looks for clues that it needs to slow down before a packet gets lost. A great deal of time in a TCP connection is spent waiting for acks and resending data, this is made worse by the typical latency across then net (if I remember right it averages 30 to 800 ms for domestic connections depending on time of day, and between 700 and like 1200 for connections abroad).

This protocol figures out ahead of time if it needs to slow down so its always getting acks back instead of waiting for timeouts. Also it avoids the binary backoff time that happens with timeouts.

So in response to many of the previous posts it loses none of the robustness of TCP. In the worst case its as slow as TCP and in the best case it should be equally as fast as TCP. In the average case, however, it shows a huge performance increase. Most of the time on the network is the average case so this is a good thing.

Re:Isn't this called UDP?

[zcat_NZ]

I can only assume that the 'description' of how normal TCP works has been 'simplified to the point of being wrong' by reporters, because it is totally wrong. The description of 'why fastTCP is better' then proceeds to describe 'normal tcp as it actually works.'

Any then they totally confuse the issue by mentioning that you can use multiple high speed links in parallel to get higher overall bandwidth. Boy am I impressed.

Re:Isn't this called UDP?

[harvardian]

I see this as a way to transmit corrupted files, not a way to speed up the internet experiance as a whole.

Without trying to be mean, you see it that way because you don't understand what's going on (mostly because the post was misleading). Fast TCP packets will still have a checksum and everything, so you're not going to get corrupted files. The change here is that normal TCP halves its "window size", or the amount of info that's out on the network at once without receiving an acknowledgement of receipt, with each error. This means that if there's one minor slowdown when 10 packets are currently out from your computer to the recipient (you've put out 10 packets without getting an ACK back yet), then your computer will reduce its window size and only allow 5 packets to be out at a time, effectively halving the transmission rate. Since TCP continually tries to get faster, it will always hit a bottleneck, resulting in your connection vacillating between optimal speed and half of that (approximately, I guess it might be worse than this on high-speed networks based on what I've read here).

In Fast TCP, they do this "congestion control" in a different way. Rather than halving the connection speed with every slowdown to ensure stability, they send as much data as possible as long as the network seems clear on the recipient's end (I think they estimate this with round-trip time of some sort).

So the "error checking" being changed by Fast TCP is NOT bit checking -- it's transmission rate checking. You'll still always get your files intact.

It's not a breakthrough, but it's good work.

[Animats]

First, this has nothing to do with removing error checking. It's about better TCP window adjustment. Read the papers.

Second, it's intended for use for single big flows on gigabit networks with long latency. You have to be pumping a few hundred megabits per second on a single TCP connection over a link with 100ms latency before it really pays off. It won't do a thing for your DSL connection. It won't do a thing for your LAN. It won't do a thing for a site with a thousand TCP connections on a gigabit pipe.

Third, unlike some previously hokey attempts to modify TCP, this one has what looks, at first glance, like sound theory behind it. There's a stability criterion for window size adjustment. That's a major step forward.

(I first addressed these issues in RFC 896 and RFC 970, back in 1984-1985. Those are the RFCs that first addressed how a TCP should behave so as not to overload the network, and what to do if it misbehaves. So I know something about this.)

Re:Isn't this called UDP?

[harvardian]

FYI, a great website for understanding how TCP congestion control works is here. It explains how TCP additively increases its window size as traffic goes through okay but then halves its window size when it runs into a problem.

And I should clarify my first post as well by explaining what a "transmission error" is that would cause the window size to halve. From the article above:

It is rare that a packet is dropped because of an error during transmission. Therefore, TCP interprets timeouts as a sign of congestion, and reduces the rate at which it is transmitting.

Basically, what I mean by a "transmission error" is a timeout -- the sender sends a packet and never gets an ACK for it. TCP works on the premise that packets are mainly dropped when congestion is high enough for routers to drop packets because of maxed buffers. Thus it makes sense to reduce transmission rate when no ACK is received to adjust to the capacity of the network.

Nagle

[zenyu]

It sounds like they are working on a replacement for the Nagle algorithm. Nagle works well on clean connections, even if badly tuned slow-start gets annoying when you have gigabit connection and it still takes a minute to ramp up to full speed on an ISO download. Where "fast tcp" would really help is on a dirty connection. I had to connect to a supercomputer a few years ago over a 100Mbps link that corrupted or lost 30% of all packets and I had to use my own streaming on top of UDP to avoid getting hammered by shrinking windows (I still needed congestion control.) On this type of connection I'd expect their "fast tcp" might give a 10x speedup. On a normal non-noisy and relatively slow DSL connection, like I have at home, I'd be surprised by a 10% speedup.

In other words the story is all wrong, but what they are doing is actually worthwhile. You sometimes have noisy networks, especially when they are wireless or in an industrial environment. The big long haul telecoms lines are better off doing error correction on line, but in the last mile you never really know the noise characteristics so this should be handled better on the TCP level. I would probably do something like FEC with the number of recoverable errors per packet and per lost packet per logical block, tuned to the error characteristics of the network. Then call it TCP2 and release an RFC and some BSD licensed source code.. (I thought of doing this as part of building an ISP friendly P2P protocol but decided I didn't have the time..) Their solution has the advantage that it works just great with regular old TCP implementations.

Re:zmodem???

[polymath69]

Note, there still is, to my knowledge, nothing slower then kermit.

At the risk that you're trolling, Kermit is actually very good indeed (after 1990 or so,) assuming you set your options correctly.

The defaults are slow, but they work; that's Kermit's raison d'etre and why it's still around. But Kermit was probably also the first protocol to implement sliding windows and configurable blocksizes; Zmodem probably got that idea from Kermit. Set your options correctly, and Kermit's damn good.

The age of the BBS is over (I ran one for about 12 years) but I'm pretty sure I'll use Kermit again before I have cause to use Zmodem again.

Re:Isn't this called UDP?

[harvardian]

This is stretching my certainty (I'm recalling all of this from a sophomore year CS networks class and I've been overwhelmed by booze in my graduating week), but here's a stab at this...

I don't think that having a long RTT (round-trip time) will have a huge effect on transmission rate in the standard TCP case. Internet traffic, if I remember correctly, is bursty (cite), meaning that a typical transmission looks like:

SEND 32 PACKETS
RECEIVE 32 ACKS

SEND 36 PACKETS
RECEIVE 36 ACKS

SEND 40 PACKETS
RECEIVE 38 ACKS

(oops! Sent 2 more than I could! Halve TX window!)

SEND 20 PACKETS
RECEIVE 20 ACKS

etc. In this case it's easy to see why having a long RTT doesn't slow things down particularly, since there can be a big gap between the SEND and RECEIVE and nothing changes.

In the case of non-bursty traffic, I don't think large RTT causes a big problem for normal TCP either. This is because even with a large RTT (if it takes 400 ms to go from sender to host, for example) ACKS will still be streaming in at a constant, if slower, rate, allowing for more packets to be sent out (this is more subtle to explain, so you might want to google more for a better explanation).

I think the reason you misunderstand this is because the New Scientist article makes it sound like you send a packet, then receive an ACK, then send one, etc. This is not the case -- you send lots of packets together...this is the principle behind the "window", that you can send out more than one packet at a time without receiving an ACK because you've been successful at that so far.

FYI, I looked up MCI's traffic times and found that transatlantic latency is roughly 80ms compared to 45ms for within-US traffic (cite). This is non-trivial, but also not huge.

If anybody disagrees with this assessment, please feel free to correct, since as I said, I'm not 100% sure that increased RTT doesn't mean lower window size.

Also, from my reading a lot of the gain was simply in the fact that halving a throughput rate of 800K/sec means you're dropping to 400K/sec when realistically you should probably only be dropping a little bit. According to NS, they improved by more than two-fold, but that's probably just because normal TCP doesn't often get to the actual max of the network, it may burp a lot on the way up and dip more than halfway than its reasonable max.

Re:Isn't this called UDP?

[tincho_uy]

No, the guy at new scientist got it right... TCP uses an AIMD (additive increase multiplicative decrease) rate control algorithm. The rate at which you send is controlled by the window size at any given time. If you detect a loss, you decrease your window, dividing it's size by 2. If packets are arriving ok, you make small increments to your window size.

This new protocol uses a different window management algorithm. It uses the acks as probes, (I guess they measure delays) and if 'the coast is clear', it maxes it's transmission speed

I do wonder about FAST TCP congestion control capabilities, thugh... As for the poster who taled about slow start, sorry pal, but slow start is just the name...At that state, the transmission rate is increased quite fast, actually

This is HSTCP - more links as well

[HydraSwitch]

I believe this is HSTCP.
For more info, you can also take a look at:
Web100 and Net100.
It basically amounts to improving the AIMD algorithm and changing the way slow start works as well. Also, whoever said it before that this will not help your DSL connection is correct. It is meant to help high speed long RTT paths. And it does so -- quite well.