Slashdot Mirror
FastTCP Commercialized Into An FTP Appliance
prostoalex writes "FastTCP technology, developed by researchers at CalTech, is being commercialized. A company called FastSoft has introduced a hardware appliance that delivers 15x-20x faster FTP transmissions than those delivered via regular TCP. Says eWeek: 'The algorithm implemented in the Aria appliance senses congestion by continuously measuring the round-trip time for the TCP acknowledgment and then monitoring how that measurement changes from moment to moment.'"
Wouldn't you need to have FastTCP routers throughout the route in order to reach the speed mentioned?
and if so why bother using the old FTP protocol instead of just making a new and more modern protocol?
They should, in fact, start supporting ECN instead of changing TCP. Implementing ECN altogether with a well tuned AQM can reduce packet losses and difference packet losses due to congestion from those due to medium (now more important with so many wireless networks). Then, they can start looking for TCP versions that can react to congestion in a different way to the way they react to packet losses.
But that's just my thought.
The problem is that "regular" TCP mis-interprets long Round-Trip-Time (aka latency) as link congestion and backs off the rate at which it is sending packets.
The bandwidth between point A and B may be rated at a high throughput, but TCP protocols such as FTP will never achieve that speed if the RTT is long. Increasing the bandwidth won't help !! So a slowdown of 20-30x is not uncommon on WAN links with high latency e.g. transcontinental, via satellite.
I've looked at technologies like Digital Fountain (and it's Java implementation, FileCatalyst) which use UDP and some clever mathematics to overcome latency, however it's not clear from TFA what FastTCP is doing underneath..
$ strings FTP.EXE | grep Copyright
@(#) Copyright (c) 1983 The Regents of the University of California.
Only for short lengths. For international or especially satellite connections you get less than 10% with normal TCP.
.. although I keep coming back to the sentence "...senses congestion by continuously measuring the round-trip time for the TCP acknowledgment and then monitoring how that measurement changes from moment to moment.".
I would imagine in the typical high-latency scenario, where regular TCP is mis-interpreting long RTT as link congestion, and backing off the rate, FastTCP is able to actually keep pushing the rate up, meanwhile keeping an eye on the RTT. I mean, the RTT shouldn't increase in line with the rate, unless the link actually *is* congested. So just increase the rate until the RTT increases, at which point you are genuinely maxxing out the link. I think that must be how it is working..
$ strings FTP.EXE | grep Copyright
@(#) Copyright (c) 1983 The Regents of the University of California.
It works fine, but we actually tend to lean toward many streams as opposed to uber-fast single streams.
i ty::RatePlots?graph=quantity&entity=dest&src_filte r=&dest_filter=Nebraska&no_mss=true&period=l14d&up to=&.submit=Update
Truthfully, you have to tweak the system pretty hard to get decent performance over a single stream (for us, 155 Mbps isn't sufficient - I work on a LHC project), especially from Nebraska to Switzerland (CERN). FAST TCP helps out a whole lot. GridFTP is the other piece of the equation - it is basically FTP with multiple data streams.
We tend to lean on hundreds of streams a whole lot more than tweaking TCP settings, and the Caltech guys give us heck for that. They're right, however - if you're getting 100s of KBps per stream to some European site, it just takes a ridiculous number of streams to get up to 100 MBps. Right now, the storage systems are behind the network, so we haven't even been able to start playing with FAST TCP yet.
http://cmsdoc.cern.ch/cms/aprom/phedex/prod/Activ
It doesn't, unless your TCP implementation is from the stone age.
I love how fastsoft likes to compare themselves to Reno. 4.3BSD "Reno" was released in 1990, and the classic Reno implementation is LONG obsolete (and does indeed suck on a wide variety of connections).
I can see how it would be quite easy to achieve 10-20 times the throughput of Reno on a high-loss or high-latency connection, in fact a stock untuned Linux stack will do so in many situations. (For example, a few months ago I was doing TCP throughput tests dealing with some faulty hardware that liked to drop bursts of packets due to a shitty network driver. A machine running VxWorks 5.4, which is pretty much vanilla Reno, could only send 160 kilobytes/second over a 100Base-T LAN to that machine due to the packet loss making it throttle back. An untuned laptop with Linux 2.6.20 managed 1.7 megabytes/second over the same connection to the same destination.)
High latency connections were a major problem for TCP prior to RFC 1323 were a problem, but TCP stack authors have had 15 years to implement RFC 1323.
FastSoft's product may have been big news in the early 1990s, but if a company has to resort to making performance comparisons against the "Reno" TCP implementation, they're a snake oil salesman because Reno is such an obsolete and shitty TCP congestion control implementation.
retrorocket.o not found, launch anyway?
Basic TCP simply ramps up the transmission rate linearly until it starts dropping packets (timeout for receiver acknowledgement), then it halves the rate and begins to ramp up again. So that means that if there is a decent amount of capacity (i.e. the receiver can ack the packets in time) then you expect to get at least half the speed the data protocol allows (this too isn't perfect but again it's not too far from Shannon). There are fiddles to deal with low capacity channels, which are pretty standard. There are a few tricks (called 'slow start'!) to get a decent transmission rate quickly from the off (if you have a 10M connection and you use the original TCP protocol you'll spend the first couple of minutes of transmission just getting up to pace). Still not very good if you have a really fast connection for big files (say a few TB of astronomical data) when you spend 20 minutes building the rate up to the channel limit only to see it reset to half.
So, there are problems. One is that if a channel gets overloaded and starts dropping packets then probably there are several TCP links going through it, most of them lose packets, drop to half rate and the link is suddenly at about 60% capacity. Another is that if you have time-critical data (videoconferencing, say) there isn't any way to protect capacity - so your videoconference gets freezes which annoy everyone because some PFY's P2P traffic is filling the channel, even though the PFY couldn't care less if his porn takes five or ten minutes to download.
There are also things you can do - for example there is nothing to stop you fiddling your implementation of TCP to only drop to 90% on a packet loss; do that and you'll get about 40% better upload speed (obviously it'll do nothing to download speed) if you're on a reasonably direct backbone connection (i.e. not a T1 or cable or whatever). But that's antisocial, and if you send enough data for people to notice then you will be very unpopular (you'll be causing far more channel overloads, and everyone else's data rates will take a big hit).
Ultimately, though, even if you cannot in theory do better than get 2x performance over TCP (it's probably a bit less than 2, I'd guess) you're still going to find it cheaper to get 1% more performance out of TCP (which certainly is possible) than to lay another 1% of fibre.
I'm Steven Low from FastSoft. We are really excited by all the discussions,
and would like to share a few things.
As several people have already pointed out that, like most TCP
variants, FastTCP is end-to-end and does not require router support,
nor does it require any hardware or software installation at the receiving
computer. It accelerates all TCP-based applications. It eliminates inefficiencies
of current TCP implementations in the presence of packet loss and long latency.
It thus provides the most benefit in transferring large files over long-fat
or lossy paths. For large file transfers, slow-start is less important
than TCP's steady state behavior (congestion avoidance algorithm)
and this is what we have optimized.
I'll be happy to answer any questions. Feel free to give me a call
at 626 357-7012, or email me at info@fastsoft.com
Steven
It doesn't, in general. There are edge-cases.
For example, most tcp-implementations use slow-start, which mean they will, regardless of latency, start with a small window, and then if that goes trough, gradually increase the window-size until no improvement is experienced anymore.
This makes a huge difference for example if your application transfers many small files, each over its own tcp-connection, which FTP will do but which I'm aware of no other commonly used file-transfer application doing. It's no accident that they use ftp as the example-protocol, while ignoring the simple fix for that case: Make a large zip or tgz-file and transfer that one large file rather than the huge collection of tiny files.
Indeed this fix is so well-known that there are ftp-servers with built in support for it: you can do "get directory.tgz" and the ftp-server will, on the fly, make a tgz and send it to you, instead of the myriad files inside that directory.
The worst though, is, like I mentioned, links that have packet-loss unrelated to congestion. Most tcp-implementations I'm aware of will interpret packet-loss as congestion, and react by throttling back. Which is disastrous if really, the packet-loss is unrelated to congestion. A good heristic for separating these two cases from eachother would indeed help a lot. (potentially several orders of magnitude), but that is a very rare situation. The more common response to links that are inherently packet-lossy is to build in error-correction on lower-levels. Satelite-links, for example, commonly do this. Instead of a 10Mbps link with 1% of all packets getting corrupted you get a 7Mbps link where 0.001% of all packets get corrupted.
For modern tcp-stacks at reasonable speeds and latencies, it shouldn't happen. We certainly had no problems at all of this type when, for example, mirroring Debian transatlantically at 100Mbps. To the contrary, if we hadn't had traffic-prioritizing the single ftp-connection would've degraded other uses of the network for the duration.