Slashdot Mirror
FastTCP Commercialized Into An FTP Appliance
prostoalex writes "FastTCP technology, developed by researchers at CalTech, is being commercialized. A company called FastSoft has introduced a hardware appliance that delivers 15x-20x faster FTP transmissions than those delivered via regular TCP. Says eWeek: 'The algorithm implemented in the Aria appliance senses congestion by continuously measuring the round-trip time for the TCP acknowledgment and then monitoring how that measurement changes from moment to moment.'"
Wouldn't you need to have FastTCP routers throughout the route in order to reach the speed mentioned?
and if so why bother using the old FTP protocol instead of just making a new and more modern protocol?
Regular TCP can't be more than an order of magnitude away from the Shannon Limit, can it?
Not a typewriter
FastTCP sounds like a fancy name for TCP Vegas (which has been around for quite some time). Window scaling and Vegas should buy you pretty much everything that FastTCP seems to be offering... Sounds like marketspeak to me.
The same amazing material that makes these so fast!
They should, in fact, start supporting ECN instead of changing TCP. Implementing ECN altogether with a well tuned AQM can reduce packet losses and difference packet losses due to congestion from those due to medium (now more important with so many wireless networks). Then, they can start looking for TCP versions that can react to congestion in a different way to the way they react to packet losses.
But that's just my thought.
Sounds like they just skip TCP slow start algorithm and stuff like that - so it's probably not faster than regular TCP after the window has stabilized. Slow-start and backoff algorithms of course cause slowdowns.
:)
Other possibility is some sort of header compression.
Anyway, to use this safely you'd need to be *sure* you know your link charasteristics. The reason TCP has the slow-start mechanisms in the first place is to make sure you don't overflow the link - that's why it's known as flow control
It was the speed of TCP which would improve, not only FTP, I guess, so therefor whatever else uses TCP should get faster aswell, shouldn't it?
Yes, you read that right - 4 Libraries of Congress per hour !!!!
See http://www.fastsoft.com/research.html
$ strings FTP.EXE | grep Copyright
@(#) Copyright (c) 1983 The Regents of the University of California.
Only for short lengths. For international or especially satellite connections you get less than 10% with normal TCP.
Think again. I suspect that you only have tried that on a low-speed link (DSL, Cable, FIOS, etc). Try thinking about 2 orders of magnitude faster.
I transfer about 20 TB / day at work, and that wouldn't be possible with a "typical FTP connection".
If you read the papers coming out of Caltech, you'd see they were optimizing for 10 Gbps lines, not residential lines. 15-20x faster is a very fair estimate; look at Caltech's presentations at SC05 or SC07.
RTFA.
"The Aria 2000, which is due in July, supports 1G-bps links. Existing Aria appliances support 10M-bps links, 50M-bps links and 200M-bps links."
10gbps my ass. The one they haven't released only does a tenth of that. And the smallest of their products barely handles my home cable line.
For what it's worth, my initial thought was that they must be targetted truly massive lines and that it would be a lot harder to truly use those. Too bad it wasn't true.
"If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
It's true that early implementations of TCP were very naive. Over time this has been fixed, but there are still a number of problems remaining, especially to do with packet loss on WIFI networks (which it sounds like this may address).
The primary problem with WIFI networks is that they naturally have a lot more packet loss than normal links. On other links, a lot of packet loss tends to indicate packet congestion, so TCP likes to decrease throughput to try to solve it. Under WIFI, that's of course unnecessary and won't solve the underlying problem.
The article is missing some important technical details and there's a little too much marketing speak, but it does clearly sound like an improved TCP implementation, and probably some kind of traffic shaping hardware on one end (so that they don't have to change the networking stack on linux and windows, patch all their machines, etc).
There were a couple of other posters that suggested that such a thing wouldn't work. One guy even suggested that it would require different routers end to end! This is of course nonsense.
1. TCP != IP. Routers don't have to know anything about TCP to work (although they generally do for NAT, ACL, and traffic shaping purposes).
2. TCP implementations have been changed a number of times in the past. Changing the implementation is not the same as changing the protocol. Nothing else on the network cares what TCP implementation you are using as long as you speak the same protocol.
Maybe they should just use good old UDP instead and implement a tweak to the FTP protocol to handle retransmit and error checking. The 'Net doesn't drop very many packets anymore, and UDP can work just fine.
I don't think you're the target market for this piece of hardware.
I believe the application is called Internet Explorer iirc.
lol: You see no door there!
Are doomed to reinvent it, poorly, to paraphrase a well known saying. I have to roll my eyes everytime I see someone recommend the use of UDP in a circumstance where the application will not tolerate data loss. In gaming and media streaming, UDP can make sense, where the receiver can gloss over the details and do something reasonable, to an extent possible interpolating the missing data or simply showing a corrupted block or having someone skip a little in an online game. The only places where I see UDP implemented in a context where packet loss without retry is tolerated is in traditionally embedded applications (i.e. service processors with IPMI, and TFTP in ethernet boot roms). Both of these protocols can start behaving very badly if packets are lost or if over a high-latency link.
TCP is the most researched most tweaked and most examined reliable transport protocol, and trying to reinvent your own over an unreliable protocol is asking for trouble.
XML is like violence. If it doesn't solve the problem, use more.
I dunno, at a previous company I built a FTP gizmo based on Windows 2k which could saturate a 1000 mile long OC3 line. That's 1.4 TBytes a day.
To transfer 20 TB/day, you need something like 1.8 Gbps sustained, not my measly 155 MBps, but that's only (only!) an order of magnitude better. TCP has shown itself quite comfortable scaling up from 300 baud modems to GigE links (6+ orders) so what's one more among friends? This is not to say TCP can't be improved: I've always thought using dropped packets to measure congestion was a but hokey, but it seems to work fine. If the fine researchers at CalTech think they can do better by measuring RTT, that sounds just great.
FastTCP isn't really a full TCP replacement but rather a congestion control algorithm. There are many competitors to FastTCP, including BIC/CUBIC (common Linux default), High-Speed TCP, H-TCP, Hybla, and many others. Microsoft calls their version Compound TCP (available in Vista).
If you use Linux, have (CU)BIC loaded, correctly setup your NIC, and tune your TCP settings (rx/tx mem, queuelen, and such) then there is be no way for FastSoft to claim a 15-20x speedup improvement. I've done full 10 gigabit transmissions with a 150ms RTT using that kind of setup. FastSoft's device doesn't even support 10 gigabit, and their 1 gigabit device still isn't released.
This article is nothing other than a Slashadvertisment.
I've seen arguments used where people say 'we don't need to worry about aspect X that TCP takes care of' and ultimately get bitten. IPMI to me is a good example. They have the notion of retries (more of an afterthought), and have sequence numbers above and beyond what UDP offers. The problem is that retries for most packets increment that sequence number, so a retry is indistinuishable from a reissuance of the same command. For some contents, this can be very undesirable.
When something with as much high-profile support as IPMI ends up with such shortcomings, it goes to show that people easily fail to understand why this aspect or that of TCP is not applicable to their use.
As to TCP over UDP, that's an example of a very bad sounding ideas. Redundant features of TCP and UDP. It's not as bad as TCP over IP over PPP over SSH which is over TCP (multiple reliable protocols on top of each other), but still, if you wanted to be a better TCP than TCP, the place to implement would be at the same layer, on top of IP protocol, not on top of UDP.
XML is like violence. If it doesn't solve the problem, use more.
It works fine, but we actually tend to lean toward many streams as opposed to uber-fast single streams.
i ty::RatePlots?graph=quantity&entity=dest&src_filte r=&dest_filter=Nebraska&no_mss=true&period=l14d&up to=&.submit=Update
Truthfully, you have to tweak the system pretty hard to get decent performance over a single stream (for us, 155 Mbps isn't sufficient - I work on a LHC project), especially from Nebraska to Switzerland (CERN). FAST TCP helps out a whole lot. GridFTP is the other piece of the equation - it is basically FTP with multiple data streams.
We tend to lean on hundreds of streams a whole lot more than tweaking TCP settings, and the Caltech guys give us heck for that. They're right, however - if you're getting 100s of KBps per stream to some European site, it just takes a ridiculous number of streams to get up to 100 MBps. Right now, the storage systems are behind the network, so we haven't even been able to start playing with FAST TCP yet.
http://cmsdoc.cern.ch/cms/aprom/phedex/prod/Activ
QoS - typically implemented not in the TCP stack but in intermediary routers that prioritize packets (important stuff goes out first and is less likely to be dropped if the connection is saturated, "bulk" data like BitTorrent goes out only if the send queue is empty at the router's WAN connection and is most likely to get dropped if a queue fills up), and in some cases artifically throttle the connection by dropping packets if the sender transmits beyond a set limit.
If you're looking for QoS in a home environment, the easiest solution is likely to replace your router with one capable of running DD-WRT. (This assumes you have a "consumer grade" router. If your gateway to the outside world is a normal PC running Linux, it's just a matter of setting up QoS on that box... Quite a few HOWTOs exist for this.)
retrorocket.o not found, launch anyway?
This could be the XMODEM killer.
I'm Steven Low from FastSoft. We are really excited by all the discussions,
and would like to share a few things.
As several people have already pointed out that, like most TCP
variants, FastTCP is end-to-end and does not require router support,
nor does it require any hardware or software installation at the receiving
computer. It accelerates all TCP-based applications. It eliminates inefficiencies
of current TCP implementations in the presence of packet loss and long latency.
It thus provides the most benefit in transferring large files over long-fat
or lossy paths. For large file transfers, slow-start is less important
than TCP's steady state behavior (congestion avoidance algorithm)
and this is what we have optimized.
I'll be happy to answer any questions. Feel free to give me a call
at 626 357-7012, or email me at info@fastsoft.com
Steven
... the the more they stay the same. And I quote:
"Many very stupid companies have tried to come up with overly clever ways to speed up TCP/IP. TCP, by its nature, is a stateful and bidirectional protocol that requires all data packets to be acknowledged; this makes the data flow reliable, by providing a mechanism for dropped packets to be retransmitted; but this also makes for a more strictly regimented flow structure involving more packets transmitted over the wire than in simpler, non-reliable protocols like UDP-- and therefore it's slower. One company that thought itself a lot smarter than it really was, called RunTCP, came up with the idea of "pre-acking" TCP packets; it would send out the acknowledgments for a whole pile of data packets in advance, thus freeing them from the onerous necessity of double-checking that each packet actually got there properly. And it worked great, speeding up TCP flows by a significant margin-- in the lab, under ideal test conditions. The minute you put RunTCP's products out onto the real Internet, everything stopped working. Which stands to reason-- their "solution" was to tear out all the infrastructure that made TCP work reliably, under competing load and in adverse conditions, in the first place. Dumbasses."
XML is like violence. If it doesn't solve the problem, use more.