Slashdot Mirror
FastTCP Commercialized Into An FTP Appliance
prostoalex writes "FastTCP technology, developed by researchers at CalTech, is being commercialized. A company called FastSoft has introduced a hardware appliance that delivers 15x-20x faster FTP transmissions than those delivered via regular TCP. Says eWeek: 'The algorithm implemented in the Aria appliance senses congestion by continuously measuring the round-trip time for the TCP acknowledgment and then monitoring how that measurement changes from moment to moment.'"
Of several research in the field. The problem is, still, microsoft being able to support it. If they crossed that bridge already, then it's great.
Mainly, SCTP, XCP, etc. are all great protocols, but if nobody supports them. In particular MS, well, it's not going to work in the long term.
saying transfers will go 15-20 times faster makes me think it'll go 14-19 times line-speed, sounds like stupid marketing bs
Ya... but what about FastSCP?
Wouldn't you need to have FastTCP routers throughout the route in order to reach the speed mentioned?
and if so why bother using the old FTP protocol instead of just making a new and more modern protocol?
Regular TCP can't be more than an order of magnitude away from the Shannon Limit, can it?
Not a typewriter
i think it's not altering the pipes / tubes =} ?.
...
... if it works like that.
the pipe stays the same (TCP/IP).
it's just a tech to fill the pipe more smartly.
on the senders end, normal tcp half's the bandwidth once
it detects a dropped packet and then tries to go faster
and faster until, again a packet is dropped. rinse repeat
my guess, this new tech is preempting the ceiling, and
trying to stay below it (ceiling being packet
gets dropped somewhere).
doesn't sound very difficult to do
FastTCP sounds like a fancy name for TCP Vegas (which has been around for quite some time). Window scaling and Vegas should buy you pretty much everything that FastTCP seems to be offering... Sounds like marketspeak to me.
The same amazing material that makes these so fast!
Does this speed up FTP or TCP?
If the latter can it speed up other protocols?
...already use a smoothed moving average of RTTs? (Remembered from a few terms old networking class)
Sounds like they just skip TCP slow start algorithm and stuff like that - so it's probably not faster than regular TCP after the window has stabilized. Slow-start and backoff algorithms of course cause slowdowns.
:)
Other possibility is some sort of header compression.
Anyway, to use this safely you'd need to be *sure* you know your link charasteristics. The reason TCP has the slow-start mechanisms in the first place is to make sure you don't overflow the link - that's why it's known as flow control
Typical FTP connections get 80% or so of available bandwidth. 15-20x faster is not possible. Maybe 1.2x if you re lucky.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
To gain that much speed, your network must be really fscked up. I can max out my 7mbps line on any FTP that has the bandwidth available. I've heard of people lines much much bigger than mine that max theirs our regularly, also. I'm not talking about short hops, either... I mean international.
The only way I could see this as being possible is if there is so much latency that it basically makes the TCP protocol think every packet is lost, and resends them... 20 times. If you are seriously on a network that is that messed up, you need to just find another network. Some silly little piece of hardware is -not- going to solve your problems.
If they had said 1.5x to 2.0x... I could believe them. It's not that hard to find network conditions that slow things that much. But 15x to 20x? No way.
"If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
This is for single-connection use of wide-bandwidth channels with long latency. If you're synchronizing two servers across a considerable distance and have more than 1Gb/s or so available, it might be useful. For anything less, don't bother.
For local connections, you don't have many packets in flight, so you don't need this. For slower connections, you don't have the bandwidth to get that useful many packets in flight, so it doesn't help there either. It's not going to help your web browsing.
An FTP session running over a 100Mbit LAN should see about 10MB/sec real data transfer, maxing out the line and accounting for overhead. They're claiming that their gadgets could move a file between each other at 150 megabytes per second over the same cable?
As the saying goes, this requires some very extraordinary evidence. Or there are a lot of missing qualifiers like "over a specific worst-case line that TCP doesn't come close to theoretical maximum performance on".
Yes, you read that right - 4 Libraries of Congress per hour !!!!
See http://www.fastsoft.com/research.html
$ strings FTP.EXE | grep Copyright
@(#) Copyright (c) 1983 The Regents of the University of California.
http://en.wikipedia.org/wiki/SCTP
- sctp/?ca=dgr-wikiaSCTP
http://www.ibm.com/developerworks/linux/library/l
And who cares about moldy old FTP anymore, anyway when there's HTTP, networked filesystems, and (name your favorite peer to peer file sharing program)?
If FastTCP is great for speeding things up over high latency links, what is there for slowing down connections? Particularly when you only want to take the remaining bandwidth and not impact users. I've seen various products that do this, but they never describe how it's done. Is it sufficient to slow down the connection when you see latency increase, or are there better algorithms?
It's true that early implementations of TCP were very naive. Over time this has been fixed, but there are still a number of problems remaining, especially to do with packet loss on WIFI networks (which it sounds like this may address).
The primary problem with WIFI networks is that they naturally have a lot more packet loss than normal links. On other links, a lot of packet loss tends to indicate packet congestion, so TCP likes to decrease throughput to try to solve it. Under WIFI, that's of course unnecessary and won't solve the underlying problem.
The article is missing some important technical details and there's a little too much marketing speak, but it does clearly sound like an improved TCP implementation, and probably some kind of traffic shaping hardware on one end (so that they don't have to change the networking stack on linux and windows, patch all their machines, etc).
There were a couple of other posters that suggested that such a thing wouldn't work. One guy even suggested that it would require different routers end to end! This is of course nonsense.
1. TCP != IP. Routers don't have to know anything about TCP to work (although they generally do for NAT, ACL, and traffic shaping purposes).
2. TCP implementations have been changed a number of times in the past. Changing the implementation is not the same as changing the protocol. Nothing else on the network cares what TCP implementation you are using as long as you speak the same protocol.
Maybe they should just use good old UDP instead and implement a tweak to the FTP protocol to handle retransmit and error checking. The 'Net doesn't drop very many packets anymore, and UDP can work just fine.
Are doomed to reinvent it, poorly, to paraphrase a well known saying. I have to roll my eyes everytime I see someone recommend the use of UDP in a circumstance where the application will not tolerate data loss. In gaming and media streaming, UDP can make sense, where the receiver can gloss over the details and do something reasonable, to an extent possible interpolating the missing data or simply showing a corrupted block or having someone skip a little in an online game. The only places where I see UDP implemented in a context where packet loss without retry is tolerated is in traditionally embedded applications (i.e. service processors with IPMI, and TFTP in ethernet boot roms). Both of these protocols can start behaving very badly if packets are lost or if over a high-latency link.
TCP is the most researched most tweaked and most examined reliable transport protocol, and trying to reinvent your own over an unreliable protocol is asking for trouble.
XML is like violence. If it doesn't solve the problem, use more.
Do any other slashdotters feel, like myself, that this device is a bit of a damp squib given that FTP is somewhat obsolete ? HTTP provides upload as well as download capabilities, and in any tests I've done I get the same download speed as with FTP. Since it doesn't have a stupid protocol I can easily tunnel it as required.
FastTCP isn't really a full TCP replacement but rather a congestion control algorithm. There are many competitors to FastTCP, including BIC/CUBIC (common Linux default), High-Speed TCP, H-TCP, Hybla, and many others. Microsoft calls their version Compound TCP (available in Vista).
If you use Linux, have (CU)BIC loaded, correctly setup your NIC, and tune your TCP settings (rx/tx mem, queuelen, and such) then there is be no way for FastSoft to claim a 15-20x speedup improvement. I've done full 10 gigabit transmissions with a 150ms RTT using that kind of setup. FastSoft's device doesn't even support 10 gigabit, and their 1 gigabit device still isn't released.
This article is nothing other than a Slashadvertisment.
Of course they'll be happy!
They'll be getting their porn 20x faster!
I've seen arguments used where people say 'we don't need to worry about aspect X that TCP takes care of' and ultimately get bitten. IPMI to me is a good example. They have the notion of retries (more of an afterthought), and have sequence numbers above and beyond what UDP offers. The problem is that retries for most packets increment that sequence number, so a retry is indistinuishable from a reissuance of the same command. For some contents, this can be very undesirable.
When something with as much high-profile support as IPMI ends up with such shortcomings, it goes to show that people easily fail to understand why this aspect or that of TCP is not applicable to their use.
As to TCP over UDP, that's an example of a very bad sounding ideas. Redundant features of TCP and UDP. It's not as bad as TCP over IP over PPP over SSH which is over TCP (multiple reliable protocols on top of each other), but still, if you wanted to be a better TCP than TCP, the place to implement would be at the same layer, on top of IP protocol, not on top of UDP.
XML is like violence. If it doesn't solve the problem, use more.
20x faster huh, so they could make my modem go at broadband speeds? i think i've seen claims like that on annoying flashing ads on the web. fuck them.
If you mod me down, I will become more powerful than you can imagine....
So basically after tahoe, reno and vegas, fasttcp is a nex gen congestion algorithm.
Benefits of fasttcp can be observed at 1gbps an up.
All this good an well but will Linux suport it ?
Not sure because it's patend pending technology.
http://www.freepatentsonline.com/20070121511.html
E-week was never known for its academic rigor. Netlab posts their published papers including FAST TCP: motivation, architecture, algorithms, performance in IEEE Transactions on Networking.
This could be the XMODEM killer.
I'm Steven Low from FastSoft. We are really excited by all the discussions,
and would like to share a few things.
As several people have already pointed out that, like most TCP
variants, FastTCP is end-to-end and does not require router support,
nor does it require any hardware or software installation at the receiving
computer. It accelerates all TCP-based applications. It eliminates inefficiencies
of current TCP implementations in the presence of packet loss and long latency.
It thus provides the most benefit in transferring large files over long-fat
or lossy paths. For large file transfers, slow-start is less important
than TCP's steady state behavior (congestion avoidance algorithm)
and this is what we have optimized.
I'll be happy to answer any questions. Feel free to give me a call
at 626 357-7012, or email me at info@fastsoft.com
Steven
... the the more they stay the same. And I quote:
"Many very stupid companies have tried to come up with overly clever ways to speed up TCP/IP. TCP, by its nature, is a stateful and bidirectional protocol that requires all data packets to be acknowledged; this makes the data flow reliable, by providing a mechanism for dropped packets to be retransmitted; but this also makes for a more strictly regimented flow structure involving more packets transmitted over the wire than in simpler, non-reliable protocols like UDP-- and therefore it's slower. One company that thought itself a lot smarter than it really was, called RunTCP, came up with the idea of "pre-acking" TCP packets; it would send out the acknowledgments for a whole pile of data packets in advance, thus freeing them from the onerous necessity of double-checking that each packet actually got there properly. And it worked great, speeding up TCP flows by a significant margin-- in the lab, under ideal test conditions. The minute you put RunTCP's products out onto the real Internet, everything stopped working. Which stands to reason-- their "solution" was to tear out all the infrastructure that made TCP work reliably, under competing load and in adverse conditions, in the first place. Dumbasses."
XML is like violence. If it doesn't solve the problem, use more.
1Mbps is prehistoric.
Where do you live, the US?
How the hell is this informative?! It's completely wrong. The sibling post by stud9920 is the correct explanation.
If we're successful in sending packets, we'll increase packet size, and we'll download multiple blocks at a time, even if the ack/nak for the previous ones hasn't been fully processed then.
:)
Hmm.. Sounds like I want my Zmodem back
Coz eternity my friend, is a long *ing time.
There are already (and have been) network accelerators that do this, and more. Cisco has a product called WAAS that incorporates this technology, and adds quite a bit more to it. As others have said here, though, I don't think that just FastTCP would get you the kinds of gains that they're talking about. Where it is good is when you have apps that don't like latency. When that's the case, throwing more bandwidth at the problem doesn't help, and instead an accelerator is the best bet. But while this company has come out with their first system, others are on their second-gen versions, and have a lot more features (like file caching...and that's not as simple as it sounds, since they can cache the main versions of files and simply send the minor changes back and forth, for example).
For your security, this post has been encrypted with ROT-13, twice.
It probably compensates for applications or servers that don't allocate a large enough TCP buffer, whose windows-size/RTT bandwidth.
(i.e. The system needs to keep a buffer of all transmitted data until it is acknowledged)
I guess it behaves as a tcp proxy, the RTT between the sending server and the applicance (on the same LAN) is small, so whatever buffer size is allocated by the server is enough.
Sam
blog.sam.liddicott.com
We already have an ability to run tcp over high bandwidth x delay-product links, its called Selective Acknowledgement (SACK), its implemented by everyone with a clue, and it works just fine.
Also, DragonFly and FreeBSD have had server-side bandwidth delay product band-limiting code in place for years to deal with the most common congestion point, which is the LANWAN interface. Combine that with fair-share scheduling and you don't have to run RED on the routers bordering your server farm. Leave RED where it should be... in the middle of the internet, not on the edges.
New Reno is possibly the most idiotic congestion control algorithm ever devised. It's not hard to do better and still be friendly to the mesh.
-Matt
Oh, it's available for Linux? Well, then, FastTCP must be a great idea. Indeed, this just shows how Windows is inferior, since Microsoft isn't shipping support for FastTCP in Windows yet...
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
Like most things, IP acceleration is better done in software. You can get all the benefit claimed for special routers or kernel drivers in user-level software by using UDP and controlling the packet injection rate and retransmission yourself. It's not even very hard to code -- unless you also want things like automatic back-off to allow incidental TCP traffic to share the line, automatic bandwidth measurement, parallel transfers (e.g. a dozen machines sharing the work to fill a 10Gbit link), file streamlining (to send whole directory trees with no delay between files), database-logged fully-acknowledged transfers, secure encryption, reliable checksumming, partial-transfer resumption, cooperative scheduled variable bandwidth limits, portability to whatever the people on the other end run, web-server and web-browser integration, GUI- or automated control and monitoring, and/or support. Then you're probably better off using a commercial product. As it happens, the movie and music studios feel that way, and they have mostly settled on the software linked from the page above.