Slashdot Mirror
Replacing TCP?
olau writes "TCP, the transfer protocol that most of the Internet is using, is getting old. These guys have invented an alternative that combines UDP with rateless erasure codes, which means that packets do not have to be resent. Cool stuff! It also has applications for peer-to-peer networks (e.g. for something like BitTorrent). They are even preparing RFCs! The guy who started it, Petar Maymounkov, is of Kademlia fame."
Now that Digital is little more than IP spread across a few different companies, maybe the holder of Decnet's patents could release the protocol under an Open Source license. If I recall correctly it was quite the networking layer.
BLING BLING. Meet the architecture that's changing everything.
TCP may be old, but it can go on for another 50 years wothout any problem.
--- "To pee or not to pee, that is the question." ---
Does it have Evil Bit implemented?
Kudos for using coral.
http://en.wikipedia.org/wiki/Kademlia
This has nothing to do with Portugal! It's useless!
The submitter says that TCP is getting old, but does that really tell us anything about how well it does its job?
If it's not an open protocol (if they charge for use) it may find niche applications. If it is, it may proliferate. I wasn't able to find details about this on the site.
The World Wide Web is dying. Soon, we shall have only the Internet.
No comments and already slashdotted.
Some inefficiencies are one thing, but you're going to need a compelling reason to get everyone to switch.
The link seems to be dead.
I guess the new protocol still has some time left in development.
Not ready for slashdotting yet.
Even if this attempt to switch to another technology is succesful, it would be painfully slow
TCP is old, but that doesn't mean it's bad or replacement is due. Some shortcomings have surfaced and been adressed. For the most part, TCP does a good job at what it was designed to do.
Please correct me if I got my facts wrong.
IP maybe is ready to be replaced.
TCP is not ready to be replaced.
TCP has solved all the major persistance problems.
Because then you're going to have the suits trying to push it down, no matter how great/useful it is in an effort to kill the possibility of coming out with something that could make pirating any easier or more efficient. That's the only way they're going to see it.
It's good to see innovation though, nonetheless.
This is hardly an innovative idea, and usually by the time you end up considering all the issues you wind up with something that looks a lot more like TCP than you had originally intended.
Plus there are already protocol stacks that work around most of their gripes about TCP (slow performance over long pipes, etc).
Here is a summary of their technology copied from their website:
Rateless Internet The Problem
Rateless Internet is an Internet transport protocol implemented over UDP, meant as a better replacement of TCP. TCP's legacy design carries a number of inefficiencies, the most prominent of which is its inability to utilize most modern links' bandwidth. This problem stems from the fact that TCP calculates the congestion of the channel based on its round-trip time. The round-trim time, however, reflects not only the congestion level, but also the physical length of the connection. This is precisely why TCP is inherently unable to reach optimal speeds on long high-bandwidth connections.
A secondary, but just as impairing, property of TCP is its inability to tolerate even small amounts (1% - 3%) of packet loss. This additionally forces TCP to work at safe and relatively low transmission speeds with under 1% loss rates. Nevertheless, our extended real-life measurements show that highest throughput is generally achieved at speeds with anywhere between 3% and 5% loss.
The Solution
By using our core coding technology we were able to design a reliable Internet transmission protocol which can circumvent both of the fore-mentioned deficiencies of TCP, while still remaining TCP-friendly. By using encoded, rather than plain, transmission we are able to send at speeds with any packet loss level. Rateless coding is used in conjunction with our Universal Congestion Control algorithm, which allows Rateless Internet to remain friendly to TCP and other congestion-aware protocols.
Universal Congestion Control is an algorithm for transmission speed control. It is based on a simple and clean idea. Speed is varied in a wave-like fashion. The top of the wave achieves near-optimal throughput, while the bottom is low enough to let coexisting protocols like TCP smoothly receive a fair share of bandwidth. The time lengths of the peaks and troughs can be adjusted parametrically to achieve customized levels of fairness between Rateless Internet and TCP.
The Rateless Internet transport is now available through our Rateless Socket product in the form of a C/C++ socket library. Rateless Internet is ideal for Internet-based applications, running on the network edges, that require high bandwidth in a heterogenous environment. It was specifically built with peer-to-peer and live multimedia content delivery applications in mind.
A slightly faster equivelent to TCP that I have to pay for and no-one else uses.
Sign me up for that sucker right now.
http://jfin.org/jFin pure java open source financial library
Considering this is slashdot and all, I was surprised that their implementation does not appear to be open source (or indeed, freely available at all), though presumably such an implementation will be possible following the RFCs. It seems to work nicely alongside TCP using UDP, quite a cool idea. The question is whether it can break TCP's de facto stranglehold on reliable Internet communication. I'd love to play with it if I could.
The link is down. The new protocol should help keep the internet running efficiently for the next decade or so.
Free Desk
Often stories are posted that refer to products or code names, with no description, which is quite annoying.
I'm glad to see this post doesn't run that risk.
Thanks for clearing that up for me.
-d
Love many, trust a few, do harm to none.
better replacement of TCP. TCP's legacy design carries a number of inefficiencies, the most prominent of which is its inability to utilize most modern links' bandwidth. This problem stems from the fact that TCP calculates the congestion of the channel based on its round-trip time. The round-trim time, however, reflects not only the congestion level, but also the physical length of the connection. This is precisely why TCP is inherently unable to reach optimal speeds on long high-bandwidth connections.
A secondary, but just as impairing, property of TCP is its inability to tolerate even small amounts (1% - 3%) of packet loss. This additionally forces TCP to work at safe and relatively low transmission speeds with under 1% loss rates. Nevertheless, our extended real-life measurements show that highest throughput is generally achieved at speeds with anywhere between 3% and 5% loss.
The Solution
By using our core coding technology we were able to design a reliable Internet transmission protocol which can circumvent both of the fore-mentioned deficiencies of TCP, while still remaining TCP-friendly. By using encoded, rather than plain, transmission we are able to send at speeds with any packet loss level. Rateless coding is used in conjunction with our Universal Congestion Control algorithm, which allows Rateless Internet to remain friendly to TCP and other congestion-aware protocols.
Universal Congestion Control is an algorithm for transmission speed control. It is based on a simple and clean idea. Speed is varied in a wave-like fashion. The top of the wave achieves near-optimal throughput, while the bottom is low enough to let coexisting protocols like TCP smoothly receive a fair share of bandwidth. The time lengths of the peaks and troughs can be adjusted parametrically to achieve customized levels of fairness between Rateless Internet and TCP.
The Rateless Internet transport is now available through our Rateless Socket product in the form of a C/C++ socket library. Rateless Internet is ideal for Internet-based applications, running on the network edges, that require high bandwidth in a heterogenous environment. It was specifically built with peer-to-peer and live multimedia content delivery applications in mind.
--- "To pee or not to pee, that is the question." ---
There's no doubt that an alternative to TCP might have technical merits. But as far as communication protocols go, TCP itself is pretty amazing. Modern TCP implementations have been tweaked over decades and have impressive performance and reliability. And modern TCP/IP stacks have rather unspoofable connection establishment, another excellent feature for security.
If you want to replace TCP, you have to do more than just develop a new protocol that is faster. It would have to outperform TCP in speed, reliability, and substantially so in order to outweigh the costs of ditching a well-established and trusted protocol.
Yay! Something to speed up my p2p client! It was almost getting to the point where getting in my car and driving to whoever's computer I happened to be downloading from was faster than downloading it. Of course, with shareaza 2.0 http://www.shareaza.com/ out, it did get a lot faster anyway (with the support for SP2 and all).
The internet will die in 2006.
That is what will happen if udp will be the basis for it. Since udp has no rate control build in this either has to be build at the application level or servers/routers will drown. Yes, ik know routers can drop udp packets if overloeaded, but this kind of procotol just sends more packets to compensate for the dropped packets.
If their technology is that good, why do they need Coral Cache to protect them from slashdotting? ;-)
Please correct me if I got my facts wrong.
While this sounds very interesting (have to re-take all those networking certification exams again, I guess), when I read this...
...my eyes told my brain this...
The guy who started it, Petar Maymounkov, is of Kademlia fame."
The guy who started it, Petar Maymounkov, is of Chlamydia fame."
I was about to wonder what sort of "fame" you could get from that. Need coffee. Need sleep.
IronChefMorimoto
you should have posted the non-coralized link so we can slashdot it
here
Can someone post the text of the article? My company's firewall won't allow non-port 80 sites to be opened, even for outgoing traffic.
Slashdot editors: Please look for this in the future before including links. Thanks.
Chip H.
R-E-S-P-E-C-T
Find out what it means to me
R-E-S-P-E-C-T
Take care, TCP
Oh socket to me, socket to me,
socket to me, socket to me...
Do really dense people warp space more than others?
Can anybody explain to me how this technology? It reads like marketing speak to me, despite the fact that it will be/is released as open source. How does the technology actually achieve reliability without retransmits? Does it actually achieve it?
Please correct me if I got my facts wrong.
1. This is coming from a company who are surely going to want to make money out of it somehow. Part of the reason TCP succeeded is there was no one to pay.
2. They don't seem to understand the GPL:
"We are planning to release Rateless Codes for free, non-commercial use, in open source under the GNU Public License."
The GPL doesn't restrict commercial use, and hence the only way that they can do this is either they try to add some conditions to the GPL, or they use another mechanism to restrict commercial use: e.g. patents.
No matter how good this technology is it's not going to get wide adoption is an alternative to TCP unless it's unencumbered.
John.
...as for UDP replacing TCP I dont care. Its cool, and im glad there are changes tryign to be made.
;)
I doubt seriously they will be done soon. Unlike going from 32 bit to 64 bit processors, going from TCP to UDP as the primary or only method seems more useless then effective in a corporate view.
I dont see people wanting to put effort into adapting any time soon. But its still great to hear that we might.
Also, thanks for the info on Kademlia, as a noob developer, I always wondered how they did that
even on BSD.
I did read their website and it looks like their revolutionary new replacement for TCP is UDP with their proprietary ECC built on top of it. However, there is a good reason why TCP never used ECC (they did exist back then).
1) The major problem a TCP packet will face is getting dropped. They mention this problem. They claim their encoding will solve this problem. It won't. No ECC algorithm will allow you to recover a dropped packet.
2) Most packets that are corrupted are corrupted well beyond the repair of most ECCs.
3) ECCs will cause packet size to increase. Not a huge problem, but why do it when ECCs don't help too much to begin with?
In research, the issue of tcp replacement is a common one. THe solutions seem to either recommend making some minor tweaks (a "light" version of TCP!) or to scrap it entirely in favor of protocol X. With that knowledge, I will keep my excitement in check as I get around to reading the article.
Can't we just leave it alone.
Oh wait...nervermind, if we did that everything would work and we'd no longer be needed.
© 2004 The SCO Group, Inc. All Rights Reserved.
I fail to see what is flamebaiting it is to say that TCP can go on for another 50 years, without problem.
Exactly the same kind of post a bit below gets 'insightful'.
It is simply true. Yes, there are some little drawbacks with TCP, but in the whole article, they do not give a compelling reason to switch, let alone why one would *have* to. I mean, RTFA: TCP is at 1-3% and the most efficient would be a throughput with 3-5% (loss)...but so what? It's not optimal, but does it anywhere claims TCP is doomed because it's not optimal in certain area's?
There are myriads of things that aren't optimal on the Net, it doesn't mean they have been here for years and will be for years to come, nor that it is a necessity to switch, if the only thing lacking is that it's not optimally suited.
--- "To pee or not to pee, that is the question." ---
the real issue with any new technology like this is will it be accepted as a standard? For that to happen it needs to be relyable, easy to get (free), and hard to exploit. This looks to have all the above so in 5 to 10 years maybe we'll see this implmented everywhere.
It doesn't seem reason enough to build a new protocol for TCP replacement just because it is "old". Protols ar not living beings you know, aging doesn't show on their cells. Of course troubles always has been popping up over the years, but nothing unsolvable (and nothing in the likings of IP).
:D ) is a bit winged to say the least.
All in all, it's good to have new alternative solutions and new technologies at hand, but to state such things as replacing TCP 'cause its age (maturity, that is
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
They say that this new protocol would tolerate higher rates of packet loss. What happens to TCP connections when a lot of connections use this new protocol? Would it create worse congestion on networks, which this protocol can tolerate, but which would greatly slow down TCP connections?
Just look at the adoption rates on IPv6. No one is going to touch a new protocol at this stage. Its not even clear that this is needed. Point me at a specific TCP pain point that is specifically and obviously reducing internet adoption...any takers?
YAWN Protocol --> Yet Another Wonderful New Protocol
ASCII is still around, despite its numerous shortcomings. There's this small thing called "backward compatibility" that people/consumers seem to love, for some reason. Well, same thing for TCP/IP. Even IPv6 has trouble taking off in the general public, despite being essentially just a small change in the format, so never mind the YAWN Protocol this article is about...
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
The GPL doesn't lend itself to ``free, non-commercial use'': it lets the licensee use and distribute freely, at any price, for any use.
Perhaps what they meant was "free, non-proprietary use," which would be GPL-compatible.
Tried their "Rateless Copy" utility, transferring a 5.8 mb binary file from my web server in Texas to my local connection in Toronto.
With Rateless Copy: time between 31-41 seconds, average of 200k/s, the resulting file is corrupted. Tried it again to ensure, same result.
Without rateless copy (http file download) 8 seconds, average of 490k/s, the resulting file works fine as expected.
Sorry, but I don't think it's all that great.
Why not SCTP ? See RFC 2960. Already in the Linux kernel, Kame, (solaris ?) and probably others.
:-/
Intro here
- SCTP can be used in many "modes"
* Provides reliable messaging (like UDP,but reliable)
* Can be used as a stream protocol (like TCP).
* One connection/association can hold multiple streams.
* One-to-many relation for messaging.
* Better at dealing with syn flooding than TCP.
Then again, I guess inveting the wheel is more "fun"
Really, is TCP flawed?
That's right brother! Down with the man! Let him make his own TCP replacement!
While we're at it let's not talk about computers because they make pirating easier. At least then slashdot can revert to its natural state: being overrun by trolls.
I submitted this story last night, and it didn't get posted.
When considering protocols for information transport, it is very important to be absolutely sure what assumptions you are making. There are a number of non-independent factors which influence the suitability (and hence efficiency) of network protocols to application demands. Bandwidth, for example, is related to but doesn't define the statistical distribution of latencies; maximum packet rate and their relationship to packet size. The channel error rate (and statistical distributions of packet failures) are again linked to fragmentation and concatenation of transmitted datagrams - and this in turn affects latencies when considering "reliable" transport protocols. Routing policy and symmetry of physical links introduces yet more tradeoffs which need to be considered - not to mention the potential problems evaluating if the burden of protocol computations outweighs the advantage of an improved strategy for a given physical link. (And I'm not even going to mention security!) When considering protocols the most important thing to consider is the model they assume of the communications infrastructure on which they are to be deployed. TCP is likely the optimal solution given the assumptions TCP makes... if you change those assumptions to more closely fit a particular network infrastructure you will likely get better performance only on that infrastructure, but far worse performance where your new assumptions do not hold. I used to be interested in the idea of dynamically synthesizing protocols to best suit the actual physical links in a heterogeneous network... however my ideas were met with extreme disinterest; I felt my critics demanded I present a protocol which beats TCP under TCP's assumptions - and no amount of hand-waving and explanation would convince them this was a silly request. I still think the idea has merit - but having wasted 3 years of my life trying to push it uphill, I've found other interesting (and far more productive) avenues to pursue.
It would have been nice if they had more technical details and less sales blurb on their website.
OT: Kind of reminded me of these "Speed up your Internet 100%!" apps targeted at those technically non-proficiant types who use Windows 98.
While there are a number of issues with TCP, I think it would be much better in the long run to work on fixing TCP rather than replace it. That way all the existing apps can take advantage of the fixes.
One thing that bothers me is I see ISPs applying policing to their subscriber's bandwidth. Policing is quite unfriendly to TCP, unlike, say, shaping. With policing, a router decides either to pass, drop, or mark a packet based on if it exceeds certain bandwidth constraints. Shaping, on the other hand, will buffer packets and introduce additional latency, thus helping TCP find the sweet spot. Of course shaping will also drop, since nobody provides infinite buffer space.
TCP is relatively easy to extend. There are still some free flag bits and additional fields can be added to the TCP header if needed.
-Aaron
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
TCP seems to be getting to their webserver just fine!
(it doesn't appear to be getting out, though)
It's only a model.
SCTP is indeed interesting. I've tangled with it when playing with SIGTRAN, i.e. SS7 over IP. The nightmares ceased a while ago. :-)
One of the more interesting special-purpose protocols I've ever messed with was the PACSAT Broadcast protocol, used for downloading files from satellites.
It makes the assumption that downloaded files are of general interest, so it broadcasts them. Which files it broadcasts is in response to requests from ground stations. Ground stations can also request fills, pieces of files they haven't received yet. The protocols have hooks to allow files to be downloaded in pieces, over several orbits, since individual passes rarely exceed 20 minutes, and the downlink bit rate isn't all that high. This all runs over a protocol analogous to UDP.
You can get files by just listening, if you want to.
...laura
(NOT) Everyone knows that TCP has problems and for many years people have been developing transport protocols that enhance or replace TCP.
These guys haven't invented anything new. There are many flavours of TCP with different congestion mechanisms and there is a special kind of transport protocol that solves most problems...
I'm talking about SCPS-TP, supported by NASA and it performs very well with high bit-error links (like satellites) and it also copes with high delay. The good thing about SCPS-TP is that it's compatible with TCP, because it basically an extension of TCP.
There is another problem with using UDP based transport protocols... they usually have low priority in routers (probably because you can use UDP for VoIP...)
Fear is the mind-killer.
Its common knowledge that all Http clients (explorer, netscape, konquerer, opera, etc) use TCP/IP. Since the WWW is synominous with the internet I think the statement is valid.
I thought this was news for nerds?
>SCTP is indeed interesting. I've tangled with it when playing with SIGTRAN, i.e. SS7 over IP. The nightmares ceased a while ago. :-)
My condolances. SCTP itself is ok, but the leap of adaption protocols to
map to SS7, and not to mention SS7 itself, makes you wonder our telephone net work at all.
Who says that rating a route based on transmission time is a bad idea? The problem isn't what it does, but what it doesn't do. That is, while speed is important you also need to consider things like the reliability (drop rate) and the throughput (windowing). What this says to me is the ROUTER should find an appropriate path by creating METRICS to compare paths, oh wait... they do but at the networking layer. What they're doing is creating a layer 4 protocol that does nothing new and making it rely on their propreitary layer 3 software for determining metrics. TCP is a layer 4 protocol, path determination is done in layer 3, isn't this an issue of not following the networking model? I may be wrong, in all fairness, but anyone have an answer to my dilemma?
-meggito
Now that I finally tought my firewall to discard all UDP packets somebody goes and makes them the kingpin of some hot new protocol. I'm not paid to keep up with all these changes, no matter how good for me or some the mythical average user they are claimed to be. Just hope they don't foist a new protocol layer or application type on us until they make it MORE RESISTANT to spoof and DOS attacks and design OUT OF IT all the possibilities for errors and botched error handling around illegal message field contents or sizes. We have certainly had our noses rubbed in what abuse is possible with TCP so PLEASE, PLEASE get it right this time around.
I'll be looking for that RFC with a microscope and a shotgun.
SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
This work seems to be about two things (which I am not sure I see a strong connection between): lowering transport latency, and using available bandwidth better. The latter has been the subject of many papers in the last few years. There are now several serious proposals of how to fix TCP with respect to long fat pipes. They don't seem to support the idea that retransmissions are harmful. So I'm going to talk about the first issue, transport latency.
The idea of using error-correcting codes (ECC) to eliminate the need for retransmissions is an interesting one. The main benefit is to reduce transport latency (the total time it takes to send data from application A to B). Here is another paper proposes has a similar idea, applied at a different level of the network architecture.
The root problem here is that network loss leads to increases in the transport latency experienced by applications. In TCP, the latency increases because TCP will resend data that is lost. That means at least one extra round-trip-time per retransmission. This "Rateless TCP" approach uses ECC so that the lost data can be recovered from other packets that were not dropped. In this way, the time to retransmit packets may not be needed. I say may, because there will be a loss rate threshold which will exceed the capability of the ECC, and retransmission will become necessary to ensure reliability. But, as long as the loss rate is below the threshold, then retransmissions will not be necessary. Note that the more "resilient" you make the ECC (meaning supporting a higher loss threshold), the more work will be needed at the ends. So you are not eliminating latency due to packet loss, you are simply moving it away from packet retransmission into the process of ECC. However, if you've got good ECC, the total latency will go down.
The ECC approach may be a nice middle ground. But, it the ultimate solution to minimize latency is probably through a combination of active queue management (AQM) and early congestion notification (ECN). Unlike ECC, this approach really would aim to eliminate packet loss in the network due to congestion, and therefore completely eliminate the associated latency. Either ECC or regular TCP would benefit. In a controlled testbed using AQM and ECN, I've completely saturated a network with gigabits of traffic, consisting of thousands of flows, and had virtually no packet loss.
It should also be noted that retransmission is NOT the dominant source of transport latency in of TCP. I am a co-author on a paper that shows another way (other than eliminating retransmission) to greatly reduce the transport latency of TCP. The basic idea is that the send-side socket buffer turns out to be the dominant source of latency (data sits in the kernel socket buffer waiting for transmission). In the above paper, we show how a dynamic socket buffer (one that tracks the congestion window) can dramatically reduce the transport latency of TCP. We allow applications to select this behaviour through a TCP_MINBUF socket option.
-- Buck
This doesn't provide anything like what TCP provides, namely a connection between two network nodes that allows transfer of arbitrary data with guaranteed reliability, with automated congestion control for optimized use of available network resources.
As far as I can tell (their website could use some more straightforward actual content), this is more like bittorrent, where a file is cut up into blocks, the blocks get distributed across the network, and anyone interested in the file then reconstructs it from available data from all sources, not necessarily having to get the entire file correctly from a single source. Only it does it more efficiently than bittorrent.
The two protocols target very different uses. TCP excels in interactive use, where the data is sent as it is generated, and no error is tolerable in the single sender-to-receiver link. Bittorrent (and other distributed network protocols) target batch jobs, where throughput is more important than reliability (because reliability can be reconstructed on the client through clever hashing schemes), and where responsiveness is entirely irrelevant.
So, this could not possibly replace TCP, since it does not do what TCP is most useful for. At the same time, the criticisms aimed at TCP by the rateless designers are valid, but well known, since TCP is indeed poorly suited for high-volume high-throughput high-delay transmissions of prepackaged data.
Still, good job to them for trying to come up with better protocols for niche or not-so-niche markets. I wish them all the best.
How does it work? Well, it's layered over Rateless Internet, in which (as we all know) packets do not have to be resent. So it carefully loses all packets and relies on Rateless Internet to make sure they arrive safely at the other side and do not have to be resent. Because no packets need to make it from A to B, you don't need any network hardware, and data can be sent just as fast as your machine can drop packets.
Guess I'd better apply for a patent...
So they use extra packets to make up for the ones that get lost. How does that improve efficiency over resending the ones that got lost? With the 1-3% loss rate they cite, I can't imagine redundancy is going to improve things a lot.
Of course, there is also the issue of rate limiting, which TCP initially did very (too) aggressively (modifications have since been proposed and implemented). Rateless reduces and increases transfer rate periodically, from what I understand. I am not very convinced that Rateless does a better job here.
Please correct me if I got my facts wrong.
That's for just them. What if all hosts on the entire Internet were by design stuffing packets at a 3-5% error rate? Meltdown, that's what. Their "real-life" measurements do not scale, suffering from the usual assumed linearity of new designs for complex systems.
Sometimes people fall in love with their new ideas, thinking that the rest of the world missed something obvious.
sigs, as if you care.
This sentence on the site:
This is precisely why TCP is inherently unable to reach optimal speeds on long high-bandwidth connections.
is complete nonsense. The Window Scaling Option in modern TCP stacks lets them use long high-bandwidth connections just fine, and the congestion control features in TCP adapt themselves to the pipe.
This is acutally a C library which uses UDP therefore if they released it under the GPL then any application that builds on it would also have to be GPL. Of course it's entirely possible for a GPL application to be commercial, but this does mean that to use this in a closed source appliation you'd have to buy it from them instead.
I've just done SCTP and SS7 for my degree. please make the bad voices go away.........
So others can have fun slashdotting other technologies, here are some websites. There are probably others, but this should keep those who do really want to move away from TCP happy.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
I can see it now.
New Guy: "It's time to go, old man. I'm the new overlord of the net now!"
TCP: "No f***ing way!!! TCP is still teh shit, ye hear? Now socket my fist and take a hanky from the packet to whipe yourself off on the way out".
It was funnier in my head.
They're doing erasure correction with an error correcting code on the complete transfer. It's sort of a good idea, but it's only useful for transferring large files, because if there are dropped packets, you don't get that data until the end. This makes it useless for anything where you want to use the data progressively, like rendering a web page, image, etc., as you download it. So this isn't going to replace TCP except in special cases.
For that matter, it would be much easier and no less effective to send data by UDP and request replacements by TCP as the packets turn out to be missing, with the replacements sent at the end, rather than ending with a chunk of ECC information suitable for reconstructing the lost packets without any feedback from the receiver.
(Of course, it is an interesting math problem to figure out what information you should send after transferring a file to optimally correct a set of deletions that only the receiver knows; but in real life, you can just send the necessary information back)
The guys ofer at Digital Foutnain have had something that does this for a year or two now. And thiers works over satelite too. Wonder if they will have any IP issues. (Intelectual Property, no pun intended).
Fly me to the moon Let me sing among those stars Let me see what spring is like On jupiter and mars
ecip.com I call it Error Correcting IP, and used it to stream live video from Sri Lanka in 1997 with Arthur C. Clarke Hal's Birthday
it was a 64K shared line with 90% packet loss, I received 60Kbps for the video stream. ( I have the video to prove it )
We even filled preliminary patents on this back in 1996 but they were never followed through with.
Luigi Rizzo (now head of the FreeBSD project)also did some excellent work on this also. http://info.iet.unipi.it/~luigi/fec.html
He calls it Erasure codes.
Which is more accurate since UDP doesn't have errors, it either come across 99.999% perfect or not at all.
So there is more information then in an error situation where ever bit is questionable.
What this means almost 1/2 the hamming distance in the codes in needed to correct an errasure verses and error.
Turns out the Error/Erasure correcting scheme it critical and not obvious. I spent almost 5 years working on this part time before it started making some real breakthroughs.
My original system was designed for 25% packet loss (not uncommon in 1996).
In the inital idea we added 1 exored packet for every three data packets, but at 25% packet loss, it turns out that it didn't increase reliablity at all! Working this out with probablities was a major eye opener!
Even when you work the problem out you realize you will still need some retransmissions to make up for lost packets, there is no possible solutions without this.
I have been trying to find people to help opensource this since I have working far too hard just to survive since 2000 to even consider taking on another task.
Anyone interested in my research and carring this forward please see my site and contact me.
John L. Sokol
I am always doing that which I can not do, in order that I may learn how to do it. - Pablo Picasso
You get datagrams like in UDP but they are sent re liably.... or not depending on what you want. I believe this has been quietly implemented in CISCO routers and Unix OS vendors network stacks over the past few years.
ayershome.org/users/eric
- They have a "TCP-friendliness" option that varies the transmission rate in a way that TCP windowing can probably cooperate with, so you can set the rate knobs to something less than full blast,
- but nothing they've documented appears to address the problem of multiple users of this application trying to use a transmission path at the same time, and
- they also don't document anything that does path rate discovery - so it may work fine if you've got a couple of small pipes feeding a fat network, but if you've got a fat pipe on the sending end and a skinny pipe on the receiving end, they don't document anything that figures out what rate is safe to transmit at.
They also don't document when you would want to use this and when you would want to use TCP and when you would want to use this on top of TCP.Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
You must be new here.
The great thing about standards is that there are so many to choose from...
apterous.org
TCP uses packet loss (NOT round trip times - where did that come from?) to signal congestion, and thus to implement congestion control. This does not work well in a typical wireless (802.11X, 802.16, etc) environment, where packet losses are to be expected. TCP also does not co-exist well with lots of UDP streaming.
So, there IS a need for a TCP replacement. However, the one being developed in the IETF is DCCP. Basically, the idea is separating congestion control and packet loss replacement.
Maymounkov and company say that they are preparing RFC's (which implies that they intend to submit this to the ITEF), but have not yet done so. So, maybe if they do, and if they offer an unencumbered license, their technology could be used, but it is way too soon to tell.
one of the problems with trying to change ther version of TCP or even replacing the protocol is that protocols that use an algorithm that is not TCP friendly usually ends up hammering or getting hammered by the network. TCP vegas is supposed to be superior to TCP Reno and Tahoe however, it does not have a chance on the same network. Therefore, even though it is better, it is useless.
The war with islam is a war on the beast
The war on terror is a war for peace
I don't know what their reasoning is, but both their claims about TCP seem incorrect.
1. TCP does not use round trip time to calculate any "congestion levels." It increases the connection rate until packets get dropped, presumably because some router in the middle got overloaded.
2. Packet loss is used as a signal to TCP to slow down because it tried to send too fast. The lost packets are subsequently retransmitted, so TCP can indeed not only tolerate but recover from packet loss. The only real case they have is packet loss due to reasons other than TCP's own aggressive sending rate, such as UDP traffic, wireless links, etc.
Given these concerns, I can't help but think that they are inventing a protocol that works well only if used on a small scale. TCP is designed to back down if it thinks it's sending too fast, and is not really optimal. One can always hack a pair of TCP nodes to not play by the rules and get more than the fair share, but the problem is that that solution wouldn't work if it were adopted network-wide.
Tsunami -- You can't bring a good wave down!
If you want to replace TCP,
IT HAS TO BE FREE. It isn't. Read their license.
I have no doubt these people have patented their ideas, too. You'll see that hit in a couple years, muzzling everyone else's ability to use these fairly simple concepts.
now just have a few million people all agree on how to convert the millions of pc's, routers, etc using tcp.
-Cnik
They've already published RFCs and had them approved. The internet is switching to Rateless Internet two weeks from next Wednesday, on November 10th. After that, no more TCP/IP.
There are no trails. There are no trees out here.
They tell you how they solved the solution, but fail to tell you how much impact this solution has overall.
They're basically stating that now they can flood the connection with packets.
But they've also told you that the packets contain your data in an error correcting encoding. What they don't mention is this:
How much overhead is required by the error correcting encoding?
How many errors can the error correcting encoding handle? (drops 1 packet = ok, drops 400 packets = bad)
How much cpu computation is required to encode and decode the payload?
How is the cpu overhead managed? (how much performance will be lost by context switching, etc.)
So they're just playing the game of distracting people with the best part of thier performance measurement without bothering to mention the performance impact of all of the other trade-offs they admitted to making.
Yeah well IPv6 might be better than IPv4, but I don't see many people using it.
This may be just a wee bit offtopic, but it may be my only chance to ask...
Who remembers HSLink?? If I recall correctly it was an add-on transfer method for Procomm Plus. It allowed two people connected over a modem to simultaneously send files to each other at basically double the normal speed. I remember thinking it had to be a scam, but me and my friends tested it and were able to send double the usual info in whatever time we were used to. (I forget, 10 minutes a meg I think)
How did this work? Were we fooled or was it for reals? Could something like that be applied to dial-up internet connections?
-Don.
Cwm, fjord-bank glyphs vext quiz
TCP doesn't use RTT to 'calculate congestion'.
This is a load of fluff, trying to capitalize on the 'p2p craze'. There are plenty of TCP replacements out there, that actually make sense. As far as TCP not being able to utilize 'today's bandwidth', again...hooey. Gigabit ethernet (when backed by adequate hardware, and taking advantage of jumbo frames) moves a HELL (two orders of magnitude) of a lot more data than your typical home broadband connection...using TCP.
The way TCP works is the only thing that keeps the net running as well as it does now.
Remove TCP, and you remove the speed limit, single hosts will easily be able to overwhelm servers, a few dozen hosts will be able to overrun multi-gig cross country links with ease.
Remove TCP, and the network admins of the world will only throttle you back in transit.
netblt (RFC) purpose is also to enhance file transfer thruput (bandwith efficiency). An implementation has also been studied.
Netcraft confirms it, TCP is dead.
That which is not TCP shall not pass through any firewall worthy of the name -- ergo, it's doomed. IPV6 may be another matter...
I'd still like to see a good protocol that doesn't require in-order packet receipt in addition to the changes that they mentioned; when transfering large volumes of data, why not?
This is exactly why FTP uses UDP for its data transfer. So use FTP. In the last decade, though, improvements to TCP stacks have mostly mooted this difference. Once upon a time, when one end of a TCP transfer NACK'd a packet, it meant that packet and every packet after it would need to be re-sent (even those which had been sent already). So if you send packets 100, 101, 102 and then get a NACK for 100, you'd have to send 100, 101, 102, 103, etc. But with modern implementations, the receiver would keep packets 101 and 102, so the sender only needs to re-send packet 100, and then proceed to packet 103. This has made TCP much more efficient over lossy networks. Deferred NACKs also deal well with the problem of out-of-order packet arrival, when tardy packets show up before the deferred NACK is transmitted.
I'm very dubious that these folks' protocol can realistically replace TCP; it simply doesn't add any must-have qualities. The only place where I see an advantage is transmitting audio and video, where you don't necessarily want to retransmit lost data, and are willing to put up with minute gaps in the data stream.
A transport protocol that really deserves more attention is TTCP (TCP for Transactions). It abbreviates the TCP connection handshake, which makes it much more efficient / better-performing for very short transactions (like typical HTTP usage).
-- TTK
UDP with rateless erasure codes, which means that packets do not have to be resent
Rubbish, what happens if i send a one packet message, and it gets lost.
You didn't see the title of the story and think it was about a new antiseptic, then?
Has netcraft confirmed it is better?
Vivin Suresh Paliath
http://vivin.net
I like
Why do people always feel the need to reinvent the wheel?
Yeah, there's no room in the Internet for new protocols to be used along side the old protocols. No one adopted AIM, Gnutella, Real, or BitTorrent because they're not backwards compatible.
Because then you're going to have the suits trying to push it down, no matter how great/useful it is in an effort to kill the possibility of coming out with something that could make pirating any easier or more efficient. That's the only way they're going to see it.
Surely if that was the case, the music industry would have been against radios... oh, wait...
From my syslog:
Oct 21 10:35:31 bonnet firefox-bin: gethostby*.getanswer: asked for "www.rateless.com.nyud.net IN A", got type "39"
They might want to change their name from "rateless" to "clueless".
http://64.233.179.104/search?q=cache:tbc9c09Q1xsJ: www.theregister.co.uk/2003/10/16/sun_gives_glimpse _of_revised/+new+TCP/IP+stack&hl=en&lr=&strip= 1
Encoded information just increases the overhead so the efficiency... will be reduced
:)
However I heard of this code:
1: enable, on
0: disable, off
TCP is just fine improvements are yet to come.
I want to be the first to invest in this new technology, and make a bazillion dollars. Better yet why not progress on to VORIv6?
Can someone explain why they chose the UDP protocol? I thought it was not a stateful protocol. And why does DNS use UDP? What is so good about UDP?
If you don't mind the overhead, add FEC at the link at the point of loss.
So where is the point of loss?
But anyway, solve [packet loss] at the point of the problem, not end-to-end.
What if neither participant in a conversation has the capability to isolate the point of the problem, let alone to fix it? What if the routers between the participants are run by faceless multinational corporations who decline cooperation with residential customers?
Lets assumes that their proposed solution runs at 100% efficiency. However, for them to be able to recover a lost packet, they will need to add redudancy to each and every packet. You can't recover data that has zero duplication. The end result is, their solution, even when working perfectly isn't likely to be better than the current solution(TCP) as far as bandwidth utilization is concerned.
Hey, I'm pretty biased having worked for DEC Field Service for 12 years. It seemed the only thing keeping DEC alive was services and the hardware engineers thinking of ways to make our lives easier. You're right that users will find a way to break things. When I was a junior tech, a senior tech once told me "every service call is a prank ... look for the easy stuff first."
Ken Olsen messed up by not embracing personal technology, other than that I had fun as a tech.
DECNet and DECNet/VMS clusters were/are great technology, easy to setup but not too scalable.
-ep
Does it support the evil bit?
Life today. Uncertainty tomorrow.
It is the old-math speak usage of "/" as in "three over four" is "3/4"
So TCP/IP is "Transmission control protocol over internet protocol" or just "TCP over IP".
While you don't see it much, this also inferrs "HTTP/TCP/IP" and so forth.
So just as "voice over IP" doesn't make voice and IP the same, TCP over IP doesn't really relate one to the other.
After all, if you use dialup, the first hop of your connection is typically TCP or UDP over PPP (Point to Point Protocol) and the ISP acts as a gateway from the PPP to the IP semantics.
(I'm guessing the immediate parent knew this, but this reply "hangs best" here in the thread. 8-)
Innocent people shouldn't be forced to pay for inferior software development.
--"Code Complete" Microsoft Press
Packets WILL need to be resent if they get lost. What bullshit. Packets generally don't get lost in this day and age, but it is POSSIBLE. So I say, Bull Shit.
--Slashdot: News for Turds. Stuff that Splatters.
It doesn't work. Somebody is full of sh*t here. And we all know who it is.
--Slashdot: News for Turds. Stuff that Splatters.
No, I say, fuque, fuque, fuque, all of this. It isn't necessary. It is people trying to gain a reputation. It isn't necessary. TCP works fine. It'a all total bullshit and we all know it.
--Slashdot: News for Turds. Stuff that Splatters.
First order of business is "What is the problem?" This is never really adequately defined, near as I can tell. Some vague references to some "problems" with TCP without any objective data to support the statements. Since a solution is being proposed to a problem, we need some way to measure the problem objectively. How else can you tell if your proposed fix is better or worse than the baseline? Since the problem is never defined in any detail, and certainly never to a point of which you can measure, and the fact that there is no objective data, makes the whole thing totally suspect right from the start. In a nutshell, extraordinariy claims without extraordinary proof
Regardless, there are some nuggets here and there that we can pick apart (from the website):
#1 Rateless Internet is an Internet transport protocol implemented over UDP, meant as a better replacement of TCP. TCP's legacy design carries a number of inefficiencies, the most prominent of which is its inability to utilize most modern links' bandwidth.
#2 This problem stems from the fact that TCP calculates the congestion of the channel based on its round-trip time. The round-trim time, however, reflects not only the congestion level, but also the physical length of the connection. This is precisely why TCP is inherently unable to reach optimal speeds on long high-bandwidth connections.
#3 A secondary, but just as impairing, property of TCP is its inability to tolerate even small amounts (1% - 3%) of packet loss. This additionally forces TCP to work at safe and relatively low transmission speeds with under 1% loss rates. Nevertheless, our extended real-life measurements show that highest throughput is generally achieved at speeds with anywhere between 3% and 5% loss.
Point #1 is a claim without any evidence to back it up. Now most of us may think there is a hint of truth to this from experience. The danger lies in making an assumption that the problem is TCP. It could be a bad implementation of the stack you're using, some kind of packet loss in the path, or you're just trying to send too much data given the conditions of the test (ie, a Pentium 90MHz trying to fill a 10Gb/sec pipe). There's just too many variables that influence the amount of link utilization to make such a generalized statement that "TCP is inherently unable to reach optimal speeds on long high-bandwidth connections". There was a time when two Sun machines sitting next to each other couldn't utilize the entire capacity of a half duplex ethernet, for example. Yet it's not uncommon to have more bandwidth to your home via cable/dsl and make full use of it even on cross country/inter continental links these days without a whole lot of change to the fundamentals of TCP. Therefore, it's unreasonable to conclude that TCP is the cause of any of the problems, perceived or real, considering the history of TCP and it's scaleability over time and total lack of any effort to rule out other causes.
Point #2 is a case of where a little bit of knowledge is much more deadly than none at all. I'd be willing to chalk this one up to a marketing person's copy than any particular design insight by the developers, but it's spooky that it's there. At a fundamental level, queueing theory tells us that the average time for a unit to be processed in a queue is related to the depth of the queue. If there's nothing, or effectively nothing, in the queue when your packet arrives, there is no delay in sending the packet. However, the delay grows as the depth of the queue grows. This shows up as variability in the round trip times of packets, relative to each other. Also known as "jitter". Therefore, no jitter means there are no queueing delays, which means that there isn't much traffic competing for access to the links. High jitter means there are a lot of packets competing for access to a link, and therefore much more likely to be c
There is also work under way to approach this sort of issue using a new protocol called DCCP. DCCP sits on top of IP rather than UDP or TCP so is at a lower level. DCCP stands for Dynamic Congestion Control Protocol and has some reasonably interesting people behind it. Here are some links referencing it: http://www.icir.org/kohler/dcp/ http://www.ietf.org/html.charters/dccp-charter.htm l
I'm actually looking to do a PhD in research on DCCP shortly so this discussion is quite interesting!
There was something about stramlining the TCP protocal which could speed up the internet and other networks. See http://www.newscientist.com/news/news.jsp?id=ns999 93799 and http://en.wikipedia.org/wiki/FAST_TCP
Murphy's Law of Research: Enough research will tend to support your theory.
TCP performance isn't blazingly fast, because as you say it's a really simple protocol. But that just means that you shouldn't use it for things it's not designed for - it's a really fine protocol for downloading operating system code to a chip that's running a minimal bootstrap environment and doesn't have anything better to do until it's booted, as long as you're running it in a relatively benign environment.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
IPv6 does provide bigger addresses, though classless addressing, firewalls, and NAT have given us an extra decade or two of slack. It's also supposed to give us mandatory built-in security, though IPSEC is a stopgap for that, and it's supposed to improve routing by providing hooks for things like geographical aggregation of IP addresses, though I don't think the real world has really deployed any implementations of that.
As you say, however, they don't provide much justification for this being better than TCP (and I couldn't find much besides the Slashdot article claiming they did), much less being a general-purpose replacement for it. In particular, it doesn't appear to do any congestion control, at least in the parts that are documented - it does have an option for varying transmission rate so that it can share bandwidth with other protocols rather than hogging the whole pipe, but there's no documentation indicating that it attempts to solve simple problems like transmission from a fat port to a skinny port, much less sharing bandwidth between large numbers of sessions or accommodating buffer congestion on the receiving device, which TCP has done lots of work on over the years.
There are some things it does that are interesting, and it may be an especially useful tool for multicasts where TCP isn't appropriate (there's been tons of work on reliable multicast protocols over the last decade or two, some of which is actually usefully implementable - this stuff may help, if integrated well, because you'd really like to minimize packet losses and retransmissions in a multicast environment, as well as not maintaining large numbers of 1:1 sessions.)
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
If the new Microsoft strategy of web-served
office applications metered by usage, in
conjunction with web-based storage (ala
WebTV), and enforced by DRM and "Trusted
Computing", why wouldn't M$ also force
NetBIOS back down our craws?
I was halfway through an medium sized SS7 project before it was canned by our client.
Happiest day of my life.
While we're developing a new protocol, it needs to be designed so that the data encapsulated within it can be easily encrypted and decrypted; with anyone from bosses and ex-lovers to the FBI trying to snoop on what people are reading, writing, or downloading, encryption at the packet level seems like the only real solution, since it's unreasonable to expect EVERY software program in the world to suddenly start supporting encryption of userdata at the program level...
Maymounkov in Bulgarian means Monkey's (of Monkey) and is pronounced My-moon-khov.
I am not monkeying around, look it up for yourself in the English-Bulgarian Dictionary.
TCP has no problems saturating links in most real world conditions.
It has problems saturating links when you have a single _short_life_ TCP connection over a very high bandwidth link and high latency link.
When you have MANY tcp connections from many users it works fine. And this is the situation for us (the many), and I forsee it will remain so - e.g. the bandwidth demands of a hundred million customers vs the bandwidth demand of a bunch of researchers.
Researchers in academic labs can go whip up whatever protocol for their own use.
So I don't see this replacing TCP, especially given the limitations of using FEC.
FEC has costs - you need to transmit redundant data - so you can't use 100% of the bandwidth. You probably need to interleave packets so that a burst doesn't wipe everything out - this leads to latency. The interleaving has to take account of the longest typical/common error burst/interruption will last.
Where I see this being useful is multicast/broadcast or extremely high latency links:
**Multicast
Say you are multicasting a 1Mbps stream to 1 million users, since you use multicast you don't need 1 Tbps. But believe me you don't want to receive Acks from 1 million users! Neither do you want tens of thousand users (with suboptimal home systems) asking you to resend packets - because you just don't have spare bandwidth to do resends.
This is where FEC can be useful. You send a stream with enough redundant data so that clients can reconstruct it. e.g. you send a 1.5Mbps stream.
However this means clients _ABSOLUTELY_MUST_ now have at least 1Mbps to 1.5Mbps spare capacity (depends on FEC used) - otherwise good luck regenerating the 1Mbps stream! Of course this is the naive case, with some math and smart people you can probably come up with a protocol to cater for users with slower connections - but there will be tradeoffs and it will be "special case" (it'll probably start to look more like download to harddisk first then playing it e.g. super high latency, and a lot less like streaming ).
This is the problem - there are many low-bandwidth clients on the Internet.
**Extremely High latency links.
If you have a space vehicle 10 light hours away, retransmissions and Acks do NOT work very well...
But if you can't wait 1 second, then FEC still isn't going to be a magic bullet to solve the problem of transcontinental link latency and still maintain high bandwidth.
You'd need MULTIPLE genuinely independent transcontinental links. Then you synchronize and send your FEC streams over those links. That way spurious interruptions and errors are unlikely to affect all the links.
But that seems like such a special case.
Maybe someone can think of how this would be useful to most of us here.
Games at the moment seem to pick one of two methods. They pick UDP, with their own reliability layer whacked on top (see: any ID software title), or they pick TCP, and suffer lag spikes (see: Ragnarok Online.)
Would this new protocol provide a way to implement reliable datagrams in a fashion that game protocol designers won't have to keep reinventing the wheel? I'm sick of all this iteration being repeated, again and again, ad infinitum. :-/
Karma: It's all a bunch of tree-huggin' hippy crap!
...so let's fix it!
Is Capitalism Good for the Poor?
they must be violating them in order to implement suspend/resume. Wonders can be done once you trash the layers. Usually to no good end...
High latency+high bandiwdth connection can benifit from large window sizes on both ends using TCP + some other optimizations, MS already tries to do away with SYN/SYN+ACK in IE in a bad attempt to accelerate the connection buildup. That said, there is nothing inherently wrong with TCP as a protocol, its just that we suffer from original implementation.
When TCP was first developed, memory and bandwidth both were at a premium. All implementations thusfar have usually simply followed the BSD code, and standard mutual understandings have been grudgingly accepted allong the way to improve the performance.
Yep, I proclaim there is nothing wrong with TCP. Only the way we use it.
Now we have high bandwidth channels for the average person. Cool. Its the high latency channels that present a problem. For this reason we end up with TCP accelerators like Mentat's SkyX product or the Via Sat Linkway or the iPSTAR useage of Nettgain. The main difference is these applications do not require you to recode your user app. And non of them would be needed if TCP were retuned on all endpoints.
Damnit, quit trying to replace TCP, and quit stepping on the layers.
Thats my TCP rant.
I think you underestimate just how much I just dont care.
Thanks.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Whoa! Even preparing RFC's! OMG!