Slashdot Mirror
BitTorrent Calls UDP Report "Utter Nonsense"
Ian Lamont writes "BitTorrent has responded to a report in the Register that suggested uTorrent's switch to UDP could cause an Internet meltdown. Marketing manager Simon Morris described the Register report as 'utter nonsense,' and said that the switch to uTP — a UDP-based implementation of the BitTorrent protocol — was intended to reduce network congestion. The original Register report was discussed enthusiastically on Slashdot this morning."
BT may have the best of intentions here in developing this experimental protocol, but this quote leads me to believe that their understanding of the problem is terribly naive:
It so happens that the congestion control mechanism inside TCP is quite crude and problematic. It only detects congestion on the internet once "packet loss" has occurred - i.e. once the user has lost data and (probably) noticed there is a problem.
Packet loss is a normal and deliberate mechanism by which TCP detects the maximum thoughput of a path. Periodically it increases the number of packets in flight until the limit is reached, then it backs off. You have to test again from time to time, in order to increase throughput if more capacity becomes available. This in no way incurs "loss of data" or a noticeable problem. Packets lost due to congestion window growth are handled by the fast retransmit algorithm, which means that there is no timeout or drop in throughput (that would be pretty stupid if the whole purpose of growing the congestion window is to _maximize_ throughput).
I wonder if Simon Morris was merely oversimplifying for the benefit of the layman, but I still find that statement disturbing. As I sugggested in the other thread, it really sounds like they're going to reinvent TCP (poorly). That's not to say you couldn't design a better protocol specifically for point-to-multipoint transfer, but I question if they're on the right track here.
...we should have known they are full of shit.
Obligatory blog plug: http://www.caseybanner.ca/
Nonsense? From The Register? Ya don't say.
How we know is more important than what we know.
My ISP, Comcast, is already on top of this new bittorrent over UDP idea and has summarily blocked all UDP traffic.
So, I wonder if I'll be experNO CARRIER
unknown host slashdot.org
Just disrupt the deflector shield with a tachyon burst.
I don't really see how this is going to kill VoIP and online gaming. Those two services are big users of UDP, no doubt, but its not like all of a sudden the explosion of UDP requests is going to sqeeze VoIP traffic out. If anything it should encourage ISPs and providers to increase the rate of their roll out of new tech.
This move should bring into focus the last mile problem that is the real source of most of the internet connection speed debate. I don't care how the solution ends up working, but I think there needs to be a plan given that most of the plans I have heard involve several years of lead time.
"Hear", not "here".
I'm so excited I just made water in my pantaloons!
we have an opportunity to detect end-to-end congestion and implement a protocol that can detect problems very quickly and throttle back accordingly so that BitTorrent doesn't slow down the internet connection
----
The major problem I see is that UDP doesn't play as nicely as TCP. Not by a longshot.
As soon as TCP notices a single packet loss, the Jacobson Algorithm kicks in and it's throttled to maybe 50-60%, and raises the limit slowly. I highly doubt that uTorrent's reworked version of UDP will play this nicely.
As soon as TCP's throttling kicks in, space will be cleared in the tubes. uTorrent will be able to send more data through UDP without noticing any loss, so it'll quickly move to fill this space. Then, TCP gets hit with more data loss - and goes slower. It seems like a vicious cycle.
everyone thought dht was great too, but I found every time I used it it caused massive headaches. I would jump on a popular torrent and for days afterward I would be having poor performance, checking logs etc would show several dozen connection attempts per second on the utorrent port, even 2-3 days after I was done with the torrent because the DHT tracker was still advertising my IP address. I'd have to release renew to bring my performance back up. This was with a fairly standard Linksys router. Any situation where the other party might not just get the message that I'm not there anymore is bound to lead to headaches on popular torrents.
I encourage the BT folks to work on new protocols and push the envelope. I'm a firm believer in TCP but I see no reason to not try and do better. This article however, doesn't tell me anything other than BT says "Will not," to the Register's "Will too!"
It can be convenient to prioritize UDP over other traffic for simple QoS at a shared broadband gateway. It catches DNS queries, VoIP, gamers, and most anything else small and sensitive.
In an attempt to avoid ISP filters the P2P users sprawl across all 64k TCP ports. Now the UDP portspace will be covered in P2P crap too. There are lots of major IP protocol numbers left... at least in my copy of /etc/protocols. I wish they'd used a new protocol for this instead of UDP.
But then again, I think we all know the real motivation for this effort is simply to make it more difficult to segregate P2P at the ISP. I don't really care about that; my problem is the collateral damage.
but a new protocol for p2p traffic sounds good to me.
It'd be nice if it allows for faster transfers and less congestion, but what I'd really like to see is something that makes it harder for ISPs to detect/penalize the traffic and more difficult for people to track what other people are transferring. Right now it's simple for your average unlicensed investigator to gather lists of IPs by monitoring torrents or just sharing out the material themselves and seeing who bites.
I wonder if you could use something like this to flood those lists with forged IPs, requesting entire files to be sent to other machines which just silently drop the traffic, all without causing bandwidth problems for anyone else. I'd hesitate to do something like that even if I didn't have to worry about causing random people connection trouble or slower file transfers, but the thought did pop into my head that if the traffic used by this protocol were made insignificant enough others might look into that sort of thing.
TCP guarantees in-order, mildly error-corrected, delivery of transmitted *DATA*. Not packets. It is a streaming protocol where the data being transmitted is of unknown and indeterminate length, or open-ended length. Since BT already doesn't care that files arrive in pieces at the beginning, middle or end (well, it does a little, but not enough to matter) then you can relax and basically eliminate one of the TCP guarantees right there: in-order delivery of data.
You can eliminate sliding windows, ACK-based retransmits, fast-retransmission, and pretty much every other mechanism that TCP uses to guarantee in-order delivery of data. You can simplify it and the application itself beautifully, and provided it correctly throttles back based on detected packet loss (it MUST be exponential back-off,) you end up with a net win for those reasons. The application can set up its own optimised data structures that don't necessarily have (but likely will end up having anyway) the overhead of an OS-backed TCP stack.
I mean who cares if you miss pieces, until the end when you can re-request them?
Heck, there are already P2P apps that use a UDP-based transfer mechanism and they are WAY less impactful on systems that a typical BitTorrent stream is. They way the hell slower, too, but that's not the point.
I do think there is a point that bears repeating: BT *MUST* have exponential back-off. If it doesn't there is logic already built-in to core routers, ISPs, and firewalls that WILL drop the connection more severely when the endpoints don't respond properly to an initial packet-drop attempt at slowing them down.
There are some really nice academic papers about it, and there are lots of algorithms and choices that companies have. They all assume TCP-like back-off on the endpoints, and they ALL uniformly punish greedy floods.
TCP, UDP who cares, I'll continue to leech off insecure wireless networks for all my piracy needs.
Oh how I love:
http://www.phenoelit-us.org/dpl/dpl.html
The probablem with BitTorrent is not that it uses a large amount of bandwidth, but that it's using the wrong bandwidth. In every country other than the United States, international bandwidth is substantially more expensive than local bandwidth, and often in short supply. Local bandwidth is cheap, or even free. Even in the US, inter-ISP bandwidth has the same cost issues, but is plentiful.
What I've never understood is what's the excuse for not implementing a peer selection algorithm that prioritises nearby users. Even a naive algorithm is going to be vastly better than a purely random selecton. Simply selecting peers based on, say, the length of the common prefix of the IP address will often produce excellent results. Why in God's name should I transfer at 0.1 kbps from some guy in Peru, when a peer down the road could be uploading to me at 500 kbps?
The truth is that the BitTorrent folks are not playing ball with ISPs. In reality, I think most major ISP could care less about copyright violation, or excessive bandwidth - it makes people pay for more expensive monthly plans - but they DO care about international bandwidth costs.
If they just took 10 minutes to revamp the peer selection algorithm, they would reduce the impact in ISPs enormously, and then they woudldn't be villified and throttled.
was discussed this morning...which means all the funny comments were expended before the really funny article was posted.
Sig this!
If they want to improve network congestion why not start by implementing a better peer selection algorithm. IIRC currently peers are selected at random. A network topology aware peer selection algorithm might improve network congestion a great deal. Currently I see peers which are on another continent being 'preferred' (to due the randomness) to peers on my own ISP's network, with which I have a 50+ mbit connection.
The truth is that the BitTorrent folks are not playing ball with ISPs. In reality, I think most major ISP could care less about copyright violation, or excessive bandwidth ...
Unfortunately, the major ISPs are components of conglomerates whose primary moneymaker is selling "content". As such they have a perverse incentive structure that can put "protecting against piracy" above the quality of the network's operation.
The networks also provided asymmetric transport and vastly oversold their bandwidth, assuming a central server / many small clients "broadcast media" model. The rise of peer-to-peer usage bit them mightily and Bit Torrent was the spearhead of that rise. So rather than spending the added billions to expand their backbones to meet their advertised service's requirements they chose to throttle it.
The ISPs were the ones to turn this into a war and fire the first shots. BitTorrent is just trying to engineer a solution on which to build peace - and is being vilified for the attempt.
Having said that, your suggestion for improving things by smarter selection of peers is good. Unfortunately the Internet doesn't have any easy mechanism to indicate which peers would be better. Good solutions would likely have to be built on additional knowledge - which implies a database to hold and serve it - which implies a new central infrastructure and queries of it - which both breaks the decentralized model and provides additional points of attack if the ISPs continue to treat this as a war and attempt to suppress "unauthorized"/"enemy" torrents.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
You'd think the VP of ANYTHING would know to properly capitalize Internet.
Assuming 1400 byte max packet data, 8 bytes of file position (i.e. files may be bigger than 4GiB), 4 bytes to give the index of a file, that's 1388 byte packets... 3094357 packets at least for 4GB, an extra 32MB.
Note that UDP packets are 12 bytes smaller than TCP, so we are already breaking even. Since we control the protocol, we could e.g. download files in 4GB chunks so that we could use an 4 bytes of file position. In principle we could also have the source IP and port number uniquely identify the file, and thus eliminate the the file index from the packet. Then we are saving 8 bytes per packet by using UDP.
Don't forget as well, that the main guy behind Hayes modems, Dale Heatherington had just a 2 year degree from a tech college, when he co-founded Hayes, and helped create the net in his own way (and Dale's also a really nice guy)
He's still someone who invents amazing things, and I always love to give his creations a look over, and despite me having a masters in robotics, his robotic creations always make mine look shabby.
Back on topic though, education isn't everything. I'm not a networking person, Bennett supposedly is. Why is it I could see what was wrong in his statements? Whats more, like the standard, I took the chance to both try the client out, and talk to the people behind it - mainly the tireless Firon, as he talked me though the protocol, and it's hopes, as well as his thoughts, as Community manager (and so the one people ask everything of from 'where do I get invites' to 'why is my utorrent sending out improperly bencoded packets') on the article. It can be read here
Wait until the bullshit report has MPAA backing
An SQL query goes to a bar, walks up to a table and asks, "Mind if I join you?"
I can't seem to find any actual info on uTP, so I pose my question to my fellow peers. Technically, how would they overcome to limitations of UDP to end up with a valid file. packet order and verification would be lost right?
I just got back from living in Japan not too long ago. Over there, we got 100 mbps up AND down for about ~$40/month. Near the end of my stay there, I got a letter from my ISP stating that they're going to start implementing a bandwidth cap, and that no user can upload 500gb/month, but downloading is still unlimited.
This year, I read on Slashdot that AU/KDDI is unrolling a 1gbps line for a similarly cheap price.
If you want my sympathy for ISPs in America, get back to me when I get even one tenth of the service I got in Japan. I'd like to extend a big middle finger of gratitude to all American ISPs. No one is spouting the gloom and doom over in Asia, and meanwhile, they're shooting ahead into the future of the internet. Asia's been rolling out fibre-optics, at great cost to them, but with spectacular results. Conversely, we sit on our asses and wonder how we can charge more money to more users to put them through the same amount of pipes without upgrading infrastructure.
**** you, American ISPs.
Signed,
Someone who's seen the other side of the world, and it was better.
UDP-based protocols tend to operate in userspace, unless this proposed protocol is bringing out a kernel module which I doubt it is. While UDP is good for lossy data like voice or video, it is a terrible idea to put all of the logic of maintaining packet order / data consistency inside userspace - because you have you to cross the kernel space into the userspace, instead of the lower level waking you up saying 'hey I have a packet for you, it is in the correct order and its been verified' - you will have to deal with 'hey I have a packet for you, I have no idea where this packet fits into the data stream and whether this is a packet that I have already given to you, but here it is anyway', it might be terribly more in-efficient than TCP based torrent transfers. Saying this, there may be some good in UDP, swarms can use true multicast (which is impossible with stream oriented protocols) to save overall bandwidth.
The problem is that any peer selection algorithm will inevitably fragment the swarm reducing the efficiency of bittorrent protocol to the extent that it's not worth it.
When you allow high number of connections on bittorrent, almost all home routers die as their session memory fills up. On Azureus I've noticed that the maximum is somewhere around 256-512 simultaneous connections and even such a low number sometimes freezes up my Netgear router, so I am forced to use Linux box instead and can now put the connection limit to 8000-16000 connections and get much higher download speeds. Switching to UDP should bypass the session limit problem on cheap router boxes, so it is all good.
- Raynet --> .
If you are Glass House you had cruel parents
uTorrent goes into niche protocol used by only few applications by endusers. It will be much easier to cut off torrent streams.
I had said this yesterday. It all sounds to suspicious for me to take seriously. It sounds like the MPAA/RIAA are trying to scare people.
Stupidity only gets you so far, then you've gotta try
Bill Gates = Dropped out after 2 semesters at a local technical college. If you don't know who he is, leave Slashdot immediately.
Don't pretend like you know what you are talking about. Bill Gates attended the "Glamorous" Haavad University
I dont know about the other but would highly doubt them due to you being pompous while saying something totally wrong.
You only have the right to be an asshole if you aren't a doofus.
Ars had an article about a technology that was made to help address this issue. Might be of interest to someone:
http://arstechnica.com/news.ars/post/20081103-comcastic-p4p-trial-shows-80-speed-boost-for-p2p-downloads.html
Why, yes I have been touched by His noodly appendage. And I plan to sue.
Unfortunately the Internet doesn't have any easy mechanism to indicate which peers would be better.
Take the ones with reasonably consistent low round-trip times. That's probably a good heuristic indicator that they're close by, which probably also means cheap.
In any case, it's probably better than picking uniformly at random.
This is an important topic.
Why was my submission on HTTP on UDP rejected on 2008-11-04
Slashdot = Sarcasm
Would you agree for BT on ICMP?
I'd like to buy homeland for our 10 million people. http://twitter.com/mahadiga