Slashdot Mirror
Why BitTorrent Causes Latency and How To Fix It
Sivar recommends an article by George Ou examining why BitTorrent affects performance so much more than other types of file transfer and a recommendation on how to fix it. The suggestion is to modify P2P clients so that, at least on upload, they space their traffic evenly in time so that other applications have a chance to fit into the interstices. "[Any] VoIP [user] or online gamer who has a roommate or a family member who uses BitTorrent (or any P2P application) knows what a nightmare it is when BitTorrent is in use. The ping (round trip latency) goes through the roof and it stays there making VoIP packets drop out and game play impossible."
Hey, I have a really spiffy idea. How about creating a router that can determine which packets take precedence? I'll make millions off that idea...
What? Oh, damn Linux! What? Oh, Windows can do it too now? Why do I always have the good ideas about 10 years too late?
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Don't download porn while playing WoW.
Do you know how many times I've died in WoW because of his porn downloading?
He's paying up, I need my epic flying mount...
My ZooLoo
Why BitTorrent causes network bandwidth to be used. And network packets to be sent & received. Really sometimes I wonder.
While I prefer Tomato on a WRT-54GL, that would do absolutely nothing at all to solve this issue. A router behind a modem can really only regulate the upload, and can't easily prevent a flood of data on the downstream side.
This issue is with the queue on the Telco's DSLAM, or on the other side of the cable from the modem. This is more like an invited DDOS, which no amount of filtering at or behind the modem can resolve, because the modem is getting the traffic from the DSLAM after it goes through the queue.
The only way to have QOS solve this issue would be to ask the telco to do the QOS for you, and the amount of processing power to do that nicely isn't trivial.
If I have nothing to hide, don't search me
Homebrew traffic shaping. *facepalm*
I don't mind traffic shaping at all, anywhere. QoS is a good thing, even when the ISPs do it. What I mind a whole awful lot is traffic blocking, ala Comcast.
Dewey, what part of this looks like authorities should be involved?
And we admit that on a small scale, we need to control our eating, but we don't want the grocery store telling us how much of things we can buy.
Traffic shaping and QOS will help a little, but the real problem is simply that you can't afford to delay priority traffic by more then one or two full-sized packets on any connection less then a few megabits (meaning: just about all home interconnects). If you wait any longer then that, it becomes noticeable.
Traffic shaping and QOS are not usually able to make that guarantee. A straight priority queue with bandwidth guarantees can, as long as you are able to actually classify the torrent traffic differently from your other traffic.
Part of the problem is that it is often not possible to distinguish between the batch and the interactive traffic with Shaping/QOS. Not only is QOS almost universally set wrong, but the simple fact is that one can mix interactive and batch traffic over the SAME ports (http, ssh, dynamically allocated ports)and that can make it virtually impossible to use traffic shaping or QOS to keep the mess away from your interactive traffic.
The best general solution is to use a straight priority mechanic with minimum bandwidth settings to separate as much of the bulk traffic out as you can, and then run fair-queueing at each priority level to take care of any that leaks through. This will do a very good job cleaning up the traffic. DragonFly has a fair-queue implementation for PF that does this. There is also at least one fair-queue implementation for PF in the wild.
Fair-queueing essentially classifies connections (the one in DFly uses PF's keep-state to classify connections), generates a hash and indexes a large array of mini-queues. One packet is then pulled off the head of each mini-queue. One enhancement I would like to make to the DFly implementation which I haven't done yet is to use the keep-state to actually determine which connections are batch and which are interactive, and have a parameter that allows the queue to give additional priority to the interactive connections by occasionally skipping the hoppers related to the batch connections. A quick and dirty way to do that is to simply check the queue length for each mini-queue.
In anycase, its a problem for which solutions are available. Regardless of what you use it has become apparent in the last few years that the only way one can classify the traffic well enough to properly queue it is by building keep-state knowledge on a connection by connection basis.
-Matt
We long ago learned that when inserting time between protocol events that it is far better to use a time randomized between an upper and lower bound than to use a repeating interval.
When fixed repeating intervals are used, separate instances of a protocol (and other protocols that use repeating intervals) slowly tend to fall into lock-step patterns with pulsating waves of traffic in accord with those patterns.
In other words, fixed protocol timers can create the traffic equivalent of the Tacoma Narrows bridge.
By-the-way, ping (ICMP Echo request/reply) is a terrible way to measure network latency. ICMP is often a disfavored form of traffic as it crosses routers, sometimes even rate limited.
There are better tools for measuring link properties, for example there is "pchar" - http://www.kitchenlab.org/www/bmah/Software/pchar/
I worked on a method to do even better measurements, but I put it aside several years ago: Fast Path Characterization Protocol at http://www.cavebear.com/archive/fpcp/fpcp-sept-19-2000.html
I have my torrents capped to 1/10 of the advertised connection speeds, but latency still affects me (very visible in ssh sessions to my remote irssi server)
My UID is prime... is yours?
It is always easier to manage uplink bandwidth from downlink bandwidth, simply by virtue of the fact that you control the actual packet queues.
Downlink bandwidth can be controlled in numerous ways. The easiest way is to actually run the incoming packets through a bandwidth limiter with a very large packet queuing capability. This will cause a ton of packets to build up in front of the limiter and eventually fill the TCP windows of the senders. The packets that get through the limiter will cause a stream of ACKs back from your machines at the desired data rate. The combination of the two will cause the remote senders to band-limit the packets they send to the bandwidth you desire.
when running incoming packets through a limiter you still need to traffic-shape/QOS, priority-queue, or priority-queue + fair-queue the packets going through the limiter. If you don't then your interactive traffic can wind up getting stuck in a packet queue with hundreds of packets in it. In addition to that you may have to control the advertised TCP window or even implement RED on your limiter to prevent the hundreds of packets built up in front of the limiter from turning into thousands of packets.
If you can classify the bulk traffic then you can use virtually any queueing mechanic. If you can't classify all of the bulk traffic then the only mechanic that will work reasonably well is, again, going to be a fair-queue.
Fair-queueing is not the holy grail but it is typically the most effective mechanism when combined with another queueing mechanic, such as a priority queue.
-Matt
Read the bloody article. He shows that bittorent traffic capped to 10% of total bandwidth still causes more latency than an http download using 90% of the pipe. The total latency hit is small, but still significant for VOIP or high intensity gaming.
--why?
That doesn't address the number of open connections issue. Bittorrent clients can often have hundreds of open connections while a browser or a game may only have 1 or 2 connections open. So when the game sends a packet, the router gets it and recognizes that it is connection 99 of 100 open connections. If the router equally prioritizes every packet, then the app that only utilizes a single connection can still wait before being serviced.
It also doesn't solve the problem of having a roommate who will leave bittorrent on indefinitely.
The real solution is to come up with a way to analyze packets and determine which packets should have the highest priority. This is called Quality of Service (QoS). Linux and routers based on linux have access to a number of different QoS schemes, but the off the shelf routers may not have good enough hardware to run it. For example I bought a ddwrt compatible router. I dumped the original factory firmware and installed ddwrt. I turned on QoS and put http and other types of traffic at higher priority than the rest. It worked great when the router could handle the traffic. I could let the bittorrent client eat as much as it wanted but when I hit a webpage, the page loaded just as fast. But every once in a while the router would crash or become really slow and inaccessible (can't access it through ssh or http). Turning off QoS alleviated that issue but of course bittorrent would starve out the other apps. In the future I plan on buying a router with a faster cpu so I can leave QoS on.
IMHO, Cisco has the best packet queueing mechanisms that I know of. I've been using their fair-queue stuff for years, and it has only gotten better with each iteration of IOS.
When I went from a T1 to a DSL line to save some money I immediately noticed the missing cisco. That little 2620 was so nice. PF couldn't hold a candle to what the 2620's fair-queue could do so I sat down and wrote a fair-queue implementation for PF (for DragonFly). It still isn't as good as what Cisco has, but it gets a lot closer then the other PF queuing mechanisms get.
I think the bit I'm missing is the batch classification. My fair-queue can still get overwhelmed by dozens of batch TCP connections if I happen to not be able to classify their traffic (and they wind up on the standard queue instead of the bulk queue). The set-up is a priority queue with minimum bandwidth guarantees plus a fair-queue at each priority level.
I keep hoping someone will take up the flag and finish it.
-Matt
When are ethical issues not directly derived from self interest? The issue with throttling at an ISP level is receiving the service one pays for. Bandwidth shaping for a personal network, deciding what one would like to do with the service they purchased, is an entirely separate issue.
http://www.cypherpunks.to/~peter/zdnet.html Schneier is a moron if he thinks telling Hollywood no will force them to use non-DRM content. All you need to do is look at the CableCard fiasco. You give Hollywood the finger and they give you the finger right back because they'd
rather NOT have any content on the PC to begin with. Like Apple, Microsoft
will humor Hollywood so they come join the party. Once they're in, they'll
get screwed out of their DRM protections because Microsoft won't patch the DRM
holes and let their customers bypass DRM. The latest DRM stripper for Windows
Media has worked for almost 2 months now and Microsoft hasn't patched it yet. Ok, so it's nasty to call someone a moron. And it's not really true either. It's ideology that causes Schneier and all the Web 2.0 'experts' to say this. He's no fool but he can't differentiate between it would be good if something being true and something being true. It would be good if Hollywood would give up on flakey DRM schemes. But if Microsoft and Apple had somehow agreed to boycott them, then Windows and Mac users would just have been left with no way to play HD content, because Hollywood is mortally afraid of people ripping HD content and uploading it to Pirate Bay. But George Ou is right that once stuff gets on open platforms like the PC it will get cracked anyway, so the OS vendors were just humouring them. And they probably knew it. FOR THE LAST TIME, I want the DRM on my system so I can play my DVDs, HD DVDs,and Blu-ray like MOST people.
You don't want it, more power to you. I've given you the links to the
software you need get avoid enabling MFPMP at all. I've shown you the lower
CPU utilizations using cheaper hardware. I don't know what else you want. ...
You know, you are a f***ing moron. End of discussion. Well, he's certainly tactless and outright rude. But he's also right about the following -
* Hollywood forced OS vendors like Microsoft and Apple to add DRM to allow playback of HD content.
* Both did, because it would be hard to sell an OS which can't play next generation content.
But this doesn't really matter because
* DRM will be cracked anyway.
* It doesn't have any effect on the OS if you don't use HD content.
He's only get flamed because he's defending Vista which is the subject of the current geek 3 minute hate. Now I don't really like Vista compared to XP, you don't need to believe that it 'causes global warming' as he puts it to dislike it.
BluRay is a product. If you don't like, don't buy and don't use the content distributed over it. I know I won't. And if you don't want Vista as a bundled OS, buy a computer it doesn't come on (like a Dell) or build your own.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
More seriously: Me shaping my own traffic is very different from someone else shaping my traffic against my will.
To borrow another poster's analogy:
I have no problem with choosing what kind of food I eat. If I had kids, I'd have no problem choosing what kind of food they eat.
I would very much not like the grocery store to choose what kind of food is best for everyone.
Fortunately, it's in the grocery store's best interest to give customers what they want. For some reason, ISPs think it's not in their best interest to do the same.
Don't thank God, thank a doctor!
Install a bandwidth management tool like cFosSpeed and you will see that latency drops down to essentially the same levels as you have without BitTorrent running without reducing the torrent speed whatsoever. This doesn't even require any of the fancy prioritization features of the bandwidth manager tool - just avoiding overloading the transmit queue.
In other words, your DSL line is perfectly capable of handling an uplink that is actually used for more than an occasional HTTP request without bogging down. The reason it doesn't do it is poor engineering of the DSLAM. With better tuning and queue management algorithms like RED (Random Early Drop) they will cooperate with TCP congestion control to avoid overloading the uplink buffers. Your DSL line will work just fine without a third-party bandwidth management tool.
Why is the DSLAM poorly engineered? The simple explanation is incompetence. Conspiracy theorist would probably claim that it's intentional because ISPs don't want you to use bandwidth-intensive applications. The truth is probably somewhere in the middle: the original flaw was a combination of lazy engineers and the fact that most users don't really use their uplink so much. It's not being fixed beacuse it serves the interests of the ISPs.
Stop worrying about the risks of nuclear power and start worrying about the risks of not using nuclear power.
Let's take the whole thing from the top.
1) Microsoft's marketing department decided that Vista needs to support BluRay.
2) The BluRay Disk association said that if they want to do this they need to support protected media paths and all the other nonsense.
3) Microsoft did that.
4) The net result is that you can Windows Vista and a software player to play BluRay DVDs. You don't need to crack anything to do this, or break any laws.
If they hadn't implemented PMP et al, you would need to crack to watch the disks because no software players would have been licensed by the BluRay consortium. I read somewhere that with DVD they originally planned not to allow software players because they were scared the keys would leak. And they were right, the Xing Mpeg player was hacked and the key was discovered.
http://en.wikipedia.org/wiki/Xing_Technology
So they sort of had a good case for only allowing hardware players. But Microsoft convinced them that PMP and so on would avoid cracks. Inevitably one of the software players was cracked.
http://en.wikipedia.org/wiki/AACS_encryption_key_controversy
Note that Windows DRM is 100% ineffective against this sort of thing, which is why PMP is a bit of a con. You can always use WinDbg to kernel mode debug a Windows machine and read every single byte of memory. But from what I can tell, the AACS key was extracted from the user mode software player, so even this wasn't necessary.
But you don't need to know the crack anything to play BluRay discs on Vista. Just use the BluRay player software that came with the machine. But that player would not have been licensed if Microsoft hadn't implemented DRM in the OS.
Now Linux can't implement DRM that will satisfy the BluRay consortium that a user won't get the keys. So to play BluRay discs on Linux you must rely on the crack. But cracked software isn't exactly user friendly. It's illegal to link to it in the US and the studio will keep tweaking the disks so it breaks and you need to download a new version.
If Microsoft hadn't implemented DRM the Windows users would be in the same boat.
Now if Blu Ray is like DVD then writable disks will only allow unencrypted content. So to copy a Blu Ray disk you'd need to crack. But just to watch a disk you don't.
Personally I pretty much rent or buy the odd DVD and watch cable. I'm in Asia and BluRay isn't too common here. I think the technology is overpriced and the requirment that the whole playback path be protected makes the whole process too fiddly. I can't see much difference in quality between HD and normal content. So I'm not going to buy it. But let's not get carried away. Windows users will watch BluRay disks in a userfriendly way. Pirates and Linux users will be able to copy/watch it too, it will just take a bit more work.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;