Slashdot Mirror
Why BitTorrent Causes Latency and How To Fix It
Sivar recommends an article by George Ou examining why BitTorrent affects performance so much more than other types of file transfer and a recommendation on how to fix it. The suggestion is to modify P2P clients so that, at least on upload, they space their traffic evenly in time so that other applications have a chance to fit into the interstices. "[Any] VoIP [user] or online gamer who has a roommate or a family member who uses BitTorrent (or any P2P application) knows what a nightmare it is when BitTorrent is in use. The ping (round trip latency) goes through the roof and it stays there making VoIP packets drop out and game play impossible."
Good routers can prioritize traffic and limit the latency increase to the time it takes one bulk traffic packet to be sent.
Most P2P programs also have ways to limit the upstream bandwidth, which is sufficient if you are in control of your side of the bandwidth bottleneck. Just set the limit so that all bulk transfers combined leave enough room for realtime applications and the latency will sort itself out (because there won't be any queues of bulk packets building up in the router).
OpenWRT or DDWRT can run some nice QoS scripts to filter based on ip/port/service
OpenWRT and Tomato feature impressive QoS capabilities as well.
L7-filter can even manage traffic at the application layer. Just set Bittorrent to "Bulk" and put Skype and Xbox live as "Premium."
Managing traffic on the router level is a lot easier than on the PC level, especially when you have several devices on a single network competing for scarce bandwidth.
While I prefer Tomato on a WRT-54GL, that would do absolutely nothing at all to solve this issue. A router behind a modem can really only regulate the upload, and can't easily prevent a flood of data on the downstream side.
This issue is with the queue on the Telco's DSLAM, or on the other side of the cable from the modem. This is more like an invited DDOS, which no amount of filtering at or behind the modem can resolve, because the modem is getting the traffic from the DSLAM after it goes through the queue.
The only way to have QOS solve this issue would be to ask the telco to do the QOS for you, and the amount of processing power to do that nicely isn't trivial.
If I have nothing to hide, don't search me
We long ago learned that when inserting time between protocol events that it is far better to use a time randomized between an upper and lower bound than to use a repeating interval.
When fixed repeating intervals are used, separate instances of a protocol (and other protocols that use repeating intervals) slowly tend to fall into lock-step patterns with pulsating waves of traffic in accord with those patterns.
In other words, fixed protocol timers can create the traffic equivalent of the Tacoma Narrows bridge.
By-the-way, ping (ICMP Echo request/reply) is a terrible way to measure network latency. ICMP is often a disfavored form of traffic as it crosses routers, sometimes even rate limited.
There are better tools for measuring link properties, for example there is "pchar" - http://www.kitchenlab.org/www/bmah/Software/pchar/
I worked on a method to do even better measurements, but I put it aside several years ago: Fast Path Characterization Protocol at http://www.cavebear.com/archive/fpcp/fpcp-sept-19-2000.html
I have my torrents capped to 1/10 of the advertised connection speeds, but latency still affects me (very visible in ssh sessions to my remote irssi server)
My UID is prime... is yours?
Wow, talk about solving a problem the hard way. Why not just use a bittorrent client which has rate limiting built-in? Which, by the way, is almost all of them? (I use rtorrent, an excellent command-line client.)
Except, wait for it, almost all p2p clients allow you to throttle your bandwidth anyway.
I want to delete my account but Slashdot doesn't allow it.
It is always easier to manage uplink bandwidth from downlink bandwidth, simply by virtue of the fact that you control the actual packet queues.
Downlink bandwidth can be controlled in numerous ways. The easiest way is to actually run the incoming packets through a bandwidth limiter with a very large packet queuing capability. This will cause a ton of packets to build up in front of the limiter and eventually fill the TCP windows of the senders. The packets that get through the limiter will cause a stream of ACKs back from your machines at the desired data rate. The combination of the two will cause the remote senders to band-limit the packets they send to the bandwidth you desire.
when running incoming packets through a limiter you still need to traffic-shape/QOS, priority-queue, or priority-queue + fair-queue the packets going through the limiter. If you don't then your interactive traffic can wind up getting stuck in a packet queue with hundreds of packets in it. In addition to that you may have to control the advertised TCP window or even implement RED on your limiter to prevent the hundreds of packets built up in front of the limiter from turning into thousands of packets.
If you can classify the bulk traffic then you can use virtually any queueing mechanic. If you can't classify all of the bulk traffic then the only mechanic that will work reasonably well is, again, going to be a fair-queue.
Fair-queueing is not the holy grail but it is typically the most effective mechanism when combined with another queueing mechanic, such as a priority queue.
-Matt
Injecting TCP RST packets is not traffic shaping. It is sneaky interference with legitimate network access.
That doesn't address the number of open connections issue. Bittorrent clients can often have hundreds of open connections while a browser or a game may only have 1 or 2 connections open. So when the game sends a packet, the router gets it and recognizes that it is connection 99 of 100 open connections. If the router equally prioritizes every packet, then the app that only utilizes a single connection can still wait before being serviced.
It also doesn't solve the problem of having a roommate who will leave bittorrent on indefinitely.
The real solution is to come up with a way to analyze packets and determine which packets should have the highest priority. This is called Quality of Service (QoS). Linux and routers based on linux have access to a number of different QoS schemes, but the off the shelf routers may not have good enough hardware to run it. For example I bought a ddwrt compatible router. I dumped the original factory firmware and installed ddwrt. I turned on QoS and put http and other types of traffic at higher priority than the rest. It worked great when the router could handle the traffic. I could let the bittorrent client eat as much as it wanted but when I hit a webpage, the page loaded just as fast. But every once in a while the router would crash or become really slow and inaccessible (can't access it through ssh or http). Turning off QoS alleviated that issue but of course bittorrent would starve out the other apps. In the future I plan on buying a router with a faster cpu so I can leave QoS on.
You forgot protocol inspection
NBAR on any current cisco IOS feature set will detect pretty much anything you need to prioritise without seriously impacting performance.
Juniper has something similar on their gear as well.
Easy QoS: Low latency queueing = fair queue with a priority queue as you described.
tag real time traffic as priority queue and allocate enough bandwidth depending on your capacity engineering. tag your important apps and put them in the second queue. Rest in default class.
This is really all you need, I have seen VOIP for over 500 extensions hold up as that sites link is over 90% for an hour And this is Cisco callmanager i.e. the remote phones and gateways bork and go into fallback mode if the keepalives are lost.
Just need to remember it needs to be end to end and in both directions
There are two key points:
For reference, here is the script that I use to set up the traffic shaping. It might prove useful to you.
That's similar to what I have, albeit with more rules and finer-grained control. Mine basically says that if the outgoing packet is > 1kb then it's probably part of a high-traffic connection and needs to be shunted to the back of the queue (low priority).
:)
The key point that I've missed is the master speed throttler at the trunk of the tree - of course the router's just throwing stuff at the modem as fast as it can so its queues are never full.
Thankyou for taking the time to reply, and making my kick myself! Greatly appreciated
Nobody else has this sig.
Any whatsoever? His part in the Maynor/Ellch debacle was a serious low point for tech journalism; he makes Rob Enderle look good, fer chrissakes. Even if the article were in fact insightful and informative, the simple fact that his name is attached to it guarantees that I'm not going to read it. Someone please tell me what it says.
http://www.cypherpunks.to/~peter/zdnet.html Schneier is a moron if he thinks telling Hollywood no will force them to use non-DRM content. All you need to do is look at the CableCard fiasco. You give Hollywood the finger and they give you the finger right back because they'd
rather NOT have any content on the PC to begin with. Like Apple, Microsoft
will humor Hollywood so they come join the party. Once they're in, they'll
get screwed out of their DRM protections because Microsoft won't patch the DRM
holes and let their customers bypass DRM. The latest DRM stripper for Windows
Media has worked for almost 2 months now and Microsoft hasn't patched it yet. Ok, so it's nasty to call someone a moron. And it's not really true either. It's ideology that causes Schneier and all the Web 2.0 'experts' to say this. He's no fool but he can't differentiate between it would be good if something being true and something being true. It would be good if Hollywood would give up on flakey DRM schemes. But if Microsoft and Apple had somehow agreed to boycott them, then Windows and Mac users would just have been left with no way to play HD content, because Hollywood is mortally afraid of people ripping HD content and uploading it to Pirate Bay. But George Ou is right that once stuff gets on open platforms like the PC it will get cracked anyway, so the OS vendors were just humouring them. And they probably knew it. FOR THE LAST TIME, I want the DRM on my system so I can play my DVDs, HD DVDs,and Blu-ray like MOST people.
You don't want it, more power to you. I've given you the links to the
software you need get avoid enabling MFPMP at all. I've shown you the lower
CPU utilizations using cheaper hardware. I don't know what else you want. ...
You know, you are a f***ing moron. End of discussion. Well, he's certainly tactless and outright rude. But he's also right about the following -
* Hollywood forced OS vendors like Microsoft and Apple to add DRM to allow playback of HD content.
* Both did, because it would be hard to sell an OS which can't play next generation content.
But this doesn't really matter because
* DRM will be cracked anyway.
* It doesn't have any effect on the OS if you don't use HD content.
He's only get flamed because he's defending Vista which is the subject of the current geek 3 minute hate. Now I don't really like Vista compared to XP, you don't need to believe that it 'causes global warming' as he puts it to dislike it.
BluRay is a product. If you don't like, don't buy and don't use the content distributed over it. I know I won't. And if you don't want Vista as a bundled OS, buy a computer it doesn't come on (like a Dell) or build your own.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
I've noticed similar problems at my place, and I think it's less about burst packeting and more about fair queuing. Bittorrent opens up tons of connections and VoIP doesn't. It's not that there's no time to send communications on a regular interval, it's that the VoIP app isn't getting them. In my case, I'd been pondering the ins and outs of Tomato's QoS but I mostly just throttled Deluge and called it a day when that did the job.
I Browse at +4 Flamebait
Open Source Sysadmin
Let's take the whole thing from the top.
1) Microsoft's marketing department decided that Vista needs to support BluRay.
2) The BluRay Disk association said that if they want to do this they need to support protected media paths and all the other nonsense.
3) Microsoft did that.
4) The net result is that you can Windows Vista and a software player to play BluRay DVDs. You don't need to crack anything to do this, or break any laws.
If they hadn't implemented PMP et al, you would need to crack to watch the disks because no software players would have been licensed by the BluRay consortium. I read somewhere that with DVD they originally planned not to allow software players because they were scared the keys would leak. And they were right, the Xing Mpeg player was hacked and the key was discovered.
http://en.wikipedia.org/wiki/Xing_Technology
So they sort of had a good case for only allowing hardware players. But Microsoft convinced them that PMP and so on would avoid cracks. Inevitably one of the software players was cracked.
http://en.wikipedia.org/wiki/AACS_encryption_key_controversy
Note that Windows DRM is 100% ineffective against this sort of thing, which is why PMP is a bit of a con. You can always use WinDbg to kernel mode debug a Windows machine and read every single byte of memory. But from what I can tell, the AACS key was extracted from the user mode software player, so even this wasn't necessary.
But you don't need to know the crack anything to play BluRay discs on Vista. Just use the BluRay player software that came with the machine. But that player would not have been licensed if Microsoft hadn't implemented DRM in the OS.
Now Linux can't implement DRM that will satisfy the BluRay consortium that a user won't get the keys. So to play BluRay discs on Linux you must rely on the crack. But cracked software isn't exactly user friendly. It's illegal to link to it in the US and the studio will keep tweaking the disks so it breaks and you need to download a new version.
If Microsoft hadn't implemented DRM the Windows users would be in the same boat.
Now if Blu Ray is like DVD then writable disks will only allow unencrypted content. So to copy a Blu Ray disk you'd need to crack. But just to watch a disk you don't.
Personally I pretty much rent or buy the odd DVD and watch cable. I'm in Asia and BluRay isn't too common here. I think the technology is overpriced and the requirment that the whole playback path be protected makes the whole process too fiddly. I can't see much difference in quality between HD and normal content. So I'm not going to buy it. But let's not get carried away. Windows users will watch BluRay disks in a userfriendly way. Pirates and Linux users will be able to copy/watch it too, it will just take a bit more work.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
Now imagine sending 10 private emails to someone (Karel Donk) and the guy continues saying annoying and idiotic things. Then imagine you lose your temper and use some profanity in a private email. Now most people can get away with that, but someone like me who is a high-profile blogger at ZDNet should have known better to write that in an email. So Donk forwards my emails to Gutmann and Gutmann posted it on that link of his pretending like I was sending Gutmann harassment email. Initially, Gutmann posted it on his University web page but he took it down because it didn't belong there. So that was Guttmann's only defense that I referred to him as a moron in some email that wasn't even sent to him.
So I used profanity in a private email and it got posted without the full context. I should have known better and I won't make that mistake again. Guttmann on the other hand never conducted a single test, never even used Vista, and he presented a bunch of web forum postings as a scientific study from a respected university. That is by definition academic misconduct.
I explain how Karel Donk is one of Gutmann's primary sources here. http://blogs.zdnet.com/Ou/?p=723)
Anyhow, thanks for being logical and email me any time.