Why BitTorrent Causes Latency and How To Fix It

Sivar recommends an article by George Ou examining why BitTorrent affects performance so much more than other types of file transfer and a recommendation on how to fix it. The suggestion is to modify P2P clients so that, at least on upload, they space their traffic evenly in time so that other applications have a chance to fit into the interstices. "[Any] VoIP [user] or online gamer who has a roommate or a family member who uses BitTorrent (or any P2P application) knows what a nightmare it is when BitTorrent is in use. The ping (round trip latency) goes through the roof and it stays there making VoIP packets drop out and game play impossible."

Comments

Re:QoS?

[pin0chet]

OpenWRT and Tomato feature impressive QoS capabilities as well.

L7-filter can even manage traffic at the application layer. Just set Bittorrent to "Bulk" and put Skype and Xbox live as "Premium."

Managing traffic on the router level is a lot easier than on the PC level, especially when you have several devices on a single network competing for scarce bandwidth.

QoS, but only on the Telco Side

[corsec67]

While I prefer Tomato on a WRT-54GL, that would do absolutely nothing at all to solve this issue. A router behind a modem can really only regulate the upload, and can't easily prevent a flood of data on the downstream side.

This issue is with the queue on the Telco's DSLAM, or on the other side of the cable from the modem. This is more like an invited DDOS, which no amount of filtering at or behind the modem can resolve, because the modem is getting the traffic from the DSLAM after it goes through the queue.

The only way to have QOS solve this issue would be to ask the telco to do the QOS for you, and the amount of processing power to do that nicely isn't trivial.

Re:QoS, but only on the Telco Side

[silas_moeckel]

Funny I'm a network guy and grok normal QOS. DD WRT and the like are capable of inbound QOS to some degree via inbound rate shaping. You loose some bandwidth and you can do the same with cisco kit and some creative use of it's rate shaping. It's not as good as QOS but it works.

Re:QoS, but only on the Telco Side

[Bruha]

That is completely false. QOS features have long been supported by CEF and many other ASIC based solutions in Cisco and many other service provider equipment. For many years now it's been there and has been ignored. At my company I have been preaching QOS to make sure that user experience is guaranteed. Routing protocols get first shot, then HTTP(S)/Telnet(SSH)/POP3/IMAP/SMTP etc etc. Every other app is regulated to bulk. Then that 95% will never see latency of problems with their web surfing and even games such as WOW DOOM, Xbox live etc can get priority queues over bulk downloads.

Once it's done at the network level the same can be applied down to the user level with the packets as they're tagged.

What we lack is ways for routers to signal upstream routers for dynamic QOS to the customer network.

Re:QoS, but only on the Telco Side

[snookums]

The article is looking at a download link that is saturated from P2P transfers from other people. In BitTorrent, the more slowly you upload, the more slowly you download. Actually, this is pretty much nonsense. In a heavily contended torrent, with more requests in the cloud than there is upload bandwidth to serve it, then often priority is given to better uploaders. However, on torrents with a good supply of fast seeds and few leechers (e.g. an old torrent with dedicated seeds provided by the content owner) it is very easy to reach the download cap you've set in your client while uploading next to nothing.

Use randomized time rather than even spacing

[karl.auerbach]

We long ago learned that when inserting time between protocol events that it is far better to use a time randomized between an upper and lower bound than to use a repeating interval.

When fixed repeating intervals are used, separate instances of a protocol (and other protocols that use repeating intervals) slowly tend to fall into lock-step patterns with pulsating waves of traffic in accord with those patterns.

In other words, fixed protocol timers can create the traffic equivalent of the Tacoma Narrows bridge.

By-the-way, ping (ICMP Echo request/reply) is a terrible way to measure network latency. ICMP is often a disfavored form of traffic as it crosses routers, sometimes even rate limited.

There are better tools for measuring link properties, for example there is "pchar" - http://www.kitchenlab.org/www/bmah/Software/pchar/

I worked on a method to do even better measurements, but I put it aside several years ago: Fast Path Characterization Protocol at http://www.cavebear.com/archive/fpcp/fpcp-sept-19-2000.html

Re:QoS?

[Dolda2000]

It sounds like you're doing it wrong. I've set up HTB shaping with tc on Linux as well, and it works very well. Flawlessly, I might even say.

There are two key points:

• You absolutely need to limit to absolute maximum outbound bandwidth (on the root qdisc, in other words) to a value slightly below your real outbound bandwidth. This point is critical. Without it, there's no point in even trying to shape the traffic, since the modem will start buffering.
• It helps very greatly if it is possible for you to classify torrent traffic into a HTB class with lower priority than whatever class the packets you care about go into. There are several possibilities for going about that:
  • If the program in question supports setting the DSCP field of the packets (where the TOS field went previously), you can use iptables with -m dscp to set the fwmark on them to classify more precisely (remember to clear the DSCP field before sending the packets out from your network, though).
  • If a program running locally on the router does not support setting DSCP values, you can create a group, set the program to SGID to that group, and use iptables with -m owner --gid-owner $GROUPNAME to set the fwmark. The same method can be used to set the DSCP field on packets from a Linux machine other than the router.

For reference, here is the script that I use to set up the traffic shaping. It might prove useful to you.

#!/bin/sh

# Current bandwidth allocation:
# 1:11 1:121 1:122 1:13 1:14 1:15 1:1
# (25 + (175 + 75) + 125 + 175 + 25) = 600

tc qdisc add dev wan root handle 1: htb default 122
# Root
tc class add dev wan parent 1: classid 1:1 htb rate 600kbit ceil 600kbit cburst 1500 burst 50kb
# TOS Min-Delay
tc class add dev wan parent 1:1 classid 1:11 htb prio 0 rate 25kbit ceil 50kbit burst 10kbit
# Bulk
tc class add dev wan parent 1:1 classid 1:12 htb prio 1 rate 250kbit ceil 600kbit burst 10kb
# HTTP
tc class add dev wan parent 1:1 classid 1:13 htb prio 1 rate 125kbit ceil 600kbit burst 50kb
# FTP (Needs iptables support)
tc class add dev wan parent 1:1 classid 1:14 htb prio 1 rate 175kbit ceil 600kbit burst 10kb
# Low priority
tc class add dev wan parent 1:1 classid 1:15 htb prio 2 rate 25kbit ceil 500kbit
burst 10kb
# TOS Max-Bandwidth
tc class add dev wan parent 1:12 classid 1:121 htb prio 1 rate 175kbit ceil 600kbit
# Default
tc class add dev wan parent 1:12 classid 1:122 htb prio 1 rate 75kbit ceil 600kbit
# TOS Min-Cost (Needs iptables support)
tc class add dev wan parent 1:15 classid 1:151 htb prio 2 rate 5kbit ceil 400kbit burst 10kb
# Auxiliary low prio bands
tc class add dev wan parent 1:15 classid 1:152 htb prio 2 rate 5kbit ceil 400kbit burst 10kb
tc class add dev wan parent 1:15 classid 1:153 htb prio 2 rate 5kbit ceil 400kbit burst 10kb
tc class add dev wan parent 1:15 classid 1:154 htb prio 2 rate 5kbit ceil 400kbit burst 10kb
tc class add dev wan parent 1:15 classid 1:155 htb prio 2 rate 5kbit ceil 400kbit burst 10kb

# Filters
tc filter add dev wan parent 1: protocol ip prio 1 handle 11 fw flowid 1:151
tc filter add dev wan parent 1: protocol ip prio 1 handle 12 fw flowid 1:152
tc filter add dev wan parent 1: protocol ip prio 1 handle 13 fw flowid 1:153
tc filter add dev wan parent 1: protocol ip prio 1 handle 14 fw flowid 1:154
tc filter add dev wan parent 1: protocol ip prio 1 handle 15 fw flowid 1:155
tc filter add dev wan parent 1: protocol ip prio 2 handle 1 fw flowid 1:14
tc filter add dev wan parent 1: protocol ip prio 3 u32 match ip tos 0x10 0x1e flowid 1:11
tc filter add dev wan parent 1: protocol ip prio 3 u32 match ip tos 0x08 0x1e flowid 1:121
tc filter add dev wan parent 1: protocol ip prio 3 u32 match ip sport 80 0xffff flowid 1:13
tc filter add dev wan parent 1: protocol ip prio 3 u32 match ip sport 443 0xffff flowid 1:13

# Leaf nodes
tc qdisc add dev wan parent 1:11 handle 2: sfq p