Better Bandwidth Utilization

jtorin writes "Daniel Hartmeier (of OpenBSD fame) has written a short but interesting article which explains how to better utilize available bandwidth. In short it gives priority to TCP ACKs over other types of traffic, thereby making it possible to max both upload and download bandwidth simultaenously. Be sure to check ot the nice graphs! Also note the article on OpenBSD Journal. OpenBSD 3.3 beta is now stable enough for daily use, so why not download a snapshot from one of the mirrors and try it out?"

Comments

like wondershaper does for months now?

[Anonymous Coward]

http://lartc.org/wondershaper/

Re:like wondershaper does for months now?

[Captain Pedantic]

From the Wondershaper script:

# To speed up downloads while an upload is going on, put ACK packets in
# the interactive class:

tc filter add dev $DEV parent 1: protocol ip prio 12 u32 \
match ip protocol 6 0xff \
match u8 0x05 0x0f at 0 \
match u16 0x0000 0xffc0 at 2 \
match u8 0x10 0xff at 33 \
flowid 1:10

We figured this before

[Thijssss]

Actually ever since my isp changed from A2000 to Chello we had the same problem as this guy has with his download being killed by his upload, a few months ago me and some friends figured the same solution but we had no idea how to actually do it on a windows based machine, anyone with a idea?

My solution:

[Black Parrot]

Put lower priorities on p0rn, MP3s, Windows viruses, and Slashdot referrals. That should speed everything else up by about two orders of magnitude.

Re:My solution:

[radish]

What is this "everything else" of which you speak?

Daniels original email

[blkwolf]

You can find Daniels original email on the subject at:
http://marc.theaimsgroup.com/?l=openbsd-pf&m= 10463 0260218727

It contains a little more of the pf rules than the article does, and has all the relevant information you need except for the nice /.'d graphs

Re:Linux solution

[pe1rxq]

No it doesn't....
It is a differend solution to a different problem caused by the same thing....

The cause is the big cache in the modem, it results in a delay on outgoing traffic.
One problem is that interactive traffic gets, well, less interactive (e.g. the echo characters in a remote shell have a delay). This is solved in the HOWTO you refered to.
Another problem is that the downstream acks get delayed resulting in less downstream data. This is solved in the mentioned article.

A combination of the two would be really great and could probably be done in both linux and openbsd.

Jeroen

Slashdotted - Mirror

[SILIZIUMM]

Since the website seems slashdotted now I've set up a mirror. You can see it there.

Title is correct!

[DarkMan]

Your correct, bandwidth is fixed. This is about better bandwidth utilisation.

The article is /.ed but the gist is that if you consider a full duplex conection, and you max out one side of that, say uploads, then the ACK packets get swamped, so your downstrem bandwidth is spent re-transmitting, or empty whilst the other end is waiting for ACKs.

The bandwidth is there, it;s just under utilised. By prioritisng the ACK's, so that they get boosted through, it becomes possible to saturate both upstream and downstream pipes at once, at peak efficency, rather than one of the coasting along, waithing for the other.

Note that this only applies for TCP/IP and similar, reliable, protocols. If you had a UDP app (e.g. media streaming done properly), then this trick won't affect it at all, as it doesn't wait for an ACK.

this may break TCP flow control!

[mekkab]

If the acks are sped up, this interferes with TCP keeping track of the statistical average Round Trip Time.

So if the network is congested and an ACK SHOULD time out but doesn't, TCP will keep on flooding the network, ruining the pool for everyone.(see: Tragedy of the commons)

Yes, I agree that this is a big-O style worse case scenario, but its something to consider.

Re:this may break TCP flow control!

[anthony_dipierro]

So if the network is congested and an ACK SHOULD time out but doesn't, TCP will keep on flooding the network, ruining the pool for everyone.

No. If the downstream is flooded, the packets won't be received, and no ACK will be sent. ACKs have higher priority, but even that can't make them appear out of thin air.

W. Richard Stevens TCP/IP Illustrated, Volume 1

[puzzled]

It seems to me that a great many /. readers have a cursory knowledge of how TCP/IP works. This is true of almost every other topic and I don't have a generalized solution for ignorance, but in this case a quick read of the first volume of Stevens' excellent TCP/IP Illustrated Series should do the trick.

Reading that book will give you a foundation to understanding how a single endpoint behaves in an IP network. If you want some understanding of the guts of a large scale internetwork I'd suggest the Cisco Press IP Quality of Service book.

There are a great many things near and dear to /. reader's hearts - the god given right to steal music by treating a retail DSL/Cable connection like a dedicated wholesale circuit being the prime example - that are more easily understood after a read of these two books.

If you're impatient you can look at my journal - I've covered some of the issues there.

Re:Security hole

[Arethan]

It's called SYN flooding. The idea is that a system only has so much memory to work with. Each established TCP session has memory overhead for packet ordering and split packet reassembly. Generally, when a system recienves a SYN packet, it assumes that the session is going to be valid, and it begins the process of completing the TCP session building, which happens to include setting aside some memory for session management.

For each SYN packet you send, you eat up a little bit more memory and CPU time on the victim. Do it enough times, and the system runs out of memory or processor time, and the system becomes unable to perform its regular operations. Effectively causing a Denial of Service.

If you're smart, you'll form the SYN packets to have source addresses that differ from your real IP, otherwise a) you're traceable; and b) your machine will be flooded with SYN/ACKS. If you are even smarter, you'll use an IP that, while valid and routable, belongs to a host that either doesn't exist, or is currently off. Otherwise the 2nd level victim recieving the SYN/ACKs from your initial target will send RSETs for every SYN/ACK, since it never requested to initial the connection. When your target gets the RSET for the SYN/ACK, it will close the session, freeing up the memory and CPU time that you are desparately trying to fill. Essentially, a non-existant host will never respond to a SYN/ACK, so the target system has to wait for a timeout duration before closing the session, which makes it easier for you to eat up CPU and memory. Unfortunately though, the fake spirce IP on your SYN packets will likely have to be within your ISP's network range, as all smart ISP network administrators perform egress packet filtering to prevent such attacks from originating within their network.

Better tactics include sending the SYNs from multiple machines that have different providers. Thus preventing load from the SYN/ACKs from filling your ISPs pipe. This effectively makes the attack a DDoS, rather than a DoS.

Either way, you can't really perform these attacks in much safety, as competent network administrators will have sniffers in place to detect these attacks as they cross their network. So #1) if your ISP admin is smart, you're busted by them regardless; and #2) if the chain of smart admins follows you all the way back to your sources, you're busted by the authorities (which if you cross state lines means the Feds, which will suck quite adamently).

So, that is how it works, but I wouldn't recommend trying it.

Not just low bandwidth

[TFloore]

Though you might see more effects of this on a low bandwidth link, it is not just for low bandwidth.

A fair number of protocols do transmit windows of a certain size. They'll send a certain amount of data, and not send more data until the oldest packest in the window gets an ACK back. You therefore only have so much data "in-flight" at any one time. Strongly asynchronous link (like aDSL and cablemodems) can require strikingly different window sizes than synchronous links.

The right amount of in-flight data is dependent on the speed of your pipe, obviously, but a lot of applications still use defaults set for low-bandwidth pipes. You can argue that the proper solution for this is to change the defaults, but if you just give ACKs priority, you don't need to worry about it, and the less you force users to change, the better. (The transmit window size has to be a user setting, directly or indirectly, either by asking a window size, or by asking "what kind of pipe do you have?" and guessing a window size from that.)

This is dependent on the protocol, true, but giving ACKs priority is actually a decent generic solution to what many consider an application-specific problem.

QOS is also often about bandwidth guarantees, not necessarily throughput. You have a 155mbit link shared among several applications, and an application that *requires* 45mbit. So you use QOS to guarantee that application gets 45mbit if it wants it, and everything else shares the remainder. If the app isn't going, then that 45mbit it requires can be made available to other apps until it is required.