Boosting Socket Performance on Linux

Cop writes "The Sockets API lets you develop client and server applications that can communicate across a local network or across the world via the Internet. Like any API, you can use the Sockets API in ways that promote high performance -- or inhibit it. This article explores four ways to use the Sockets API to squeeze the greatest performance out your application and to tune the GNU/Linux® environment to achieve the best results."

Comments

Re:Don't forget to wipe - and stay off the ice pip

[heauxmeaux]

'aunses'

Did you mean anuses? Or the correct pluralization:
Anii ?

Usage: You are an anus! You and your kin are anii!

Summary ripped directly from article (again)

[sczimme]

Here is the summary:

The Sockets API lets you develop client and server applications that can communicate across a local network or across the world via the Internet. Like any API, you can use the Sockets API in ways that promote high performance -- or inhibit it. This article explores four ways to use the Sockets API to squeeze the greatest performance out your application and to tune the GNU/Linux® environment to achieve the best results.

Here is the first paragraph of the article:

The Sockets API lets you develop client and server applications that can communicate across a local network or across the world via the Internet. Like any API, you can use the Sockets API in ways that promote high performance -- or inhibit it. This article explores four ways to use the Sockets API to squeeze the greatest performance out your application and to tune the GNU/Linux® environment to achieve the best results.

Unless Cop (the submitter) is actually M. Tim Jones (the article author), Cop didn't write a darn thing.

Didn't we just have this discussion on /. a few days ago?

No mention of alternatives to select?

[complexmath]

Tuning socket parameters is great and all, but the real performance problem with socket IO has to do with using select and poll. There are high-performance alternatives (which admittedly tend to vary from OS to OS) that are so far superior that I wouldn't even consider the default methods unless complete code portability were a crucial factor.

Re:No mention of alternatives to select?

[hackstraw]

Try this:

http://www.xmailserver.org/linux-patches/nio-impro ve.html /dev/epoll

The website is hideous, but there used to be benchmarks against different polling/selecting methods. If I remember correctly, its kinda trial and error, YMMV, kind of stuff. Its worth a look.

Re:No mention of alternatives to select?

[statusbar]

This page, while out of date, and referenced earlier during this discussion, needs re-emphasis. I hope it gets updated soon:

http://www.kegel.com/c10k.html

Very awesome paper. How do _you_ make a server that handles 10,000 connections?

--jeff++

Re:GNU/Linux®?

[wfberg]

Most probably it's just IBM policy to always acknowledge some one else's trademarks, so as not to get in trouble. Both GNU (yeah, I know! I knooow..) and Linux are registered trademarks (... of their respective owners, of course..)

Re:I've always wanted to know if it is possible

[pclminion]

Netcat might be what you want. It has two modes, a "client" and "server" mode. In client mode, it connects to an IP/port that you specify, then reads data from stdin and sends it through that socket. In server mode, it listens on a port you specify, and prints any data it received to stdout.

Is that what you're looking for?

Re:Code Portability

[complexmath]

There was a Boost library in the works to encapsulate all of this rather nicely, but I'm not sure if it ever made it out of beta. ACE is another option, though that tends to be overkill for some projects. I rolled my own class wrapper around this stuff, but then I enjoy library programming.

Re:Hello 1995

[pthisis]

In the same line - where is the discussion of different FD table polling mechanisms? select() versus poll(), and wheres the writeup about Linux's epoll(). I would have been interested in an epoll() article, especially how it compares to FreeBSD's kqueue().

For the overview, you want Dan Kegel's c10k page:

http://www.kegel.com/c10k.html

Math error in paper?

[Stiletto]

throughput = window_size / RTT

110KB / 0.050 = 2.2MBps

If instead you use the window size calculated above, you get a whopping 31.25MBps, as shown here:

625KB / 0.050 = 31.25MBps


That's funny, I get 12.5MBps

???

Re:Nagle's algorithm

[buck68]

You may be interested in a paper we wrote a few years back [1]. We also started with the premise that some applications require both minimal latency and maximal bandwidth. In our case the application was our own media streaming system. We came up with our own patch to TCP (in Linux). The patch provided a new socket option, we call TCP_MINBUF. The idea is that you need a certain minimum amount of buffer allow TCP's congestion window to function, but no more. Indeed, in the paper we show that the delay due to socket buffer beyond the congestion window is often by far the dominant source of latency--not retransmissions, or delayed acks, or all the other more commonly cited things. So basically what TCP_MINBUF does, is dynamically size the socket buffer to follow the congestion window size. It had a huge impact on latency.

[1] "Supporting Low Latency TCP-Based Media Streams", Ashvin Goel, Charles Krasic, Kang Li, and Jonathan Walpole. Tenth International Workshop on Quality of Service (IWQoS), May 2002.

http://www.eecg.toronto.edu/~ashvin/publications/i wqos2002.pdf

Re:Hello 1995

[AKAImBatman]

The Linux kernel automatically doubles the buffer for its own use. In the article:

Within the Linux 2.6 kernel, the window size for the send buffer is taken as defined by the user in the call, but the receive buffer is doubled automatically. You can verify the size of each buffer using the getsockopt call.

From the MAN page:

NOTES

Linux assumes that half of the send/receive buffer is used for internal kernel structures; thus the sysctls are twice what can be observed on the wire.

The article could have better explained that in context. For the most part it's automatic though, so don't worry about it.

Re:Be aware

[jas0n]

Looks like a rip off of an OnLamp article from a few months ago, and not a very good one at that! At least the OnLamp article explained how to tweak a few more OS's and the math was correct. And just to add insult to injury the article on OnLamp was written by one of those Berkeley guys ;-)