Slashdot Mirror
Have Sockets Run Their Course?
ChelleChelle writes "This article examines the limitations of the sockets API. The Internet and the networking world in general have changed in very significant ways since the sockets API was first developed in 1982, but the API has had the effect of narrowing the ways in which developers think about and write networked applications. This article discusses the history as well as the future of the sockets API, focusing on how 'high bandwidth, low latency, and multihoming are driving the development of new alternatives.'"
You mean like this? http://en.wikipedia.org/wiki/Unix_domain_sockets
Been there, done that. Apple (once again) had a great implementation of an alternative technology, that it finally abandoned when it didn't feel like fighting any more.
Open Transport (the PPC stack used in the Classic Mac OS) was fast, efficient, and cool. And based on the STREAMS methodology, the only real competition to Berkeley Sockets.
Choice is good, mmmkay?
Hire a Linux system administrator, systems engineer,
This guy's worried about "narrowing the ways in which developers think about and write networked applications" in a world where people are reinventing wall(1) as twitter, IRC as friendfeed, and other web 2.0 'innovations.' You want to widen developers' thinking about networking? Leave sockets alone and close off port 80.
REM Old programmers don't die. They just GOSUB without RETURN.
BSD sockets have a limitation of only a single stream at a time (for example, if you are loading a website over HTTP and you get stuck loading a huge image, you have no choice but to open up another socket connection or else wait). They are also stuck around the paradigm of only supporting byte streams, which means that users are always forced to write the same code over and over to create packet headers or delimited messages.
I would highly recommend checking out Structured Stream Transport. I'm not from MIT and I wasn't entirely satisfied with their sample implementation, but the paper is really insightful and explains how you can develop basically a smarter version of TCP that is both more efficient and also more flexible. And I'm sure there are other systems being developed with similar ideas in mind.
We definitely need to keep bsd sockets, if not just because I'm a regular user of netcat :-p, and also because they are what allow the creation of more advanced protocols, but I don't think most applications should still be using such low-level protocols today.
I am developing SCTP applications and has contributed to the linux implementation, and I think that one of the advantages of the socket API is that it is usable with select()/ and poll(), ie. it is file descriptors you can pass around.
But for SCTP there are things that don't fit nicely into the socket API, especially when using one-to-many socket types. For instance for retrieving options for an association you have to piggyback data in a getsockopt() call by using the output buffer also for input. It works, but it is not nice. Also, for sending/receiving messages you have to use sendmsg/recvmsg with all the features including control data, and the ugly control data parsing.
or person getting a hard attack
Viagra overdose?
This is not funny. It's called priapism and can result in impotence or worse.
Windows' solution is pretty nice. You can pass a pre-created socket handle to accept_ex, which automatically accepts an incoming connection using that socket handle, so that you don't have to use two system calls (select and accept). You can also pre-accept multiple sockets, instead of having to make the system calls under load.
Sockets can also be closed with a "re-use" flag, which leaves the handle valid and saves making a system call to create another.
You then associate the sockets with an "IO completion port", which as best as I can tell is a multithreaded-safe linked list for really fast kernel to user program communication. To receive from the socket you make an async receive call, giving a pointer to a buffer to receive into.
Whenever data is received on those sockets (and has had a corresponding async request made for it already) the kernel automatically queues the socket handle to that linked list. If you associate a socket with the completion port before you accept a connection with it (i.e. you're using acceptex) it also triggers when the socket accepts a connection.
In the user code, you run multiple threads listening on the completion port (you can also use the completion port in the thread pooling API, which runs two threads to each cpu core by default). When a message arrives from the kernel, the most recently finished thread wakes and processes the received data, which will already be in the user-space buffer you provided in the original receive call.
If all threads are busy and there are messages in the completion port they will bounce right off of the completion port, picking up the next bit of completed IO they need to process without making a system call.
The socket API... or rather the UNIX file descriptor API... has been extended many times. Sockets are already one such extension, and there's no reason you couldn't do something like mmap() a socket to map the buffers into user space directly. Heck, udp sockets already diverge from the read/write paradigm.
The problem with sockets is at a higher level. They're not mapped into the file system name space. You should be able to open a socket by calling open() on something like "/dev/tcp/address-or-name/port-or-name" and completely hide the details of gethostbyname(), bind(), and so on from the application layer. If they'd done that we'd already be using IPv6 for everything because applications wouldn't have to know about the details of addresses because they'd just be arbitrary strings like file names already are.
The Berkeley socket API has stood up very well against the tests of time, and it is fairly lean and quite versatile, but yeah, there's definitely room for newcomers.
For example, when it comes to high packet rates - say, thousands of VoIP RTP streams - the length of the typical path a packet takes through the kernel layers becomes quite prohibitive.
I've been trying to reach gigabit ethernet saturation with G711 VoIP RTP streams (that is, 172-byte UDP packets @ 50Hz per stream), which works out to a theoretical maximum of 10500 streams - 525000 packets/second. My initial speed tests, with minor tweaking, got me around 1/10th of that, thanks to all the kernel overhead, and the lack of control over how and when packets will be sent.
So I wrote my own socket-> UDP-> IP-> ARP-> Ethernet abstraction which hooks directly into the PACKET_MMAP API (as used by libpcap), with the TX Ring patch, and with all the corner-cutting I managed to achieve 10000 streams (500k packets/sec) which equates to about 95% of the theoretical peak.
In short, we probably need more widespread support for different network programming APIs which address more specific needs - BSD sockets are too generalised sometimes.
You mean, like pipes?
Pipes for local communication and tubes for global communication. Seems like a winner.