Have Sockets Run Their Course?

ChelleChelle writes "This article examines the limitations of the sockets API. The Internet and the networking world in general have changed in very significant ways since the sockets API was first developed in 1982, but the API has had the effect of narrowing the ways in which developers think about and write networked applications. This article discusses the history as well as the future of the sockets API, focusing on how 'high bandwidth, low latency, and multihoming are driving the development of new alternatives.'"

Comments

Re:whats really needed...

[Anonymous Coward]

You mean like this? http://en.wikipedia.org/wiki/Unix_domain_sockets

Open Transport, Part II

[Etcetera]

Been there, done that. Apple (once again) had a great implementation of an alternative technology, that it finally abandoned when it didn't feel like fighting any more.

Open Transport (the PPC stack used in the Classic Mac OS) was fast, efficient, and cool. And based on the STREAMS methodology, the only real competition to Berkeley Sockets.

Choice is good, mmmkay?

Re:Open Transport, Part II

[Shin-LaC]

I wrote networking code using Open Transport before I ever touched sockets, so I think I have a view as unbiased as you can get (or perhaps biased in favor of OT). I didn't mind OT, but I when I moved to sockets I was impressed with how easy and comfortable it is to work with them. The limitations of the classic Mac OS architecture probably made writing Open Transport code thornier than it would have been on a moden system, so I won't discount TLI or STREAMS in general, but I have to say that the particular implementation that was Open Transport on Mac OS is inferior to sockets on a UNIX system.

Re:Open Transport, Part II

[mzs]

Sure it was cool how you could push and pop drivers (say you wanted a different line discipline) but please tell me how it prevented any copies? The AT&T implementation also had two extra context switches.

This is what was bad about STREAMS:

In early implementations there was no notion of multithreading so a bad thing happened later. There was a time when the STREAMS drivers and demultiplexers assumed single threaded so the kernel had to pass off to a single worker thread in the kernel everything STREAMS related. So yo had some big iron box of the time with say four processors and IO performance was just balls until the STREAMS drivers were rewritten. But then you still had that worker thread around, so one thing was that those were broken out, so there was an extra two thread switches there. Then they did some stuff variously called something like Fast STREAMS where the fast paths would not switch. So all this optimization work went in to making STREAMS fast and they were still slow. It turned-out that the reason for that was due to the complexity of the STREAMS subsystem and all the layering that caused so many extra function calls per driver. STREAMS have largely been relegated to legacy and conformance at this point.

Which sockets API?

[PhrostyMcByte]

There are Berkeley sockets which are relatively portable, and then there are extremely platform-specific APIs for high performance and scalability. The old API might have run it's course, but most of the new ones are still relevant. Things like asio are helping to merge all the differences into one nice API.

Structured Stream Transport

[ace123]

BSD sockets have a limitation of only a single stream at a time (for example, if you are loading a website over HTTP and you get stuck loading a huge image, you have no choice but to open up another socket connection or else wait). They are also stuck around the paradigm of only supporting byte streams, which means that users are always forced to write the same code over and over to create packet headers or delimited messages.

I would highly recommend checking out Structured Stream Transport. I'm not from MIT and I wasn't entirely satisfied with their sample implementation, but the paper is really insightful and explains how you can develop basically a smarter version of TCP that is both more efficient and also more flexible. And I'm sure there are other systems being developed with similar ideas in mind.

We definitely need to keep bsd sockets, if not just because I'm a regular user of netcat :-p, and also because they are what allow the creation of more advanced protocols, but I don't think most applications should still be using such low-level protocols today.

Re:Structured Stream Transport

[Anonymous Coward]

TCP != BSD Sockets

No matter how much abstraction you pile on top to open multiple streams, automatically add headers, communicate a fix message size to avoid in-band delimiters, etc., you'll still have to send all those messages over linear octet streams when using TCP.

Now you could choose not to use TCP -- UDP lets you send non-linear messages of arbitrary size without delimiters. And there may be other newer, better options available as well. But you can do both TCP and UDP (as well other other comm types) using the same sockets API.

Re:Structured Stream Transport

[Anonymous Coward]

BSD sockets have a limitation of only a single stream at a time (for example, if you are loading a website over HTTP and you get stuck loading a huge image, you have no choice but to open up another socket connection or else wait).

No it doesn't. This is a limitation of TCP. You could just as easily use a different protocol (e.g., SCTP) with sockets.

STREAMS?

[$lashdot]

Macs used STREAMS from system 7.5.2 onwards. Was kind of sad to see that go away with the switch to OS X.

Re:SCTP an interesting example

[QuoteMstr]

It works, but it is not nice.

So use a wrapper, like sctp_send from libsctp. There's no reason the kernel proper has to export these interfaces.

Re:wrong

[TheThiefMaster]

Windows' solution is pretty nice. You can pass a pre-created socket handle to accept_ex, which automatically accepts an incoming connection using that socket handle, so that you don't have to use two system calls (select and accept). You can also pre-accept multiple sockets, instead of having to make the system calls under load.
Sockets can also be closed with a "re-use" flag, which leaves the handle valid and saves making a system call to create another.

You then associate the sockets with an "IO completion port", which as best as I can tell is a multithreaded-safe linked list for really fast kernel to user program communication. To receive from the socket you make an async receive call, giving a pointer to a buffer to receive into.
Whenever data is received on those sockets (and has had a corresponding async request made for it already) the kernel automatically queues the socket handle to that linked list. If you associate a socket with the completion port before you accept a connection with it (i.e. you're using acceptex) it also triggers when the socket accepts a connection.
In the user code, you run multiple threads listening on the completion port (you can also use the completion port in the thread pooling API, which runs two threads to each cpu core by default). When a message arrives from the kernel, the most recently finished thread wakes and processes the received data, which will already be in the user-space buffer you provided in the original receive call.

If all threads are busy and there are messages in the completion port they will bounce right off of the completion port, picking up the next bit of completed IO they need to process without making a system call.

Unix always had it

[mangu]

some way to seamlessly connect LOCAL processes to each other

You mean, like pipes?

Plan 9

[Anonymous Coward]

Already does that.

Re:Really...

[Anonymous Coward]

You think the "or worse" is a joke? Apparently you didn't read the link:

Potential complications include ischemia...the ischemia may result in gangrene, which could necessitate penis removal.