Scalable, Fault-Tolerant TCP Connections?

pauljlucas asks: "My company is developing custom server software for an instant messaging type server (under Solaris). Every client maintains a TCP connection to the server when it is 'logged in'; the server maintains state of who's logged in where. For large-scale deployment, there are two problems: scalability and fault-tolerance. A single server can handle at most around 64000 open sockets. To go beyond this, you need many servers. Another way would be to 'fake' a TCP stack in user-space (by reading/writing raw TCP packets) thus not having one real socket per connection. For fault-tolerance, ideally one would like N servers to maintain the exact state, at least for the server process, so that if one goes off line, the other(s) can pick up seamlessly. I'm thinking that both of these issues must have already been solved without having to write lots of custom software. Is anybody aware of off-the-shelf software and/or hardware solutions (either commercial or freeware)?"

Comments

Why TCP?

[Anonymous Coward]

The easiest solution would be to use UDP, and have the clients switch servers when they can't get a response. As long as the servers have a common state, the client should be able to switch between them without too many problems. And since UDP is connectionless, it will scale better - you won't have to open a socket for each client.

It would be possible to implement a server-side solution, using TCP, but you'd need to keep track of things like sequence numbers and window sizes to move a TCP connection seamlessly between two computers (meaning you'd probably need to write a TCP stack).

You want linuxvirtualserver

[pthisis]

First off, the Kegel's c10k page referenced earlier is definitely worth a read. And if you're under the impression that having 2^16 TCP port numbers limits you to 2^16 connections, that's not accurate. You can have hundreds of thousands of connections to one machine, presuming you manage them properly (as the c10k page points out).

More importantly, you should check out http://www.linuxvirtualserver.org/ ; it's aim is exactly what you want:

"The Linux Virtual Server is a highly scalable and highly available server built on a cluster of real servers, with the load balancer running on the Linux operating system. The architecture of the cluster is transparent to end users. End users only see a single virtual server."

Sounds like a perfect match.

Sumner