Writing High-Availability Services?

bigattichouse asks: "I have a project coming up that will require some serious load capabilities accepting socket connections. while I have a design that can be distributed over multiple servers (using queued reads/writes to the db) and is as low-overhead as I can make it - I am concerned about falling into common problems that may have been overcome in many other projects. What strategies (threading, forks, etc) give the best capability? What common pitfalls should I avoid?"

Comments

What kind of system?

[Pyromage]

In general, there are many things you can do. Pooling, caching, etc. can help in many situations. But what situation are you in?

Are you writing a web app where you have to hold session data across TCP connections?

Are you writing an app that will have sustained connections (more than one request per connection?)?

These different situations require different strategies.

DB reads more common or writes? How big's the difference?

What kind of system is your target? Can you trade memory for speed (caching)?

Take a look at SEDA http://seda.sourceforge.net. While you probably won't be rewriting your app to use this framework, many of the strategies may be useful and applicable to your app.

Also, just note the difference between efficient and scalable: some designs will take longer than others on short loads, but many of those make tradeoffs that are only noticable under high stress. Consider what tradeoffs you've made so far: some may be good or bad, and more may need to be made.

All this was said without knowledge of what you app is other than a DB app. I am not an expert, but I doubt an expert could say all that much with that little information.

In one case....

[OwnerOfWhinyCat]

Though Pyromage's criteria requests are vital to making good suggestions, I had a high-burst rate problem for a server application that I solved slightly "out of the box." Since I wrote the client as well, I switched from the "connectioned" TCP interface to the "connectionless" UDP one. Since my application had to track the state of every pending request in any case, going to the connectionless protocol only meant adding 4 more states. This cut the kernel overhead significantly, and the total packet counts went down by half.

If you can supply the rest of the data it's likely that other good tradeoffs can be suggested.

Re:Less fluff, more detail

[Twylite]

Ah, telecoms :) Is this the industry/application in question, or just hypothetical? There was mention of throughput (indirectly) and availability, but not of latency in the original question. Also there was mention of queuing queries to a back-end database ... this doesn't sound like a minimal-latency scenario.

Anyway, the technologies you mention are not likely to be acceptable in such a scenario -- but MOM is quite likely to be appropriate. In fact many cellular services are based on MOM (conceptually, although they may not use commercial MOM products), as you receive requests and send responses based on discrete messages from/to the handset.