Efficient Use of Network Load-Balancing w/ SSL?

vw asks: "I was wondering if anyone has setup a webserver farm that uses both SSL and zero-affinity network-load-balancing. (i.e. requests in an SSL channel can be handled by any server in the farm) I have been having a very difficult time locating information on this specific combination of features in a web server product. The closest I found was WLBS, which implies that there must be an affinity toward a particular server for a given client IP. I understand the problem has something to do with sharing SSL certificates between physical machines. Any suggestions?"

Comments

F5 Big ip

[m_c_rose]

you might want to look into F5 www.f5.com . They sell a quality load balancing platform. you can generrally find them on Ebay.

here

[anthony_dipierro]

#!/usr/bin/perl

use CGI;

$q = new CGI;
if (rand()<0.5) {
print $q->redirect("https://www1.yourdomain/");
} else {
print $q->redirect("http://www2.yourdomain/");
}

Re:here

[Anonymous Coward]

You're joking, right?

The whole point is to allow a single session to be handled by the same server over multiple requests when using SSL, to avoid the overhead of generating new session keys, etc, with each page load (including any images that are also referenced from the page). Using rand() in the manner you suggest will essentially guarantee that any efficiencies will be lost -- not to mention your "load balancing" is being done in the ugliest possible manner. Visitors who bookmark pages will end up with a URL to a specific server -- which could be down for maintenance, failure, or simply decommissioned (or maybe you change your naming scheme for the individual servers at some point).

A good load balancing solution will recognize SSL sessions and forward them to the same internal machine with each request. And any load balancing solution that has a feature like that will also include all the basic failover, redundancy, etc. features larger sites absolutely require.

One of the biggest benefits (aside from additional performance) of a true load balancing solution is the ability to virtually eliminate any downtime because of maintenance, upgrades, or server failures. The product my company uses also provides a hot backup for the load balancer itself. When I need to update the web apps on the servers I manage, I can shut down Apache on any of the servers and know that the load balancer will now take that server out of the pool until it comes back up. I can repeat the process over each server behind the balancer and customers will never even know servers were being shut down.

If the balancer goes out (it's only happened once since it went in to place a couple years ago), the hot backup picks up immediately and service is uninterrupted. Everything is completely transparent to the visitors, and it also allows us to reorganize our internal networks to meet our ever changing whims.

A good load balancing solution is more than just spreading requests across a group of servers. It provides scalability, performance, uninterrupted service, easier management and updates, network abstraction, monitoring, and an additional layer of security.

Re:here

[anthony_dipierro]

Visitors who bookmark pages will end up with a URL to a specific server -- which could be down for maintenance, failure, or simply decommissioned (or maybe you change your naming scheme for the individual servers at some point).

So always remember to point decomissioned servers DNS records back to the main IP address.

A good load balancing solution will recognize SSL sessions and forward them to the same internal machine with each request.

This does that. You only redirect once, when you hit www.yourdomain.

And any load balancing solution that has a feature like that will also include all the basic failover, redundancy, etc. features larger sites absolutely require.

This can easily be extended to do failover.

A good load balancing solution is more than just spreading requests across a group of servers. It provides scalability, performance, uninterrupted service, easier management and updates, network abstraction, monitoring, and an additional layer of security.

You sound like a commercial. In any case, if you have one of those huge sites with tons of money to waste, go and get a commercial load balancer. Two of them, actually. And don't make slashdot do your job for you. You can afford to hire a professional to do the job.

i have....

[Zurk]

its easy. just use SSL from a main "proxy" server (or better yet a 2-3 server set sharing the certs) that takes the SSL requests and handles em as well as opens standard http connections to the webservers.
the proxy server set will handle your SSL encryption while your web servers in your cluster serve webpages.
ive only done it with apache on linux so i dont know how any other webserver will work.

Re:i have....

[frankske]

More than 1 server handling the SSL session is actually bad!

1. Image me (the client) connection to secure.yoursite.tld. Using your round-robin DNS, I end up at server1.
2. We both setup a session (involving key interchanging etc) and then you give me your first page (eg a login form).
3. I fill in the form and send it back. If then, I got connected to server2, we'd have to do recreate the SSL session keys etc, losing lots of time (both real and CPU).
   
   
   So whatever you use as first load-balancer, make SURE it knows about SSL sessions! But then you have a complex loadbalancer, which then becomes a single point of failure ...

Re:i have....

[Anonymous Coward]

sticky load balancing takes care of your problem. RRDNS is a bad way of doing load balancing anyway.

Re:i have....

[Zen Sandwich]

But 'sticky' isn't 'zero affinity', is it? So what you really want it what the original poster suggested, a SSL-speaking proxy (eg Squid in SSL accellerator mode) that terminates the SSL session and forwards the request inside it to a cluster of non-SSL webservers (using RRDNS perhaps, or LVS if you want a 'smarter' solution). The downside there is your squid proxy is doing a lot of work, so you probably want to have a backup one and use something like heartbeat to fail-over to it if there's a problem with the first one.

Apache + Splash is the solution

[Crutcher]

This is a solved problem, check out the splash project http://anoncvs.aldigital.co.uk/splash/ for details.

Basically, it is an Apache module which uses the spread http://spread.org secure mesaging server to synchronizes ssl conection information.

Re:Apache + Splash is the solution [more info]

[Crutcher]

Splash provides a distributed database using Spread. Apache-SSL has a plugable SSL conection cache layer, and there is a Splash implementation available. So, when a client connects to an Apache-SSL server in the pool, and the server checks the cache for SSL cache information, it is checking a DB it shares with all the other machines in the pool. This is exactly what is being requested for this Ask Slashdot.

Somewhat Simple -- perhaps non-obvious.

[rtaylor]

Setup an OpenBSD box with a crypto card. Let it do all of the dirty work. That box should proxy requests to the load balancer and other webservers in unencrypted form.

Lets you buy a single cert, and with hardware crypto you should be able to handle enough for a modest load -- more than a few million pages daily.

Re:Somewhat Simple -- perhaps non-obvious.

[Anonymous Coward]

OBSD rocks. Crypto cards are nice, inexpensive, and available. Overall, not a bad solution except for one thing...what happens if the OBSD box dies due to a nonredundant part fails, like the motherboard goes? You have a single point of failure, which is not ideal.

Re:Somewhat Simple -- perhaps non-obvious.

[wdr1]

I have to admit that I'm somewhat cringing at that solution.

I think it will work in a lot of enviroments, but if you're heavily concerned with security (e.g. you're passing around credit card numbers), and it sounds like the poster is, relaying information, at any point, unencrypted is a bad idea. If you can't get around (e.g. sometimes front end webservers must talk to DBs) just to limit that network channel to just those devices to reduce the odds of penetration.

Having said that, your problem shouldn't be too bad -- just make sure each physical server has the correct certificates for www.mydomain.com. Did you run into problems with that?

-Bill

Re:Somewhat Simple -- perhaps non-obvious.

[rtaylor]

I'm going with the assumption the links between frontend and live webservers is on a safe network (private IP space, restricted access, etc.). Ie. Load balancers are doing nat, etc.

There is more SSL traffic than a single box can handle, and load balancers are already involved so it's probably a pretty valid assumption.

Re:Somewhat Simple -- perhaps non-obvious.

[rtaylor]

Like you said, they're cheap.

If he has 2 load balancers, then he'll get two of these possibly each with their own cert.

That said, 2 load balaners, 4 switches (infront and behind nat'd load balancers / firewalls), N webservers and probably 2 netapps of somekind isn't cheap -- so 2 crypto boxes probably won't make a dent as they can be fairly lightweight.

Re:Apache + Splash is the solution [more info]

[Anonymous Coward]

That's a cool idea. I'll have to look into it further. I'm curious tho, does the overhead of hitting the DB for SSL key cause any issues?

I used to use a Cisco LocalDirector for load balancing a few years ago. Great device, but I could never properly achieve what this poster is looking for.

I like the idea (in some other post) of proxying the requests on the front, handles the SSL there, and then passing the requests to the real web servers. You still need to have a decent failover system though. Might be more trouble than its worth.

You can also do it with Cisco hardware.

[Mordant]

The CSS 11500 & SCA 11000 will do this.

Re:You can also do it with Cisco hardware.

[Anonymous Coward]

Actually, you cannot. The CSS's do not handle SSL, and therefore, could only load balance based upon rudimentary information - tcp/ip headers, basically.

Repeat?

[FreeLinux]

I addressed an earlier question about load balancing that is very pertinent to your question.

Put simply there is no better solution than a hardware one, for this situation. This Nortel solution does it all in one box. However, if your traffic load is great enough you might want to look at splitting out the functions across multiple boxes like these Nortel/Alteoon 180 Series switches. There is also a separate SSL concentrator/accelerator.

Re:Repeat?

[Anonymous Coward]

The iSD doesnt load balance, but it does handle SSL proxying. It's most commonly used with an Alteon Switch, like an AD4. This works well. The F5 is an "all in one box" solution, and afaik, the only load balancer on the market that can do SSL internally.

I'm sure you're right.

[FreeLinux]

But, for some reason, in my mind, "quality load balancing platform" and eBay just don't go together.

Do they have sthe support contracts on eBay too?

Re:I'm sure you're right.

[NetJunkie]

You can buy the hardware from anywhere. They'll happily sell you a support contract for it. They aren't going to refuse your money.

The contract is the same as if you bought the hardware new and tacked on the support.

it's not simple...

[trims]

Given the technological design behind SSL (in that it operates immediately above IP, and below TCP/UDP), the summary you provide above is true.

That is, you can set up a cluster of web servers behind a typical redirector. Say Server A, B, and C. Each machine has a different IP address, which is a routable address (let's call it w1.foo.com, w2.foo.com, w3.foo.com). There is a single IP address for the "master" domain (say www.foo.com). All machines are set up to serve www.foo.com, both HTTP and HTTPS. For normal HTTP requests, the redirector does it's thing, with EACH http request coming to www.foo.com being served by the next available machine. Any connection a client makes can potentially be served by any machine.

However, since SSL operates below the TCP/HTTP header, a client making a SSL connection to https://www.foo.com will initially be load-balanced across the machines. However, for the rest of that SSL session, it will continue to communicate ONLY with the machine which initially answered the beginning SSL connection.

This isn't a real big problem for most web apps, since each connection is really just getting a bunch of web pages. If the server goes down, you simply have to push RELOAD on the client, and it re-initiates the HTTPS session, getting another machine to answer. It can be a serious problem with other SSL apps, as the termination of a session by the loss of the answering machine may make it difficult to programatically re-start the session.

Just to make this clear: each time a client initiates an SSL session, it gets bound to a single server. This server may be different for each session, but remember that a "session" may include many data requests.

There is no problem sharing certificates between multiple machines (as the cert is tied to hostname, and it's easy to duplicate across multiple web servers). It's an SSL protocol limitation.

-Erik

Re:it's not simple...

[Anonymous Coward]

> Given the technological design behind SSL (in that it operates immediately above IP,
> and below TCP/UDP), the summary you provide above is true.

That's incorrect. SSL traffic is carried within a reliable connection (TCP), so it operates above the TCP transport layer.

There are several reasons to have clients being "stickily" bound to particular physical servers in a virtual server group, most of which have nothing to do with SSL. One is that there is some transient session data living on the server which wouldn't be available on other servers -- such as Java servlet state. Another is to maximize the efficiency of whatever caches might be in play, be they disk or database or SSL session state. This session stickiness is generally accomplished by hashing on the addresses, or by parsing the incoming HTTP request for some data to indicate which server to use -- usually a cookie header. The latter isn't possible with SSL unless the load balancer is actually terminating the SSL connection. These are often called SSL or "e-commerce" accelerators.

-- Robert

People always try to do it the hard way.

[Anonymous Coward]

Check out Novell's iChain product. The client connects to iChain via SSL and iChain can connect to any number of back-end servers. One iChain box can easily handle dual T3's so it'll probably work for you.

There is a paper on this topic

[Sitaram Iyer]

.. Here [ps.gz] published in NDSS '02.

Hmm

[The AtomicPunk]

I think you're either over-complicating this or I'm not quite following what you're asking.

If you said up a brain dead load balancer, the only real impact you'll have is that it'll have to restablish the SSL connection each time it hops servers. Not wonderful, but not the end of the world.

That's assuming you ARE sharing SSL certificates across machines - and if all the servers are supposed to be serving one secure domain (ie https://www.mydomain.com) then I'm not sure why you wouldn't.

A better solution is to use a load balancer that will support 'sticky' based on the SSL session ID, that way the sessions are kept open, saving both ends from a lot of uneccessary work - but this apparently doesn't suit your needs. :)

Re:Hmm

[Anonymous Coward]

SSL Session ID's won't cut it because shitty web browsers like IE V5.X.X renegotiate SSL session ID's every two minutes.

Linux Virtual Server with ip affinity

[danpbrowning]

The Linux Virtual Server with ip affinity is a great way to go. Very fast and effecient, stable.

Re:Linux Virtual Server with ip affinity

[Zen Sandwich]

Except the original question asked for a 'zero affinity' solution. Or does that imply that LVS' 'ip affinity' is actually no affinity?

Re:Linux Virtual Server with ip affinity

[danpbrowning]

Ha ha. You're right. But there penalty for not having ip affinity is $$$, and it isn't worth it (don't believe them when they say all the AOL users will hit one box -- ip affinity works fine). Look at the cost of F5 iron. Ouch. (Compared to a $50 Pentium 300 doing the same work even faster using ip affinity).

Fastest answer...

[ebrandsberg]

Check out the Netscaler 9400 or 9800. The 9800 is a gigabit load balancer that can do up to 4400 RSA handshakes per second and up to 450-500Mb/s of bulk decryption per second on one box. The 9400 is the fast ethernet version, so it is pretty limited to fast ethernet speeds for everything.

Erik