Best Solution For HA and Network Load Balancing?

supaneko writes "I am working with a non-profit that will eventually host a massive online self-help archive and community (using FTP and HTTP services). We are expecting 1,000+ unique visitors / day. I know that having only one server to serve this number of people is not a great idea, so I began to look into clusters. After a bit of reading I determined that I am looking for high availability, in case of hardware fault, and network load balancing, which will allow the load to be shared among the two to six servers that we hope to purchase. What I have not been able to determine is the 'perfect' solution that would offer efficiency, ease-of-use, simple maintenance, enjoyable performance, and a notably better experience when compared to other setups. Reading about Windows 2003 Clustering makes the whole process sounds easy, while Linux and FreeBSD just seem overly complicated. But is this truly the case? What have you all done for clustering solutions that worked out well? What key features should I be aware for successful cluster setup (hubs, wiring, hardware, software, same servers across the board, etc.)?"

Comments

Re:Is It Mission Critical?

[Errtu76]

It's not overboard. And even with a hosting provider you're still dependent on hardware problems. What you can do to realise what you want is:

- buy 2 cheap servers with lots of RAM
- set them up as XEN platforms
- create 2 virtuals for the loadbalancers
- setup LVS (heartbeat + ldirectord) on each virtual
- create 4 webserver virtuals, 2 on each xen host
- configure your loadbalancers to distribute load over all webserver virtuals

And you're done. Oh, make sure to disable tcp_checksum_offloading on your webservers, else LVS won't work that well (read: not at all).

we run a nonprofit with 100m+ visitors a day

[midom]

Hi! we run a non-profit website that gets 100 million visitors a day on ~350 servers. we don't even use any "clustering" technology, just replication for databases, and software (LVS) load balancer in front of both app (PHP) and squids at the edge. but oh well, you can always waste money on expensive hardware and clustering technology. and you can always check how we build things

STOP. You have no idea what you're doing.

[Enleth]

I'm sorry, but I have to say that. Don't be offended, please - sooner or later you will look at your submission and laugh really hard, but for now you need to realise that you said something very, very silly. A few people already politely pointed out that 1000 visitors a day is nothing - but seriously, it's such a great magnitude of nothingness that, if you make such a gross misintepretation of your expected traffic, you need to reconsider if you really are the right person for the job *right now* and maybe gain some more experience before trying to spend other people's money on a ton of hardware that will just sit there, idle and consume huge amounts electricity (also paid by other people's money).

I'm serving a 6k/day website (scripting, database, some custom daemons etc.) from a Celeron 1.5GHz with 1GB RAM, and it's still doing almost nothing. If you really have to have some load balancing, get two of those for $100 each.

Re:1000+ a day isn't very much

[Bandman]

HA isn't there just for load issues. It's there to guarantee availability. 1,000 users might be peanuts, but I've got a site that only gets a couple hundred visitors a day. That site has clustered load balancers which talk to redundant app servers, which talk to redundant web servers (connected via redundant switches). It's really important that the site be there for those couple of hundred visitors.

The number of visitors isn't as important as the importance of the visitors.