High-Performance Web Server How-To

ssassen writes "Aspiring to build a high-performance web server? Hardware Analysis has an article posted that details how to build a high-performance web server from the ground up. They tackle the tough design choices and what hardware to pick and end up with a web server designed to serve daily changing content with lots of images, movies, active forums and millions of page views every month."

Comments

High-performance web server

[quigonn]

I'd suggest everybody with the need of a high-performance web server to try out
fnord. It's extremely small, and pretty fast (without any special performance hacks!), see here.

Re:High-performance web server

[Electrum]

Yep. fnord is probably the fastest small web server available. There are basically two ways to engineer a fast web server: make it as small as possible to incur the least overhead or make it complicated and use every possible trick to make it fast.

If you need features that a small web server like fnord can't provide and speed is a must, then Zeus is probably the best choice. Zeus beats the pants off every other UNIX web server. It's "tricks" include non blocking I/O, linear scalability with regard to number of CPU's, platform specific system calls and mechanisms (acceptx(), poll(), sendpath, /dev/poll, etc.), sendfile() and sendfile() cache, memory and mmap() file cache, DNS cache, stat() cache, multiple accept() per I/O event notification, tuning the socket buffers, disabling nagle, tuning the listen queue, SSL disk cache, log file cache, etc.

Which design is better? Depends on your needs. It is quite interesting that the only way to beat a really small web server is to make one really big that includes everything but the kitchen sink.

my $0.02

[spoonist]

* I prefer SCSI over IDE

* RedHat is a pain to strip down to a bare minimum web server, I prefer OpenBSD. Sleek and elegant like the early days of Linux distros.

* I've used Dell PowerEdge 2650 rackmount servers and they're VERY well made and easy to use. Redundant power supplies, SCSI removable drives, good physical security (lots of locks).

Re:my $0.02

[Door-opening Fascist]

RedHat is a pain to strip down to a bare minimum web server, I prefer OpenBSD [openbsd.org]. Sleek and elegant like the early days of Linux distros.

OpenBSD doesn't have support for multiple processors, which are a necessity for database servers and dynamic web servers. I'd say FreeBSD is the way to go.

Re:my $0.02

[SuiteSisterMary]

If your server isn't designed with 'security' in mind, including the ability to padlock the chassis, and at least send an SNMP trap when the chassis is opened, then you need to learn that as far as 'computer and data security' is concerned, protecting from external network attacks is actually quite low on the totem pole.

Or, "If Joe Random Idiot can walk in and rip out the hard drive, who cares how 3117 your firewall and other network protections are."

Re:10'000 RPM

[khuber]

10k drives are LESS reliable, since they move faster.

Okay, well ,you can use ancient MFM drives since they move much slower and would be more reliable by your logic.

Personally, I'd take 10k SCSI drives over 7.2k IDE drives for a server, no question.

-Kevin

Re:So fast and soo goo...

[irc.goatse.cx troll]

Server has nothing to do with it.
10,000 slashdotters * 500k pages = 5gigs in about an hour.
these figures are both estimates, but you can see that network congestion is obviously more of a bottleneck than their performance server.

Alternative HowTo

[h0tblack]

1. goto here
2. click buy
3. upon delivery open box and plugin
4. turn on Apache with the click of a button
5. happily serve up lots of content :)

6. (optional) wait for attacks from ppl at suggesting using apple hardware...

Why Apache?

[chrysalis]

I don't understand.

Their article is about building a high performance web server, and they tell people to use Apache.

Apache is featureful, but it has never been designed to be fast.

Zeus is designed for high performance.

The article supposes that money is not a problem. So go for Zeus. The Apache recommendation is totally out of context.

Server running at near 100% load

[ssassen]

From the SecureCRT console, connected through SSH1, as the backend is giving me timeouts. I can tell you that we're near 100% server load and are still serving out those pages to at least 1500 clients. I'm sure some of you get timeouts or can't even reach the server at all, for that I apologize, but we just have one of these, not a whole rack full of them.

Have a good weekend,

Sander Sassen

Email: ssassen@hardwareanalysis.com
Visit us at: http://www.hardwareanalysis.com

Re:Building a Better Webserver in the 21st Century

[khuber]

I hate to do this, but actually MS has put out some good stuff that's relevant to larger sites.

http://www.microsoft.com/backstage/whitepaper.htm

-Kevin

how to build a high performance/reliable webserver

[jacquesm]

1) use multiple machines / round robin DNS
2) use decent speed hardware but stay away from
'top of the line' stuff (fastest processor,
fastest drives) because they usually are not
more reliable
3) replicate your databases to all machines so
db access is always LOCAL
4) use a front end cache to make sure you use
as little database interaction as you can
get away with (say flush the cache once per
minute)
5) use decent switching hardware and routers, no
point in having a beast of a server hooked up
to a hub now is there...

that's it ! reasonable price and lots of performance

Re:"Three times the power?"

[NineNine]

"Microsoft Windows 2000 Pro"

I got a good laugh out of this... W2K Pro is the desktop version, not the server version. Wow. Great article. Really well informed author.

Re:But any web server is high-performance

[NineNine]

Our databases are tuned. Some apps would just need to transfer too much data per request for a SQL call to be feasible.

I had this problem for a while... Sloppy coding on my part was querying 65K+ records per page. Server would start to crawl with a few hundred simultaneous users. Since I fixed it, 1000+ simultaneous users is no problem at all.

Re:That "howto" sucks

[jimfrost]

High traffic sites, the ones that are really dynamic anyway, do more than that.

They start with a load balancer at the front end, or possibly several layers of load balancer. If they run a distributed operation they'll use smart DNS systems or routers to direct requests to the most local server cluster. The server cluster will be fronted by a request scattering system.

Behind the request scattering system you'll find a cluster of machines whose job it is to serve static content (often the bulk of data served by a site) and route dynamic requests to another cluster of servers, enforcing session affinity for the dynamic requests.

Behind the static content servers are the application servers. They do the heavy lifting, building dynamic pages as appropriate for individual users and caching everything they can to offload the database.

Behind the application servers is the database or database cluster. The latter is really not that useful if you have a highly dynamic site as there are problems with data synchronization in database clusters (no matter what the database vendors tell you). But that's ok, single databases can handle a lot of volume if built correctly and caching is done appropriately at the application level.

And there you have it, the structure of a really large site.

Re:OK so where do I start?

[ssassen]

People are negative because the server has been unreachable for some, but they tend to conveniently forget that we did not design for 2000+ simultaneous clients, just a couple of hunderd really. Just thought I'd let you know, as we only have one of these whereas most websites (like Anand and Tom) have a rack full of them. Still we're handling the load pretty well and are serving out the pages to about 1500 clients.

Have a good weekend,

Sander Sassen

Email: ssassen@hardwareanalysis.com
Visit us at: http://www.hardwareanalysis.com

Re:But any web server is high-performance

[jimfrost]

I've seen both kinds and take it from me, many small servers is more of a headache than the hardware cost savings is worth. Your network architecture gets complicated, you end up having to hire lots of people just to keep the machines running and with up-to-date software, and database connection pooling becomes a lot less efficient.

You save money in the long run by buying fewer, more powerful machines.

Re:"Three times the power?"

[(H)elix1]

That was total FUD. The two operating systems have comparable performance on the same hardware.

Win2k pro limits you to 10 concurrent TCP/IP connections, Win2K Server has no (artificial) limit but won't cluster, Advanced Server can cluster but I don't know a thing about it..

Linux has no (artificial) limit... not sure about clustering options there either.

Found out about the TCP/IP limit when I added SP2 and trashed my evening counter-strike server - this makes a HUGE difference.

Re:But any web server is high-performance

[Hast]

How about reading the FAQ before you start giving out "facts"? Slashdot is running on:
* 5 load balanced Web servers dedicated to pages
* 3 load balanced Web servers dedicated to images
* 1 SQL server
* 1 NFS Server
Either the "little 4 way intel" you mention has a serious case of shizofrenia or your just full of it. (Guess which theory I'm going for.)

Besides the poster mentioned that those sites /are/ bigger than Slashdot. E.g. the mention that "Getting your URL posted during Friends" is nothing like getting it posted on Slashdot.

I know I shouldn't feed the trolls, but someone might actually belive this tripe.

Re:But any web server is high-performance

[Aldurn]

Aside: Is there any open source software that manages session affinity yet?

Yes. Linux Virtual Server is an incredible project. You put your web servers behind it and (in the case of simple NAT balancing) you set the gateway of those computers to be the address of your LVS server. You then tell LVS to direct all IPs of a certain netmask to one server (i.e. if you set for 255.255.255.0, 192.168.1.5 and 192.168.1.133 will connect to the same server).

The only problem I had with it was that it does not detect downtime. However, I wrote a quick script that used the checkhttp program from Nagios to pull a site out of the loop when it went down (these were Windows 2000 servers: it happened quite frequently, and our MCSE didn't know why :)

There are higher performance ways to set up clustering using LVS, but since I was lazy, that's what I did.

Re:"Three times the power?"

[Magila]

Win2k pro limits you to 10 concurrent TCP/IP connections.

Whao! bullshit meter rising! While Win2K does have a limit on TCP/IP connections, it is in the thousands. A limit of 10 would be totaly ridiculous, it would cripple the OS for MANY people. Also, most of the traffic for a CS server is UDP so the TCP/IP connection limit isn't going to affect that much at all.

Re:"Three times the power?"

[elemental23]

The maximum number of other computers that are permitted to simultaneously connect over the network to Windows NT Workstation 3.5, 3.51, 4.0, and Windows 2000 Professional is ten. This limit includes all transports and resource sharing protocols combined. This limit is the number of simultaneous sessions from other computers the system is permitted to host.

From Microsoft Knowledge Base Article Q122920.
(Warning: The page layout is broken in Mozilla)

It's an artificial limitation. The idea is that if you need more simultaneous connections you should buy Win2k Server. In other words, MS wants you to spend more money.

Re:server load

[1110110001]

Maybe the article Handling the Loads, describing how Slashdot kept their Servers up at 9/11, is a bit of the thing you're looking for. b4n