Slashdot Mirror
Building a Better Webserver
msolnik writes: "The guys over at Aces' Hardware have put up a new article going over the basics, and not-so-basics, of building a new server. This is a very informative, I think everyone should devote 5 minutes and a can of Dr Pepper to this article."
But don't.
Actually a very interesting article, to be honest, in my 1 year of building webserver applications. I haven't gone through a process like this once. Usually we make a rough guess about how the application has performed (or more usually underperformed on existing servers, and just scale a percentile. As you can imagine, this is hardly realistic. Thanks for the read!
Lets see exactly how long their lovely new webserver stands up to a slashdoting... :)
(Maybe they just sent this so they could test it? Plan.)
Instant traffic to your site, no advertising!
The above post is an editorial, the poster cannot and will not be held responsible for all or in part for it's contents
Microsoft has written several white papers of this sort already. Of course, they're Microsoft, so that means I can kiss my +2 bonus goodbye. Seriously, though.
If you fall off a building, go real limp, because maybe you'll look like a dummy and people will be like hey, free dummy
but I figure it's best to spend the money early on, get a good setup going that can handle high volumes
Throwing money at the problem is exactly the WRONG approch. You need to start by spending time PLANNING and RESEARCHING the best way to do things.
For example, if you are setting up a dynamic site like ./, which is serving 100 pages/second. It obviously needs to be dynamic, so you need a database to store all the comments in.
There are two ways to do this, one is to serve content straight out of the database, but this means that for every page you serve up, there needs to be a database query. (the database queries are the expensive part in terms of time it takes to serve a page). The other way would be to serve the articals as static pages which are generated every minute or so by a process on the database and pushed down to the web server, which serves these up as static pages.
The advantage of this is that insted of 100 database queries per minute, you end up with, maybe 10 queries per minute to populate the static pages. Sure, you site is no longer 100% dynamic, but it is a whole lot faster, and you have saved thousands of dollars to boot!
This is just one small off-the-top-of-my-head example of where PLANNING sould become way before spending any money.
...the one with a lot of mirrors.
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
"Real multithreading" is really no panacea. See the notes from John Ousterhout's talk, Why Threads Are A Bad Idea (for most purposes).
of Dr. Pepper? I expect to go through a whole case waiting for the slashdot effect to wear off...
Once upon a time, we had 1 web server that did everything, so it needed to be able to do everything. Now everytime we do something new we toss out a new webserver (or 2 or 10 of 'em). And they all basically need to do one thing (webmail, portal, whatnot) and do it well and that's it.
So we've got a whole bunch of Apache servers which a bucket load of apache processes who basically spend all day doing little more than exec'ing the same CGI over and over (and copying the data around a couple of extra times).
I'm pretty much now convinced that would my next step is going to be to franken-meld my cgi with something like mini-httpd so it is a single, persistant, app.
I'm certainly not redoing the whole thing in Java though! :)
Shut up, be happy. The conveniences you demanded are now mandatory. -- Jello Biafra
Is it just me, or do most folks confuse these two. If a popular website only has a 9 Mbps pipe to the Internet, it doesn't matter how many Crays they have running their webserver farm, they're only going to be able to churn out 9 Mbps (minus overhead). Granted that the converse is possible... gobs of bandwidth, but a slow server... but I would imagine that bandwidth is the limiting factor of at least 99% of websites.
Consider a user with a typical analog modem that has an average maximum downstream throughput of, say, 5 KB/s. If this user is trying to download the general message board index page, about 200 KB in size (rather small by today's standards), it will require a solid 40 seconds to complete this single download.... To maximize the efficiency of the network itself, we can compress the output stream and thus, compress the site. HTML is often very repetitive, so it's not impossible to reach a very high compression ratio. The 200 KB request mentioned above required 40 seconds of sustained transfer on a 5 KB/s link. If that 200 KB request can be compressed to 15 KB, it will require only 3 seconds of transfer time.
Except that 56 Kbps modems get 5 KBps thoughput by compressing the data! If the client and server compress, the modems won't be able to; the net effect is lots of extra work on the server side, and probably no increased throughput for the modem user.
The server might or might not see a decrease in latency, and in the number of sockets needed simultaneously; it depends on how much it can "stuff" the intermediate "pipes". The server will see an overall decrease in bandwidth needed to serve all the pages.
Ironically, broadband customers (who presumably don't have any compression between their clients and Internet servers) will see pages load faster. (And the poor cable modem providers from the previous story will be happy.)
Stupid job ads, weird spam, occasional insight at
SPARCs come from Sun, everybody makes a PC - so guess which is cheaper? We see some reasons why they went for the Blade (a nice machine, but rather more expensive than a couple of PCs).
Please get this right, I'm no x86 fan, but I love the competition going through the line from the processors, chip-sets, mother-boards, etc. This has got to ensure that unless you really want the 2GHz Pentium 4, you have plenty of choice.
As for reliability, I don't know the Blade, but the SPARC 20s used to give some headaches over their internal construction. It always seemed a little complicated with the daughter boards and they seemed to lose contact after machines were moved around.
See my journal, I write things there
One thing that does seem to work against the onslaught is a throttling webserver. If you haven't got the bandwidth etc to serve a sudden onslaught of requests, probably the best thing to do is to just start 503'ing -- at least people get a quick message 'come back later' instead of just dead air.
Shut up, be happy. The conveniences you demanded are now mandatory. -- Jello Biafra
The SPARCstation 20 was one heck of a great machine back in the day, especially for its size (a low profile pizzabox). The design was a lot like it's older brother (the SPARCstation 10 from 1992)... that is, two MBUS slots (for up to 4 CPUs) and 4 SBUS slots (Sun Expansion cards, 25 MHz x 64 bit = ~ 200 MB/sec each, but 400 MB/sec bus total).
I remember using a Sun evaulation model at Rice many years ago... the machine had two 150 MHz HyperSPARC processors (though 4 were avilable for more $$), a wide SCSI + fast ethernet card, two gfx cards for two monitors, and some sort of strange high speed serial card (for some oddball scanner, I think). Not to mention 512 MB of ram, in 1994! The machine was a pretty decent powerhouse and sooo small! I sort of wish the concept would have caught on, given how large modern workstations are in comparison. Heck, back then an SBUS card was about 1/3 the size of a modern 7" PCI card.
Then there's the other end of the spectrum... one department bought a Silicon Graphics Indigo2 Extrme in 1993. The gfx cardset was three full size GIO-64 cards (64 bit @ 100 MHz = about 800 MB/sec), one of which had 8 dedicated ASICs for doing geometry alone. 384 MB of RAM on that beast. Pretty wild stuff for the desktop.
Ahh, technology. I love you!
Ousterhout says threads are bad for apparent concurrency but good for taking advantage of multiple processors, and for building scalable servers.
In other words, with the right hardware architecture, threads could be very useful for sites such as Ace's Hardware (though they happened to go with a uniprocessor) and Slashdot.
Java threads are also easier to program than C and C++ threads, though not easy. (Manual memory management is hard; thread programming is hard; manual memory management in a threaded program is very hard. I'm not speaking hypothetically on the last point; I've really envied Java programmers the last few weeks.)-:
Stupid job ads, weird spam, occasional insight at
Actually, that's a terribly wasteful way to go. If you work on an easily-scalable infrastructure, then you can pretty much purchase capacity as it's needed, which not only frees up capital for a longer time, you end up spending a lot less, as the price of computers is always dropping, and the performance is always going up.
steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
In a part about databases and persistent connections they confuse the issues more than a bit. The real problem is not too many processes, what automatically makes threads look better, but the symmetry among processes -- any request should be possible to serve by every process, so all processes end up with database connections. This is a problem particular to Apache and Apachelike servers, not a fundamental issue with processes and threads.
In my server (fhttpd I have used the completely different idea -- processes are still processes, however they can be specialized, and requests that don't run database-dependent scripts are directed to processes that don't have database connections, so reasonable performance is achieved if the webmaster defines different applications for different purposes. While I didn't post any updates to the server's source in two last years (was rather busy at work that I am leaving now), even the published version 0.4.3, despite its lack of clustering and process management mechanism that I am working on now, performed well in situations where "lightweight" and "heavyweight" tasks were separated.
Contrary to the popular belief, there indeed is no God.
Speaking as the maintainer of a site that is periodically slashdotted...
e d, everyone waiting in queue gets a periodic update, at a certain point the load of generating the updates swamps the machine. I have to limit the number of people in queue.)
Yes, a throttling server is a great idea. If you recognize that there will always be a load too high for you to handle (10 requests per minute for my site, yes minute, it is a physical device), then you must either decide to deal with the load or let the load crush your machine.
Consider a typical web server. When it gets overloaded it slows down, each request takes longer to handle, there are more concurrent threads, overall efficiency drops, each request takes longer to handle.... welcome to the death spiral. (on my site-which-must-not-be-named-less-it-be-slashdott
The key decision is to determine how many concurrent threads you can handle without sacrificing efficiency and then reject enough traffic to stay under that limit.
This is where optimism comes in and bites you in the ass. You remember that every shunned connection is going to cost you money/fame/clicks whatever so you set the limit too high and melt down anyway.
There are more factors than just CPU and Bandwidth... like what's between the two. A new coworker recently told me of his major learning experiences in the mid 1990s running several popular news websites durring the beginning of the web boom. One of the more popular sites he ran originally had a T1 routed through a Cisco 4000 router. Things worked great until he had an additional, load balancing T1 added for added thruput and redundancy. Things didn't feel much faster, in fact, they were almost slower. After much investigation he learned that the router didn't have enough RAM or CPU to handle the packet shuffling that intelligent multihoming routing requires. A similar instance happined with a friend's company when they tried to run a T3 through their existing router. While the old cisco had enough cpu and ram in theory, its switching hardware and thruput couldn't handle the full number of packets the T3 was providing thru the shiny new HSSI high speed serial card.
Now, I realize modern hardware (Cisco 3660 and 7x00 series, and pretty much any Juniper) can route several T3s (at 45mbps each direction) worth of data, but older routers and minimally configed routers do exist.
There are MANY bottlenecks in hosting a website. Server daemon, CPU, router, routing and filtering methods, latency and hops between server and internet backbones, overall bandwidth thruput, and much more.
It's not as simple as "lame server, overloaded CPU, should have installed distro version x+1".
...is crap crap crap crap crap. The Blade 100 is one of Sun's IDE-based machines. Because Sun expects that such boxes are going to be used either as low-end workstations (the Blade 100) or disk-avoiding compute farm servers (the Netra X1), the disk subsystem of them is painfully below par: Sun routinely ships under-speced IDE disks (remember the 4800 RPM drive? Sun does, and they've got a whole warehouseful to refresh your memory with!) and compounds the problem with Solaris 8's ATA/IDE drivers, which are the worst in the known universe -- watch your entire system drag to a half as the OS attempt to write out a .5MB file! Whee!
I don't know what Ace's traffic numbers are normally like, but using a Blade 100 for anything other than a small, personal website is flat-out folly. At a minimum, they should have been using a Netra T1/AC200 ($3k, nicely configured, and a 1U rackmount machine to boot), and I would probably have thought seriously about scrounging a used E250 or E220R off of Ebay.
News for Nerds. Stuff that Matters? Like hell.
If you haven't noticed by now, Ace's Hardware has a neat little indicator on each page that shows time processing and queue time it spent getting to you (very bottom left-hand corner of each page). Most are about 74ms - 112ms for me. This, plus the result of some pings and traceroutes leads me to belive they're heavily BANDWIDTH bound right now, not CPU bound. I do hope Ace puts up a summary of the Slashdot effect as well as some other data for us to pour over. Some MRTG router graphs of the bandwidth usage would be *really* nice, too.
If you can't live without Apache, there's always mod_bandwidth.
Not quite as elegant a solution, but it's nice for preventing your web server from taking all of your bandwidth (if, say, you run it off your cable modem, and wish to continue gaming...).
quote:
::same:: address space, and only get new memory when they try to touch a page that is write only (ie they can run and run and run, but once they try to access their memory they get new memory space with the contents copied). It saves time and memory.
:(
This means an Apache web server using keepalives will need to have more child processes running than connections. Depending upon the configuration and the amount of traffic, this can result in a process pool that is significantly larger than the total number of concurrent connections. In fact, many large sites even go so far as to disable keepalives on Apache simply because all the blocked processes consume too much memory.
::end quote::
lets see, anyone here hear of COW (copy on write) Linux uses this idea to save time on fork'd child processes, they get the
The only setback is when a process fork's a child, its current time slice is cut in half with half given to the child, so the main proc will run aground if to many requests come in and the server has more processes to worry about.
-ShadoeLord
this is my sig, there are many like it, but this one is mine.
When I discussed this issue with Thau (or to be precise, he did most of the talking) he gave the reason for using processes over threads as the awful state of the then pthreads packages. If Apache was to be portable it could not use threads. He even spent some time writing a threads package of his own.
I am tempted to suggest that rather than abandon apache for some java server (yeah lets compile all our code to an obsolete byte code and then try to JIT compile it for another architecture), it should not be a major task to replace the Apache hunt group of processes with a thread loop.
The other reason Thau gave for using processes was that the scheduler on UNIX sux and using lots of threads was a good way to get more resources, err quite.
Now that we have Linux I don't see why the design of applications like apache should be compromised to support obsolete and crippled legacy O/S. If someone wants to run on a BSD Vaxen then they can write their own Web server. One of the liabilities of open source is that once a platform is supported it can end up with the application supporting the platform long after the O/S vendor has ceased to. In the 1980s I had an unpleasant experience with a bunch of physicists attempting to use an old MVS machine, despite the fact that the vendor had obviously ceased giving meaningfull support for at least a decade. In particular they insisted that all function calls in the fortran programs be limited to 6 characters since they were still waiting for the new linker (when it came it turned out that for functions over 8 characters long it took the first four characters and the last four characters to build the linker label... lame, lame, lame)
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/