Slashdot Mirror
Scaling Server Performance
An anonymous reader writes "When Ace's Hardware's article Hitchhiker's Guide to the Mainframe was posted on Slashdot, they got 590,000 hits and over 250,000 page requests during one day. This kind of traffic caused only a 21% average CPU load to their Java-based web server, which is powered by a single 550MHz UltraSparc-II CPU. In their newest article, Scaling Server Performance, Ace's Hardware explains how this was possible."
Are we supposed to be impressed with a computer that can serve 8 hits and 4 pages per second?
they got 590,000 hits and over 250,000 page requests during one day. This kind of traffic caused only a 21% average CPU load ... they didn't respond to any of them.
Reliable, Great Value Hosting: $7.95/mo 2.4G/120G
Of course, it is incumbent upon all of us to rush out and try to the link to the article. And some of us to actually read it as opposed to just reading the title.
SLASHDOT THEM AGAIN!!!
When I was benchmarking web servers in *1994*, servers could handle 100,000/hr, which is only about 30/sec. You may need a T3 to handle the bandwidth, but any server that can't handle it today is misconfigured.
seeing as it took Slashdot 35 seconds to serve me up this comments.pl?op-Reply page, yes, i think we are supposed to be impressed.
Cretin - a powerful and flexible CD reencoder
Which is very funny: this is an article explaining how a web site survived the /. effect, thus trying to catch the /. readers back for a second round, and getting lots of advertising hits at the same time. If only that server could keep up. /. I saw a report about a 200Mhz (?) PC running Windows 95 and with about 30 hard disks, that also seemed to do very well under the /. effect.
Now, a while back on
Sig for sale or rent. One previous user. Inquire within.
I would be more interested in stats on a webserver that took a puke. It would be interesting to see what started the dominos falling and what ultimatly brought it down. It would be as good a learning experiance as this article is.
Not to be cynical, but serving (nearly) static pages shouldn't be a huge load by any standard. Even with dynamic (fully dynamic) pages, 250,000 isn't a huge number.
As an example, I run a pretty popular site that pumps out about 250,000 as well, all CGI-created and database fed pages. This is being served by two 1ghz web heads and a 1ghz db server. Granted that those three machine run at 100% load during peak hours, it's still not a huge deal (this is because I haven't finished the local caching mechanism yet). Did I mention that the two webservers also toss 1 million images a day too?
Of course, I don't wan to belittle the article that much -- If anything, it shows the preformance gains one gets when you use efficient hardware (I have no doubt that their 550 mhz Ultrasparc II has nearly the same horsepower as a 1 ghz x86) and efficient caching (caching data in RAM and serving from there, avoiding disk access penalties, is a huge performance increase).
Hilary Rosen's speech was about her love of money and her desire to roll around naked in a pile of money.
OTOH, my puny little SDSL connection was seriously maxed out.
Even old hardware can happily serve up hundreds of documents a second, if the pages are static.
>Garth Brooks covers this in his famous book "The Mythical Man Month" where he proves in a controled lab environment that Java under X86 runs on the order of Olog(n) slower than it does on a RISC chip like an UltraSparc.
WTF? Fred Brooks wrote this book, and I don't seem to remember RISC or UltraSparc chips, not to mention Java, in 1974. Garth Brooks is (AFAIK) a country music singer. Try again.
Sig for sale or rent. One previous user. Inquire within.
I never really thought that the problem lied with the server's hardware, but in the bandwidth to the host. Shouldn't an article be written about how to conserve bandwidth during a slashdot effect? Even older servers should be able to handle 100 requests per second. I think most FPS's are alot more taxing than that.
If you limit yourself to static stuff then you can cram any amount of traffic out of very limited boxes.
Even google does everything they can to cache stuff and turn dynamic requests into static ones, and they actually have a reason (lotsa traffic, complicated requests).
The fact that you can use java to write speedy code doesn't prove a thing either, it only says that it is now no longer a bottleneck.
You can probably saturate a decent sized pipe using -- aaarghh -- VB or something asinine like that as long as you do 'pictures and pages'.
MP3 Search Engine
"The page cannot be displayed".
The lesson here is: put your money where your mouth is and you may end up eating it.
Lots of people could use this type performance. I only had a chance to use JSP on one project, a while back. Tomcat was notoriously difficult to install back then. But when it was up, the difference between JSP application server and PHP become apparent. Application servers can make quite the difference.
Just having an application scope for variables saved us a trip the the ldap server per request. PostNUKE, squirellmail, and lots of other large PHP apps could be sped up drastically if some of those features were available in the PHP engine.
Based on upvotes, Ageism is the only "-ism" Slashdotters care about and think isn't SJW
I prefer reading "The Mythical Man Month" in the original latin. Brooks' translation leaves much to be desired.
Of course, there are more complex applications where data caching can be implementing, such as discussion forums where multiple users can be adding, editing, and deleting messages simultaneously. But that's a topic for another article.
Most of the applications I write involve updating data almost as often as fetching it from the database. In an environment like Apache where you have individual processes serving content (and database connections are process-centric), implementing caches that are updatable becomes a very complex excercise, without implementing an additional layer.
eToys used a b-tree (Sleepycat?) database layer situated in front of the database layer - they would store objects in the b-tree, and fetch them from there if they had not expired. Once cache amongst all the servers made this worth doing; a Java web server can do something similar, since the objects are stored in memory shared between the various serving threads. The end result is similar to what Ace's Hardware has done.
What have other people done? Since I use Apache, I'm leaning towards a disk-based caching system.
Reloading their page a couple times (2nd page of the article, not the one slashdot linked to), I'm getting occasional 503 errors, and the rest are taking a very long time to load. Usually the page comes up with some "broken" images that didn't load.
At the bottom of each page, there's a number that seems to indicate the time they believe their server spent serving the page. Usually is says something like "2 ms" or "3 ms"... That may be how long their code spent creating the html, but the real world performance I see (via a 1.5 Mbit/sec DSL line) is many seconds for the html and many more for the images, some of which never show up, and sometimes a 503 error instead of anything useful at all.
So, Brian, if you're reading this comment (which will probably be worthy of "redundant" moderation by the time I hit the Submit button)... it ain't workin' as well as you think. Maybe the next article will be an explaination of what went wrong this time, and you can try again???
PJRC: Electronic Projects, 8051 Microcontroller Tools
We have OLD Cobalt Raq3's (300 MHz AMD K6, 128 MB Ram, single IDE drive) running the latest Cobalt OS, and we JUST had one of these boxes get hammered this week; in a 12 hour period, it handled 625,000 hits (mostly CGI's, but it had a reasonable amount of static content), and at the same time handled 35,000 POP requests, sent 4,500 emails, and did some other random functions (and things like hostname lookups are enabled for weblogs, FTP uploads are happening for weather-wite webcams that were associated with the heavy traffic, etc, so there's obviously not a huge amount of "tune it till it's ONLY gonna do one thing" going on here). Now, the box was taking a whipping compared to it's normal load, but c'mon. I can't say the "Poor little 550 MHz UltraSPARC story" makes me tear up :-)
Tuesday, November 27, 2001 8:07 AM EST
------
It was published over a year ago, and undoubtedly was based on their spring/summer 2001 trials. Even then this info wasn't revolutionary, and is even less so now.
creation science book
They've really simply discovered that dynamically generating essentially static content is a bad idea : the 'dynamic' pages they are talking of are just articles which once written stay the same, and so are serving identical pages to each user.
Using scripting with database look ups to create such pages is obviously not good - much better is to compile your data in to static pages and serve those. I have done this for my own website using XSLT to generate the html pages with consistant links and menu's etc. - but you do have to remember to re-build it after making any changes or adding new content (I use gnu make to handle the dependancies of one page upon another so it doesn't rebuild the entire site everytime.)
They've taken the alternative approach of still using a database for the requests, but then caching future requests for the same page-id's, which has the advantage of being compatible with their original dynamic generation system, but they don't mention how they handle the dependancy / cascading alterations problem if they change the content (though they could always flush the entire cache of course....).
Neither of these approaches can help you though if you have real dynamic pages where every request is unique or there are are too many possible pages for caching to be feasible (for example amazon or google).
Unfortunately, the comment on the processor isn't quite right.
The UltraSparc II only goes up to 480MHz, and the UltraIII starts at 750. In between is the grey area of the IIe and IIi, and the ONLY Sun box with a 550MHz processor is the SunBlade 100/150.
If that's their web server, then the CPU is the least of their worries--the thing has internal IDE drives, two (only) 33MHz narrow PCI slots, and not much else. Assuming that one of the PCI slots is used for a faster and/or redundant network connection (QFE card most likely), then the other one is the only connection to SCSI disks. That CPU, low-end as it is (for Sun), is definitely going to spend its time waiting for the rest of the system.
(And yes, I know that was your second point--I just wanted to back it up with some detail)
"People who do stupid things with hazardous materials often die." -- Jim Davidson on alt.folklore.urban
I remember reading that original article, and yes, I was impressed at the responsiveness of the server. But before they are congratulated so much, consider this. The original story was posted on slashdot at 1AM.. so the initial spike of activity resulting from the linking being in the top few on Slashdot was directly proportional to the number of people on Slashdot at the time. As you can see from their graphs (if they're showing up for you) that traffic spiked, then continued on during the day.
This time around, the link got posted at 2PM not 1AM, and so far as I can see, they handle this flurry of hits much less gracefully than the previous ones! There are a lot more people online at 2PM than 1AM (all arguments of nocturnal nighthawks and people in other time zones aside).
Seriously ... the numbers aren't that great. I used to admin a DEC Alpha Digital Unix server running at a whopping 300Mhz and it routinely served over 1.5M hits per day along with email, authentication and accounting for over 5,000 people and we rarely if ever saw it over a 0.5 load average. This was 4 years ago.
It's not apples to apples, since we weren't serving the same set of pages (we had around 500 personal homepages, each with a varied combination of static HTML, images and CGI programs) but honestly, if the numbers in this article are supposed to be impressive, we've grown too accustomed to web server feature bloat.
It is more productive to voice thoughtful opinions (reply) than to judge (moderate) others.
Queuing approaches have proven to be much more scalable in other areas - no reason to think it wouldn't work for web servers. Check out SEDA: An Architecture for Highly Concurrent Server Applications for a working implementation in Java that outperformed Apache [insert benchmark caveat here].
More on event-driven servers that minimize data copies and context-switching here.
Some static story pages? Who cares?
It all depends if you are actually doing something of interest.
Like the comments in Slashcode, most apps go from static, to dynamic, to static caching of dynamic pages.
At DTN we served up customized portal pages to people with commodity and equity quotes, news, graphs, etc. Since they didn't have any money we had to use a load balanced Pentium Pro and a Pentium II. The app had no problem serving the load, and it was fast.
Now that I work for companies that have money, our apps run really slow. Developers get expensive machines and don't know how to optimize any more.
I remember back in 1986 my Web-Enabled Elmo Daemon (WEED) was pretty lazy. But he sure could take hits. I think somewhere along the lines of 500 a day.
You're nothing; like me.
Have you ever tried a test where the clients kept their connections open for a reasonable length of time??
In the real world, virtually all clients are connected via links ranging from slow dialup to 1.5 Mbit/sec. They hold connections open and tie up server memory resources for a lot longer than a fast-as-possible benchmark running on the same machine or over fast ethernet.
Any server running on a single box is probably going to have trouble with 17000+ pages per seconds to modem users, who require many seconds to transfer the page. If the average connection open time is 2 seconds, that's 34000 open connections. Even if the server used only 32k of RAM per connection (barely enough to buffer a few packets and allocate "window" inside the TCP layer in the OS, and maintain OS-level info and buffering for the open file), that'd be over 1 gigabyte of memory. I suspect a combination of Windows (TCP/IP & file I/O), IIS, and ASP.NET uses a lot more than 32k per connection.
PJRC: Electronic Projects, 8051 Microcontroller Tools
Ace's Hardware needs to research real servers before talking about their "scalable" servers. Their numbers are really saying that their box performs like a dog.
For those of you interested in this topic here is a few pointers and words of wisdom.
Server scalabilty and performance has three basic metrics, thruput (urls/sec), simultaneous connections, and performance while overloaded. Of course, you could add latensy but I'd argue that with the correct design latency is directly proportional to the real work you are doing, bad design insertes arbitrary waits.
I know of a HTTP Proxy by a large ISP that does user authentications & URL authorization (re: database), header manipulation, and on-the-fly text compression at 3000 urls/sec for 2000-4000 simultaneous connections and maintains that performance under load by sheding connections, all this on a dual 1GHz Intel PIII box running a Open Source OS that starts with "L". That is a maximum of 260 Million URL/day, three orders of magnitude greater performance than Ace's Hardware stats.
The simple answer to the question "How do I create a scalable fast network server?" is Event-driven GOOD & Threads BAD. Event driven network communication is two to three orders of magnitude better performing than thread/thread-pool based network communications. See Dan Kegel's C10K web page. That means you must use non-blocking IO to client sockets and databases. Once you accomplish that small feat, dynamic content just consumes CPU; with 2.8 Ghz Xeon processors you have plenty of cycles for parsing HTML markup or whatever. Threads cause cache thrashing, and context switching. While thread programmers don't see the cost in their code, just read the kernel code and you'll see how much work HAS TO BE DONE to switch threads. Event driven programming just takes some state lookups (array manipulation) and a callback (push some pointers onto the stack and jump to a function pointer).
Desgin is FAR MORE IMPORTANT than which runtime you use (execution tree, byte code, or straight assembly). I have done some very high load network programming with Perl using POE.
Python has Twisted Python
Java has the java.nio and the brilliant event/thread hybrid library SEDA by Matt Welsch.
I am also looking into the programming language Erlang which builds concurrancy and event driven programming into the language. Further, Erlang is used by some big telco manufacturers to great effect (high performance and claimed 99.9999999% nine-nines reliability on a big app).
-- I am not a fanatic, I am a true believer.
138974 ms
A little over their 4 ms goal. Specifically, 138,970ms.
________________________________________________
suwain_2
Basically, the whole article has 1 message: you should cache stuff. I couldn't agree more. Why doing a database request every time a page is hit? Even if you're going to show the same information say 1000 times? By combining dynamic and static elements, the "server load" part of the slashdot effect can be eliminated, I think slashdot also does this, but differently.
Obviously, if you don't have enough bandwidth, you are screwed anyway, but usually it's the server load that is the problem.
MfG shurdeek
As a bunch of people have pointed out, it is unlikely that the /. effect is a matter of "crashing" servers. It is much more likely that most of the "slashdotted" sites on the front page on a given day involve a server which is doing just fine and a bandwidth pipe which is seriously about to burst.
You can saturate most any small-business-affordable pipe with a Pentium classic machine as a Web server. Or to put it another way, there's no point sticking a dual-P4-Xeon Web server with 4GB memory and a RAID-5 on a DSL line.
The computer I'm using right now (a PIII system) could run Apache very nicely in the background and would likely survive quite a hitrate without too much trouble. But if even just a few thousand people were to hit it all at once, there would be a traffic jam, some people wouldn't get served, and the ISP would probably close me down, because I'm only sitting on a 256k pipe.
STOP . AMERICA . NOW
This story is dopey. If you have a web server and it is hitting a CPU bottleneck, you have done something wrong.
Ok, if the server actively plays chess against a hundred people, I'll let you be cpu bound.
They said that the peak load was 11 hits per second, with 4 pages being served. They also said that their CPU was 21% loaded to serve this much traffic.
/. readers to hit them all at once, or perhaps they need a bigger pipe connecting them to the Net.
This says nothing about what they can serve under ideal conditions; this is what they actually served up during an actual slashdotting. If you want to max out their server, you will need to get more
Read the article; on ApacheBench with one particular page they tested, the server tested out at five dozen pages served up per second.
I don't know about you, but I was somewhat impressed by all this. A $1000 Sun does seem to have been a wise choice for them.
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
I hate to do this, and get into some kind of "look at my l33t skills" type thing...but seriously, those numbers are just nothing to be impressed with. As several people have pointed out, usually the limitation on a well configured server is the bandwidth available. I have a buddy who runs a few adult sites, and I go ahead and keep his machines updated, optimized, etc, etc. On one web server alone, with simply rebuild Apache with a higher HSL and streamlining only essential services this *one* server is handling an average of 16,000,000 hits per day. (avg approx. 16,000,000 hits, 5,000,000 pageviews, 450,000 unique visitors per day). In fact, only last month did we set up a separate database server in anticipation of him getting even more traffic (I wanted to separate the web server from the db server esp. if we were gonna move to load balancing)...even still the cpu load was consistently low and the site was/is serving dynamically generated content (php) and is all driven by a mysql content management system. I have yet to even max out the usage of the server and do some ulimit type stuff or hard adjustments via kernel changes.... so what is the big deal about this article. I think it would be good to put up an article about how to optimize your web servers both in layout and actual configurations to allow for Slashdot levels of traffic. I doubt this will happen, just as the mirroring content on featured stories to help ease bandwidth or other similar suggestions. The saddest part is that once you spend the time to really optimize a machine or machines...it takes far less time to maintain them.
This is why people should set an expiration time on their static content. If, for example, I set up the images to expire one hour from the access time, multiple visits to the page (and images shared between multiple pages) would only be requested once. An ISP's proxy servers down the chain would only help in this regard.
In addition, for static content, "LastModified" is easy to compute. Clients can request a page, send an "If-Modified-Since" header with the timestamp of the static item, and if the item hasn't changed, return a 304 response and no data.
The same can be done for dynamic content, but it requires a bit more work. Most web servers do these things for static content out of the box.
As was said in the article, the fastest request is the request that never has to be made.
- I don't need to go outside, my CRT tan'll do me just fine.
The article pretty much said two things:
-- Caching objects saves a database hit and makes things fast.
-- Resin scales better than Apache.
That's great and everything, but it really doesn't help anyone else. Ok, so now I want to apply this object caching to my own application. Where does this cache live? If I'm not running Resin, then I guess every apache process has one. How do I handle dirty objects which need to be written back to the DB? What if they have been dirtied in two different processes? If I am using some sort of service external to the web application to do the caching, how fast is that? Faster than the database? Perhaps, but now it has to scale too, and it STILL has to consult the database, only for writes, which are worse than reads.
This happened to work for their application, but in order to be applied more generally, it needs lots more explanation.
Any object persistence mechanism which is smart enough to handle caching in a read-write system with any level of configurability is going to be a large, complicated piece of software itself, and will have its own issues to bring to the table.
I have read both articles from Ace's Hardware about how they built their "killer" web server. And in all that talk about the server and the applications that run on it they make no mention of what they did to the OS and the machine itself other than putting faster drives in it. They show an Ultra 30 as a server running a GUI, if I wanted "killer" performance CDE would be the first thing to go! They also don't mention any tweaks to improve system and network performance (and there are a few I can think of). Hell I'm willing to bet they didn't change the jumper setting on the Blade to get the memory to run at 100 MHz over the default 84 MHz as shipped! they also don't do anything like multipath or trunk network adapters (which you could easily do with Solaris). It looks like they took a machine out of the box, did a default install of Solaris, loaded applcations and "plugged it into the net"! I wonder how much better their performance would be if they tuned their server like Colin Bitterfield (of Sun Microsystems) did?
It's not about hits per second, it's about efficiency, and I think he (Brian I guess) did it very well. Some stupid idiotic slashdotters say, my servers handle this and that, but the point is, HOW MUCH DOES IT COST? HOW MANY MACHINES DOU YOU NEED? WHAT OS YOU WILL USE? WHAT'S THE BANDWITH SPENT? Let's not forget also the attitude to optimize the content to the modem users, well they also need to be taken in consideration. About the cache use, very intelligent, WE USE CACHE EVERYDAY all the time. Let's use it with efficiency. The 2 points where maybe some people are missing are, APACHED WAS ASS KICKED in the ass by this server RESIN, honestly I've never heard of it, I'm off web development since a long time... and PHP and APACHE can't be a good combination when things get ugly, and you need scalability. That's what I think was the most useful information, along with the IPC info. Well I'm impressed, It loads fast for me and I have a Half T1, so I could se if it was slashdotted, like somebody said, if /.ed it's a bandwith problem, never the CPU SETUP.
Kudos to Solaris with it's threads and Sun servers(Blade isn't even a server!! IT's a workstation) with it's architechture.
And I tought JAVA was slow, maybe for other uses, but in web development, it seems to be the best option yet.