Slashdot Mirror
Web Server Comparisons
Anonymous Coward writes "ZDNet is running a story today (yes I know it's Christmas) about several web servers. They have apparently benchmarked these servers. They tested Sun, Linux and MS servers. I'm not sure that I like the way they tested, they didn't include BeOS or tune the configs. Check it out at their web site"My fault - this is totally out of date. Blame it on the egg-nog.
... so you know what web server to get for Christmas. :-)
Quando Omni Flunkus Moritati
May 6 1999? That's today?
;)
Man, I better get ready, the 4th is coming up soon!!! Gotta find the best fireworks vantage point...
I wasn't too concerned that they didn't include BeOS, as it's more of a desktop O/S than a server. FreeBSD on the other hand should of been included, as it's definately a good server O/S, and is used to power busy web sites such as yahoo and hotmail.
This rewiew is quite old.. it says May 6, 1999. Also they used 2.0.35 on their linux machine.
I'm not going to be a troll and say M$ was bought off but ZDNet doesn't exactly have a reputation for being very objective about things. I wish I had the real numbers handy at this moment but I wasn't imagining things when I saw a benchmark showing Linux beat NT under high loads (scalability)
Blender And Linux Fan
Is BeOS often used as a web platform? I would have liked to see them include FreeBSD, personally.
----
Celebrate the finer things in life
Nope, this isn't cutting it... I'm not that big a FreeBSD lover, but I am taking exception to the fact that it and its brethren are continually left out of these benchcrafts...er, marks...
Because of its speed, ease of setup and administration, and a dizzying array of add-on and development products from Microsoft and third parties, we award Microsoft's Web platform--Internet Information Server (IIS) running atop Windows NT--our Editors' Choice.
Of all the reasons NOT to choose a web server, I think they hit them all. Granted, speed is important. But ease of setup? administration? (maybe) add-on and development products? (must mean ASP, & other proprietary compoonents... that "mixed web" environment they spoke of earlier in the article must not be that important)
Whatever happened to security, or - since "ease of setup and administration" are such a factor, how about security "out of the box"? Granted, I'm glad to see Linux listed, but (no offense) Caldera's not exactly the distro I'd pick for my web serving. Nor would I use Stronghold. (personal prefs, made through experience)
Just seems like ZDNet refuses to just get things right. Ordinarily I hate to be the crybaby bitching about the testing, the methods/materials, etc..., but I'm really becoming more and more disappointed in ZDNet's lack of integrity.
Flame away...
- Mindcraft is known for making benchmarks to suit the manufacturer.
- Benchmarks can be wildly manipulated... - Hence we should call this practice benchcrafting!
Why do stories like this get posted to slashdot? This is months old and was posted on various linux sites back in May, perhaps even Slashdot. I personally have submitted numerous recent stories that have been rejected for various reasons, but this makes it? Who's running this thing anymore?
----
Celebrate the finer things in life
The tested using a 2.0.35 kernel. This is about 1+ year old and doesn't contain support for upgraded SMP performance. While an accurate measure of Linux then, it's far from accurate now.
Figures lie, and liars figure. What else do I have to say?
"A few weeks before our testing began, Linus Torvalds (creator and keeper of the Linux kernel) released the Linux 2.2 kernel..."
This article is a bit dated - May 6, 1999; Put into perspective by the upcoming 2.4.x release
-----Transmission Complete----- If you want to email me...Don't
I didn't see any mention of BeOS and MacOS10, unless maybe I didn't look hard enough. BeOS *COULD* be a decent web server I'd imagine, after all it is loosely based on UNIX. MacOS10, havn't heard much mention of that these days but I know it too has a web server and also is based on UNIX. It would be nice for once to see an objective report that doesn't waste all it's time rambling about windows and give equal coverage to all these solutions.
Ooh, I'm feeling insightful today
Blender And Linux Fan
Why is ZDNet comparing NT 4.0 and the latest versions of other operating systems to Linux 2.0.35? That kernel is quite old and lacks a lot of optomizations that the later kernels have.
They "tuned the cache" to the point that none of the servers went to disk for their workload. That right there gave NT an advantage. In a real world scenario I'll be out to the disk all the time. I'll be there to get my data during misses, to write my state info for transactions, to log things.
How much logging was the NT server doing? If it wasn't a lot then they took out the disk subsystem from the equation.
The whole reason they used the 2.0 kernel was fishy at best also. 2.2 TCP stack broke communications with their win 95 clients?
Finally, why on earth would I want to do hundreds or thousands of SSL transactions in software. If you are doing more then a few a second you really need a hardware SSL brick or card, which works with all the tested platforms. These people obviously don't understand but one set of solutions.
_________________________
they repeat bogus reasons for using Microsoft servers ("if you have existing business logic, such as pricing strategies, written in Visual Basic..." -- hello, you want to call Visual Basic script, written by an accountant's assistant from HTTP server in secure environment?), measurements made with CGI scripts on Unix (heard of anything else?), their WebBench tests that have nothing to do with real-life environments other than that it uses HTTP, never use Gigabit ethernet cards in their low-latency tests, "demonstrate" that SSL servers are dominated by Microsoft IIS by splitting branded Apache-derivatives into different categories, don't include Unix servers other than Apache and Netscape, omit *BSD, etc.
In other words, usual advertisers-driven "Editors' Choice" stuff from ZDNet.
Contrary to the popular belief, there indeed is no God.
I don't understand how the 2.2 kernel didn't work for win 95 clients (as stated in the article). Was this ever true, and if so, for what version of the 2.2 kernel?
Also, what is the state of threading in apache? Is there currently any support for it (the article states the 1.2.x version they used didn't support it at all)
Not a very bad article (it is somewhat old now), but maybe not to relevant any longer. (And they seems to thing that web based administration is easier than config files... well, whatever floats your boat.)
"It's overkill, of course. But you can never have too much overkill." - Anonymous Slashdot Coward
Sorry for the offtopic, but:
Christmas is celebrating a pagan festival (winter solstice), hence the tree and other rituals (gifts perhaps?) So, for all the christians who have a tree, are they sinning by worshipping a false God?
And dude, I repent already! Forgive me and forever hold your peace. PS. hey-sus is at least 2003 years old...
"It's overkill, of course. But you can never have too much overkill." - Anonymous Slashdot Coward
Linux really didn't seem to do poorly. In fact I couldn't tell a difference between the platforms..
When using CGI, they all seemed to do about the same, yes, the two that ran NSAPI and ISAPI kicked the living crap out of the ones running CGI scripts, but I'll bet if they'd slapped mod-perl onto the Apache server, Linux would have caught right up..
greg
dronf!
I've personally used NT for a lot of development and wanted to have a site hosted on Windows NT, because I knew some ASP and was familiar with ISAPI. But where I live (Europe, I've checked most countries here) Windows NT hosting is usually three times the price of Linux/BSD hosting. So I went for a Linux server and discovered that (after trying out some other hosters for other sites) having a database or other extensions installed is usually also very expensive on NT (try getting them to let you install your own ISAPI extensions) and that most Linux/BSD hosters would happily let me use their (MySQL/PostgreSQL)database and stuff like PHP and my own CGI-scripts.
Administration-issues are only important for a small bunch of people.
2000th birthday? Heck, and I thought being off-by-one on the millennium was bad, but this takes the cake. Jesus wasn't born two thousand years ago, you know. It was in error when they picked the year. In fact, he's already passed his bimillennial birthday.
And if there is no Armaggedon, do you promise to repent your irrational ways, and come over to the bright side? The light of reason shines more clearly than the shadow of self-deception. One of the seven deadly sins of the Buddhists is ignorance. Heh. -freehand, an unrepentant servant of clarity. Ye must be prepared to die to know the truth (if you're not, you cannot consider the possibility that you're wrong).
People who don't sign their names are in no position to call others names.
We admitted it for some class of setups after the Mindcraft benchmarks, and worked to improve the areas of deficiency. That still didn't make the benchmarks useful to 95% of people needing to deploy web-servers.
And "wussies" is spelled "wussies". Although I'll probably be woosy come New Year's Eve...
"It's overkill, of course. But you can never have too much overkill." - Anonymous Slashdot Coward
What a load of bull! How much do you want to bed that they didn't turn on the SMP option in the kernel?
There's no reason for a sig here.
They don't show the formula that gives the number, but from similar web benchmarking reviews, I know that even the worst ones are serving up hundreds of page-views per second. The best are maxing out multiple 10Mbps Ethernet cards - i.e. you need a T3 line to actually provide the bandwidth you're serving.
If you can afford that, you aren't reading Ziff-Davis to make your product decisions or even find your shortlist.
These kinds of servers are only needed by the big ISPs and the eBays of the world - the whole review is only of interest to a few thousand webmasters.
My employer is a city government serving some 860,000 people with a mostly static, partly active web site about all their city services and taxes and utility bills - and it rarely exceeds a few tens of pageviews per second.
Forget all the sniping about tuning and benchmark methodology; the really stupid thing about these product comparisons is that they imply that more than a fraction of one percent of their audience should even care about which one wins. For the rest of us, a free product running on a free OS and hardware that costs less than the monthly cost of our Internet bandwidth can meet all our needs.
well, what can I say?
:)
Lets drink to yet another microsoft victory!!!
hahaha...
well maybe thye didnt configure the other boxes properly and whatnot, and maybe microosft's solution is not the best (no dooubt you all think so, to prove it, this will be down to -1 in minutes!) but one thing I have to say is that Microsoft certainly seems to be getting its act together quite well lately!
I have to say that Windows 2000 is an absolutely magnificent improvement on previous operating systems. I've been running it for months, ever since the first relatively stable releases, and I still havent seen the blue screen of death! (Well once, but that was cause I was messing with my hardware and my RAM was screwy...) Its much more stable, I'm getting uptimes ive never dreamed before! I just ended an uptime of more than a month! and I dont even try to run my system nonstop (so all you Linux buffs up for 318 days can shut it!)
In any case its an improvement on Win98... the defult background color is a welcome change... (oh and they even changed the tint of blue background used on the BSoD, it too looks better now)
But one definite loss...one definite indelible black mark on microosft's good name... I was much dissapointed to find that Windows 2000 Advanced Server does not ship with that silly pinball game... (no wait, I probably havent seen it around since I specifically didnt install the games... Im starting to miss solitare to tell you the truth
But dont flame too hard, because I do run Linux, and I do think its a much more hardcore OS when you get to now it. In fact the only reason I dont use it fulltime is becuase I cant be bothered to switch, and my current linux box (an IBM compatible DX/50) isn't really a supreme specimen to learn on. But I promise you, that as soon as I get that new 1 gigahertz box I've always wanted, I'll retite this one to an exclusive Linux box, and then I'll really be able to use Linux!!
(Apparently linux doesnt like the PS/2 architecture... or old ATAPI SCSI CD-Roms for that matter... took 24 solid hours of moving hardware around from different computers to finally get redhat installed... and even now it refuses to do some reasonable things! Im putting it all down to the oldness of the computer its on [rather than my lack of competence] -- that sig on the bottom is not a joke, after repeated attempts I couldnt get it to know my network card, so I eventually gave up...)
They recommend NT & IIS as their editor's choice but Netcraft says:
www.zdnet.com is running Netscape-Enterprise/3.6 SP3 on Solaris
Did anyone notice this?
A few weeks before our testing began, Linus Torvalds (creator and keeper of the Linux kernel) released the Linux 2.2 kernel. It promised to overcome the lack of multiprocessor scaling that has hampered performance on our previous tests.
How long has this review been going on? How often have they upgraded? It sounds like they never upgraded Linux. You can bet they upgraded their Windows.
Unfortunately, we could not test the potential improvement in processor scaling, as the 2.2 kernel included a TCP/IP stack improvement that broke its communication with our Windows 95 clients. This fix was not in the 2.0 kernel, which was the latest available in standard distributions during our testing, so we tested with Caldera's 2.0.35 kernel distribution.
And this part is even more confusing. They purposefully used an old version of Linux for their test? They couldn't pick up a more recent kernel that would improve probably all of the results?
Very strange.
Stumpy
OK.... "A complement to threading in Windows NT is its ability to achieve asynchronous I/O. Asynchronous I/O lets a threaded Web server process requests at the same time it performs file or network I/O. Without this capability, the Web server would sit and wait while the disk subsystem pulled a file from the hard disk or passed the requested file to the network. Instead, the server can work simultaneously on requests from other clients while the I/O subsystem handles the file or network I/O. A similar feature is available in Solaris but has yet to be fully implemented in Linux."
According to historical evidence, Jesus was a pharisee rabbi crucified in 88 B.C.E. If he was 33 at the time, this means he is 2121 years old now.
OK.... "A complement to threading in Windows NT is its ability to achieve asynchronous I/O. Asynchronous I/O lets a threaded Web server process requests at the same time it performs file or network I/O. Without this capability, the Web server would sit and wait while the disk subsystem pulled a file from the hard disk or passed the requested file to the network. Instead, the server can work simultaneously on requests from other clients while the I/O subsystem handles the file or network I/O. A similar feature is available in Solaris but has yet to be fully implemented in Linux." OK correct me if I'm wrong - but linux uses processes to do pthreads.... so if a 'thread' makes a disk request then it will block while the request is carried out and another process will take over.... so that statement is not entirely true... Anyway who cares it#'s christmas - and I'm happy that my new Dell laptop is running linux far better than it is running windows (suspend crashes windows, dos sets the wrong screen mode and all sorts of otehr problems...)
I just did a "view source" (yea. I'm on my windows box behind my linux firewall).. and the frikkin a META tag said "META http-equiv="expires" CONTENT="Tue, 20 Aug 1996 01:00:00 GMT"" It expires Aug 20, 1996.. whats today? You guys need to research stuff before you post it.. your integrity is starting to drop... kinda like shugashack.com .. sCary is frikkin worthless now.
If only speed matters, and not security, why on earth are they benchmarking SSL? Hello, anybody home?
Linux still has catching up to do to be on par with IIS in some types of static content; and we can again thank ZD for a meaningless benchmark that is no help in such development efforts.
Personally I can't wait to see what will happen with the 2.4 kernel with khttpd and phttpd on the horizon...
Its true, this was an old article which was probably biased in lots of ways. They might have been payed off, or maliciously constructed their methodology to deface Non-MicroSoft products. I don't know so much about servers or programming, but i try to have common sense. And common sense tells us that the whole point of an open source system of software is to have everyone try to help improve an existing system. Not just talk about how good the good points are and deny any flaws it might have. Take their complaints and listen to them. If they're valid use them to improve on it. The good points will already be there, just try to remove any leg they have to stand on with their criticism.
"BCE" my butt. Politically correct bullshit is deeply idiotic.
there's other people ofended by ZDNet's dirty ads for the X10 camera showing BUSTY girl and say "hide this camera anywhere" I just readed a article about it on ZDnet how come girls who work at ZDnet or a man with a dauhgter put up with this? I see the ads every time I go there for the last 2 month in the Xmas shopping time too.
Maybe you'll consider (Score -1: Troll), because I'm suggesting rational discourse over religious whining.
It appears from this thread of comments that the slashdot community is unhappy about all sorts of things that don't seem central to the issue. The comments becry ZDNet's advertising vs testing integrity and methodology, Caldera distro's shortcomings, Stronghold's performance vs other Apache releases, and why they didn't choose the EndOS-BeOS of solutions.
The issue, as I would see it, is "what can Linux do, to fare better in third-party comparisons?"
It appears from the article that there are several reasons that ZDNet listed why they felt Linux/Apache/Stronghold was limited. Let's start with those.
Why did this give NT an unfair advantage? Why does Linux (or particularly the Caldera distro) solution not deal with RAM-rich servers as well as NT? I think the poster who complained of this meant the inverse: if it were a RAM-poor server, Linux would have the advantage in disk accesses.
All religious handwaving aside, why did NT fare better by spinning threads than Apache could do by spinning processes? What is the big bottleneck in managing a process, that managing a thread doesn't have? They were using a brand-new MP kernel straight from Linus. Will the Linux kernel mature to deal with SMP situations and massive numbers of similar threads or processes better?
Though I think they should have configured some PHP or Mod_Perl into their mix, just as they had to bend to the SSL3-only constraint for another platform, they have a point. Writing modules is the way to go, to get inside the server and run fast. Besides PHP and Mod_Perl, where can Linux go to improve?
Most of these seem to suggest that ZDNet could have configured their Linux servers more like real-world Linux admins would, and would find better performance. This does talk poorly of Linux's ease-of-use, though they lay it on thick when they cry about config files. This is an education issue.
Slashdot is already slanted (pardon the pun) towards the Linux solution. Not every problem is a nail, and there are even different hammers for different nails. Let's be objective and constructive, instead of whining about every possible outside excuse. Improve the tool, and the tool will become the standard.
[
Remember that this article was reviewing e-commerce site tools, not just regular site tools. They made it pretty apparent in the beginning of the article that Linux has a virtual lock on things that don't involve e-commerce.
What's your damage, Heather?
It seems that ZDNet doesn't like to acknowledge any non-commercial software, and went out of their way to test commercial products wherever possible. CodeWarrior won't run on Caldera, AFAIK. :)
Gates' Law: Every 18 months, the speed of software halves.
Hey, if they did their web server evaluation using REAL WORLD standards (for instance, security gets the highest rating with stability a close second) then their "reviews" would have some substance. As it stands, they seem to evaluate all the products and then pick their criteria for "best" out of the catagories that MS excelled in. Does anyone think that IIS is the "best" web server platform? If so, why is Apache kicking its butt according to netcraft? All of their "comparisions" seem to, unreasonably (IMHO), favour the MS solution. Even when the MS solution is demonstratably flawed. Remember the "crack Win2000" site MS put up. And then "lightning" took it down. And then there were other problems. Yep. I'd want to run my business on a web server that's down more than it's up.
very well said. This is the most reasonable argument i have seen regarding web-serving benchmarks. if my login account had any moderation i would mod this up
Right on. I'll also add a few things:
One of the knee-jerk posters complained that ZD didn't use the latest kernel, then somebody re-read it and realized that this article was written way back in May when that kernel wasn't available. Even if it had been written yesterday, the NT camp could then legitimately complain that ZD should have used Windows 2000, which is a better web server and is theoretically available today (check your newsgroups.)
Every time one of these gets posted, people scream and moan about the box not being configured correctly. "If only we'd have been at the helm, we'd have won that race," they say, complaining about how ZD didn't do hours and hours of tuning work to get the Linux box just right. You know what? They didn't put hours and hours into tuning the NT box, either, and the result still stands.
What's your damage, Heather?
hemos didn't say that zdnet was running it today, that stupid anonymous coward said that they were running it today. hemos was not wrong in this case. he just was kinda lazy to edit the post or add a disclaimer "actually, it's may 6th.." but, in any case, may 6th, 1999 was a long time ago. that probably explains why they were running linux 2.0.35 and not something reasonably current (i.e. 2.2.x).
-- adraken
First, the story is from may, so it doesnt even count for xmas in july...
Second, what is stronghold and why didnt they use apache? Was stronghold a release name for an old apache version? am I missing something? They used IIS which is standard for NT, isn't apache the standard for linux servers?
I'm sure had they used MSFrontPage Personal WebServer for their NT benchmarks, the scores would have been more comparitive.
Ah well, standard benchcraft from the king of benchcrafting.
Merry Xmas!
NH
Hemos, Look at the date, then lay off the crack pipe. May 6th is not Christmas! Duh!
If you've ever used BeOS you will know that its more a single user OS in the same class as Windoze but with a better architecture. I mean theres no login and everyone is root by default. The only good thing is the POSIX compliant shell and ease of use. Mac OS X is based on BSD (and basically the same as any BSD). Note that this article is dated and not a serious benchmark efort in any case.
With regard to said sermon, a correction has been issued. Instead of 2000 years, the author meant, uh, 2121. Note also that said sermon was a joke, and that this is Christmas Day, and that some moderators have no sense of humor.
I am the plausible religious zealot troll. No sermon or good wishes, express or implied, were meant.
(C) Plausible Religious Zealot Troll, 1999
All Rights Reserved
No comment at this time
If comparing CGI scripts with ISAPI modules is acceptable, then I can easily "prove" that anything running in Linux is much faster than the equivalent program in Windows 2000(TM). Just compare an interpreted GW-Basic program in W2K with the same algorithm compiled in C on Linux.
These benchmarkets are just thinly disguised press releases, not responsible journalism.
The question you propose on "what can Linux do, to fare better in third-party comparisons?" is asked by thousands of users worldwide, not with marketing in mind, but with the objective of making our own systems run better and faster. Free code allows one to improve our systems, and many of us do so, or try to.
where is the proof that running web servers on linux offers a 500-50000% speedup??
apache runs almost 800times faster on linux than any other unix
why dont we see mention of this?
linux is the only choice for the enterprise
IIRC They combined some features of netBSD and FreeBSD to come up with what they used in Mac OS X
I will absolutely take the advice of someone like yourself as the truth.
Nearly all UNIXes are rock solid stable. BSD doesn't crash, Linux doesn't crash, and commercial UNIXes don't crash. So arguing that my UNIX OS is more stable than your UNIX OS is just plain stupid. If reliability is your concern, pick any UNIX.
If you want to run a single OS on every platform in your company to reduce support costs, pick Linux. It runs on everything from Palmtops to desktops, to servers, to IBM S/390's. And I see several entries on the top 500 supercomputers that run Linux, are there any BSD entries? I don't think so.
So for maximum portability pick Linux.
Want to avoid having multiple OS'es in your company? Don't pick BSD. Every vendor that suppies a product running BSD is running that vendors version of BSD. And there are at least 4 versions of open source BSD out right now. Forget about portability BSD only runs on a handful of processors. Is BSD even running on the next Intel chip (IA64) yet? Linux was the first OS to run on that powerhouse of a chip.
Not to mention the elitism of the developers and supporters of BSD. Who wants to associate with people who can only promote their product through FUD?
So the choice of the future is clearly Linux. Don't worry, BSD will have a small footnote in history as a curiosity that held on in small comunitites similar to the TS-1000 and C64 users groups that still meet today.
Oh, and by the way, I am not a kid.
ancient kernels available for download. 2.2.0 .... :-P
---
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
Ignoring the veracity of your post, "don't flame me while I present my unbacked up opinions" is probably not the best way to avoid flaming.
Recentally, I setup a Linux router with 8 10Mb/s feeds (full duplex)inbound with one 100Mb/s feed going out. What happened is that at about 20K interrupts a second (about 70Mb/s), the system, which was running fine (about 97% idle) started sucking up cpu time. By 30K interrupts, it had saturated the CPU. In this setup, I made use of the vlan code (it was attached to a vlan switch), and used an Alteon AceNIC which does interrupt coalessing. The AceNIC under the same loads ran with about 1600 interrupts a second, and continued to run with about 97-98% idle time at the same traffic levels that had pegged the system before. I'm wondering if special drivers that are doing interrupt coalessing are making the difference on the NT boxes. Assuming that each transfer generates on a WebBench 20 interrupts (or more), then the 1K connections a second and the CPU load are really understandable.
On another note, why not have different groups do benchmarking with a fixed $$ amount that they can use to purchase equipment, as well as fixing a $$ amount per hour that they have to spend configuring the servers. This would be a MUCH more realistic benchmark scenario as the cost of equipment and time are realistic factors in the real world.
Erik
Linux network code has a bottle neck that only allows it to send to one network card at a time. So even though there were 4 cards in the machine, Linux is effectively limited to the bandwidth of one card.
With this problem fixed Linux will be able to send out 4 times the throughput that it can now.
This weakness is clearly going to be exploited in every M$ benchmark until we fix it.
Who is working on this problem and can we help with cash or moral support?
--
There are three kinds of lies: Lies, damn Lies and benchmarks. - Maddog
Not being from the US, I can't quite figure out what the "E" should stand for. And I can't come up with anything sufficiently idiotic either.
The fact that they came up with a BS reason for using an old kernel, even when that was written back in May is what people are really complaining about. I haven't heard any reports of any of the 2.2.* series' tcp-ip stack "breaking" connection with win95 boxes. They should have used the latest RELEASED kernel, or just gotten whatever updates were availible from caldera. Seeing as Windows 2k hasn't been RELEASED yet (I don't care to use pirated beta version from a newsgroup) there is no good reason for them to have used it. I do agree that linux might be a little harder to configure, but i prefer having one or two text files to tweak my webserver as opposed to 10's of tiny menus and trying to remember where each option was.
Before the Common Era (IIRC), its been a while since i gave two shits about being politcally correct
They could have tested Solaris on an Intel box and they could have also tested Apache on NT. Obviously, the multi-processor issue with Apache will be sorted out in the future ... the Linux kernel just became multi-processor friendly.
But, they fail to mention these issues with their tests. But, what can you expect from a pop-culture magazine like PC Magazine?
NT and IIS might not be the best, but there is something to be said about ease of use. I can set up an e-commerce site that supports a high amount of traffic - with a minimum of server tweaking - in a very short time with a minimum of hassle.
Linux is a great... until you need to use it
crazy dynamite monkey
Anyone else notice that this review is terribly dated?!? They are talking about 2.2.0 only being a week old!
uhm. windows 2000 has been released. it'll be on computers shipping feb 17, but it will be in stores and online well before that and close partners have it already (The final code).
And 4 diffrent Open Source versions of BSD? I thought there was only 3- FreeBSD,NetBSD and OpenBSD. How many diffrent Linux distro's are there again?
I too would like to see some real benchmarks with NT vs Linux vs FreeBSD. Though I run linux on my PC's I have tryed FreeBSD, and it is faster. Linux kiddies like your self would learn a little more if they just tryed something OTHER then linux for once. This If-Its-Not-Linux-then-it-SuX thing most /.ers got going is kinda funny in a way. I always thought Open Source users had a open mind...but I guess thats not the way it is on /. :)
I have to return some videotapes...
this will probably get marked as flamebait, but.. i have got to say this.
in response to zdnet's remarks, I would like to offer the following opposing points. Let me cut to the chase: The word on the street is that it is naive to think that ZDnet wouldn't displace meaningful discussion of an issue's merit or demerit with hunch and emotion if it got the chance. In public, ZDnet vehemently inveighs against corruption and sin. But when nobody's looking, ZDnet never fails to pooh-pooh the reams of solid evidence pointing to the existence and operation of an irritating coterie of insurrectionism. Take a good, close look at yourself, ZDnet. What you'll probably find is that you're iconoclastic. As amazing as it seems, ZDnet is not only immoral, but amoral. In a nutshell, ZDnet's cronies increase people's stress and aggression simply because they think it's fun.
Don't be a dumbass. Can't you see that they didn't even use the 2.2 kernel? In fact, if you read the article, the 2.2 kernel was too good for the test! It's improved stack implementation _BROKE_ the Win95 boxes! Why did they use WinBlows clients then if they didn't work properly? Use some common sense.
In most cases, yes, but wouldn't it be silly to say that he was crucified 88 years Before Christ, since that is what 'BC' stands for, you know. Not just politically correct, but also logically.
Pay attention to the context next time, not just your flame-trigger keywords.
Good... bad... I'm the one with the gun.
ProofReading Markup Language - and yes, I find typos.
What did they hope to prove by pitting the 2.0 kernel against NT and Solaris rather than the 2.2 kernel? Why don't they pit NT 3.5x against 2.0 and see which comes out on top. I think 2.4 will do some real damage though. Hmmm it would be easy to see BeOS go against NT..... no competition there as a workstation OS goes.
why don't they ever test it on anything but x86? one of the strengths of linux is that it runs on many different archs, so you can use decent archs and aren't stuck with x86 (as you pretty much are w/ windows). this seems like a huge oversight to me.
The best are maxing out multiple 10Mbps Ethernet cards - i.e. you need a T3 line to actually provide the bandwidth you're serving.
Without addressing the issue of whether *this* benchmark is valid, what if you are providing web services over a local intranet with a gigabit backbone and most clients on switched-100? What if that web server was the web interface for a heavily-used ERP application, or a Product Data Management solution over HTTP-DAV? Wouldn't the web server that was able to perform at the high-end matter in this case?
You don't need to be a big ISP or an eBay to make use of such bandwidth - if it's local. This scenario describes the company I work for, and we have only ~200 employees.
banner ads are getting out of hand. all these sites have about 15k of sidebar links, 10k or so banner ad, then usually less than 1k of actual content. often it's just two paragraphs before you have to click "next". another cheat trick is to have a top 10 list split into 10pages + intro + conclusion. all these banner ads and blatant tricks make tv ads appear suttle(sp).
Also, what is the state of threading in apache?
:-)
Apache 2.0 will be a hybrid forking/threading server thus giving it some of the speed advantages of threading while maintaining the advantage of multiple processes that every one of these benchmarks never mentions: stability.
If for some reason one of the threads in a multi-threading server crashes, it can bring the whole server down with it. If one of Apache's child servers crashes... Apache forks a new one to replace it. The new design will be a compromise with several preforked children, each of which is multi-threading. Then let's see what the benchmarks look like
Chris
San Francisco values: compassion, tolerance, respect, intelligence
Notice that he/she implied that this May 6th article was posted today, and actually stated that there was _no_ tuning at all (at least slightly contrary to the article). This was from back in the mindcraft days (or a bit prior), which in internet time is nearly forever. Yeah, the article itself has a bunch of holes in it, but being from almost 8 months ago isn't it just water over the dam?
Here's the link to info about Apache 2.0 which I actually wanted to include in my post above:
http://www.apacheweek.com/features/apac he20
Chris
San Francisco values: compassion, tolerance, respect, intelligence
junkbuster
..
"We must move forward, not backward, upward not forward, and always twirling, twirling, twirling towards freedom."
Bitchslapped? Give Rob a bitchslap from bitchslapped.com.
Ironically, since there is no year zero, Christ was born either *before* or *after* himself. My proposal for calendar reform is this: 1 before Christ is the year before he was born. He was born in year 0 DC (zero *during* Christ), and crucified in year 33 DC. The first year after his crucifixion would be 0 AC (zero after Christ). C-language style for array numbering works good for us Unix hackers. According to the canonical dating used by Dionysius Exiguus, we would be now in year 1965 AC, which would leave me enough time to retire without having to worry about Y2K. Merry Armageddon and a Happy Looting to all.
FYI, 2/17/2000 is the day that it will be released to the public. Read for yourself if you don't believe me.
..
"We must move forward, not backward, upward not forward, and always twirling, twirling, twirling towards freedom."
Bitchslapped? Give Rob a bitchslap from bitchslapped.com.
Maybe this was a sponsored test from M$. That would explain the bios of this test. And it would explain why FreeBSD wasn't in the test...
NT4.0 can't beat Hotmail. Hotmail needs FreeBSD. Then the conclusion could hardly conclude that NT4 is better than FreeBSD.... when NT4.0 can't run hotmail
how can you find http://ads.x10.com/zdnetmacro/nov19m1.gif offensive? Are you amish (omish?) or something? Its just a good looking girl with a white shirt on.
I have to return some videotapes...
AD stands for anno domini, "in the year of our lord."
"The best way to do mathematics is to be creatively lazy." -I. M. Isaacs
I presume all those who have been flaming it made no attempt to suggest that Linux was a better web server platform than NT/Solaris in May '99?
All i see is benchmarks on x86 hardware. They tested solaris on x86 and compared it with NT. Why don't they test Solaris on UltraSParc's Irix on MIPS , NT on x86 AND alpha and linux on and do benchmarks? That should be a far more realistic test for the ones who actually CARE to see what platform to use. If NT is such a damn knockout why are they themselves using Solaris? WHy is MS themselves using FreeBSD on their hotmail service ????? How can you have the FACE to suggest NT when you who do the test and you who MAKE the damn product don't even use it sir? Can someone explain that to me? You want real-world tests? Look at the REAL world. Look at cdrom.com look at Hotmail , look at yahoo and some heavy-duty sites who are actually ONLINE and running and look at what THEY use.... All these benchmarks smanchmarks don't impress me ONE bit. Put the thing ONLINE and let it proove itself in real-world situations. Let me see NT take the pounding cdrom.com takes on the same hw cdrom.com has...... Let me see NT run yahoo , or even MS's OWN hotmail service .... How in the name of god can they have the nerve to market and promote NT when MS and Zdnet themselves DON'T FSCKING USE THE DAMN THING ON THEIR OWN SITES?
Though this isnt as nice as junkbuster it is easy as hell to setup. http://www.ecst.csuchico.edu/~atman/spam/adblock.h tml
I have to return some videotapes...
Read the flippin post you replied to. He said that the worst was performing several hundreds of hits a second. If you have only 200 users, how are you going to be generating that kind of load?
M$ Can tweak NT/W2k so it serves pages faster, but it is still unreliable for most applications...
People like to state the NT can be very stable, how many apps do you have running? How much downtime per year do you have? How much time do you spend fixing problems?
My biggest grip was you couldnt telnet into NT and restart processes. W2K has that. NT can try to become more like *nix platform, it still has a long way to go..
Im going to stick with Solaris and Apache. Im under staffed, and over worked.
NT is not the answer.
Also another sad part is most E-Commerce packages are for NT.
Well, lets see how many sad geeks are not getting drunk, and are reading a survey that will tick them off? well its nolonger xmas day here, and i've got a hang over, but im going to cure that by drinking some more, i suggest for anyone who is remotely upset about this, go get some beer!
Apparently
www.zdnet.com is running Netscape-Enterprise/3.6 SP3 on Solaris.
Why did they choose that platform if the IIS on NT solution earns the editors choice?
Or the Webmasters of Zdnet.com disagrees with the Editors?
Well, At the homepage of Apple Computer Inc., they claim that the AppleShare IP 6.3 provides a good web server. Is there any result of comparison including the AppleShareIP?
Yeah, except for the part about doing work. NT may crash, but it does actually have useful programs (photoshop, MSOffice, etc.) as opposed to beos which has...hmm...Gobe Productive?
Jeremy
Looking for a Python IRC bot?
Here's a project that promises up to 70% better Apache performance on Linux: http://oss.sgi.com/projects/apache/patchinfo.html Also, is sendfile() in Apache? Who has some info about this?
There is absolutely no historical support for such a statement. I'm sure all the people who read this and assume that "it's posted on slashdot, it must be correct" are thankful for your complete misinformation.
I can find myriad historians who would date Jesus' birth to approximately 5-6 BC. I challenge you to find any substantial contingent of historians with proper credentials (ie. not "Harry's House of History" degrees) to state that Jesus was born prior even to 10 BC.
Oh, and supposing you can find such information, how do you explain the fooling of billions of people into thinking that he was born ~115 years before they think he was?
Jeremy
Looking for a Python IRC bot?
Probably because people don't bother tuning the Linux kernel for good SMP performance on web tasks.
Why would that be? Because it doesn't make sense to pay the premium for four processors in a box when for less money you can get four processors in four boxes and quadruple your disk and network bandwidth. People use SMP on Windows mostly because of software licensing costs, because of per-box colocation costs, or because it sounds good. In my experience, SMP on Linux is mostly used for getting really high performance on numerical and scientific tasks.
Apache has a perfectly good in-process module API; mod_perl is written in it. You can write that kind of code yourself if you want to write C/C++ extensions to the Apache server.
I consider it pretty foolish, however, to write web server extensions for an E-commerce site in a language without runtime error checking or fault isolation. Applications specific modules simply can't receive the testing and debugging that Apache itself has received. You are lucky if the server crashes due to a bug; more likely, you are going to ship an unpredictable quantity of widgets to an unpredictable address as a stray pointer leads to overwriting some of your order data.
The performance difference between native code and Perl, Tcl, Python, Java, or PHP3 simply don't matter in most web applications: applications are generally bandwidth or database bound. Given that simple fact, you should use the safest and easiest language to program in. For single programmer projects, that's likely to be a scripting language. For large, component-based multi-programmer projects, that's likely to be Java, OO Pascal, Eiffel, or something of that kind.
Of course, most ppl posting here, dont know much about the various platforms, they just want what is not MS. BE/OS by their OWN admition, (if you care to read the BE website), is not suitable to be used as a large scale network server. They say, on their site, its more tuned for personal use and media use. NOT network use. BE.com does NOT run BeOS, or Linux, they run FreeBSD. a stable proven, underhyped OS. Please. get real. BE is not a web server platform contender at this point in time.
I think someone touched on this, but this is IMPORTANT since even ZDnet said this:
Apache and linux's process based system works better by adding boxes than adding processors/resources in a single box.
So here's the deal: figure out how much it costs to buy OS, software, hardware, and setup the system. Determine its optimal performance (which may NOT be peak performance). If Linux still doesn't have SMP at a cost effective level, spec out single CPU boxes and save the cash for entire other systems. (Don't flame me for saying it doesn't work, I'm saying it might be cost effective to not use it until 2.4) If Solaris' cost-performance is better on Sun hardware (duh) then use it.
Then, take those numbers and ask ZDNET to append them to their article.
We know linux has an automatic $1000 price advantage over NT/Solaris which is about a third the cost of adding another server. I'm not sure about Stronghold's cost vs IIS/NS server/etc, but I'm guessing it's not as expensive as IIS. And with the exception of the "custom" API's (NSAPI, ISAPI), Linux performed as well as the other servers, even with the use of a twitchy old version of caldera.
Personally, I'd want to see the addition of a mod_perl'd server to represent the Linux equivalent of NSAPI and ISAPI, but with the bonus that perl stuff is PORTABLE, much more so than NS or IIS-only scripts.
If we suggest this right to ZDNET, when they review the new W2k's web server and put it up against a decent server linux (w/kernel 2.4 we can hope) they may add a price/performance index.
I've been on slashdot so long I'm starting to get out of touch with the cool stuff if it ain't on slashdot.
The AC who SUBMITTED the article said ZDNet was running it today, not Hemos. Hemos checked. The italics mean the AC said it. Got it?
:-)
crazy dynamite monkey
Why does it matter what year he's born in? Man, I hate it when Christian zealots get caught up on little details...
They only show Apache running on Linux, but seem to overlook the fact that it will run on other UNIX (like Solaris 2.7 on their Sun E-250 that they used in the test)...
Is the Sun Web Server that much better tuned for Solaris? In the static page view comparison it blew just about everything else away, but they conveniently decided not to include Apache.
If you have only 200 users, how are you going to be generating that kind of load?
By uploading and downloading multi-hundred megabyte CAD drawings and other such bandwidth consuming activity - such as video conferencing - over HTTP. Or by being in a much larger company with a similar, high-bandwidth infrastructure.
My point is that the firm I work for only has 200 users, yet we have more than enough bandwidth to saturate the web servers tested in the benchmark. There must be many companies with larger user bases that are capable of generating this kind of traffic over the LAN, without being eBay or a huge ISP. Not all web servers publish over the Internet, many publish over an intranet.
Our company may not be able to afford a T3, but we don't need one if the client is in the same building, capiche? Good. Such performance issues are relevant to more people than the poster three levels up described.
Mac OS X is based on BSD (and basically the same as any BSD)
Not entirely true. MacOS X is based on a Mach microkernel, which is entirely different that the classic-style Unix kernels used by the BSDs. This is going to give it very different performance characteristics as a web server than any BSD.
The *BSD stuff is in there to provide Unix compatiblity in the user space (to run Apache with, for instance.) From what I've heard, the "Yellow Box" GUI environment sits directly on Mach, and the whole BSD bit will probably be an optional install on OS X Client.
--
Business. Numbers. Money. People. Computer World.
...please say that post was a joke.
"Nobody owns the fucking words man." - James Dean
I believe that's spelled "woozy"
Did anyone else notice that when the Windows NT review was talking about stability in Windows NT, it mentioned major sites that use NT, including Ebay?
Does everyone else think that is a pretty good example? Ebay is great, but as an example of stability, I'd prefer an airplane with one wing split in half. Or is that just me?
Also, does Ebay even use SSL? I notice that the benchmarks do not neccicarly compare to the examples used in the review.
Little Brother, watching the watchers
And will remain so.
More tools, faster development time, more mind share, better managed company.
The noveau riche Linux companies like RH won't know what to do with their billions and the ball dropping and stock price corrections are going to hit hard soon.
The only real (but weak) threat to NT as the dominant web platform for e-business comes from MacOS/X server. But Apple should write their own web server. Apache is over-hyped and under performing.
Folks, Apache is not "cutting edge" and never has been. It's pretty solid as web server for silly "pictures of my cat" sites but it doesn't run in e-commerce environments. Microsoft *completely* dominates the e-commerce field and Apache will not catch up unless big players (like IBM) get really serious about it. I don't think they are going to because there are limits to just how supportive of free software companies like IBM and SGI can be and still benefit the bottom line.
The idea that FreeBSD and Linux (and Unix) is cutting edge now just because it's free is ludicrous. We need to examine the facts and the technology more honestly.
"A complement to threading in Windows NT is its ability to achieve asynchronous I/O. Asynchronous I/O lets a threaded Web server process requests at the same time it performs file or network I/O. Without this capability, the Web server would sit and wait while the disk subsystem pulled a file from the hard disk or passed the requested file to the network. Instead, the server can work simultaneously on requests from other clients while the I/O subsystem handles the file or network I/O. A similar feature is available in Solaris but has yet to be fully implemented in Linux."
What a load of bollocks. Let's get the facts straight. Any IIS development where a request takes more than a fraction of a second forces the ISAPI extension developer to spawn their own thread and implement the request asynchronously themeselves. Why? Because IIS has a limited number of threads in its pool and when they are all used clients start receiving "Server Busy" messages.
In my experience developing for IIS, if you want asynchronous disk I/O and asynch network I/O, you have to do it all yourself. This is not a standard feature of NT by a long shot. Writing your own multithreaded code adds considerable time to development. It's not necessarily trivial, especially if you have to debug due to race conditions (which might not even show up for a long time). Spawning a thread is easy enough, but ensuring that everything is thread safe and correctly synchronised could be turn out to be time consuming if you make just one simple mistake. These fools at ZDNet make it sound so trivial and simple: they obviously don't understand what they're talking about. A complex multi-threaded system that has been badly designed and implemented is a nightmare, especially if you have a large number of developers working on it (in my experience, the majority of software engineers out there just cannot develop safe multi-threaded code).
As many other readers pointed out, this report is really outdated. I really would love to see a more up-to-date web server benchmark including more platforms/solutions - such as the kHTTPd + say Apache.
kHTTPd is a statical page only web-server running as kernel module and is found in latest Linux 2.3 distributions.
Are your parents burnt-out stoners?
Not just benchmarketing, but stale benchmarketing!
There's likely to be some good data buried in there somewhere. But, for once, it would be nice for a benchmark crew to actually provide some honest results, so we don't have to sift through the wreckage for trinkets of relevance.
All this "study" tells me is that CGI is slower than NSAPI/ISAPI, I already now this. It should be since ISAPI and NSAPI is multi threaded and CGI is not. Nobody interested in speed uses CGI, they use modPerl or modJServ. This would make a way more interesting study. What do you expect from PC Magazine.
With hundreds of third-party products plugging into IIS, you could theoretically replace Windows components at almost any level. For example, you could bypass COM for some purposes and implement a third-party CORBA broker to talk to external systems. This versatility lets you opt for Windows NT as your Web platform without locking you into a Microsoft-only solution.
:-) was quietly released.
"And you, Mr Reviewer, are addressing us from the surface of which planet?" He who pays the piper calls the tunes; this has two implications here: at the presentation level (1) this particular piper is being paid by...? and at the content level (2) you'll find out just how MS-independent you are when you install Service Pack 6, and suddenly discover why SP6a (-: God forbid that they should call it "SP7"!
I think the first component to replace would be the entire web service (with Apache, much less prone to running the command of your choice than IIS), and the second would be the entire OS (with almost any Unix-ish). Then you could safely say "I am not locked into a Microsoft-only solution" without me laughing long and loud at you.
Got time? Spend some of it coding or testing
wos = Waste Of Space.
Therefore I think wosies would be the correct spelling.
Old COBOL programmers never die. They just code in C.
Earlier this month, there was a Slashdot article which mentioned that consumer reports was been sued by Suzuki. I don't have all the details on the lawsuit, but Suzuki is claiming that Consumer Reports subjected the Suzuki Sidekick to a biased test which was specifically designed such that the Sidekick would fail.
I think that this case might have some extremely interesting legs. Most of us who work in the industry are well aware that there are testing houses that can be relied on to bias there testing processes. This is by no means restricted to the high tech industry (ever wonder about some of those "quotes" that movie advertizements include? What about the unbiased research conducted by the Tobacco Institute or some of the more interested funded research into global warming these days)
If Suzuki is able to successful sue Consumer Reports, this might set a very interesting precedent that could be extended to other potentially biased studies and reports.
Please note, I don't think that there is necessarily anything wrong with presenting a strong arguement in favor of a particular point of view. In my mind, the probalem comes about when individual bodies are supposedly acting as dispassionate observers when, in fact, they clearly have an institutional bias.
Richard
What about reliability? The ZDNet test misses the whole issue.
Most servers are never going to get the waves of traffic you see at the high end of their testing scale. But any Web server, no matter what its traffic, has to stay up.
That's where NT falls on its face. NT servers go down about once a week; Linux boxes can go for 2 months or more without a reboot.
This review is old, but it's missinformation still lingers.
I was actually referring to the adjective "woozy", as in dizzy, rather than the noun "wussie"/"woosie"/"woosy", which means that you get sand kicked in your face regularly
Hi Ed.
ZDNet chose to tune ALL the servers to have 68Mb of web source material and at least 68Mb of memory disk cache.
Why did this give NT an unfair advantage? Why does Linux (or particularly the Caldera distro) solution not deal with RAM-rich servers as well as NT?
Well, I don't think you have it quite right. It is not that they are exposing a Linux weakness in handling RAM rich servers, it is that they are hiding a major performance bottleneck of NT -- its file system.
On the other hand, you get something for your performance hit -- filesystem journaling. It will be interesting to see what happens when ext3 and Reiser FS become more common.
In any case, I run a small web site that is 160MB in size, not counting databases. This is too large to be cached in the setup described, although not unreasonable to be entirely cached. However, if I was doing serious corporate intranet or a major ecommerce site, I would expect it to be much, much larger.
I am not a conspiracy theorist, but does it not seem a tad unrealistic to devise a web benchmark test which totally discounts disk access?
All religious handwaving aside, why did NT fare better by spinning threads than Apache could do by spinning processes? What is the big bottleneck in managing a process, that managing a thread doesn't have? They were using a brand-new MP kernel straight from Linus. Will the Linux kernel mature to deal with SMP situations and massive numbers of similar threads or processes better?
Well, on Unix, forking is cheap -- very cheap. On Windows, launching a new application instance is very expensive. Therefore multithreading is a huge performance win -- multiprocessor or no -- on windows, but a relatively smaller one on Unix. If you think about it, it hardly seems worth multithreading unless the threads are updating some common memory. If you write a thread with no critical sections, it may as well be a process on Unix, but on Windows it benefits from getting access to memory pages that have to be laboriously set up (they are simply copied in a Unix fork).
I am guessing that multithreading Apache speed improvements in Unix will scarcely be measurable unless nearly all the data being served out is cached in memory.
ZDNet suggested that in-process programming worked better for all the hairy e-commerce they decided to test.
... they have a point.
Well, that is a matter of opinion. Ebay may use IIS, but they do everything through CGIs.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
Let's do something about this!
Can we organize our own unbiased testing, disclosing hardware, OS, all configuration tweaks, patches, etc. and come up with a standard testing method that's doable with reasonably available hardware (that people have available at home or work)? Between all of us loyal readers, I'm sure we've got access to just about any configuration possible.
I'm not looking to determine which config can serve up 1 billion hits per second and 2 terabytes per minute here.. Let's just see where the loads/cpu times start to peak out, if we can establish that.
We all know the costs of most of the hardware and software, so we can make our own cost comparisons. I'm just really interested to see how the different platforms stack up. So far I'm really happy with our UltraSparc Solaris running NES, but I'd prefer Linux/Apache for development. I'd like to see some numbers on this, and I'd like to help generate them.
Slashdot isn't just a collection of links with funny polls.. The reader base is far too well educated and experienced for that. Let's get something accomplished!
Anyone have input on this, or am I asking too much?
They should have rigged the tests to favor Linux. Everybody knows that whatever OS is trendy and 31337 this year must be the fastest on earth for every purpose, right?