Slashdot Mirror
OSS Web Stacks Outperformed by .Net?
Gimble writes "eWeek has an article up that looks at the performance of portals using open source stacks and comparing them to their MS equivalents. The article's conclusion is that .Net outperforms the open source stacks, mainly because of its tighter integration, but also notes that running the open source stacks on Windows (WAMP) delivered strong performance." From the article: "Based on our forays into user forums for many top open-source enterprise applications, there are many IT managers attempting to run open-source products on Windows servers--attracted, no doubt, to the benefits and efficiencies of using open source without having to become Linux administrators. The results of our WAMP stack tests indicate that these folks might be on to something."
From the article: "The criticism we expect to hear most is of the stacks we left out--including commercial J2EE platforms, such as those available from BEA Systems, IBM, Oracle and Sun Microsystems, as well as the many other database and server platform permutations." I can't believe they came to this conclusion on such little data. They did, however, create a blog to disparate results can be shared.
"The mind works quicker than you think!"
I'm still not exactly sure what they tested. They have vague terms like "Request per Second" and "Throughput", yet they don't actually say what each page that is being requested is actually doing.
.NET tests they say they used "Sharepoint". Huh? For what? Considering that Sharepoint is *extremely* complicated and has incredibly rich functionality they should be very clear as to what they used it for.
For the
Not to mention the fact that using a portal application in your tests means that there is really very little way to isolate if it was a poorly written portal application or a crappy framework that the portal application was built on that's causing perf issues.
It is very difficult to test framework vs framework, but this is just about the worst way one could even attempt it.
At absolute best, this compares portal frameworks on various platforms. Even if they were trying to do that, they did a piss poor job.
How can you test the performance of a stack and compare it to others when the back end database servers, portal software and web server software is different?
.NET stacks are faster true when it could be the implementation of SQL being faster than MySQL? This test just doesn't make sense to me.
How is the statement that
Not to start a war here, but ASP.Net is a pretty damned nice environment to work under... I've used a lot of PHP, Cold Fusion, some JSP, and Classic ASP in the past. ASP.Net is my favorite.. I've been peeking in with Ruby/Rails but just haven't had the time to dive in much. of the above Ruby/Rails is probably the closest competitor on an ease of development/functionality level.
Michael J. Ryan - tracker1.info
What was the set of measures? For me, "performance" has more to do with uptime, reliability and security. Those are the performance standards I care about.
So, they ran an outward-facing Zope server (after being explicitly told not to) and the performace was lackluster? Go figure. In the real world, they'd run Zope behind an Apache or Squid proxy (as per every installation recommendation I've ever seen) which would immediately boost throughput by an order of magnitude. In short, using Zope to dynamically generate static content instead of caching the results whenever possible is insane, and pretty much no one does it. They also apparently forgot about ZEO, although I'm not sure how you can be savvy enough to get Zope up and populated without knowing about it's built-in clustering.
Apparently they had no interest in any tuning whatsoever, to the point of de-tuning it by installing it in an explicitly unrecommended configuring. And then it lost. Go figure.
Dewey, what part of this looks like authorities should be involved?
Justified perhaps, but automatically accurate... not necessarily.
Ok, fanboy.
So when MS doesn't add new features they are slammed for not innovating enough, and when they do add new features they are slammed for contributing to bloat that you don't want. People bitched about IE6 not having tabs, etc. Firefox came out and MS finally realized it had to update IE so it added a lot of features people were asking for and the most-heard comment on Slashdot after IE7b2 was released was "it's ugly". Face it: Microsoft just can't win.
You sorta' answered yourself there.
Not every product is a winner. MS historically doesn't release every single product as a beta and quietly stop promoting the ones that suck. Instead they release final versions and some fall on their face. No company has a perfect record.
The problem is not that criticism isn't warranted, it's that MS can't win no matter what. If they release a weak or buggy product they get slammed, but if they take too long to release they get slammed. If they don't add new features they get slammed, but if they add new features it's called bloat. If an MS product gets bad reviews the reviewers are being honest, but if they get good reviews the reviewers are obviously being paid. For years MS got slammed for security issues, and they beefed up SP2 and suddenly there were waves of "but it broke my application" complaints. The list goes on.
Microsoft has gotten so big that they are in the impossible position of trying to keep everyone happy. I'm not particularly a Microsoft "fan", but I hate this wanton "Micro$oft is teh suxors!1!" b.s. OSS fanboys need to grow up and realize that Microsoft can't go back in time and correct the sins of the past, and since it is a monopoly it can't just genuinely screw its customers and break every file/application by releasing a new version of Windows that corrects all the problems of the old versions but offers no legacy support. They have a tough balancing act to do and, while they're not perfect, they're getting better.
That's a poor analogy. Microsoft software is a Ferrari, and by comparison Open Source is a Mini? I don't think so.
A better analogy is this, because they refused to do any tuning on the OSS technologies: They bought a Mustang GT and a Nissan 350z and put them in a race. Then they told the driver of the Nissan that he could only use first and second gear. It would be nice if the Nissan was so dramatically superior that it could win even with that handicap, but since even a Ferrari would lose to a Mustang GT with just two forward gears at its disposal, it didn't happen.
As recently as last month, one of our customers corrected their very peculiar connection issues by replacing their windows server with a linux server. For some reason the windows server was changing the acknoledgement part of the TCP header- for the same client- at the same computer- for every transmission.
Windows Servers/App languages doesn't seem to scale well. It's a *great* way to get your business up and running asap. But you run into growth problems and need to switch to an enterprise solution (oracle, as/400, java, linux, etc.) once you reach a certain size. I still prefer windows as a desktop OS for now. I still slightly prefer windows office to openoffice. I think part of that is years of using office makes me comfortable but openoffice gets closer every day to replacing it for my home and personal use. I will probably not buy another version of office unless it is super cheap ($50/included free on the PC I buy).
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
It depends on a lot more than just "is it IIS/.Net or is it LAMP?"
.Net itself vs. php or .jsp - it depends largely on several things:
.Net because performance (of at least 1.x.x versions, I haven't worked with it since) absolutely sucks to almost any PHP application.
.Net can provide blazing performance. Your code has to be very efficient and you often need to throw a real large amount of RAM at it. Also, when you start a .Net application, it can take a while for it to serve up the first page - a concern for Tuesday updates when you were forced to reboot for patches to finish installing.
.Net application? Say, OS Commerce vs. a custom single-purpose .Net eCommerce application? I'd darn well expect .Net to blow away the LAMP solution, because OS Commerce is an inefficient beast - the only thing going for it is is that it's a "swiss army knife" open source eCommerce application. It does EVERYTHING, but sacrifices efficiency due to its hacked-together design. (yes, I'm using "hacked" as in "that guy is a hack"). On the converse, if you compare, say, DotNetNuke to mambo or Drupal and set out to provide the same or similar functionality, chances are that the LAMP solution will blow away .Net by any load test metrics you can come up with.
.Net? Licensing. Hardware costs. Have you ever set up a Windows cluster? It's not cheap, and even if you can come up with a cheap way to do it, you don't WANT to because it will be unreliable. SQL Server licensing is EXPENSIVE (or if the db is real advantage of Microsoft's .Net platform? The development tools, NO one has an IDE which is better than Microsoft's. Zend's PHP Studio is darn good, but the level of integration with documentation, the debugging environment, the code completion, and keyboard navigation does not match Visual
On equivalent hardware, with equivalent RAM, if you're running MySQL with the MyISAM engine it will blow away SQL Server performance for most queries, but at a cost: You do not have stored procedures or transactions. When you switch to InnoDB you gain those but take a performance hit and that advantage over SQL Server disappears. On the plus side, MySQL is free/free (unless you need the commercial license - which in turn depends on whether you bundle it with your application AND how you interface with MySQL AND the "license" of your software).
As far as
- What is the architecture and how efficient is the code? If you use, say, DotNetNuke as an example, it's a good argument AGAINST using
- How much RAM can you throw at it, and is caching appropriate for your application? With fully dynamic web sites where pages may display random content (rotating images, randomized quote of the day, etc.) unless you override the caching mechanism, the cache can effectively break your web site. However if caching IS appropriate and/or you override it when necessary,
- (related to above) are you comparing a poorly-coded, inefficient, beastly PHP application to a lightweight highly-optimized
What is my point? Unless you are comparing apples to apples, it's FUD or at best an amateur comparison. The way to test it is to implement the same task with a similar (as possible) architecture on each platform, with the application on each fully optimized, with both IIS and apache/tomcat/whatever fully tuned and streamlined to pull out every bit of performance from it, then load test both of those using the same sorts of tests. Comparing a highly-optimized single-purpose application to a general-purpose portal platform is not a fair test.
Also: Even though the article says "Even the most ardent PHP fans will admit that PHP is not designed with performance in mind," PHP performs damn well considering it's heritage, the fact that its primary platform is "a patchy server" and that it is FREE.
Why would one choose LAMP over
The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
Comparing Sharepoint to Zope is beyond silly.
Zope is an object relational application server, making it slower than anything else running standard DB's. Technologically wise Zope is ten years ahead of Sharepoint - this is payed for with performance hoging and heavy-weight memory usage. 2Gigs is not enough for running Zope/Plone in a serious production enviroment.
Sharepoint is a monolithic built-to-fit solution that was grown over the course of almost a decade and finally has turned into something that doesn't crash every odd hour and - at last - performs the way it was supposed to back in 2001.
Keeping in mind that Zope was allready working back in 2001 and actually hasn't changed all that much since then. The entire redo - Zop 3.0 - still is in developement.
Sharepoint is usually used for CMS purposes, while Zope is usually used for highly abstracted business application developement.
Nobody in his right mind would get the idea to build an ERP system with Sharepoint.
Bottom line:
These guys didn't know what they where testing.
We suffer more in our imagination than in reality. - Seneca
To test the .Net stack, we ran Windows Server 2003 R2, SQL Server 2005 and SharePoint Portal Server 2003. Across the board, this configuration performed very well, with the top overall average throughput (by far) at 4.59M bps.
Quick check.....
$2,792.00 (Froogle Directron) Microsoft Windows Server 2003 R2 Enterprise, 25 Clients
$5,489.18 (Froogle Non Academic) SQL Server 2005 Complete
$5,619.00 (MS Website Retail) SharePoint Portal Server 2003 Server License with 5 CALs
$1,124.00 (Dell) Suse Enterprise Linux 9 With Server Hardwarex ?c=us&cs=555&l=en&oc=MLB1580&s=biz Couldn't find Suse Enterprise 10
Integrated LAMP Stackhttp://www.novell.com/products/linuxenterpris eserver/lamp.html
http://configure.us.dell.com/dellstore/config.asp
Hmmm, Could train a couple of Windows Admins with $11,000. Better yet just Hire a good Linux Admin.
To a large degree, we credit this strong showing to the high level of integration that exists among the components of this stack. While most of the open-source and Java systems are developed independently of each other, each of the .Net components is designed specifically to integrate and perform well together.
Even if the .Net stack had bombed convincingly in these tests, it would probably still maintain popularity in many companies.
Some people (PHBs) will never come around.
But its strong showing should give companies confidence that the .Net stack will handle most high-level enterprise needs.
For more than $12Grand it better blow away the Free Alternatives and configure itself and require zero admin.
I know I will get slammed for not using TCO but I don't believe those numbers at all. In my experience it takes the same amount of time for day to day maintenance. And when there is a problem (and there will be, no matter which one you choose) It costs me less time and therefore money to bring back up the Linux box.
Cost is not the only factor in a buying decision but is a factor, and if performance is arguably equal than it is a huge factor.
OSGGFG - Open Source Gamers Guide to Free Games
Even if that was true I'd still rather have a proper UNIX environment anyways. Not to mention a real multi-user OS without shelling out huge dollars. Oh and the development tools and ...
... you're wasting your money.
Sorry, but with modern windows, what exactly is missing that dis-qualifies it from being a real multi-user OS? And FWIW, I've found VS2005 to be much better than any development studio I've tried for linux.
If you buy Windows to run AMP servers
So you think its better to have servers which don't integrate with your corporate network? $400 for Web edition isn't a whole lot to anyone running web farms.
Linux and BSD are more efficient to work with in server contexts
Your opinion, there's no fact to that. I replace my linux server at home with SBS 2003 because its easier to manage the network using SBS2003 than it was with Linux + SMB.
work better with 64-bit processors [Win64 is a huge compatibility joke atm]
I've heard quite differently; indeed, some people claim XP64 is the best desktop OS, even better than 32bit XP. I don't think the 64 bit servers are suffering huge problems either.
don't require you to call India each time your HD breaks and you need a re-install.
A bunch of FUD here; mearly replacing an HD doesn't require calling anyone, and alot of shops image the drives every night so that if there is a failure, they slap on the saved image and are up and running again in no time.
Heck, in BSD/Linux doing a ghost of your system is as simple as burning a tarball to a DVD.
You can easily save an image of a Windows installtion as well.
No need for 3rd party ghosting tools and praying that Windows lets you "get away with" using your OS...
Go ahead and spend hours and hours tarballing your server; the fact is that it would be faster to just create an image of the HD and burn that to a DVD. There's no praying involved; my former employer had great success doing such restores. And putting down a new image is faster than un-tarring a file to disk again.
Ok here's the problem. Most posts I've seen point out how great linux is. I don't think linux users take criticism well. If you all feel that the benchmark is wrong, do one yourself. Prove its close or faster. I have serious trouble believing that WAMP is faster than LAMP myself. I do think a properly configured ASP.NET/IIS server vs a linux/apache server would show dynamic content faster on the windows box. The reason is most open source languages require you to do a lot of research and read to get real performance. Anyone can sit down with .NET and get something quick running. This IS a flaw in open source and it needs to be addressed with better documentation and developers realizing they need to attract everyone. Having a hard to configure piece of software or hard to use library doesn't make you a good programmer. It makes you a bad programmer. Some linux users want more people to use linux and that means the TARGET AUDIENCE has changed. Software must be updated to cope with it. Further, I personally feel a linux/unix box takes more setup time than a Windows server even counting registry tweaks and configuration adjustments most windows admins ignore. The advantage is if the box is going to stay that way for a long time. In that case, linux/unix win hands down because configurations don't change with good software. This does not hold true for linux on desktops though. Graphical environments often require crazy upgrade procedures and often break resulting in a recompile of the whole damn system. (yes gnome developers, i'm talking about you)
Further, the real problem is the open source technologies for web development are lacking. Ruby on rails has promise. Like many open source problems, there are too many languages and little consensus. I won't bother complaining about PHP again because people who need to listen don't. Its not as user friendly and consistant as ASP nor is the documentation. (as a brief summary)
Performance can be a real world issue. If someone is on a limited hardware budget and can't just throw hardware at the problem like many linux people tend to do, its a big problem.
Cost of ownership (TCO) is relative. When Microsoft promotes TCO many scream their heads off on slashdot. Its not cheaper to run commerical linux over windows. Last I checked suse or redhat charge around the same for a license as microsoft. (especially when you don't need a rediculous number of users.. aka small to medium businesses and individuals) Yes, they could download gentoo or whatever. When I look at cost, I look at the cost to get a sysadmin (windows ones are cheaper), the cost of the hardware (usually same), the cost of the software and the amount of time spent on administration during the life of the system. Redhat has just as many patches as Microsoft does. Watch their list sometime. You can argue there are less (insert your system component here), but a good sysadmin updates everything on the box that could possibly get executed/exploited not just what he uses. If you can't delete notepad or X11, you must update it. It can mean the difference between someone on your box and someone with root on your box. Most os vendors ship a similar number of patches if you count software that is optional but still dangerous. (apple, microsoft, sun, etc)
Terms and vendor support are where you start to make sense. Microsoft has poor support. Oracle claims redhat does. My personal experience has been that the linux community is rude on forums and other types of support.
I've professionally administered several types of servers, and to this day I still think FreeBSD is the cheapest server platform. I recommend/use windows as a webserver or file server only.
Sorry, but with modern windows, what exactly is missing that dis-qualifies it from being a real multi-user OS? And FWIW, I've found VS2005 to be much better than any development studio I've tried for linux.
:)
;)
/etc or /var. It's not like you're stuck with tar either - disk imaging still works fine. When you can tarball up only the parts of the machine you need to keep, the idea of disk imaging seems redundant (for most cases).
I think the grandparents point is that though it is technically a multi-user OS, it isn't a very good one. The kernel level schedulers on Windows give a really poor response under multiple sources of medium to heavy load.
I'm not aware of that many "development studios" for Unix. There's a couple, like eclipse and such, but many Unix writers tend to work a different way. In line with the Unix philosophy (rules 1, 3, 4, 5, 7 and 8 if you're interested) there's a lot of people who keep the editor, compiler, linker etc all seperate. If you're looking for a "development studio" for Unix, I think you're looking for the wrong thing. We don't have a Start Menu either.
So you think its better to have servers which don't integrate with your corporate network? $400 for Web edition isn't a whole lot to anyone running web farms.
Well, I'm sure anyone could have pointed out that large companies with large budgets can afford the $400 dollar cost. But that $400 dollars is a cost, and to a lot of companies, it's a big cost. It's an even bigger cost when many difference licences have to be purchased. (Obviously, Unixes have costs of their own, but the costs tend to increase in line with the use being made, or the complexity of the "solution" etc, and up-front licences payments (which are immoral, in my opinion) normally end up being a far less weildy solution than Free Unix is) And in my, and the grandparents opinion, it's an unjustified cost.
Your opinion, there's no fact to that. I replace my linux server at home with SBS 2003 because its easier to manage the network using SBS2003 than it was with Linux + SMB.
Of course it's opinion! The whole damn subject is opinion! As if a (very very poorly run) test on the speed of a server is the last word on the subject! There's little universal fact to your personal experience either.
I've heard quite differently; indeed, some people claim XP64 is the best desktop OS, even better than 32bit XP. I don't think the 64 bit servers are suffering huge problems either.
Have you actually used it? WinXP 64bit really doesn't work very well at all. I wouldn't know about 64 bit servers, but 64 bit WinXP is really not something that works very well.
Go ahead and spend hours and hours tarballing your server; the fact is that it would be faster to just create an image of the HD and burn that to a DVD. There's no praying involved; my former employer had great success doing such restores. And putting down a new image is faster than un-tarring a file to disk again.
It's possible to use either image or tar with Unixes. Tar is useful because it's easy to use diffs and delta compression on backups then. Tar's also useful when you only want to backup a certain area; such as