Slashdot Mirror
OSS Web Stacks Outperformed by .Net?
Gimble writes "eWeek has an article up that looks at the performance of portals using open source stacks and comparing them to their MS equivalents. The article's conclusion is that .Net outperforms the open source stacks, mainly because of its tighter integration, but also notes that running the open source stacks on Windows (WAMP) delivered strong performance." From the article: "Based on our forays into user forums for many top open-source enterprise applications, there are many IT managers attempting to run open-source products on Windows servers--attracted, no doubt, to the benefits and efficiencies of using open source without having to become Linux administrators. The results of our WAMP stack tests indicate that these folks might be on to something."
The .NET CLR runs compiled bytecode and IIS runs in kernelspace, the only httpd I know running in kernel space on nix is tux (redhat content accelerator) and nobody in their right mind is going to serve dynamic content with that. Did they test PHP with an opcode cache and the CLR running a dynamic language? This isn't even apples and oranges, it's apples and teapots. If I wanted performance, I wouldn't be running either Apache or PHP!
But this is a very close second place.
#1. NO tuning was done on the LAMP stuff. None at all. They ran the stuff "out of the box".
#2. They didn't write their own app. That means they didn't test the SAME processes on each system.
#3. They didn't bother to find WHERE the differences were. Is it in the IP stack? Is it in the OS? Is it in the scripting language? Is it in the app?
How bad can "research" be and still be published in "eWeek"? There wasn't any research done for that article.
Microsoft has, in the past, taken various short-cuts when IIS was the server and IE was the browser. Is that the case in this "study"? Are the other "stacks" "slower" because they follow the protocols?
You won't know because they'd didn't LOOK for the REASON behind their "results".
At least MindCraft was paid to do poor research.
Too true... i came off of struts and tag libraries to .net and it was/is a much better environment. I am not sure what the state of the art is in Java any more but I am glad I switched.
Classic ASP is a horror from hell and I think soured many from using MS web solutions.
Please sign petition to restore sanity to our banking system!!!
http://financialpetition.org/
Listen to Dell, RIAA, MPAA, etc etc....uhm... no they don't. They just don't beat them as mercilessly as the home consumer, because they COULD swing back and do some damage over time.
To Dell/HP/Etc - You must not sell naked or Linux systems or your the price of OEM Windows gets larger. RIAA/MPAA - If you don't do what we like...we won't play with your DRM schemes. Government - If you stop pressuring us we will donate to campaign funds and let you keep using Office.
Go look at MS campaign fund history...right up until the antitrust thing they really didn't give anyone any money, once the antitrust thing kicked off....big dollars to everyone running.
The only change I can believe in is what I find in my couch cushions.
They seem to have a generalized poverty of data. Their charts seem absurd to the point of being straw men. I mean, come on - I don't think there's anything seriously wrong enough with Linux that WAMP would have a score of 12 transactions/sec, competing with Windows, whereas LAMP would have a performance of 2. My experience with Windows vs. Linux has always been that they are similar in terms of speed from pure processing tasks to 3d games. Sometimes Windows does a little better, sometimes Linux is better. But they're usually in the same ballpark. The numbers are just too neat. It's like they put up a chart saying that Republicans, Germans, Koreans and Canadians have sex once a month, whereas Democrats, Brazilians, and the British have sex five million times per second.
Moreover, the whole rest of the article is morass of poetic circumlocution. My gut feeling as somebody who works with words a lot is that they're trying to obfuscate something with a giant wall of banal text. I don't know exactly what that is, because I don't feel like reading all of it, but if I had to guess I'd say that the real thing to take away from this article is that anybody can set up .NET and a Windows box, but that it requires a little bit of patience and research to make Linux work properly - research that these people were not willing to do.
Am I the only person who has never heard of the word "stack" in this context?
Wikipedia: stacks - Nope
Google definition of stack - Nope.
Urban Dictionary: stack - Nope.
Dictionary.com - stack - Nope
Google search "IT stack" - Only hit is the eweek article.
I think they made up this term.
s/stack/platform/g
or
s/stack/framework/g
Here's the price breakdown for a SINGLE webserver that allows external connections to authenticate (non-domain, say a e-commerce site with user accounts) against a SINGLE SQL 2005 Database. Sql Express is free, however it's not licensed for unlimited users in a production environment.
Web Server (Prices from CDW.com):
- 1 Copy Windows 2003 R2 (5 CALs): $959.99
- 1 External Connector License for Windows: $1,969.99
- Total: $2,929.98
SQL Server:- 1 Copy Windows 2003 R2 (5 CALs): $959.99
- 1 Copy SQL Server 2005 Workgroup Edition 1 Processor License: $3,819.99
- Total: $4,779.98
Grand Total for a single web and sql box: $7,709.96.And don't forget that you'll need SOME hardware to run that OS. Even barebones boxes with no data protection will run you $500 a box.
So, to start a basic e-commerce site on the legit, you're talking roughly $9,000 for windows and $1,000 for Linux/OSS.
TOUGH sell for Microsoft for the little guy.
"Everything MSFT does is to benefit the stock holders through locking the "customers" into their system.
Isn't this what buisness is about? Don't get me wrong I abhore some of M$'s tactics but they are in buisness to make money not necessarily please customers. In an ideal world the two go hand in hand but for the vast majority of the non-techie users out there M$ does please them. They can send their gran a birthday letter in word and she can open it on her PC because she has Windows/Office too."
But, to get back on topic, this is WHY we distrust any benchmark about Windows winning over Linux. MS has a monetary need to make it look good. Linux has a monetary preference (some preference, but not to the level of MS).
You raise some excellent points. If I may be allowed to expand upon them ...
#1. Set a price limit. You can set multiple limits ($1,000 vs $5,000 vs $25,000 vs $100,000 vs $1,000,000+). The key concept here is that you get different characteristics as your budget increases/decreases. What characteristics does each "stack" offer in each price range? Yep, this does give the advantage to Free stacks (Free like speech, free like beer). Deal with it. In the Real World the bottom line is the bottom line. Each team gets to spend the money however they want to.
#2. Get the "experts" to tune each stack. BUT they must document each modification they make, including WHY they made that modification (what testing did they run and how did those test results tell them what mod's to make) AND they are only allowed to make mod's that can be found via public websites (no secret tuning parameters that are only known to the organization writing that software) AND they aren't allowed to touch any source code. They get what everyone else gets.
#3. The fun part. Each team gets to pull apart the work of the other teams. Even if your solution is faster for the specs given, how much wiggle room do you have? Is faster and fragile better than slower and stable? How much "slower" is acceptable for how much more "stable"? Can the other team defeat your security (network access only)?
#4. Freeze those systems. Then, over the next year, patch them and re-test them. Do the patches break the "tuning" that was done?
Now that would be an informative test process (and would result in lots of articles and interviews for the magazine publishing it).
Yeah, you can run Linux / Apache / MySQL / perl on a single drive workstation and get damn good performance for less than $300.
But that will be completely different from Oracle / Java on a cluster of Suns costing $10,000,000. And not just in the number of boxes you'd be running.
A curious mix of flawed logic, marketing waffle and technical language.
.. mix of a Windows server and open-source components .. businesses should seriously consider the combo for their enterprise applications."
.Net stack performed very well in our tests, clearly showing the benefits of the tight integration among each of the stack components"
.. JBoss Portal on Windows performed considerably better than JBoss on .. CentOS"
.Net components is designed specifically to integrate and perform well together"
.. on Windows .., no doubt, to the benefits and efficiencies of using open source without having to become Linux administrators"
"Probably most surprising was the solid performance that came from the
fud.alert: LAMP runs better on Windows.
Why would anyone move to Windows to use Open Source? Don't you still have to pay per simultaneous connection.
"Microsoft's
How does 'tight integration', which is a function of how easy the sysop maintains the system, affect the efficiency of a running 'stack'. Does the stack know it is better 'integrated' and therefore runs like a happy bunny?
"JBoss Portal is relatively immature
fud.injection: JBoss on CentOS is immature. JBoss on Windows is better.
"we credit this strong showing to the high level of integration that exists among the components of this stack. While most of the open-source and Java systems are developed independently of each other, each of the
fud.injection: open-source and Java don't perform well together. Open Source runs better under Windows. Oh please Mr. Manager don't move off that Windows boxen.
"Neither the open-source nor the Windows communities seem to be able to accept a marriage of open-source server components and Windows operating systems"
What licensing restriction do the current ms.EULA put on Open Source projects developed with\and for Windows? Name any possible benefit that would be obtained by running Open Source on Windows? And don't mention the ease of use GUI. A proper sysop writes scripts to maintain the system.
"there are many IT managers attempting to run open-source
How by any logic is it easier on Windows? This totally fails the logic test. Apache on Windows requires the same kind of config as Linux. Name any Open Source app that is easier to maintain under Windows. Provide concrete examples not opinion.
"JBoss on Windows far outpacing its Linux brethren"
I'm sure Marc Fleury would be interested in how Microsoft managed to get JBoss running faster on Windows.
"Enterprise IT managers shouldn't hesitate to look into the option of deploying open-source stacks on a Windows Server platform."
Yea, remember you still have your yearly tithe to pay Redmond. That's five seperate times in that article that you advised people to stick to 'open-source' on Windows. I do believe we have now all fully gotten the sub.text.
davecb5620@gmail.com
Actually, I'm speaking specifically about authenticated users that are not part of your domain. This is a common scenario for an e-commerce site, or any site that users can create accounts on. A bulletin board is another example.
/ Pur%20Archive/MicrosoftProductUseRights(Worldwide) (English)(July2006).doc#_Toc127699990
In this scenario, you must use an external connector license because those users are not authenticated, and the Web Edition of Windows 2003 does not have an applicable External Connector License.
You can read more here:
http://www.microsoftvolumelicensing.com/userights
Pages 43 cover Web Edition of 2003 Server. Page 19 talks about External Connector License.
Now, I know that Plone is a dog, and XOOPS may be popular on Sourceforge, but I don't think it's the most obvious choice for building a high performance portal using PHP. So, using these two as the basis for testing is silly.
The fact that JBoss Portal on Windows outpaces JBoss Portal on Linux has a simple reason: JBoss isn't fully open source; crucial parts of it (namely the Java runtime itself) are under Sun's control, and hell will freeze over before Sun bothers to do a good job implementing Java for their competitors' Linux systems.
As for things generally running faster on Windows, that's implausible. Differences between raw Windows and Linux system performance are at most in the single digit percentages, so if they saw any significant differences between the same applications running on top of the two platforms, either the application vendor spends more time tuning for Windows (as in Sun Java), or the testing labs screwed up.
In fact, the whole test is really ill conceived: none of the "portals" they compared provide the same functionality; it just doesn't make sense to test them against each other. Overall, this test mostly seems to test the competency of eWeek, and they aren't doing too well.
Default for MySQL on Linux: low RAM shared use box, minimal RAM use. Default for MySQL on Windows: ask some questions about use and make fair use of the resources of the system. On the Linux box you have to copy a sample configuration file by hand and that step wasn't in the installation instructions.
Net result: use default MySQL and you get a setup that inherently strongly favors Windows, because Windows setup is optimzed while the other isn't.
I don't know whether they made this mistake or not. If they did, as it appears from their results, the comparisons that include a database component are meaningless.
The 2003 article Is the Portal Dead? discusses Gartner's announcement that "the portal is dead, long live the portal." And more recently, Portals are Dead reiterates that. But of course, there's plenty more about the death of the portal.
So you think its better to have servers which don't integrate with your corporate network? $400 for Web edition isn't a whole lot to anyone running web farms.
WTF? What planet do you do business on? On my planet, Earth, adding THIRTY TO FIFTY PERCENT to the fixed cost of a rack unit is a pretty big deal. Have you ever compared prices on Windows and Linux hosting? That huge pricing difference is there for a reason.
No, I'm not a Linux fan, not at all. But talk sense. Windows is better and it costs more. The two are related IMO.
That's the beauty of the Microsoft culture. Had they used GTK+ or QT or god forid ported GDI to Linux they'd be set. But because they use their platform dependent system and user libraries they lock themselves into their own damn OS.
.net crap and what not, at least not at first. I'd pay a decent coin just for the IDE and integration with GCC/GDB. The editor [and the RAD tools] are actually very well put together (at least for VC6, the later VSes are less and less friendly IMHO) and useful.
And note, they wouldn't have to port all the
Kdevelop sucks because it makes horrible looking projects by default and requires KDE libs (I hate KDE in general).
Jedit is about the best thing down [in OSS land]. It requires Java and ANT but runs pretty much anywhere and has decent source editing tricks.
The trick is MSFT would have to work with others. I mean GCC and GDB already exist, why not profit from that?
Tom
Someday, I'll have a real sig.