Red Hat Finishes Last

JTMatrix writes "RedHat takes last place [in an IDG Network Operating Systems showdown]." The information on how they benchmarked everything is readily available on the site. Go check it out. Update: 01/26 01:07 by H :Check out this link for more technical information.

Comments

Mentioned scripting but didn't rate it

[dsplat]

The text of the article mentioned that Linux provides the ability to use the standard Unix tools in scripts to automate tasks across a network. As far as getting consistent system administration done quickly across a large network, that is much more useful than running a GUI for each one.

They mentioned scalability, and one important factor with scalability is how administration scales with the number of servers. I don't expect to see many benchmarks that do it, but I would like to see a real scalability test with 1, 10, and 100 server configurations. The ability to learn the details once, and then automate them out of your way is a big plus with a rich, mature CLI environment.

I don't mean to say that there is no place for a set of GUI system administration tools. The single server in a small business will be easier to maintain that way. The file server at home serving my machine, my wife's and my kids' would be easier as well. It opens doors at the low end of the scale, which represents a larger number of sites. If you are a captive of the GUI for every configuration task, it slows you down significantly as the number of servers grows.

Not really "last place"

[FreshView]

I wouldn't say Red Hat came in "last", because they weren't really rating them apples to apples. They said Red Hat was best for some things, Windows 2000 was best for others, and so on.

I actually thought the article was somewhat complimentary towards red hat. The benchmark they needed to run, however, was Quake3Arena servers on each. : )

Abortive closes ARE supported on Linux

[X]

I'm surprised that nobody has pointed this out yet: abortive closes ARE supported on Linux. They just aren't enabled by default, and that's a good thing.
Abortive closes are great if you're a client running a benchmark, but if you're a server, you could receive packets at a port from a previous connection that will now appera to be coming from a new connection! Not a good thing.

People don't know what "enterprise" means

[DonkPunch]

(Big, frustrated rant ahead)

It really burns me when I see technical pundits talking about "enterprise-class" systems when they clearly have no idea what an "enterprise" is.

Here's a big, fat, spelled-out clue for them: "Enterprise" means more than just "really big". "Enterprise" means more than "lots of bundled pretty lights".

1. Enterprise systems have to be able to handle a tremendous load without sweating. This means lots of processes and lots of threads running smoothly at the same time. When overloaded, enterprise systems degrade gracefully.

Let me spell that out in caps -- ENTERPRISE SYSTEMS DEGRADE GRACEFULLY. They may refuse additional client connections, they may log error messages, but they may not EVER collapse under pressure. Anything less is not an enterprise system. It is a toy. Period.

2. Enterprise systems integrate with existing systems. A REAL enterprise often has legacy systems -- some of which have been running since before the new web developer was out of diapers. Companies offering enterprise solutions like to talk about how well their products work with your existing systems. Companies selling toys also want to help you with "updating", "migrating", or "replacing" your existing systems (which were working just fine before you strolled into my office, twerp). Consider that a red flag.

3. Enterprise systems stay up. In a real enterprise, rebooting costs money. Usually it costs BIG money. A company who doesn't understand that doesn't understand what an enterprise is. Beware -- toy-makers will try to sell you aftermarket add-ons for clustering, failover, or maintaing your "quality of service". Don't be fooled. You will pay more to maintain "quality of service" than you would pay to get a solid system in the first place.

I am so tired of magazines pandering to managers who think that they're running an "enterprise". Real enterprises already HAVE professionals to do these comparisons. They have no choice. In the long run, having a professional who is accountable is whole lot cheaper than trusting some twit at IDG, CNN, or ZDNet.

Now, with that perspective, I ask: do any of these NOS qualify as "enterprise-class"? If not, which ones come closest?

But it's FIRST in the online poll...

[maynard]

...with over 40% of the vote. And interestingly, many of the statements in this article are pretty subjective opinion. For example:

Microsoft's Windows 2000 edges out NetWare for the Network World Blue Ribbon Award. Windows 2000 tops the field with its management interface, server monitoring tools, storage management facilities and security measures.

Yet they admit several paragraphs down:

Windows 2000 demonstrated poor write performance across all our file tests. [...]

And even after turning off forced syncs after writes:

This second round of file testing proves that Windows 2000 is dependent on its file system cache to optimize write performance. The results of the testing with the write-through flag off were much higher - as much as 20 times faster. However, Windows 2000 still fell behind both NetWare and RedHat Linux in the file write tests when the write-through flag was off.

So, even though Win2000 is the slowest of the bunch (even slower than SCO's UNIXWare, according to this artile), it "Tops the field," but the benchmarks tell the true story. So, if you just skim the first few paragraphs of this article you'll walk away thinking Win2000 is the OS to beat. But by actually reading the article, you'll see the whole picture. Why do I think this is more of an advertisement for Win2000, than a serious article?

Huh?!

[BJH]

Where in that article (which I read a couple of hours before it was posted on /.) does it say that RedHat finished last?

I'm going to rant a bit here - Could the posters please make sure that the comments they post (either their own or those the submitter putin) are at least vaguely accurate and not likely to cause a goddamn flame war? This comment was completely gratuitous.

Back on topic: I actually found the article to be reasonably fair (if a bit clueless in places - the "RedHat only" poll comes to mind), but it covered some pretty deep material for CNN; stuff about Winblows NT's multi-threaded TCP stack, the stuff about Samba, etc.

Can we do without the endless flames of CNN now? Please?

Guessing versus Knowing

[rjamestaylor]

When we examined the Samba file system code, we found that it too honors the write-through flag. The Samba code then finds an optimum time during the read/write sequence to write to disk.

This jumped out at me because it so obviously points to a (if not the) significant benefit of Open Source (or, of at least having the source code, open or otherwise): not guessing and inferring about a black box. Microsoft 2000 "appeared" to be the only system "honoring" the performance-hitting flag of the benchmark suite. That was the argument MS gave for why Win2K's write performance was 10% of read. But the testers could infer Netware honored the flag by running the suite without it and noticing the performance increase. Nice to know. Great to be able to change inputs, observe outputs and infer process.

But with RedHat (Samba, specifically) no guessing was needed. Just look at the code! There it is. No mystery.

This suggests to me that the real SPAM threat has nothing to do with email. It has to do with closed-source systems: mystery meat. Usually nasty things are contained in mystery meat (no offense to Hormel...I'm sure Spam is fine and I remember my bachelor-days of fried Spam with Mustard on Toast fondly).

Fight spam: insist on the source! Can you imagine eating something that didn't come with an ingredient list? Why use an OS that isn't OS?

Sorry... got carried away, but the point is clear: those who care about their systems will demand access to the source.

:-only kona in my cup-:
:-robert taylor-:

Why no "interopability" score?

[rogerbo]

Why do these reviews never include an "interoperability" or a "plays nicely with others" score?

they always seem to test with all win98 clients or
all NT clients. Ok, I don't know maybe most places are all Microsoft nowadays but in my environment we have mac, unix and NT clients and that's not going to change anytime soon. We have applications that we need access to on all platform's.

Then all these issues like microsoft's "enhancement" of DNS in Windows 2000, their deliberate breaking of samba authentication in NT SP3 and all sorts of other cases where MS toys "do not play nicely with others" would get mentioned.

And MS would get dead last in this category every time.

These are factors real sys and network admins need to know about.

It's a good article, read the whole thing

[DocJohn]

You don't buy or judge an NOS based upon a single benchmark result. Read the whole story and you'll see why RH Linux didn't quite make it to the top, mainly because of its poor user management abilities, monitoring tools, and lack of other niceties expected from an enterprise NOS these days.

Car comparisons...

[DrCode]

Let's see, the RedHat car is:
More reliable (1 point)
Faster (1 point)
Cheaper (1 point)

The MicroMobile car has:
A better radio (1 point)
White lettering on the tires (1 point)
A bigger speedometer (1 point)
Corinthian leather shift knob (1 point)

Therefore, we conclude that the MicroMobile is the clear winner, with a 33% higher rating than the RedMobile.

Umm, actually it wasn't last...

[jht]

SCO UnixWare came in last, not Red Hat. And file server performance isn't the top reason I'd run Red Hat, anyhow. Basically, the way I see the choices are:

If you run a pretty much homogenous network of Windows (95, NT, and/or 2000) clients, then Windows 2000 isn't a bad server, really. Where Windows 2000 starts to suck hard is if you have to support other platforms in either the desktop or the server space. But it's actually a pretty solid OS, and a "safe" pick for a Windows shop.

If speed is what matters, you run mostly Windows at the desktops, and you're not looking for an application server (because nobody in their right minds develops NLMs), NetWare is fast, efficient, and has the most robust and complete directory services out there. Not to mention that there's a tremendous amount of trained, experienced NetWare CNE's to draw upon. It's fast, it's stable, and it's not Microsoft.

But if you want to run the most stable platform of all, and you want the power of Unix's tools and services, then Red Hat is ideal. It's easy on the wallet, too. Combine Red Hat and solid hardware that has multiple power supplies and ECC RAM and you'll probably never have to reboot it. And Linux is a lot easier for a network administrator to handle than it used to be.

And if you're on crack, you'll pick UnixWare - which sucked when Novell had it and still sucked the last time I got a look at it (a year or so ago). Some of the features of Red Hat, a much higher price, and closed source. Yum.

- -Josh Turiel

The above should be moderated up

[aheitner]

Because it's something we always need to remember.

It's nice to know Linux is fast (and it's no shame to get beaten out by Novell; they have a lot of experience in the area).

But for 99% of the server tasks people have in this area -- the interoffice server, sharing files and providing print and mail services, you could buy a meatier machine if you needed it. The real issue is reliability and ease of management. You need the thing up, period, because the whole office stops if it's down. And you probably prefer if you didn't have to have a tech for your department just to babysit that one machine. Ideally, your central tech support for all departments (or your part time tech support guy, if you're small) should be able to keep it running with minimal effort. We are, after all, looking for core services here, not cutting edge stuff.

IDG gave Linux the props it's due: Linux will beat out NetWare when it comes to building funky custom solutions. NetWare is very good at what it does. But you have to pay for every server module you want, and they're of course not open and flexible like the Linux ones are. NetWare would make it much harder for you to have that central office machine also be the web development machine for the office -- i.e. not only serve the files, but allow you to update them. And I don't know anything at all about adding database functionality to NetWare to drive a fancier website -- all very easy in Linux, and all there as soon as you want it.

This is one of the most balanced reviews I've seen. I may not agree with their choice of winner, but I can't criticize IDG's fundmental strategy of "choose the best NOS for your capabilities and your needs".

Of course SCO is worthless; and Solaris must be considered for its impressive scalability. Linux is fine for most scalability tasks, with the exception it seems of multiple NICs (which is a weird case anyhow. Rarely does a server need more than a single 100mbit link, and a quad-Xeon Linux box will chew up heavy duty database stuff very sweetly :).

Good test Bad conclusions

[YAH00]

After reading the article I got a really uneasy feeling as if the author did not want to get flamed for not doing the tests properly, but still *wanted* to show MS Windows 2000 as the best and Red Hat as the wrost.

It seems that the tests results completly belie the conclusions drawn... Here are some things I noticed...

File system performance
-----------------------
Test Conclusion : Windows 2000 sucks, RedHat is pretty good

Overall conclusion : Looks like no weightage was given to the overall conclusion based on these results

TCP Performance
---------------
Test Conclusion : W2k seems to be the best, RedHat sucks but will get better

Overall conclusion : AHA... Something in which w2k is the best. Now I will be vindicated in calling w2k the best NOS and Redhat the worst

Management Tools
----------------
Test Conclusions : W2k tools are really polished and provide setting many system settings. No remote management. Redhat is definitly more klunky, but lets you do pretty much whatever you want to, and from where ever you want to.

Overall Conclusions : Since w2k tools look so much better, they must be better

Monitoring tools
----------------
Test results : w2k has graphical clients to monitor CPU/Memory usage. RedHat does not

Overall conclusion : We used the w2k graphical clients for monitoring the system resource usage while our tests were being conducted. We did not even bother to look at linux tools like g-top which do exactly the same thing, and more. Like process level control. Since we did not use these tools, they do not exist. Ergo w2k is better. So there. Also although Unix utilities are a lot more flexible, and can be scripted, since they are too complex for us, they are pretty much useless for everyone else as well

Client Management
-----------------
Test Conclusion : w2k provides active directory and users and groups and oh my!!! Redhat only provides unix level control. Ha that old security model can't do users and groups and... well it can't do active directory so there!!!

Overall conclusion : w2k does active directory!! NAH NAH NAH NAH

And so on and so forth!!!!

It looks like they did put in some effort in doing the tests right. But it looks like they also put in a lot of effort into making the results fit their apparently pre-drawn conclusions