Slashdot Mirror
Open Source OS Benchmarking Competition
BenchmarkingFreak writes "OSnews is running a story about a new benchmarking competition: OSU Open Source Lab wanted to take the concept of benchmarking a little bit further with the Beaver Challenge 2004. In this competition they will be allowing a community of experts in each OS to tweak their configurations to ensure maximum performance. And they are running it all on wicked machines, just imagine... well you know."
* Debian GNU/Linux
* Fedora Linux
* FreeBSD
* Gentoo Linux
* NetBSD
* OpenBSD
* Red Hat Linux
* Slackware Linux
* SuSE GNU/Linux
Where's Mandrake?
It will be decided once and for all that Gentoo offers no discernable advantage over more conventional distros. At which point, Gentoo-zealots will only have portage to pull in converts.
For me, that's enough. Portage doesn't have *half* the dependency problems of apt, or up2date. I was a long time RedHat user, and I've tinkered with Debian here and there (can't stand it actually). But now I'm a certified Gentoo Zealot because of portage alone.
"Ignorance more frequently begets confidence than does knowledge"
- Charles Darwin
Maybe, maybe not.. that's the point of this exercise. Each distro gets a chance to optimize as much as possible. The nice thing about is everyone has to share what they did to make it faster... As a debian user, I can't wait to see what they do..
Given OSNews' recent penchant for poorly-done benchmarks (e.g. 1, 2), I'm glad to see them run an article about someone else's (hopefully well-done) testing. By having expert teams who know what they're doing tweak the configurations, this should be a much more representative result. Hopefully OSNews will learn some methodology from these guys...
"You can never have too many elephants on your team."
The only way to get real world benchmark results would be to distribute the benchmarks setiathome style and then do some funky statitical analysis. But you'd also need a way to verify the actual hardware used for each test.
Otherwise, we still end up with data that needs to be heavily interpreted to get any take on what will happen in production on any given hardware.
Oh yeah, not to mention that for real world production, performance is also dependant on maintainability, uptime and a variety of other factors.
Still, this will result in a bunch of pretty graphs and nice rants and raves and dick waving for years to come, even though the data will only be good for one given point in time, for one given hardware platform, for one given configuration.
I don't think it's all about a winner or a loser. As some others have pointed out each distro has it's quirks. This will show raw speed, as well as how it was accomplished, but there are plenty of other reasons to pick one over the other... Preference of package management, security, easy of maintaince. setup time etc and so forth. I personaly can't wait to see how this turns out...
I have to confess to being more interested in the universality of the testing framework than any of the results. Whatever is done I hope it leads to some standards for future, lower profile but perhaps more useful benchmarks.
An accepted cross distro testing criteria would be nice.
ls
Come on, if it's built by Dell in 3min the
PS/2 & USB controller will die, 3hrs the HDD controller will fail & wipe each HDD, but in 3 days they'll replace them with the equivelant Compaq or clone machine....
Flamebait I know... Am I Ashamed? No
: )
I wonder how much variation there will be in each team's ability to optimize the boxes, ie, which team is the better tweaker. Maybe this will be more of a determining factor in the outcome than the distro or OS used.
Beautiful.
Back a while, when gentoo was still had the smell of pop novelty, you would hear all this great stuff about how gentoo distros were the fastest, something about being able to specify --funroll-all-the-bad-loopies and --enable-r0xor-opts and --omit-random-instructions to the compiler. Of course, all these claims of gentoo's speed have never been backed up. On the contrary, the only results we've seen published tend to indicate that you average gentoo distro is composed of slower-than-average or average applications.
These days, we hear the new mantras of the gentoo-fanboys: it's not the speed (good thing!) that they use gentoo for, but instead the ease of use or robust package management or configuration flexibility. That's great and all, but it's all a bunch of green-is-my-favorite-color kind of advocacy: opinionated, unsupported, and unconvincing. People who've gone through the long laborious pain of installing gentoo (reminiscent of slackware 3.0 and libc upgrades, what year is it again?), and then having wasted the effort on a system that will probably spend more cycles compiling itself than serving the users, they justify the waste with a belief that their system is better managed or more finely tuned or whatever. Emphasis on whatever.
Of course, none of the supposed benefits of gentoo are backed with anything approaching rigorous analysis. Instead we get vague anecdotes and slashdot fanboyism. When we inevitably learn that the gentoo portage system is riddled with problems, conflicting package maintenance mechanisms and policy, broken and overtweaked package scripts, and that the whole thing needs a certain amount of voodoo to work, the gentoo boys will probably come up with some other reason why it is the one distro to rule them all.
The rest of will just wait for the results of your empirical studies with smiles on our faces!
XML causes global warming.
"When we inevitably learn that the gentoo portage system is riddled with problems, conflicting package maintenance mechanisms and policy, broken and overtweaked package scripts, and that the whole thing needs a certain amount of voodoo to work"
You know this is one of the better descriptions of portage/Gentoo I have heard. If I had the time/resources I would re-write portage using a bette langauge and more sane feature set. Portage was a good idea, and is a HORRIBLE implementation. however it still beats RPM.
PS somebody mod the parent up, I would have modded you up, but I already posted to this topic.
"Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe." --Albert Einstein
It depends on the exact function of the machine, but it's important to remember that different parts of a disk perform differently.
Why put /boot at the beginning? It only gets accessed once during a run and unless you're rebooting constantly, shouldn't be used very much at all. So, put /boot in the slowest performing part.
How often do you use swap? Depending on the OS and the purpose of the machine, swap might never be used, or when it gets used it's a Bad Thing (e.g. you should have enough RAM to handle 99% of situations). So, swap might not need to be in the best performing part either.
If the machine is a mail server, you might want /var/spool/ or /var/mail or the equivalent in the fastest part. If syslog server, then /var/log gets best performance.
This link shows some stats for older disks. My own tests on more recent disks (seagate 200mb, 8mb cache and wd 250mb,8mb cache ide drives) show similar results - a 2:1 difference from the fastest to slowest sectors of the disk.
Once we start talking about RAID, things get even more complex. Are you sure the same zones are being used from one test run to the next? Your software volume manager might be able to tell you, but hardware raid can be more difficult to tell.
Huh? Part of the reason I switched over to NetBSD from Linux years ago (for the most part, there are still times when a quick-and-dirty Slackware box does a special trick or two) was because NetBSD (and FreeBSD is very similar) was FAR easier. There's a learning curve involved, but there's 'one way' that things are done and it's the classic Unix way. You can pick up an O'Reilly book from 1993 and the info in it closely applies. Linux, on the other hand, is a big snarl of forks, each distro doing each task and configuration in it's own way, everybody contending that THEIR way is BEST, and as a consequence, no clear straight-forward anything, except gui buttons in places where GUI buttons aren't needed.
/etc/ files on a NetBSD box is a converging process. You learn more and more as you work with it, and it doesn't change when Johnny volunteer coder at Distro X learns Python and gets tricky with pretty buttons on a control panel.
Learn how to configure the
Anyway, harumph. BSD is NOT less user-friendly. Perhaps it has a smaller userbase, but if you're reading and commenting on this article, you're capable of working with it.
---
Advice: Read the HUNDREDS of posts on the gentoo forums about this.
The fact of the matter is that portage is plenty fast. Any speed boosts given to the actual emerge program set would be negligable because of the sheer amount of time dedicated to compiling.
More Advice: Stop trolling.
When you say that "Portage was a good idea, and is a HORRIBLE implementation" you really need to enumerate WHY it's a horrible implementation.
Take this common troll as an example.
Example 1: Windows is a HORRIBLE OS.
Example 2: Windows is a HORRIBLE OS because being locked into the choices Microsoft made in my "interests" are usually counter-productive.
See the difference? Example 1, while in many people's oppinion is valid, leaves people wondering why Windows is horrible. Example 2 gives anybody reading specific evidence and also allows anybody that wants to defend the point areas to do so.
I love the way that gentoo handles packages. I, admitingly, have a BSD bias, but it still allows my system to be what I want.
The feature set is anything but 'insane,' but once again, I have no idea why you think so, so I can't exactly defend that against any reasons you have.
Reply to this post and we might actually have some decent points to give to the gentoo team to make inprovements.
Not Free(as in beer). Free(as in "I'm free to beat you over the head for being a dumbass")
I agree with all of the above, but as a gentoo fanboy I run Gentoo for two reasons
1: Portage
2: To update all the software on my system to something that was released less than 72 hours ago with one command (emerge world)
Of course this has absolutely wrecked my installation on a number of occasions, but I kind of like it. It keeps the skills sharp and lets you figure out how to solve problems that you never knew existed in linux (gnu/linux whatever).
BBH
1. Unless you're using a computer more than 10 years old, you're not going to be waiting for Portage to calculate dependancies for any significant amount of time. I wouldn't mind Gentoo developers optimizing Portage for speed, but only trolls or the most anal-retentive of users will be put off by it's "inefficiency".
/usr/portage; /var/tmp/portage
2. RTFM. Gentoo docs are very easy to follow, and it only needs to be done once. I'm sure that typing in a command or two is not going to kill you.
3. Emerge, as far as the end user is concerned, is the Gentoo package manager. The scripts that power Portage are completely behind the scenes. I've been using Gentoo since it was an early beta, and I've never had to go in and modify a Portage script in order to get it to work.
4.
Yep. Really disorganized.
5. Several options:
a.) tell portage to ignore the dependancy and install anyway
b.) remove java functionality from db
c.) install db manually and inject the package into portage
6. Gentoo is a source-based distro. It doesn't have very many binary packages, nor does it claim to. If binaries are so important to you, Gentoo offers a physical product that you can buy with all major packages precompiled for all major CPU architectures.
7. I'll give you this one.
And, seeing as I managed to not read the topic of your post before I went off and posted myself, I misinterpreted what you said.
First of all, Gentoo is good.
Second of all, Gentoo isn't the easiest thing to setup.
Third of all, it is a matter of taste. And just as we shouldnt judge the general users of GNU/Linux on the mad zealots that shouts the loudest, you shouldnt judge Gentoo on the Gentoo zealots.
I use gentoo and I love it. On the other hand, Ive also used pretty much any other distro out there. IMHO, the one that was most pain-like to set up was debian. Though it is, a matter of opinion.
Now, I will go hide under a rock for a month.
Hmm, interesting, did not know about that. Mandrake does use Debian's menu system, but the first version I heard about was 5.1, and it was already based on Red Hat, which I think was at version 5.0 at that time, thus the version number.
Michel
Fedora Project Contribut