Slashdot Mirror
NetBSD 2.0 vs FreeBSD 5.3 Benchmarks
diegocgteleline.es writes "According to OSnews, Gregory McGarry benchmarked NetBSD 2.0 against FreeBSD 5.3 and found that NetBSD 2.0 surpasses FreeBSD 5.3 in most of benchmarks. The machine used for benchmarks is a 3 Ghz P4 so it doesn't reflect the improvements of FreeBSD 5.3 in the SMP arena, which is where their developers have put their efforts in the last years and where NetBSD is still using a "big-lock" model. Newsforge is also carrying a interview with some NetBSD developers about the technology behind NetBSD 2.0."
FreeBSD's focus is typically on overall throughput of massive server, with responsetime as a tradeoff, while this benchmark with all of its timings of single OS operations is more something realtimish?
If NetBSD aspires to be a RT focussed distro/OS, they'd better have benchmarked it against some Linux with RT patches?
As an OS junky, I have used both of these BSDs, and am very impressed with them. I use FreeBSD on my P4 3GHz laptop, as Project Evil lets me use the WiFi card nicely, and ACPI kind of works. I use NetBSD on my Sparc Station 5 and my Athlon desktop, and I have found both to be wonderful for desktop use and as servers. I salute both of the groups of people who make these OSes, and wish that I had more time to contribute.
I would assume that FreeBSD has a "default install" as tested that supports at least 2-way SMP, given the focus that has been made in FreeBSD. This compared with NetBSD could make a huge difference, is NetBSD's default install also setup for SMP. It is a known fact that using an SMP compiled kernel on pretty much ANY OS will result in slower performance on a single processor, as the added overhead of locking data structrues will have to occur.
It is important to note that all of the tests that were performed where done on a uni-processor workstation.
The blanket statement that "NetBSD 2.0 has better threading and process management and network latency than FreeBSD 5.3" does not hold water when the test suite is run on 2 and 4-way SMP systems. FreeBSD 5.x is an amazing engineering effort in which various parts of the kernel have been locked down and decent thread concurancy can occur on multi-proc machines. Part of the latency that you see in these benchmarks are due to the mtx_lock() and mtx_unlock() overhead that is incurred.
It is also important to note that P4-systems with HyperThreading (As the one used in these tests) have been the "bastard child" on FreeBSD. For the longest of times, compiling anything with CPUTYPE=p4 would produce broken code (In all fairness, this was mostly due to a set of bugs in GCC 3.x). Significant work was put into 5.3 to ensure that the Pentium 4 lived up to the Tier-1 platform robustness standard. In short, it would be fun to see these benchmarks be run on i386/pentium3, i386/Athlon-MP, amd64/opteron and Alpha as well!
In the freebsd mailing list there was a troll using this test to come down on a couple of highly skilled developers. /. folks to enjoy:
;-)
Being lazy by nature I copy/paste my respone in the mailing list for the
Benchmark are made to be put into perspective, although everybody has a right to say what (s)he wants to say, this doesn't mean that you have to say it.
It seems to me that FreeBSD is focusing it performance onto MP 64bit processors. As we can see in the benchmark it has in comparison to other projects a negative impact on UP system.
But just put it in the perspective of processor developments, AMD (followed by Intel) is heading towards a multi-core 64 bit systems, what probably becomes mainstream at the end of next year.
With this technology the FreeBSD model could have a winner on there hands.
Doing the same job but not having the same philosophy on it, is always inefficient, but in the real world it leads to the Darwin effect.
What means that the best solution gets there chance of survival against the test of time.
Luckily these are all BSD's, good solution will spread, just take a look at PF.
OpenBSD has a good user base but not compatible to the sum of user base of the other BSD's. Still PF has spread there wings beyond the user base of OpenBSD.
FreeBSD is just a name for an OS, if any other OS can give me more "bang for the buck" and provides a full solution, I will use it.
Be it DragonFly/Free/Open/Net, MacOsX, GNU+Linux, Windows or any of the other hundreds of OS'es out there.
I like the BSD license so I will tend to stick to "gratis" BSD OS'es.
All of the disagreements in development is a healthy process to make sure the sort "BSD" an not the specie *BSD will survive.
Sure I have my disappointments about some decision, but hey so is live, this ain't a fan club for next biggest boy band (he he BSD-Boyz), where using an OS to provide solution for our technologic problems, you favor your solution but don't blind yourself.
And when you don't blind yourself you re-evaluate your situation and move forward with the best solution for your problem.
Sure it is a pain to migrate my boxes to another OS (well that is the fun part) and do some massive rewriting of my documentation, but thats my job and I tend to like it. Just standing still and not progress has its attractiveness when you had a very rough ride, but it gets dull very soon and then you find yourself back on the dirty tracks.
But these are my opinion only, however I like to share them
I've been an avid follower of the developments in FreeBSD for around 5 years now, so my overview of the entire history of "glue that binds" FreeBSD together isn't complete. That said, I've come to be a bit disappointed at how events in the last 18 months or so seem to be pushing the project in a direction that has made things more difficult, instead of more successful, that has shown distain for experience and quality and made FreeBSD a platform for large ego's to push their personal projects down everyone's throat.
The statistics sample from 2001 over a year was a cheap attempt to minimize Matt's contribution to the project. The reason why he has been mostly silent is probably one of the most prominent signs of his superior maturity. The fact that the official defense (mostly fronted by Greg, atm) he wasn't such a substantial committer is crap, for the most part. If one wanted to go by the stats, Jeff Robertson (sorry if I munged the spelling) would be one of the key committers, and his UMA system isn't even entirely ripe yet, it's just been committed within the sample timeframe. That suddenly phk is at the top of the list, is simple a result of his newest attempt to add another large chunk of bit rot to the project that he can later claim not to have time to maintain "unless someone is willing to pay for my time" (like the atm bits, the half-finished devd monster, et.al.) One can hardly get him to look at his malloc bits, that put his name in lights at some point in the long past.
Matt didn't contribute because he was convinced that that the smp development direction that was chosen (my impression at least from the archives and my fading memory) was overly complex, too complex for the number and talent level of the contributers involved, and that it would delay a release from the -current branch significantly. So he was right. I'll almost bet that that was a constant sore for John, who still hasn't gotten his long-promised, but little delivered re-entrant work done, but he always had time enough to object to any other commits that might help along the way. Strangely Julian and Matt could work together. One might attribute certain commits to both Matt and Julian (if that would matter anyway, since -core is interested in proving the opposite statistically).
If the issue here had anything to do with IPFW, then you all better get out your C-coder hats and take a little more time to fix that rotting pile of muck that has been the standard broken packet filter interface for FreeBSD long past its possible usefulness. A packet filter with no central maintainer which is subject to once yearly random feature bloat through some wild university project from Luigi. The brokenness that Luigi introduced (and the repository bloat through backing out and recommitting, ad absurdum) was probably no less a threat to security than anything Matt did. If the security officer was to be blatantly honest with himself, ipfw would be marked broken for either a full audit or full removal (just port obsd's pf or something that someone actually actively _cares_ about).
You've alienated Jordan, Mike, Bill Paul (for all I can see), Greenman, you constantly rag on Terry, even though he's seen and done more with FreeBSD than most of you, O'Brien is on the verge of quitting (since he, like I, am not convinced that GEOM is anything more than an ego trip that will never be completely maintained or usefully documented). There are certainly others, too, that have attempted to make technically correct contributions, but didn't fit into the sort of paranoid "glee club" that core would like to have around them. You guys lack the talent to steer the positive from Matt into the project and let the crap fall by the wayside. I'm not saying Matt's rants are the most intelligent thing he's done, but he's sat by the wayside and watch the superstars beat up the code to a point where it's less stable, slower, and more bloated than it ever was. I, for one, can understand his frustration (as I can with Mike's, Jordan's, and a few o
Wow.. nice job, mod. :-/
I just went from FreeBSD 5.3BETA3 to 5.3-RELEASE-p2 on my fileserver, and moved to a new hard disk at the same time. Somewhere between beta3 and -release, the decision to make SCHED_4BSD the default scheduler was made, and SCHED_ULE won't compile because it has an #error macro.
Anyways, the point of this is that because I was going to a new hard drive, I was doing a lot of dumping and paxing to transfer things over, some before installworld, and some after. I noticed once in 5.3-release that heavy disk activity made the system far less responsive, although the throughput did in fact seem a little higher. This _is_ all subjective talk, but it was serious enough that opening a new ssh session on the machine took substantially longer than before.
I want to see some benchmarks on how various systems behave when the disk activity is up, and how much throughput they can actually accomplish. I know that schedulers give disk-bound processes more run-time in order to finish quickly, and it seems almost like sched_4bsd is exaggerating that. It would be interesting see some numbers attached to my predictions.
Yeah Yeah
Just open the main page and what you'll get?
Another root exploit for linux
hehehe
so much for the 0.17 bugs
according to Carnegie Mellon University's CyLab Sustainable Computing
Got a link for this claim?
3 people just replied to a disinforming troll.
Finally, a *BSD related discussion on /. that's constructive and objective, with people giving valid points of view and opinions.. Keep it up!
I just want to inform whoever modded this crap up (at +3!) that this is an old troll. It first appeared on the FreeBSD mailing lists in january 2004 with the false signature "Maxim Hermion", in order to imitate the name (and the email address) of the FreeBSD committer Maxime Henrion.
This is what the real Maxime Henrion says about it, in a reply to that troll message.
Please check the sources before modding things up...
For those bitching that a microbenchmark's not worth beans, I ask that you beat this with FreeBSD (it's a bit old, but hey):
. se/LSR3-m/
Aparently the folks from the Swedish University Network (SUNet) at Lulea managed to break their previous Internet 2 Landspeed record for both single and multiple streams, using NetBSD again. Comparison:
Old record:
* 838860800000 bytes in 1588 real seconds = 4226 Mbit/sec o
* Distance: 16,343 km (10,157 miles)
* 69.073 Petabit-meters/second (12% increase)
New record:
* 1966080000000 bytes in 3648.81 real seconds = 4310.62 Mbit/sec
* Distance: 28,983 km (18,013 miles)
* 124.935 Petabit-meters/second (78.6% increase)
The big difference in distance and thus the record itself is due to suboptimal routing, crossing the ocean three times. Nonetheless, thanks to a newer version of end machines' operating system -- a prerelease of NetBSD 2.0 -- and some newer routers, this record was achieved on a production network just in the previous case. See the project pages for single stream and multiple streams for more information!
http://proj.sunet.se/LSR3-s/
http://proj.sunet
At the risk of burning my karma, I don't think this is acceptable.
:-/
The parent (modded at +4 right now!) made its first appearance on the FreeBSD mailing lists, with the false signature "Maxim Hermion", in order to imitate the name (and the email address) of the FreeBSD committer Maxime Henrion.
Here's what Maxime Henrion (the real FreeBSD committer) writes about it, in a reply to this troll message.
This being said, one can suppose that nobody who is intellectually honest would call this anything other than a defaming piece of crap, authored by a worthless troll.
Still, on this forum, a piece of crap like this has mysteriously managed to reach the +4 level, by doing nothing else than gratuitously defaming FreeBSD.
To the people that modded this up: nice job indeed... Maybe, according to your agenda, it really *is* a nice job.
--
Being able to read *other people's* source code is a nice thing, not a 'fundamental freedom'.
How does a recent OpenBSD (3.5 or higher) on a typical non-SMP PC compare to a recent NetBSD or FreeBSD?
I hear it is slower but I'm wondering if anyone has any actual experience.
http://www.thebricktestament.com/the_law/when_to_
Someone pointed out that I was wrong about the origin of that post - the original was actually written 2 years ago (Feb 2003) by Shaun Jurrens, a FreeBSD user (here's what he says about the troll I previously linked).
I apologize, but I googled before talking, and the oldest result I found was the troll I linked in parent (Jan 2004).
Still, I don't think that the text of a 2-year-old rant, copied and pasted by a troll, is objectively worth any modding up. And FWIW, what I think of the ones who do it doesn't change very much.
- NetBSD (fastest)
- OpenBSD (happy medium)
- FreeBSD (slowest)
Of course, by this time next year the rankings might change, as all of these operating systems are still under development.Thanks a lot. I kinda figured this, but I'd heard such bad stuff about OpenBSD's performance relative to FreeBSD, I figured it might still be last.
But have you actually run them youself on similar hardware?
http://www.thebricktestament.com/the_law/when_to_
since OpenBSD doesn't have UBC yet (NetBSD has had it since 1.6, linux has had it since 2.4, and FreeBSD has had it since 2.0.5), OpenBSD is slower than even FreeBSD 5.x for applications whch need large file cache. e.g. web server, nfs server, etc....
I am not trying to sling shit on OpenBSD here, I've been using it as my primary desktop, net connected servers and firewall OS for very many years and will continue to do so. However I had to do something new recently that caused me to switch to NetBSD 2.0 for this project due to massive performance gains that could not be ignored (this may be due to my scripts being far from optimum at the moment, but)...
I have been doing some intensive data mining on some large datasets (500MB - 1GB) on a 2GB RAM PC.
OpenBSD 3.6 was taking days to do what NetBSD 2.0 was doing in under an hour and it appears to have mostly come down to NetBSD's disk caching. Multi passes at the large files show the disk access light come on just once on NetBSD and then the analysis flies. After seeing OpenBSD 3.6 performing so much slower with this task, I tried Fedora Core 3 and found it to also perform very much slower than NetBSD. In both cases, disk light is hard-on for the very slow duration of the task (which I did not finish for the Fedora machine because it was quickly apparent that it was also very slow).
Both BSD installs had noatime, softdeps and same partitioning for the filesystems being thrashed. I'd be curious to see if the gap closes much when I switch this over to perl. Disk performance even for single pass time trials in NetBSD are better than OpenBSD.
From now on, NetBSD will be my number cruncher. I am really shocked. I expected a difference, but not such a huge difference.
I thought Fedora Core with a 2.6 Linux kernel and disk caching would have kept up or perhaps even been better. Any config tips for that? I'd like to give SuSE 9.2 a go too... Regardless, NetBSD has really won me at the moment for disk intensive tasks.
As an IDS running snort, I've had problems with the FreeBSD nge driver. I need these NICs for monitoring gigabit links. Simply "upping" the interface caused FreeBSD to panic. I posted here and there, and opened a problem report, but got no replies. FWIW, I never saw a kernel panic until I used 5.3, but I do acknowledge that the technology added is new and results may very.
In the end, I am unable to use FreeBSD 5.3 for servers. I think the new SMP stuff is broken and unproven. I will continue using the 4.x FreeBSD line since it's much more stable.
One last comment, I must admit that in order to get something stable for my IDSs, I installed Gentoo. NetBSD might be a reasonable choice too.
Same old FUD, that has been disproved countless times...
OpenBSD doesn't turn on softupdates by default - you have to do it manually.
Neither does NetBSD actually. And if you look at his post another time you'll notice he acknowledges their functionality. The hard fact is NetBSD is a speed daemon.
Sam ty sig.
It's a pity that DragonFlyBSD wasn't benchmarked in place of FreeBSD.
{{.sig}}