Slashdot Mirror
Two papers On Performance Tuning FreeBSD
avleenvig writes "Finally I have completed my paper on tuning the FreeBSD system for various common tasks, to compliment the paper on compiling and tuning the FreeBSD kernel. These papers were written against FreeBSD 4.x systems but should be equally applicable to 5.x systems on almost all counts. Compiling and tuning the FreeBSD kernel:
http://silverwraith.com/papers/freebsd-kernel.php
Performance tuning FreeBSD for different applications:
http://silverwraith.com/papers/freebsd-tuning.php. You will find within, details on tuning your network, disks, sysctls and kernel for maximum performance under various conditions. "
*BSD is dying
Yet another crippling bombshell hit the beleaguered *BSD community when last month IDC confirmed that *BSD accounts for less than a fraction of 1 percent of all servers. Coming on the heels of the latest Netcraft survey which plainly states that *BSD has lost more market share, this news serves to reinforce what we've known all along. *BSD is collapsing in complete disarray, as further exemplified by failing dead last in the recent Sys Admin comprehensive networking test.
You don't need to be a Kreskin to predict *BSD's future. The hand writing is on the wall: *BSD faces a bleak future. In fact there won't be any future at all for *BSD because *BSD is dying. Things are looking very bad for *BSD. As many of us are already aware, *BSD continues to lose market share. Red ink flows like a river of blood. FreeBSD is the most endangered of them all.
Let's keep to the facts and look at the numbers.
OpenBSD leader Theo states that there are 7000 users of OpenBSD. How many users of NetBSD are there? Let's see. The number of OpenBSD versus NetBSD posts on Usenet is roughly in ratio of 5 to 1. Therefore there are about 7000/5 = 1400 NetBSD users. BSD/OS posts on Usenet are about half of the volume of NetBSD posts. Therefore there are about 700 users of BSD/OS. A recent article put FreeBSD at about 80 percent of the *BSD market. Therefore there are (7000+1400+700)*4 = 36400 FreeBSD users. This is consistent with the number of FreeBSD Usenet posts.
Due to the troubles of Walnut Creek, abysmal sales and so on, FreeBSD went out of business and was taken over by BSDI who sell another troubled OS. Now BSDI is also dead, its corpse turned over to another charnel house.
All major surveys show that *BSD has steadily declined in market share. *BSD is very sick and its long term survival prospects are very dim. If *BSD is to survive at all it will be among OS hobbyist dabblers. *BSD continues to decay. Nothing short of a miracle could save it at this point in time. For all practical purposes, *BSD is dead.
*BSD is dying
NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER NIGGER
But if you ever had analysed *nix systems for performance bottle necks this could only help for some little problems.
If he had read the tuning(7) man page, there would be more information for other parts of the system.
But he had done that what I didn't: put up a website with informations for anybody. So overall: Great Work!
Thanks for the good word :-)
I tried hard to cover things not mentioned in tuning(7). That page alreasdy has a lot of good stuff in it that I couldn't really improve.
I felt there was something missing though and when I first started performance tuning I found that the data I've written about isn't in one single location.
Of course, I'll keep adding to it as I learn more.
Hopefully in the next week I'll have a paper up on what you can do to your server to keep it up during DDoS attacks (or if you're slashdotted ;)
This has gotta be one the easiest ways to boost the performance of your system.
The best place to look for reasons as to why this helps out so much is the FreeBSD Handbook: 9.2 Why Build a Custom Kernel?
(\(\
(^.^)
(")")
*This is the cute bunny virus, please copy this into your sig so it can spread
Damned fool thought he knew something
He just fixed his web site,
he had Raid 0, as Mirroring, Raid 1 as striping.
Someone send this fool to RAID school!
Damned fools!
I wish there were more usefull step by step how-to stuff on the web like this. Too often I look for how to do something and I have to figure it out in bits an pieces.
It difficult for the people in the land between complete noob, and guru. It's like I'm an advanced user that has to admin his own box. ]:3}>
Pretty Pictures!
Some of the procedures described in these so-called "papers" (they are really too short to merit the name) are in direct contravention of the FreeBSD Project's recommended and supported procedures.
The recommended way to build a kernel is with the 'make buildkernel' command, after a successful 'make buildworld'.
Optimization levels higher than plain -O (such as the -O2 and -O3 levels recommended by the article) are known to trigger bugs in some of the inline assembler code in the FreeBSD kernel. In previous FreeBSD versions (that shipped with older gcc versions), they were also known to trigger compiler bugs.
The TOP_TABLE_SIZE option is irrelevant to system performance. Likewise, the NFS_NOSERVER option, although it reduces the size of the kernel, does not affect performance. Conversely, none of the truly important kernel options are explained or even mentioned.
The author mentions kern.ipc.maxsockets and kern.ipc.nmbclusters, but fails to mention kern.ipc.nmbufs which must be tuned to match kern.ipc.nmbclusters (the rule of thumb is one cluster for every other mbuf). Also, the suggested values (2048) are very low - lower than the default (which is computed at boot time as a function of physical memory size) and much lower than what a busy network server will need.
Admins who are truly concerned with the performance of their FreeBSD systems are advised to read the tuning(7) manual page, as well as some of the excellent FreeBSD books available from e.g. O'Reilly.
OK, I know this is very much OT but a busy site such as Slashdot should be able to help me out here. Bear in mind that I'm not trying to start a flamewar or anything; just want some good reasoned responses. Right...
Why should I use FreeBSD over Linux?
The reason I'm asking is this: despite having used Linux for many years, I'm constantly being told by FreeBSD fans to switch to their favourite OS. Some make pleasant suggestions, others act with great zealotry and tell me things I know aren't true. The way I see it is as follows:
Stability - Various BSD fans have told me that it's "more stable" and "crashes less". I can safely say that my Debian and Slackware boxes have _never_ crashed or kernel panicked in five years of use; yes, in comparison to a bleeding edge desktop distro such as Mandrake, FreeBSD is bound to be more solid, but proper, well-designed and thoroughly tested distros like Debian and Slackware are totally rock-solid.
Performance - I've been told by FreeBSD users that their OS is much faster than Linux. To make this judgement myself, I performed a few benchmarks with FreeBSD 4.8 and Linux 2.4.20, and also FreeBSD 5.1 and Linux 2.6.0-test. The differences were negligible, although on my 2-CPU box Linux was the clear winner. 2.6.0-test also showed more responsive behaviour on the desktop.
Hardware support - I had troubles getting FreeBSD running on my laptop. Linux supported the hardware much better, and has a significantly broader range of x86 support.
Software support - It's so much easier to find software that will compile natively on Linux. Yep, Ports are good, but they're nowhere near as tested and integrated as, say, Debian's stable repositories.
Security - Both OSes are pretty secure by modern standards, but I can't see the value in FreeBSD's updating method. With Debian, one simple "apt-get" command is needed to get the latest security fixes. With FreeBSD, a tiresome chore of CVSuping, compiling and installing is required, which is doubly annoying on lots of boxes.
Community - Even when I've researched my problem and read up on the docs, I've had BSD fans act incredibly obnoxiously towards me. That's not good at all.
Long term support - FreeBSD only supports each release for 12 months; this means that users have to upgrade. And although upgrading isn't too difficult, the end result is a slightly different system and difficult to target apps against (new features/bugs/changes is newer Ports releases etc). Meanwhile, Debian has over 2 years support for each release, and Red Hat offer 5 - perfect for corporate adoption.
So those are the criteria I judge an OS on, and while many BSD fans keep telling me to use FreeBSD, I can't see what it offers in the real-world over Linux (subjective licensing issues aside).
What concrete benefits does FreeBSD actually offer? Serious question. It appears that Linux wins in the above areas, but any input would be good to hear.
The papers are great but I'm missing a important aspect of it.
I don't know what kind of performance gains I can expect. It might not be worth the trouble if you gain a few percentage points compared to an unoptimized system. For me if an optimization can make you gain a 10% extra performance (or more!) it begins to be interesting.
The handbook mentions three reasons to compile your own kernel. None of them really makes much difference to performance:
Faster booting: yes, but that's not really performance in the sense being considered.
Saving memory: yes, but it'll only be a megabyte or two. Insignificant on most machines where performance is important.
Hardware support: yes, but that's not performance improvement.
So build your own kernel - it's educational if nothing else - but don't expect anything to run faster as a result. The only reason I build my own kernel is to apply some useful patches: one to spin down ATA disks when they're idle, and one to enable power-saving in the VIA chipset when the machine is idle (makes it *much* cooler).
Roy Horn of the famous "Siegfried and Roy" magic ensemble was interviewed today from his hospital bed. Mr. Horn is recovering from a life threatening tiger attack. When asked about his condition, Roy had this to say,
Hang in there Roy. We're all pulling for you!
what will they do about the smell ?
Due to the troubles of Walnut Creek, abysmal sales and so on, FreeBSD went out of business and was taken over by BSDI who sell another troubled OS. Now BSDI is also dead, its corpse turned over to yet another charnel house.
The numbers continue to decline for *BSD but FreeBSD may be hurting the most. Look at the numbers. The loss of user base for FreeBSD continues in a head spinning downward spiral. In truth, for all practical purposes FreeBSD is already dead. It is a dead man walking. It's a fact: FreeBSD is dying.
The record is clear on one thing: no operating system has ever come back from the grave. Efforts to resuscitate *BSD are one step away from spiritualists wishing to communicate with the dead. As the situation grows more desperate for the adherents of this doomed OS, the sorrow takes hold. An unremitting gloom hangs like a death shroud over a once hopeful *BSD community. The hope is gone; a mournful nostalgia has settled in. Now is the end time for *BSD.
Pre-5.2 RELEASEs ship with uniprocessor kernels. If you want SMP support, you have to recompile.
5.2, and future RELEASEs, will ship with SMP kernels. Due to the added overhead of kernel locking, this cuts kernel performance by about 20%. If you've got a uniprocessor machine, and you're doing kernel-intensive work, youll probably want to recompile.
Tarsnap: Online backups for the truly paranoid
Due to the troubles of Walnut Creek, abysmal sales and so on, FreeBSD went out of business and was taken over by BSDI who sell another troubled OS. Now BSDI is also dead, its corpse turned over to yet another charnel house.
The numbers continue to decline for *BSD but FreeBSD may be hurting the most. Look at the numbers. The loss of user base for FreeBSD continues in a head spinning downward spiral. In truth, for all practical purposes FreeBSD is already dead. It is a dead man walking. It's a fact: FreeBSD is dying.
A full grown stallion's cock, when fully erect, will measure some two to three feet long. It can be three to six inches thick at the base, to about two inches thick at the head. Horses are somewhat different from other animals in the way their cock head works
Horse semen is extremely viscous, if you touch your finger to a pool of it you can draw a thin string of it five to six feet long!
Horse cum has a nice flat taste to it...not at all bitter like man's cum. You can easily drink cups of it with no discomfort.
Young fillys have no objection to someone playing with their pussy's. I have walked up on a pen full of strange fillys at night and they came right up to me and I petted them and felt up their pussys and they just lifted their tales and seemed to enjoy it.
Wild mares or those that have been artificially inseminated are usually reluctant to have sex. The wild ones are used to violent horsecock and the others have had peoples arms in their cunts so they can be apprehensive about sexual events.
Well, that depends on on what you're currently doing with your system. For example on one system which is pretty old (a P233), applying the recompiling performance optimizations let X run much faster. I think they'd be less obvious on a faster box. It really depensd on what bottlenext you're currently hitting.
What We Can Learn From BSD
By Chinese Karma Whore, Version 1.0
Everyone knows about BSD's failure and imminent demise. As we pore over the history of BSD, we'll uncover a story of fatal mistakes, poor priorities, and personal rivalry, and we'll learn what mistakes to avoid so as to save Linux from a similarly grisly fate.
Let's not be overly morbid and give BSD credit for its early successes. In the 1970s, Ken Thompson and Bill Joy both made significant contributions to the computing world on the BSD platform. In the 80s, DARPA saw BSD as the premiere open platform, and, after initial successes with the 4.1BSD product, gave the BSD company a 2 year contract.
These early triumphs would soon be forgotten in a series of internal conflicts that would mar BSD's progress. In 1992, AT&T filed suit against Berkeley Software, claiming that proprietary code agreements had been haphazardly violated. In the same year, BSD filed countersuit, reciprocating bad intentions and fueling internal rivalry. While AT&T and Berkeley Software lawyers battled in court, lead developers of various BSD distributions quarreled on Usenet. In 1995, Theo de Raadt, one of the founders of the NetBSD project, formed his own rival distribution, OpenBSD, as the result of a quarrel that he documents on his website. Mr. de Raadt's stubborn arrogance was later seen in his clash with Darren Reed, which resulted in the expulsion of IPF from the OpenBSD distribution.
As personal rivalries took precedence over a quality product, BSD's codebase became worse and worse. As we all know, incompatibilities between each BSD distribution make code sharing an arduous task. Research conducted at MIT found BSD's filesystem implementation to be "very poorly performing." Even BSD's acclaimed TCP/IP stack has lagged behind, according to this study.
Problems with BSD's codebase were compounded by fundamental flaws in the BSD design approach. As argued by Eric Raymond in his watershed essay, The Cathedral and the Bazaar, rapid, decentralized development models are inherently superior to slow, centralized ones in software development. BSD developers never heeded Mr. Raymond's lesson and insisted that centralized models lead to 'cleaner code.' Don't believe their hype - BSD's development model has significantly impaired its progress. Any achievements that BSD managed to make were nullified by the BSD license, which allows corporations and coders alike to reap profits without reciprocating the goodwill of open-source. Fortunately, Linux is not prone to this exploitation, as it is licensed under the GPL.
The failure of BSD culminated in the resignation of Jordan Hubbard and Michael Smith from the FreeBSD core team. They both believed that FreeBSD had long lost its earlier vitality. Like an empire in decline, BSD had become bureaucratic and stagnant. As Linux gains market share and as BSD sinks deeper into the mire of decay, their parting addresses will resound as fitting eulogies to BSD's demise.
Both articles give nice ideas, but it may not be a good idea to blindly follow their advices.
For instance, polling doesn't always improve things. Past the theory, I tried to enable device polling on a web server running FreeBSD 4.9-STABLE and the performance significantly _decreased_.
Also one of those papers suggests playing with extra GCC optimization flags. Well. First, don't expect your system to get significant speed improvements except for some very specific applications (usually not servers, rather intensive maths work).
Next, FreeBSD is not Gentoo Linux.
Gentoo has tons of users heavily testing all possible GCC optimization and discovering breakages. Those breakages are solved by adding extra patches, by backporting patches from CVS trees or by filtering annoying gcc flags for some piece of software. The whole thing works and it is even reliable because Gentoo Linux was designed that way and people use it that way.
On the other hand, almost no FreeBSD user use special flags. Even nothing but "-O" is explicitely recommended. There is no Gentoo-like workarounds for specific optimization flags. Nothing is really tested with extreme compiler optimization flags. So if you start playing with this, you may hit strange bugs and not a lot of people would have encoutered the same bug. So you're on your own. Your system may be unstable while your actual gain of performance will be near zero (really in a real world, bottlenecks are often disk/io, ram and network latency).
{{.sig}}
anyone who says that never walked into an isp. they must run one linux box at home, with a windoze machine for games. serious comuting there.
i wish anyone who said fbsd was dead/dying just fucking try it for a month. then they would change their mind.
In the DEVICE_POLLING part of the article, he says setting HZ=1000 in the kernel config slows town the clock, but that's wrong... it speeds it up... it ticks ever 1 ms instead of FreeBSD's default ever 10 ms. Last I checked that was speeding up not slowing down.