Slashdot Mirror
Measuring The Benefits Of The Gentoo Approach
An anonymous reader writes "We're constantly hearing how the source based nature of the Gentoo distro makes better use of your hardware, but no-one seems to have really tested it. What kind of gains are involved over distros which use binary packaging? The article is here."
ccache and distcc are software, and they can make a huge (as in several orders of magnitude for things like mozilla) difference
The best way to optimize gentoo is too bootstrap it yourself. This yields impressive performance gains. In my case I settled on these flags:
-mcpu=pentium4 -march=pentium4 -mmmx -msse2 -Os -fomit-frame-pointer -pi
pe -fforce-addr -fforce-mem -ffast-math -mpush-args -mfpmath=sse
The other reason I use gentoo, is that it's easy to stay current. Once a week I do an emerge sync, followed by an emerge -u --deep world. The Gentoo package management system rocks.
Portage can be used to install binary (precompiled tbz2 packages of ebuilds).
.deb packages or slackware .tgz packages.
From emerge --help:
--usepkg (-k short option)
Tell emerge to use binary packages (from $PKGDIR) if they are available, thus possibly avoiding some time-consuming compiles.This option is useful for CD installs; you can export PKGDIR=/mnt/cdrom/packages and then use this option to have emerge "pull" binary packages from the CD in order to satisfy dependencies.
--usepkgonly (-K short option)
Like --usepkg above, except this only allows the use of binary packages, and it will abort the emerge if the package is not available at the time of dependency calculation.
You can also, of course, emerge rpm and install any RPM packages. I'm not sure about debian
Gentoo is also accept pre-orders for it's upcoming 1.4 release. Information can be found here, at the Gentoo Store.
They even have precompiled packages optimizaed for Athlon-XP's - drool!
I have been using Gentoo for months now and will never turn back. Little of this has to do with performance and 99.9% of has to do with Portage. Package management, dependency checking and the lot are SO great. Secondly is where performance comes in. Without proper CFLAGS you might as well ignore this. On my Athlon XP 2800 I have this:
CFLAGS="-march=athlon-xp -O3 -pipe -fomit-frame-pointer -mmmx -msse -m3dnow"
In some simple tests I have done I have seen this as worth while. I have two pages I have created that might be worth a read:
CFLAGS Guide
Is -mmmx and such worth it?
Hope you enjoy these reads.
Unstable Apps: Our Android Apps Don't Suck
Looks like someone failed to set their USE flag properly. If you have it set right you will get support for all you want. Or if you do "emerge -vp packagname" before doing an actual emerge you can see what optional flags aren't getting used. People that use Gentoo but don't read the portage/emerge/use documents are asking for this. Gentoo isn't for all, it is only for the willing.
Please go here and reas as much as possible for installing Gentoo so you don't do something stupid.
Unstable Apps: Our Android Apps Don't Suck
It's not part of the main distro, but there is a kernel-multimedia-2.4.21.0.16mdk-1-1mdk.i586.rpm in Mandrake contribs. Check it out if you want a more responsive kernel.
One of my favorite uses for Gentoo is optimizing for size rather than execution speed. As you say, the CPU is rarely the bottleneck these days, but loading files from the disk can be a factor in a program starting up. I've done the benchmarks, and some rather large programs see significantly reduced load times when optimized with -Os.
Watch this space for an excellent Gentoo installation tool. This guy is doing some pretty great work (pretty much by himself) making the gentoo installation quite literally a five step process. He plans on adding GUI support soon and gentoo has a hardware autodetection kernel used by glis, so you're set.
The biggest problem with APT-rpm that I have is that APT isn't very "apt" when it comes to installing the new redhat kernels. I prefer yum (http://linux.duke.edu/projects/yum/). I haven't used apt in a while, but I got tired of typing rpm --rebuilddb after 75 updates I installed on my system. I don't know if they ever fixed that bug.
After running yum for most of a year, I definitely prefer yum to apt-rpm.
That said, I find it interesting that there's really no reason an APT-rpm or Yum can't be modified to do essentially the same thing that emerge does with Gentoo.
I've had my share of source RPM's that wouldn't compile correctly without some modification to the spec file. But I would like to see someone modify yum to use the source rpms, install all the devel packages and then install it. As an added benefit red hat, mandrake would make better source rpms explicitly for that purpose.
You obviously don't use GIMP or analyse OS mapping data much. I have the RAM to get the info into memory, IO is not an issue for much of my work.
Having said that, portage is the main reason I've converted all my machines to Gentoo; it's just not a serious option to go back to RPM based systems after using it for a week or so.
TWW
"Encyclopedia" is to "Wikipedia" what "Library" is to "Some people at a bus stop"
Mandrake while my favorite choice, doesnt include the best pre-emptive kernels.
L icense: GPL :
You mean like this one (from contrib for 9.1)?
Name : kernel-multimedia-2.4.21.0.16mdk
Group : System/Kernel and hardware Source RPM: kernel-multimedia-2.4.21.0.16mdk-1-1mdk.src.rpm
Packager : Danny Tholen
URL : http://www.kernel.org/
Summary : A preemptible Linux kernel, which reduces the latency of the kernel.
Description
This kernel includes patches useful for multmedia purposes like:
preemption, low-latency and the ability for processes to transfer their
capabilities.
The preemtion patches allow a task to be preempted anywhere within the kernel,
using spinlocks as markers for non-preemptibility regions. The resulting
system response is greatly increased, with measured average latencies under
1ms. Andrew Morton's low-latency patches fix the remaining points in the kernel
that cause latency. The setpcap patch allows suid root processess to transfer
capabilities to non-root processess, and so making it possible for user
processes to run with realtime priority.
[some uninteresting fields removed in aid of the lameness filter]
The next one for 9.2 contrib will most likely have the O(1) scheduler also.
Preemptive multi-tasking just means that a process can be interrupted at any point and another process, or the OS, can make use of the CPU. Preempting the kernel means that actual kernel code can be interrupted.
There are other distributions that include kernel patches as packages. Debian, for example, let's you patch your kernel with a compile option passed to make-kpkg.
Changing the CPU multiplier will not give you what you are looking for as it will likely change your FSB speed and L1/L2 cache access rates. The most common bottlenecks on systems are heirarchical IO bandwidth related. For example, having scads of RAM for buffer/cache will help you with disk IO woes.
Compiling binaries with optimization for a particular processor help with i-cache and d-cache utilization. The fewer instructions fetched (or the order in which they are fetched) makes a big difference in performance.
Boosting CPU cache size (up to practical cache limits), increasing FSB speed and avoiding disk IO are much more significant than CPU M/GHz.
Compilation, especially optimized (-0X) compilation is _VERY_ CPU intense. If you have enough RAM to avoid the disk thrashing caused by writing numerous intermediate files you will peg your CPU.
Most user activities (aside from games) on computers are not bottlenecked by CPU but by various heirarchical IO constraints and hence the previous poster was correct that the CPU is not a significant bottleneck on modern systems.
Here's a script that helps you determines which CFLAGS are compatible with your CPU:
Speaking as someone who used Debian ("unstable") for years and switched to Gentoo a few months ago....
/usr/portage. Ebuilds are fairly straight-forward; it doesn't take too long to learn how they work so that you can make simple edits, or even create your own. Also, ebuilds for old versions usually hang around for a while, so if a program is working fine for you and then there's a bug in a new version, you can downgrade easily.
/usr/local/portage, and they will be handled gracefully by emerge. And /usr/local/portage takes precedence over the stuff in /usr/portage, so you can use it to replace existing ebuilds in a way that won't be overwritten next time you "emerge sync".
First of all, Gentoo has definitely improved since you tried it. It has worked quite well for me, and I can't recall anything failing to compile. I imagine that if something as significant as gcc were broken, the problem would probably be rectified rather quickly. I worry more about fringe programs that few people are using.
Anyway, my answer to your question of whether Gentoo is like the "unstable" branch of Debian would have to be "yes and no".
It's like the unstable branch in the sense that the newest versions of software are easily available to you soon after they are released. It's also like the unstable branch in the sense that you can try out that new software before it has undergone weeks or months of testing with the rest of the system, and like Debian unstable it is best for the geekier crowd who can deal with unexpected problems here and there (I'm being somewhat hypothetical here, because I haven't really had any such problems in Gentoo).
It's also like Debian unstable in the sense that it's actually usually more stable than the released versions of many of the rpm-based distros. (Though it's been a while since I last used Mandrake or RedHat, so perhaps they have improved their stability as well?)
It's unlike Debian unstable in the sense that you have more power to fix the problems that may come up (at least problems that can be blamed on the distro and not on the software itself). I felt kind of helpless with Debian unstable when dpkg would choke on a deb. You can't do much (that I know of) to edit an already packaged deb, and getting the src deb to try to find the problem seems like kind of a daunting task for a non-Debian-developer.
In Gentoo, if there's a problem in an ebuild and you don't want to wait for it to get fixed... well, you have all of the ebuilds right there on your hard drive under
One of the features I like is that you can put your own ebuilds in
In Gentoo you are really never helpless.
You very well may know this already, but Daniel Stone has an unofficial apt repository with XFree86 4.3.0 packages. I've been using them for months with great success.
:).
http://penguinppc.org/~daniels/xfree86/README
If you just like compiling X, or have some other reason to, forget I said anything
Game... blouses.
As some worthless anecdotal evidence, I have never had a problem with the stable packages, and i've been using Gentoo for nearly a year. As for the unstable, most of the problems I have had are build/configuration errors.
debs are actually renamed ar archives that conform to a specification and have been renamed. I have come across guides on unaring a deb,altering it and then packing it back up. I've never had to do it but the guides on how to do it didn't seem too bad.
I've never had a dependency problem I couldn't fix in Debian with a little noodling around with the system. On the other hand, I do sometimes recompile source debs to get options that I need. For instance, I deploy Netatalk with dhx authentication. I have to install the crypto devel libs and tweak the rules file to get it. Thankfully, I can build on one box and deploy the custom deb whereever needed.
I suspect that I could get most of the benefit of Gentoo by rebuilding the kernel, glibc, and maybe the xlib source debs with i686 options. At some point, I would like to see where the most benefit is reaped from custom compilation.
Actually I think the key sentence of the article was this:
Doh! No wonder it sucked.
O3 turns on things like inlining that are only worthwhile in certain circumstances, but are often counterproductive. So the results aren't surprising in the least.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
Seriously though, doubling the access speed of your RAM is likely to do more good on that sort of task than doubling the CPU speed.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
What exactly are you trying to say?
-Os decreases cache misses, and that's just as important on a P4 as on any other CPU.
Opening applications is something most users probably do a lot more often than running bzip.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
prelink can help with this. google for it and you should find out about it or look on gentoo.org under the docs section. Prelink support is built into portage :)
I usually say: Gentoo is for users who can read.
Less is more !
Agreed. Back when I was a hardware tech in the early 90's, I recall building pools of identical machines for customer orders. For burn-in, I would loop benchmarks for 24 hours before shipping them out. There was typically 1-2% difference in identical systems.
People tend to forget the complexity of a PC and the inevitable, microscopic differences each part made. Thus differences in resistance, heat generated, and performance.
Also, I don't recomend Gentoo for most users- I am not a fanatic, I just like configuring things myself so when something messes up, there's only one person to blame (go slackware!).
I'm all for folks recommending things they enjoy. The only thing that upsets me is the repeated number of Gentoo users that come on Slashdot or Usenet and make claims that simply are not true about the benefits distro. I see a *ton* of Debian fans pushing their own distro, but usually it comes down to someone claiming an ideological difference or liking the distro's packaging policy -- you don't see a bunch of claims that Debian is much faster or more stable than other distros. It hurts all Linux folks to have a lot of false information floating around, and it really doesn't do Gentoo any more good than it does to have Linux folks making outrageous claims about Linux's capabilities.
What you should note is that there ARE performance gains, though slight.
[shrug] That's probably true. -fomit-frame-pointer, at the least is pretty sure not to hurt you aside from the inability to debug.
I didn't know there were this many gentoo-haters out there, but I definitely see only good things from this distribution. No need to bash it nor me. That's getting a little childish.
I don't think I was bashing Gentoo -- I specifically said that it was a "worthy distro". I may have been slightly harsh towards you, simply because of my frusteration with the behavior of a large number of Gentoo users on various forums, but I don't think it was particularly out of line, and I pointed out that I've been in the same boat WRT compile flags.
What I was saying was Gentoo is definitely a distribution to try for new and intermediate users to get to know how their system works and what can be done (e.x.- what services are unneeded) to speed it up.
[shrug] I'm not saying it's *bad* for them, though every distribution I know of (well, I don't know precisely how Debian deals with source, so perhaps not Debian) provides a pretty good learning system. On rpm based systems, you can just create ~/.rpmrc, modify buildarchtranslate, fill in an OPTFLAGS field, and then use rpm --rebuild on a SRPM to do the same rebuilds that you'd do on Gentoo.
My other real irritation is the number of (a long time ago, mostly Slackware, now mostly Gentoo) users that run around telling would-be Linux gurus that any advanced users *clearly* use a system in which they build their own packages. This tends to put these people in a position of feeling that they need to use a particular distro to prove their tech balls. It's not a particularly friendly environment for any new users. (They ignore, of course, the fact that Linus uses KDE on SuSE, and Alan Cox GNOME on Red Hat -- and both of them seem comfortable).
Does that make Gentoo bad at all? No. Of course not. I'd just like to keep the twin nastinesses of "Bob sucks, because he only uses distro X" and "Distro Y beats the shit out of other distros when it comes to performance/stability/other inflated claim" to a minimum.
May we never see th