Slashdot Mirror
Optimizations for Source-Based Distributions?
Kenny Mann asks: "I currently run a Linux distribution called Lunar Linux and it is a source based distribution branched from the original Sorcerer GNU Linux. I've done a bit of research on compiler optimizations and such and was wondering what kind of performance is there really to be had for setting these options? I know that the more options the greater chance of unexpected failures, so my next question is what about optimizing your kernel?" Optimization is tricky, and I think the answer to this question is more complex than "yes, optimize" or "no, don't optimize". Rather there might be classes of applications that are safe to optimize and classes of applications that are not. How do those performance hounds out there feel about optimizing the kernel, however?
Could these distros possibly have geekier names? Why not just name them 'Knights Who Say "Ni" Linux' or '+5 Elven Cloak of Invisibility Linux' and be done with it? At least other distros are named after animals, hats or nothing. fp?
This thread on forums.gentoo.org discusses to death what compilation flags are good to use (mostly gcc 3.x+). Even though you're using a different source based distro, the concepts are all the same.
When a project is given to the opensource community, the optimizations that work should be submitted as well. Possible instability due to such optimizations should be noted, and where expedient, should be fixed.
This is usually taken care of in the form of a makefile. If the author didn't intend any optimizations to be run against the code, they didn't put them in the makefile.
It would be great, however, if there were a project to find out what instabilities happen due to specific optimizations, and either fix GCC so it can more intelligently tell when to optimize and when not to, or get code developers to adopt safe coding practices which will allow the optimizations to be made without problem.
-Adam
Latest versions of GLIBC and Kernel and XFree 4.3 are more important than optimisations.
I'm not an expert on anything, but a friend of mine teaches at a local university, and when we were at my birthday party he mentioned having compiled an old encryption proggy he made (codigo pro) both under the ancient Turbo C and the most recent version of GCC. I dont know what settings he used, but he said it was like 3 times faster for the same code running on the same platform. :)
Of course he made the software ages ago when he knew very little about programming compared to now, and the source was probably lacking every possible optimization that an expert coder would introduce, so YMMV.
I never mess with settings when compiling programs under Linux, since I'm pretty much a newbie and often have trouble installing things from source.
However, an expert friend of mine who has a very old PC always goes for an LFSish setup where he compiles everything and tweaks all the settings by hand. He claims it works miracles, and I believe him. I know from my old MS-DOS graphics programming experience that small source and compiler tweaks could be the difference between a professional-looking program and a crappy amateurish app full of flicker.
I could ramble endlessly about all the optimization success stories from my youth, starting with the classic "DEFINT A-Z" QBasic trick, then progressing through Turbo Pascal compiler tweaks and finally achieving C + ASM goodness, but I shall not bore you any further
Could someone explain the point of source distros to me? It's about optimization, which equals saving time, right? How much time has this person spent downloaded megabytes of source code and poring over the docs trying to get the compiler options just so? And all to save a measily .001 microsecond per day?
Check out other distro's forums and mailinglists. Sorcerer, Gentoo, and many others of these squeeze-everything-out-of-the-compile distros have great resources for this type of question. The Gentoo forums (https://forums.gentoo.org/) have quite a bit of talk about this sort of thing, but are more geared toward the portage system. However, threads about -fomit-frame-pointer and -funroll-loops can be enlightening - do they really give a speed boost, or do they just take up more disk space? Search away :)
I have seen a thread about optimizing the kernel on the Gentoo forums (guess which distro I spend time with?), but these seem to be much more hassle than they're worth in the long run in terms of segfaults and crashes and the like.
Someone already commented on this somewhere, it could have been FidoNET or Slashdot, so I'll paraphrase. Anyhow, the upshot is that there are two things about Linux.
The lady or gentleman who finds it more entertaining to tweak the kernel than play Quake is that much geekier and worthy of respect in my book. It's just important that such a person realizes, aside from gaining some small amount of technical knowledge and problem-solving skills, such an act is also little more productive than a fragfest.
You like splinters in your crotch? -Jon Caldara
There should also be some sort of massive Linux bugzilla, with maybe an automated reporting agent like Windows and Netscape have. Applications can be written to call that program when an error occurs, or whenever the user wants to file a report. The data wouldn't even have to be looked at by humans, just be collected to tell what the most common problems are or maybe for a developer to search through later. I suppose the massive bandwidth costs would prohibit a project of this sort, but I still think it would be useful.
"I assumed blithely that there were no elves out there in the darkness"
2. Do re-compile your C library however. Most of your applications spend a lot of their time executing code from the C library.
You are being MICROattacked, from various angles, in a SOFT manner.
First answer your question on kernel - the kernel optimization by default is good enough, e.g. it uses -Os instead of -O3 because some program like kernel usually run faster with less memory trace. You might want to optimize individual modules, though.
:), you could try some crazy optimization. The hardest thing to decide is that which optimization flags in gcc work best for your system. Should you use all optimization flags? Will these flags break your system?
:)
For the rest of the packages(I know you didn't ask, but it doesn't stop me.
Inspired by rocklinux, I've tried to benchmark individual optimization flag, i.e. test each flag and discard those flags which don't give your system performance gain. Of course, the script used in link above is pretty old and you must modify for gcc3.2+. Thanks to lameass filter I won't post my script here.
That sound like wasting of time but the result is satisfying. The max. yield I could gain is as much as 19% in comparing to plain -O3 optimization. Here are the result:
vendor_id : GenuineIntel
model name : Mobile Pentium MMX
flags : fpu vme de pse tsc msr mce cx8 mmx
gcc version 3.2 (i586-pc-linux-gnu)
Result: '-O3 -march=pentium-mmx -fomit-frame-pointer -finline-functions-fcse-follow-jumps -funroll-loops -frerun-cse-after-loop - frerun-loop-opt -fno-cprop-registers -funroll-all-loops -maccumulate-outgoing-args -fschedule-insns'
Performance gain(compare to -O3 only) ~ 9.9%
vendor_id : GenuineIntel
model name : Pentium III (Coppermine)
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse
gcc version 3.2 (i686-pc-linux-gnu)
Result: '-O3 -march=pentium3 -fomit-frame-pointer -finline-functions -funroll-loops'
Performance gain(compare to -O3 only) ~ 13.7%
vendor_id : AuthenticAMD
model name : AMD Athlon(TM) MP 2000+ (a dual CPU system)
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse syscall mmxext 3dnowext 3dnow
vendor_id : AuthenticAMD
model name : AMD Athlon(TM) MP 2000+
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse syscall mmxext 3dnowext 3dnow
gcc version 3.2 (i686-pc-linux-gnu)
Result: '-O3 -march=athlon-mp -fomit-frame-pointer -finline-functions -fforce-mem -s -funroll-loops -frerun-loop-opt -fdelete- null-pointer-checks -fprefetch-loop-arrays -ffast-math -maccumulate-outgoing-args -fschedule-insns'
Performance gain(compare to -O3 only) ~ 19.6%
19.6%!! If you asked me, it worths it to optimize your desktop; but to the server, you'd like to have it running stable than to have it running 19% faster, you can trust me on that.
PS. In the processing of testing, I found some flags are dangerous and better use with care: -fmove-all-movables, -frename-registers and -malign-double. I suspected that they broke my file-util, which corrupted my entire fs. Just be careful.
delayed branching? Now that's something that can save a whole lot of time in some kernel code. It can also confuse the hell out of someone looking at assembly code!
I agree with you: everything's relative.
gcc optimzations for the kernel were discussed recently on this thread
One place where gcc isn't quite standards-compliant is with IEEE floating point math. This isn't a problem usually, but for a library I'm writing, I have to enable -ffloat-store or I'll get slightly wrong results. Intel's C compiler is worse on this front by default though, since you have to enable two or three flags to get standard behavior
The ocean parts and the meteors come down
Laid out in amber, baby.
Sure, just write in the "correct" way, the code will perform well even without any tricky optimization. However, although you know what the computer will do to execute your C code, many new programmers do not. They will just use doubles when ints should be used, use pow() when calculating 1 Knowing how to write "clean" code is the most reliable way to give your program acceptable performance. Compiler optimization is much less important for many people (for GCC, you can't get much faster than -O2 or -Os for most code), comparatively, and one definitely shouldn't rely on them (like changing a/5.0 to a*0.2) too much.