Slashdot Mirror
High Performance Linux Kernel Project — LinuxDNA
Thaidog submits word of a high-performance Linux kernel project called "LinuxDNA," writing "I am heading up a project to get a current kernel version to compile with the Intel ICC compiler and we have finally had success in creating a kernel! All the instructions to compile the kernel are there (geared towards Gentoo, but obviously it can work on any Linux) and it is relatively easy for anyone with the skills to compile a kernel to get it working. We see this as a great project for high performance clusters, gaming and scientific computing. The hopes are to maintain a kernel source along side the current kernel ... the mirror has 2.6.22 on it currently, because there are a few changes after .22 that make compiling a little harder for the average Joe (but not impossible). Here is our first story in Linux Journal."
Compilers shouldn't need to be compatible with each other; code should be written to standards (C99 or so) and Makefiles and configure scripts should weed out the options automatically.
IMHO This is a great development, for one important reason.
Portability of the kernel.
GCC is a great compiler, but relying on it excessively is a bad thing for the quality of kernel code, the wider range of compilers used, the more portable and robust the code should become.
I know there will be the usual torrent of its-just-not-open-enough rants, but my reasoning has nothing to do with that, it is simply healthy for the kernel to be compilable across more compilers.
It also could have interesting implications with respect to the current GCC licensing 'changes' enforcing GPL on the new plugin structures, etc.
GCC is a wonderful compiler however it has in the past had problems with political motivations rather than technical, and moves like this could help protect against those in the future (some of us still remember the gcc->pgcc->egcs->gcc debarcle).
Of course no discussion of compilers should happen without also mentioning LLVM, another valuable project.
You see, I'm a consultant and am paid by the hour.
:)
I think the point is that ICC has been made "gcc compatible" in certain areas by defining a lot of pre-baked defines, and accepting a lot of gcc arguments.
In the end, though, autoconf/automake and cmake and even a hand-coded Makefile could easily abstract the differences between compilers so that -mno-sse2 is used on gcc and --no-simd-instructions=sse2 on some esoteric (non-existent, I made that up) compiler. I used to have a couple of projects which happily ran on BSD or GNU userland (BSD make, GNU make, jot vs. seq, gcc vs. icc vs. amiga sas/c :) and all built fairly usable code from the same script automatically depending on the target platform.
The over-reliance of the Linux kernel and it's hardcoded options for GCC means you have to port GCC to your platform first, before you can use a compiler which may already be written by/for your CPU vendor (a good example was always Codewarrior.. but that's defunct now)
Of course there is always configure script abuse; just like you can't build MPlayer for a system with less features than the one you're on without specifying 30-40 hand-added options to force everything back down.
A lot of it comes down to laziness - using what you have and not considering that other people may have different tools. And of course the usual Unix philosophy that while you may never need something, it should be installed anyway just because an app CAN use it (I can imagine using a photo application for JPEGs alone, but they will still pull in every image library using the dynamic linker, at load time.. and all these plugins will be spread across by disk)
A few years ago someone figured out that Intel's compiler was engaged in dirty tricks: it inserted code to cause poor performance on hardware that did not have an Intel CPUID.
http://techreport.com/discussions.x/8547
But perhaps they have cleaned this up before the 10.0 release:
http://blogs.zdnet.com/Ou/?p=518
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
They should think about moving to a Java kernel. They could just bootstrap one of the new, clever "Just-In-Time" Virtual Machines at powerup.
These JVMs are able to dynamically optimize the running code in real-time, far beyond what could be achieved by C or C++ compilers, without any performance degradation.
A Java kernel would likely run at least 50 times faster then the very best hand coded assembler - and since the language is completely type-safe and doesn't implement dangerous legacy language features such as pointers or multiple-inheritance then it would be unlikely to ever crash.
Unfortunately, writing an OS inherently requires making use of functionality not addressed in the C standards. If you stick only to behavior well defined by the ISO C standards you *can* *not* write a full kernel. Doing stuff that low level requires occasional ASM, and certainly some stuff dependent on a particular hardware platform. I think that being as compiler-portable as it is hardware-portable should certainly be a goal. The ability to build on as many platforms as possible certainly helps shake out bugs and bad assumptions. But, just saying "clean it up to full C99 compliancy, and don't do anything that causes undefined behavior" would be ignoring the actual reality of the situation, and makes as much sense as porting the whole kernel to Java or Bash scrips.
We were not impressed.
-- Erich
Slashdot reader since 1997
There isn't one, so what you do is use pragmas (I remember #pragma pack(1)) or attributes (__attribute__((packed)) or something similar.
Of course they're compiler-specific but there's no reason that code can't be written wrapped in defines or typedefs to stop compiler-specific stuff getting into real production code nested 10 directories down in a codebase with 40,000,000 lines.
Linux does an okay job of this - but since coders usually reference the compiler manual to use these esoteric pragmas and types, they are usually told "this is specific to GCC" (GCC does a good job of this anyway) so they should be wrapping them by default to help their application be portable and maintainable to future compilers (especially if they change the attribute name or the way it works - as has been done on many a GCC, let alone other compilers).
What usually nukes it (and why linux-dna has a compiler wrapper) is because they're hardcoding options and doing other weird GCC-specific crap. This is not because they are lazy but because the Linux kernel has a "we use GCC so support that, who gives a crap about other compilers?" development policy and it usually takes some convincing - or a fork, as linux-dna is - to get these patches into mainline.
Java is not a "systems language", meaning you don't write operating systems and systems level code in it for very good reasons.
Funny cause Sun already did that like 13 years ago.
One of them being, name me a processor that can run Java bytecode nativly.
The ARM9E.
Depends on the CPU... gcc has reasonable performance on x86, but on ia64 or ppc the vendor supplied compilers have a big advantage. even on x86 icc leads by a considerable margin in some areas, especially on very new processors.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
On the contrary - there is no support for threads or interrupts whatsoever in C99. Sure, there's pthreads and the like - but those are not part of C99, nor can you implement them in pure C99.
C itself (all versions) tries very hard to avoid tying itself to any specific hardware or OS. It even supports weird things like platforms with more than 8 bits in a char, or with reserved bits in their integers. But as a result, it has only the bare minimum featureset common across all platforms imaginable, and this is why it's very hard to write anything useful with only pure C. (No networking, no listing the contents of a directory, no executing any other programs except via system()...)
For most userland applications, C plus some OS-dependent libraries are good enough, of course. Things like the POSIX API can't be implemented in regular C (at some level you have an assembly call to the OS's syscall interface), but if you treat it as opaque, no problem.
But for an OS kernel, things aren't that easy. In the quest for high performance, Linux does all kinds of neat hacks, including things like inlining assembly code into C functions - and later rewriting that code on the fly (google for 'smp alternatives' for more information). It also makes use of CPU-level atomic operations - and exactly which ones are available depend on the architecture. Because of these kinds of hacks, which produce noticeable speed improvements, it is utterly impossible to stick purely to standards like C99.