Slashdot Mirror
Linux 2.6.17 Released
diegocgteleline.es writes "After almost three months, Linux 2.6.17 has been released. The changes include support for Sun Niagara CPUs, a new I/O mechanism called 'splice' which can improve the performance greatly for some applications, a scheduler domain optimized for multicore machines, driver for the widely used broadcom 43xx wifi chip (Apple's Airport Extreme and such), iptables support for the H.323 protocol, CCID2 support for DCCP, softmac layer for the wireless stack, block queue IO tracing, and many other changes listed at the changelog"
I have this now installed in my dual core AMD and the difference is noticable. X is noticable faster, as is my video editing stuff. Good work guys!
Humor from a Genetically Molested Mind
It's good to know that even in this day and age of faster and faster computers there are still people who care about speed and efficiency instead of simply waiting for hardware to solve their problems for them. I do have one tiny complaint though, and it is that some of the performance gains are only possible by using new system calls. This is bad for three reasons:
1- More work for developers, some of whom may never learn about these faster calls.
2- Old applications can't benefit
3- Applications that wish to be backwards compatible can't benefit
Obviously though it is necessary to write new functions on occassion; for example when the new function is worse than the old function is under some circumstances. It may be that all the new functionality is of this type, but I don't have enough information to know for sure.
Philosophy.
Modules... Only the modules (read: 'drivers') that are needed are loaded. It needs to be in the kernel because it accesses the hardware (the net card) at a fairly low level.
I.O.U One Sig.
Changelog
Lone Gunmen crew.
Some stuff I found interesting on the human-friendly changelog.
/proc file /proc/self/mountstats, where mounted file systems can export information (configuration options, performance counters, and so on)
Block queue IO tracing support (blktrace). This allows users to see any traffic happening on a block device queue. In other words, you can get very detailed stadistics of what your disks are doing. User space support tools available in: git://brick.kernel.dk/data/git/blktrace.git
New
Introduce the splice(), tee() and vmsplice() system calls, a new I/O method.
The idea behind splice is the availability of a in-kernel buffer that the user has control over, where "splice()" moves data to/from the buffer from/to an arbitrary file descriptor, while "tee()" copies the data in one buffer to another, ie: it "duplicates" it. The in-buffer however is implemented as a set of reference-counted pointers which the kernel copies around without actually copying the data. So while tee() "duplicates" the in-kernel buffer, in practice it doesn't copy the data but increments the reference pointers, avoiding extra copies of the data. In the same way, splice() can move data from one end to another, but instead of bringing the data from the source to the process' memory and sending back to the destination it just moves it avoiding the extra copy. This new scheme can be used anywhere where a process needs to send something from one end to another, but it doesn't need to touch or even look at the data, just forward it: Avoiding extra copies of data means you don't waste time copying data around (huge performance improvement). For example, you could forward data that comes from a MPEG-4 hardware encoder, and tee() it to duplicate the stream, and write one of the streams to disk, and the other one to a socket for a real-time network broadcast. Again, all without actually physically copying it around in memory.
factor 966971: 966971
Even under windows drivers load into the kernel and normally become part of the kernel proper. Things under linux are similar, but differ from Windows in very important ways. First of all, the overriding philosophy behind the linux kernel is the GPL (well a modified version of the GPLv2) license for the source code. This states that the source code for the kernel and parts of the kernel should always be available. Also, for philosophical reasons associated with the licensing issue, Linus has said that he does not care as much about a binary stable driver API (ABI) in the kernel. Since the source code to the kernel is always available, if you want decent drivers, they should be placed in the kernel source code tree, since drivers really ought to be free and open. Unfortunately this means that a binary kernel driver from one version of the kernel may not work on another. This is done partially to encourage open source drivers and partially to prevent the kernel developers from being tied to design decisions that later prove unwise. But this does pose a problem for folks that want to implement their own third-party drivers in a propriety fashion. NVidia writes a special open source driver that implements a special, stable ABI that it's proprietary, closed-source video driver talks to to overcome this.
Many have argued that Linus needs to stablize the kernel driver ABI. But on the other hand, by not doing so and encouraging drivers to be open source and in the kernel source tree brings us a large amount of stability that Windows just cannot achieve. Most windows stability problems are not caused by the kernel, which is as stable as Linux, but by third-party device drivers. Anyway it is a trade-off, and one that is hotly contested. Personally, everything I currently use has open source drivers that come with my kernel bundle (Fedora Core). They are loaded on demand, so they don't cause memory bloat. If I was to compile my own kernel, I could choose not to build many of the drivers, reducing the disk bloat too.
One of the biggest things for me in this kernel release is the Broadcom wireless driver. Kudos to the team that clean-room reverse engineered the driver.
A hell of a lot of this stuff seems to me to be the sort of code that should be going into the 2.7 stream, not 2.6. The earliest days of Linux had revisions X.Y.Z. If Y was even, it was a "stable" branch, and could generally be considered safe for production work. If Y was odd, it was a "development" branch, and could break things badly.
...
This was a major boon for Linux: if you needed the bleeding edge, you could get it, whilst acknowledging the risks in doing so. If you needed something stable, again, you could get it. Now? It seems that the supposedly stable kernel is right out there on the bleeding edge
While the "who cares about software efficiency, the hw is getting faster" attitude might be OK for desktop PCs, it does not apply to handheld/mobile devices (which make up a huge, and ever growing, % of all Linux devices). Being able to use a slower CPU (or use a fast one very efficiently) makes for reduced power consumption == smaller devices == longer battery life. Nobody wants a cell phone with a 2 pound battery that only runs for 1 day.
Engineering is the art of compromise.
This is the essence of the Microkernel debate. http://en.wikipedia.org/wiki/Microkernel/ The truth is that the Microkernel model probably is a better design, but in terms of when the Linux kernel was starting out - its implementation simply wasn't pratical. It didn't help that the people who thought they knew how to build a better kernel decided to try and intellectually brow-beat Linus into doing it instead of implementing it themselves and putting it under the GPL. This led to a lot of bitterness and resentment between the two camps. The HURD http://en.wikipedia.org/wiki/Hurd project is a GPL microkernel project, but it simply wasn't managed as well as Linus managed Linux.
I think over time, things eventually will move to a microkernel model even though there are other ways to emulate some of their security and flexability benefits - like xen http://en.wikipedia.org/wiki/Xen
It's worth pointing out that pretty much every remotely mainstream OS *except* Linux manages to work (and work well) with a stable kernel ABI. Including ones considered at least - if not more - stable than Linux, even by Linux zealots, like FreeBSD and Solaris.
Insightful? How about Kino or Cinelerra or Lives or Mainactor?
Sounds good -- how much does it cost?
Sincerely,
The New Guy
2.6.16! You're crazy. I'm still holding back on 2.4.10 until the dust settles.
The "splice" system call seems to be an answer to one of Microsoft's bad ideas - serving web pages from the kernel. At one point, Microsoft was claiming that an "enterprise operating system" had to be able to do that. So now Linux has a comparable "zero copy" facility.
"Zero copy" tends to be overrated. It makes some benchmarks look good, but it's only useful if your system is mostly doing very dumb I/O bound stuff. In environments where web pages have to be ground through some engine like PHP before they go out, it won't help much.
The usual effect of adding "zero copy" to something is that the performance goes up a little, the complexity goes up a lot, and the number of crashes increases.
This is really just a way for app code to manipulate data without needing to have it copied or memory-mapped.
Linus refused the FreeBSD-style zero-copy because it is often a lose on SMP and with modern hardware. Page table and TLB updates have huge costs on modern hardware.
If you do like the Microsoft way, use Red Hat's kernel. The in-kernel server works very well.
I read that as ip conntracking to allow videoconferencing devices that follow the h.323 standard to be natted.
:)
obtw: your pedant bit is apparently stuck high. just a fyi -- didn't know if you realized it.
try hitting '/' on make menuconfig, type ov511 hit enter. That's a hot tip that's saved me quite a bit of time...
It'll find it if it's there.
The above comment has been marked WORKSFORME, and is now closed.
-:sigma.SB
WARN
THERE IS ANOTHER SYSTEM
You listed I/O schedulers. I think the multi-core bit talks about a PROCESS scheduler. Two different things. Linux already has specific support for Intel's HTT bullshit and understands NUMA. Understanding multi-core is a good move up.
If you have a 2P dual-core setup the best performance for two independent tasks would be spread to both chips. Specially in the AMD camp. That means each task gets a full memory bus to themselves. The trick is to pick up when two tasks have shared memory between each other and schedule that for one chip. Specially on the Intel side of things with their massive shared L2 cache.
Tom
Someday, I'll have a real sig.
I'm somewhat shocked that nobody else has pointed out the new Broadcom 43xx/Airport Extreme support. That's the one thing that grabbed my attention in the whole paragraph. Not having support for Apple's built-in wireless hardware has been a showstopper for a lot of people to even consider trying out Linux on a Mac, especially the portables. This driver will open up several million possible new computers for Linux to be installed on, since at this point the wireless hardware was about the last incompatible piece of hardware on the Mac side. This is a very big deal for anyone with Mac hardware or anyone planning to buy a Mac, and for all the geeks who are already running Linux on their Mac.
Very cool.
For kernel operations, you want everything pretty efficient. You want it as fast as possible and you don't want a lot of extra code hanging around. Unfortunately, the higher level a language you use, the more inefficency there is. For most programs it doesn't matter. They are either not the sort of thing that needs speed (like a word processor) or one where you can optimize the small part of the code that takes most of the time (like a game). However the kernel is a little different. Everything in there is time critical essentially.
C is the best compramise. While assembly might give you the theoritical best code, it'll big a giant mess to try and totally unmaintainable. Might actually be slower and larger for it. C is pretty good because it's easy enough to generate deceant code in, but it isn't much higher up the abstraction chain so it compiles quite efficient.
You have to remember that object orientation and such are all human creations. Processors don't think in objects, for that matter they don't really even think in functions. They think in memory locations, and jumps to those locations. Doing OO code means a whole messy layer the compiler has to go through to translate that in to something the processor actually understands.
there is still not a true, separate driver interface API.
Sure there is. There's just not a consistent ABI, and that's on purpose.
If you're contributing a driver, GREAT. It'll compile against the currently installed kernel just fine.
If it's closed-source, go die. The kernel's GPL, not lGPL
Haven't tried the release of 2.6.17 yet, but rcX versions required extracting the firmware for your Broadcom card from a binary such as bcmwl5.sys (Windows driver). The tool bcm43xx-fwcutter does this.
I'm not an Ubuntu guy, but this reference might be useful to anybody trying to make the new Broadcom Wifi driver work in Linux. Very easy steps, and most non-Ubuntu users should find it easy to adapt for their specific distros.
There's no place I could be, since I've found Serenity...
Life is just nature's way of keeping meat fresh.
It's worth pointing out that pretty much every remotely mainstream OS *except* Linux manages to work (and work well) with a stable kernel ABI. Including ones considered at least - if not more - stable than Linux, even by Linux zealots, like FreeBSD and Solaris.
FreeBSD example then just proves that a stable ABI won't bring more drivers to Linux, thus destroying the GP argument that Linus needs to stabilize the kernel driver ABI.
Sure there is. There's just not a consistent ABI, and that's on purpose.
If you're contributing a driver, GREAT. It'll compile against the currently installed kernel just fine.
Untrue I'm afraid. If your modules aren't in-tree then they *will* break every so often because the kernel API is not stable. Especially under the 2.6 development model - under the previous 2.4/2.5 model you were pretty much guaranteed that API breakages would only be happening in the 2.5 tree, now they happen at any point in the 2.6 tree. (Yes, I do know this stuff - I work on out-of-tree kernel code).
There is some arguement that all drivers should be in-tree, and for common hardware it is definately a Good Thing to have the drivers in the tree - as the API changes then the person implementing the API change will fix up all the in-tree code that uses that API.
For very specialist and expensive hardware it poses a problem though: the person who does the API change won't have the hardware to test with, and probably all the people who use that hardware are using enterprise distributions so breakages to the module won't be spotted for a long time. It's hard for the hardware vendor to track these kinds of updates and perform the necessary regression testing.
http://blog.nexusuk.org
It's worth pointing out that pretty much every remotely mainstream OS *except* Linux manages to work (and work well) with a stable kernel ABI.
Also worth pointing out that much of the stability trouble in Windows is caused by shoddy drivers - FOSS drivers are traditionally more stable than closed drivers (not least because when bugs are found, people with a vested interest in fixing them will often do so rather than waiting for the manufacturer to get their finger out).
Whilest a stable ABI may result in more drivers being made available, I fear it could lead to a lot of "Windows quality" drivers. And if closed drivers are officially legitimised, many companies will refuse to release open drivers since there is very little in it for them. At the moment, many of the open drivers are there because the vendor believes that releasing a binary driver is legally dubious at best - legitimise binary drivers and this motivation goes away.
Anyone who's dealt with bugs in the nVidia drivers will know of the problems of closed development - I've reported bugs that have taken years for nVidia to fix which I would've been happy to try and fix myself if only the code was open.
http://blog.nexusuk.org
I agree that using GOTO is a bad idea when another control structure is adaquate,
but, at least in C, there are times when using GOTO is the most natural and,
unequivically, the best choice.
Off the top of my head, I can think of two situations where using a GOTO is
the best solution:
1. breaking out of nested loops. In C, the break command can only break
out of a single loop level. If you need to break out of 2 or more loops, you
can play an ugly game of setting and checking state flags at each level
of looping or you can simply create a label at the exit point and use
GOTO to get there. (sometimes you can wrap your loops as a function call,
but that's often the ugliest solution)
2. shared cleanup code. In a function with multiple exit points, instead
of doing cleanup at each exit point, it is often clearer to set your
return value and then GOTO a label that handles all cleanup before
returning.
Be cautious when using GOTO, but don't be afraid of it. Learn to
recognize when GOTO is appropriate and when it should be avoided.
*sigh* back to work...