Linux 2.6.17 Released

diegocgteleline.es writes "After almost three months, Linux 2.6.17 has been released. The changes include support for Sun Niagara CPUs, a new I/O mechanism called 'splice' which can improve the performance greatly for some applications, a scheduler domain optimized for multicore machines, driver for the widely used broadcom 43xx wifi chip (Apple's Airport Extreme and such), iptables support for the H.323 protocol, CCID2 support for DCCP, softmac layer for the wireless stack, block queue IO tracing, and many other changes listed at the changelog"

Comments

linux

[Anonymous Coward]

But does it run linux ???

Re:linux

[moro_666]

Will ubuntu have it soon ?

They have included much of the stuff into their version of 2.6.15.23,
but ofcourse that doesn't have everything. The broadcom driver that came
with ubuntu(same sources, maybe earlier version) has somesort of issues
with my BCM4318 :( , so it just doesn't work, people claim it has something
to do with soft interrupt stuff.

ps. broadcom, next time make interrupts stiff.

Re:linux

[ElleyKitten]

The broadcom driver that came with ubuntu(same sources, maybe earlier version) has somesort of issues with my BCM4318 :( , so it just doesn't work,

Try this.

Really helped

[drsmack1]

I have this now installed in my dual core AMD and the difference is noticable. X is noticable faster, as is my video editing stuff. Good work guys!

Re:Really helped

[jnik]

My opinion: Linus has; Schilling refuses. Slightly closer to "fact": DVD-R Tools works.

what

[Anonymous Coward]

is this for computers?

Re:Video Editing?

[Anonymous Coward]

Final Cut Pro on OS X is my guess, at least if he's doing anything new and interesting. It's widely known that Windows- and Linux-based nonlinear editing tools are designed for accountants and squares.

Question for the masses.

[SynapseLapse]

I'm still pretty new to the Linux scene (So far I've done a FreeBSD, Ubuntu and Fedora Core 4 installation.), but I do have a question.
Why are the network drivers part of the kernel? It seems like this would make it more difficult to adopt newer hardware types. Also, since most computers have 1-2 NICs at the most, wouldn't that clog up the kernel with tons of drivers for hardware you'll never use?

Re:Question for the masses.

[shird]

Modules... Only the modules (read: 'drivers') that are needed are loaded. It needs to be in the kernel because it accesses the hardware (the net card) at a fairly low level.

Re:Question for the masses.

[caseih]

Even under windows drivers load into the kernel and normally become part of the kernel proper. Things under linux are similar, but differ from Windows in very important ways. First of all, the overriding philosophy behind the linux kernel is the GPL (well a modified version of the GPLv2) license for the source code. This states that the source code for the kernel and parts of the kernel should always be available. Also, for philosophical reasons associated with the licensing issue, Linus has said that he does not care as much about a binary stable driver API (ABI) in the kernel. Since the source code to the kernel is always available, if you want decent drivers, they should be placed in the kernel source code tree, since drivers really ought to be free and open. Unfortunately this means that a binary kernel driver from one version of the kernel may not work on another. This is done partially to encourage open source drivers and partially to prevent the kernel developers from being tied to design decisions that later prove unwise. But this does pose a problem for folks that want to implement their own third-party drivers in a propriety fashion. NVidia writes a special open source driver that implements a special, stable ABI that it's proprietary, closed-source video driver talks to to overcome this.

Many have argued that Linus needs to stablize the kernel driver ABI. But on the other hand, by not doing so and encouraging drivers to be open source and in the kernel source tree brings us a large amount of stability that Windows just cannot achieve. Most windows stability problems are not caused by the kernel, which is as stable as Linux, but by third-party device drivers. Anyway it is a trade-off, and one that is hotly contested. Personally, everything I currently use has open source drivers that come with my kernel bundle (Fedora Core). They are loaded on demand, so they don't cause memory bloat. If I was to compile my own kernel, I could choose not to build many of the drivers, reducing the disk bloat too.

One of the biggest things for me in this kernel release is the Broadcom wireless driver. Kudos to the team that clean-room reverse engineered the driver.

Re:Question for the masses.

[hcob$]

2 options:

1:
Compile everything you need for your machine to run into the kernel... no more, no less... then you're good to go. No clutter, no loading at runtime... nothing.

2:
You have no idea what you actually need past boot(and root) FS, cpu, and hard drives. Compile everything else as a module(driver) to be loaded when you need it, and voila, no bloat to the kernel, but a few dozen MBs taken up on the HD.

In the grand scheme of things, a few extra modules for network cards will cause you no trouble. And for reference, most Enterprise level computers have upwards of 4-6 network adapters. Wonderful for redundancy and resiliancy.

I'll leave you with one bit of parting advice:
Assuming a minimum configuration on any OS leads you down the path of Microsoft. Knowing your hardware and tweaking it yields unprecidented pride and performance... as well as a unwarrented self importance. The more you learn about OSes... the more you see they are all alike.

Re:Question for the masses.

[Anonymous Coward]

Okay, so how frequently do you feel the need to swap out NICs, sound cards, video adapters, or anything else?

In the grandparent's instance, his hardware may not change for months or years on end because, well, he dosen't want to shut his computers or servers down to experiment with random hardware... Because of this, it might make sense for him to compile the drivers directly into the kernel for a tiny boost in performance and memory utilization... That would make sense for embedded computers, too obviously, and it might be a desireable thing to do.

For everyone else there are modules, drivers that the kernel can load automatically for hardware that it automagically detected, and it won't load drivers for devices that aren't detected or drivers that the kernel wasn't specifically directed to load. From a user perspective, you can have ALL of the driver modules compiled and most will just eat up a few megs of space in /usr, and everything will work the way it's supposed to, so it's worth eating up a few megs of drive space to be compatible with most everything Linux supports.

Re:Question for the masses.

[billstewart]

So you've learned what RTFM means by now? :-) Ok, it's been a while since I've read up on kernel structure either.... but you _should_ do so. Linux is rather famously not a microkernel architecture that lets you partition off little pieces into user space - it's a big honkin' kernel plus loadable modules that let you add even more things. There are hardware-dependent and hardware-independent parts of the kernel. Device drivers inherently hardware-dependent, and sharing address space with the kernel makes it easier to do things like DMA without having to do a lot of data copying.

As far as network drivers in particular go, the layers that use them, such as IP, live in the kernel, so it's rather annoying for them to talk to drivers that are up in user space. Specific network cards, especially wireless, might have bits that live up in user space, such as user interfaces for loading in crypto keys, but the bulk data transfer applications normally belong down in or near the kernel.

Why are there a whole pile of network card drivers in the kernel when you'd normally use only one or two? Same reason there are a whole pile of drivers for other devices in the kernel, when you've normally only got one graphics card and one sound card. If you're shipping a pre-compiled kernel, you want it to support as many different users as possible, and all it costs you is some RAM to store the code you're not using, or if you can handle them as loadable modules, it only costs you the work to keep track of those. But if you want to compile your own kernel for specific machines, and leave out the drivers you don't want, and while you're at it compile all your applications programs with the level of optimization your hardware supports, get a copy of Gentoo Linux and have fun learning lots more detail about Linux internals.

Re:Question for the masses.

[drsmithy]

Many have argued that Linus needs to stablize the kernel driver ABI. But on the other hand, by not doing so and encouraging drivers to be open source and in the kernel source tree brings us a large amount of stability that Windows just cannot achieve.

It's worth pointing out that pretty much every remotely mainstream OS *except* Linux manages to work (and work well) with a stable kernel ABI. Including ones considered at least - if not more - stable than Linux, even by Linux zealots, like FreeBSD and Solaris.

Re:Question for the masses.

[dhalgren]

It is freedom. Freedom does not mean "the developers accept every suggestion and all criticism". You are free to use it, use something else, dig in there and change what you don't like, and even fork it and publish it the way you think it should be. That's freedom.

And of course, Linus is free to do what the hell he wants. He doesn't owe us a thing.

Re:Question for the masses.

[Lobais]

But on the other hand, there are nearly no drivers for FreeBSD and Solaris.

Re:Question for the masses.

[ookaze]

It's worth pointing out that pretty much every remotely mainstream OS *except* Linux manages to work (and work well) with a stable kernel ABI. Including ones considered at least - if not more - stable than Linux, even by Linux zealots, like FreeBSD and Solaris.

FreeBSD example then just proves that a stable ABI won't bring more drivers to Linux, thus destroying the GP argument that Linus needs to stabilize the kernel driver ABI.

Re:Question for the masses.

[FireFury03]

It's worth pointing out that pretty much every remotely mainstream OS *except* Linux manages to work (and work well) with a stable kernel ABI.

Also worth pointing out that much of the stability trouble in Windows is caused by shoddy drivers - FOSS drivers are traditionally more stable than closed drivers (not least because when bugs are found, people with a vested interest in fixing them will often do so rather than waiting for the manufacturer to get their finger out).

Whilest a stable ABI may result in more drivers being made available, I fear it could lead to a lot of "Windows quality" drivers. And if closed drivers are officially legitimised, many companies will refuse to release open drivers since there is very little in it for them. At the moment, many of the open drivers are there because the vendor believes that releasing a binary driver is legally dubious at best - legitimise binary drivers and this motivation goes away.

Anyone who's dealt with bugs in the nVidia drivers will know of the problems of closed development - I've reported bugs that have taken years for nVidia to fix which I would've been happy to try and fix myself if only the code was open.

Re:Question for the masses.

[DamnStupidElf]

It's worth pointing out that pretty much every remotely mainstream OS *except* Linux manages to work (and work well) with a stable kernel ABI. Including ones considered at least - if not more - stable than Linux, even by Linux zealots, like FreeBSD and Solaris.

This is why OpenBSD is having more success than Linux in getting specs from vendors, right? Because they just let BLOBs run in the kernel? Oh, wait...

The only secure way to run binary drivers is to run them in usermode with ACLs on each hardware port and memory region they use, with safe wrappers around DMA. In fact, it's probably not even safe to trust binary drivers in this case. What if the hardware doesn't respect your computer and decides to snoop the bus on its own? It can leak information back to the driver through side channels, or worse act as a backdoor. Suppose your wireless chipset maker gets a visit from a TLA who wants immediate access to any wireless system. The driver is the place to do that, since it runs in kernel mode. Makes you wonder about all those "closed source wireless drivers are the *only way* to satisfy the FCC!" arguments.

Go Linux!

[Umbral Blot]

It's good to know that even in this day and age of faster and faster computers there are still people who care about speed and efficiency instead of simply waiting for hardware to solve their problems for them. I do have one tiny complaint though, and it is that some of the performance gains are only possible by using new system calls. This is bad for three reasons:
1- More work for developers, some of whom may never learn about these faster calls.
2- Old applications can't benefit
3- Applications that wish to be backwards compatible can't benefit
Obviously though it is necessary to write new functions on occassion; for example when the new function is worse than the old function is under some circumstances. It may be that all the new functionality is of this type, but I don't have enough information to know for sure.

Re:Go Linux!

[freralqqvba]

sendfile(2) is now a call to splice() so programs that use the old syscall will benefit as well and without modificaiton.

Re:Go Linux!

[ip_fired]

The kernel is written in C, and so are those system calls. I don't believe you can overload a C function.

Re:Go Linux!

[Umbral Blot]

No you can't, although that fact had slipped my mind. How sad for C. The real question here then is: why isn't there a better language than C for creating OSs in? A real macro system and overloading would probably be nice for kernel dev.s everywhere.

Re:Go Linux!

[TCM]

It's good to know that even in this day and age of faster and faster computers there are still people who care about speed and efficiency instead of simply waiting for hardware to solve their problems for them.

Another way of saying this: It sucks to know that even in this day and age of faster and faster computers there are still people who cut corners and use specific hacks to gain speed instead of simply building clean and well-designed systems and let the hardware do the work.

Just saying..

Re:Go Linux!

[ip_fired]

Because C is fast, and there is so much already written in C, that it would be a pain to move over. There are a lot of hacks in the linux kernel (they actually use GOTO statements! *gasp*). I can tell you it was a real eye-opener for me when I started looking at things there. I'm sure the reason nobody has moved over to a better language though is because of the massive amounts of work that would have to be redone.

Re:Go Linux!

[Umbral Blot]

How could anything be better than flint for making arrowheads?

Re:Go Linux!

[pavon]

Obviously though it is necessary to write new functions on occassion; for example when the new function is worse than the old function is under some circumstances.

That is exactly why it was done. More information about can be found at kerneltrap: here, and here. It was also previously on slashdot, although you would be best to skip that - it has more misinformation than the other kind.

In short, all the known ways of implementing zero-copy within the existing API's cause the most common usage cases of those API to be slower than they are now. Therefore, it made more sense to export this new API for the applications where speed is critical.

In the the first kernaltrap article, Linus also explains why splice is different from sendfile, contrary to the posts here claiming they are essentially the same.

Re:Go Linux!

[iabervon]

In this case, the new syscall is because a specific situation can be optimized compared to using the existing functions, but the more efficient function only works at all for certain special (but important) cases. In this case, the optimization is that copying data from outside of the program to outside of the program is more efficient if the data doesn't have to go through the program; obviously, this can't be used for the common case where the program is trying to use the data. The case that it helps a lot is when a program is sending data from the disk to the video card, or network to the disk, etc. With the new syscall, the data doesn't have to be copied into the program's address space (or the program's page tables changed to bring it in, which generally costs a TLB flush).

Programs that want to be backwards compatible can rely on getting an error result of ENOSYS when they try, and can then fall back to using the traditional method.

Re:Go Linux!

[waveclaw]

The kernel is written in C, and so are those system calls. I don't believe you can overload a C function.

There is no overloading going on here. Overloading is to create a new function with the same name, but taking different parameters.

Ahem. The original function, sendfile(2), was rewritten to call splice() instead of doing something else.

Everybody that wrote code that used the old function now has to deal with splice() running instead of the old function's logic.

Just to hammer it home:
Old - app -> sendfile(2) -> some logic -> return to app
New - app -> sendfile(2) -> splice() -> splice's logic -> return to sendfile(2) -> return to app

With the Linux kernel, as this exepmlifies, you can improve the original code and get everyone (well, those to lazy to revert the changes) to use it. In this case you have a fixed API (sendfile(2) which is well known and published) so you don't just want to tell everybody to recompile with called to splice().

See the difference? Feel the difference.

The kernel is GPL and thus the actual source code used to compile the binary kernel you use is available to you. With a closed source kernel you might be able to purchase an SDK with linkable binaries and some (probably undocumetned) header files. Programmers in this situation need things like function overloading and class inheritence just to do anything. One way of looking at the history of languages like C++ is as a technical solution to the ethical problem of closed source programming. Those languages focus on extending on the outside. With OSS you can usually replace, fix and improve on the inside. BSD and GNU differ on a the point of GNU wanting everyone to share the source to those fixes if they share the resulting binaries. But I digress.

And I can't wait to see if this breaks something.

Re:Go Linux!

[jrockway]

> A real macro system and overloading would probably be nice for kernel dev.s everywhere.

Like LISP? That's what they used to use, but C was chosen for UNIX, and UNIX caught on big time, so C is the language now. I think it's about time to write an OS (kernel + tools) in LISP, so we can return to the good-old-days of Lisp machines.

Re:Go Linux!

[SpinyNorman]

Overloading is just syntactic sugar - it doesn't give you any fucntionality.

There's no functional difference between using an overloaded name f(a), f(x, y), f(p, q, r) or three separate ones f_a(a), f_xy(x, y), f_pqr(p, q, r).

If you want default arguments C has them, and if you want polymorphism then C has it too (function pointers).

Re:Go Linux!

[pclminion]

Another way of saying this: It sucks to know that even in this day and age of faster and faster computers there are still people who cut corners and use specific hacks to gain speed instead of simply building clean and well-designed systems and let the hardware do the work.

Why do you assume that all optimizations are hacks? Lifting an invariant calculation out of a loop can potentially make things MUCH faster, yet is hardly a "hack." Or how about strength-reducing "2 * x" into "x + x," is that a hack? Same goes for a change of data structures (say, switching from a linear search in a linked list to a constant-time lookup from a hash table). Are hash tables "hacks?"

Man, I can't wait for the day when the hardware is smart enough to morph my linked lists into hash tables for me!

I think you have no clue what you're talking about.

Re:Go Linux!

[pooly7]

Actually if you are thinking about high-end server, the few % you get with optimized calls, can be translated in $ with several digits. So if you paid someone less than that number, everybody wins.

In case of slashdot, break mirror

[TrueKonrads]

Changelog

some highlights from the changelog

[doti]

Some stuff I found interesting on the human-friendly changelog.

Block queue IO tracing support (blktrace). This allows users to see any traffic happening on a block device queue. In other words, you can get very detailed stadistics of what your disks are doing. User space support tools available in: git://brick.kernel.dk/data/git/blktrace.git

New /proc file /proc/self/mountstats, where mounted file systems can export information (configuration options, performance counters, and so on)

Introduce the splice(), tee() and vmsplice() system calls, a new I/O method.
The idea behind splice is the availability of a in-kernel buffer that the user has control over, where "splice()" moves data to/from the buffer from/to an arbitrary file descriptor, while "tee()" copies the data in one buffer to another, ie: it "duplicates" it. The in-buffer however is implemented as a set of reference-counted pointers which the kernel copies around without actually copying the data. So while tee() "duplicates" the in-kernel buffer, in practice it doesn't copy the data but increments the reference pointers, avoiding extra copies of the data. In the same way, splice() can move data from one end to another, but instead of bringing the data from the source to the process' memory and sending back to the destination it just moves it avoiding the extra copy. This new scheme can be used anywhere where a process needs to send something from one end to another, but it doesn't need to touch or even look at the data, just forward it: Avoiding extra copies of data means you don't waste time copying data around (huge performance improvement). For example, you could forward data that comes from a MPEG-4 hardware encoder, and tee() it to duplicate the stream, and write one of the streams to disk, and the other one to a socket for a real-time network broadcast. Again, all without actually physically copying it around in memory.

Where is 2.7?

[Anonymous Coward]

A hell of a lot of this stuff seems to me to be the sort of code that should be going into the 2.7 stream, not 2.6. The earliest days of Linux had revisions X.Y.Z. If Y was even, it was a "stable" branch, and could generally be considered safe for production work. If Y was odd, it was a "development" branch, and could break things badly.

This was a major boon for Linux: if you needed the bleeding edge, you could get it, whilst acknowledging the risks in doing so. If you needed something stable, again, you could get it. Now? It seems that the supposedly stable kernel is right out there on the bleeding edge ...

Re:Where is 2.7?

[x2A]

The stable/development branches might be a nice idea in theory, but in practice it doesn't work. Distros would ship, for example, a "stable" 2.4.xx kernel, except it wouldn't actually be that. They would spot nice features in the 2.5 kernel that they wanted to offer their users, and so back-port them... and any other nice patches floating around the net while they're at it. The result being that the kernels that ship with distros were so heavily modified, that stability (from one machine to another) went right out of the window. You couldn't go to kernel.org and download an updated kernel, as without all the patches, it wouldn't work. So you had to stick to the distro's kernels.

So instead, the 2.6 goal is to have development/stable parts of the cycle, rather than seperate branches. Roughtly: patches that could break things get submitted at the beginning of the cycle, and -pre1/-pre2 tarballs are released. If you want bleeding edge, you go here. Release candidates are released, where developers get chance to fix bugs etc in the code. Then, any code that's still [known to be] buggy gets dropped for the final release (eg, 2.6.17). The developer can work on it, and try add it again during subsequent cycles. When it works, it can be included in a final release.

During this cycle, security and other urgent bug fixes take place in the ultra-stable branch, with version such as 2.6.16.1, 2.6.16.2.

(This is the rough idea I believe, there could be some slight inaccuracies in how it actually takes place, I haven't followed it 100%, but this should be close enough to get the right idea).

Re:Where is 2.7?

[iabervon]

That was the theory. But in practice, if Y was even, the kernel was obsolete, while if Y was odd, the kernel was broken. Except, of course, 2.even.0, which was actually stable, but broke compatibility with the previous kernel that worked. And occasionally, 2.even was kept up-to-date because nobody could use 2.odd for development, because it didn't work at all. You could tell that the old model didn't actually work, because no distribution shipped any kernel that used that model; they all shipped 2.even with an arbitrary set of patches (generally hundreds) from 2.odd and elsewhere. With the new model, distros are shipping kernels with only a few patches, and those patches are getting merged upstream.

The stable kernels aren't remotely on the bleeding edge; they contain only features which have been tested over the past three months, after being filtered out of the bleeding-edge development as being things that have already stabilized and stand a good chance of being proven in three months. It's effectively very similar, except the development series isn't left known-broken and the stabilization process happens on a quick schedule, with stuff that isn't ready pushed off to the next cycle rather than delaying the current cycle. Also, the version numbers change by less (development gets -mm, -rc, or -git; stable series change the third digit by one instead of the second by two; and bugfix releases change the fourth digit instead of the third).

Re:Where is 2.7?

[10Ghz]

This was a major boon for Linux

Or bane. The "old way" meant that the vanilla-kernel (kernel offered by kernel.org) was stable. But new features took a LONG time to appear in the vanilla-kernel. But users and distros still wanted those advanced features that were not part of the kernel (yet). What happened was that distros offered their own vendor-kernels, that were VERY different from vanilla-kernel. Distros then spent their time and energy fixing their own vendor-kernels, instead of vanilla-kernel.

This new system changes things so that new features are added to the vanilla-kernel, which means that the difference between vanilla and vendor-kernels is not that big. The distributors can focus on stabilizing the kernel, instead of adding new features to it. And porting those fixes to vanilla is a lot easier than porting changes in the old system. This means that if you want to use REALLY stable kernel, you should use the vendor-kernel.

In short: this new system means that things progress a lot faster for everyone, with new features appearing in the kernel. And we can still have the stability we want if we use the tested and patched vendor-kernels.

Linux is for other devices too

[EmbeddedJanitor]

While the "who cares about software efficiency, the hw is getting faster" attitude might be OK for desktop PCs, it does not apply to handheld/mobile devices (which make up a huge, and ever growing, % of all Linux devices). Being able to use a slower CPU (or use a fast one very efficiently) makes for reduced power consumption == smaller devices == longer battery life. Nobody wants a cell phone with a 2 pound battery that only runs for 1 day.

Microkernel anyone?

[argoff]

Why are the network drivers part of the kernel? It seems like this would make it more difficult to adopt newer hardware types. Also, since most computers have 1-2 NICs at the most, wouldn't that clog up the kernel with tons of drivers for hardware you'll never use?

This is the essence of the Microkernel debate. http://en.wikipedia.org/wiki/Microkernel/ The truth is that the Microkernel model probably is a better design, but in terms of when the Linux kernel was starting out - its implementation simply wasn't pratical. It didn't help that the people who thought they knew how to build a better kernel decided to try and intellectually brow-beat Linus into doing it instead of implementing it themselves and putting it under the GPL. This led to a lot of bitterness and resentment between the two camps. The HURD http://en.wikipedia.org/wiki/Hurd project is a GPL microkernel project, but it simply wasn't managed as well as Linus managed Linux.

I think over time, things eventually will move to a microkernel model even though there are other ways to emulate some of their security and flexability benefits - like xen http://en.wikipedia.org/wiki/Xen

Re:Microkernel anyone?

[ArbitraryConstant]

Why are the network drivers part of the kernel? It seems like this would make it more difficult to adopt newer hardware types. Also, since most computers have 1-2 NICs at the most, wouldn't that clog up the kernel with tons of drivers for hardware you'll never use?

This is the essence of the Microkernel debate.

It has nothing whatsoever to do with the microkernel debate. The two issues are completely orthogonal to each other. The microkernel proponents claim the driver should be a process running outside the kernel, which has nothing to do with the number of drivers you have sitting around, or how difficult it is to adopt newer hardware types.

Either way, you've got a ton of drivers sitting around that you'll never use. They don't clog up the kernel, since the kernel image rarely contains many drivers. Instead, most Linux distros use modules that get loaded as needed. On a microkernel, they would be driver binaries that would get run as needed. They clog things up to exactly the same extent; they sit around on the hard drive doing nothing.

Either way, it's hard to add new drivers to old kernels. This is not a result of the fact that drivers are in the kernel, but of the fact that Linus refuses to use a stable driver API. This would preclude driver compatibility between versions just as effectively on a microkernel as it does now.

As I said, the two issues are unrelated.

Re:Microkernel anyone?

[argoff]

Either way, you've got a ton of drivers sitting around that you'll never use. They don't clog up the kernel, since the kernel image rarely contains many drivers. Instead, most Linux distros use modules that get loaded as needed. On a microkernel, they would be driver binaries that would get run as needed. They clog things up to exactly the same extent; they sit around on the hard drive doing nothing.

The point is not a resource usage point, but a flexability point. If you want to add a new driver or even change one, it pretty much takes co-opting the entire kernel tree, or compiling the driver live against the current kernel, or having pre-compiled modules against every possible kernel out there.

Either way, it's hard to add new drivers to old kernels. This is not a result of the fact that drivers are in the kernel, but of the fact that Linus refuses to use a stable driver API. This would preclude driver compatibility between versions just as effectively on a microkernel as it does now.

The driver API wouldn't need to be as stable in a Microkernel design. (I'm not a kernel guru, so you may know better)

Re:Video Editing?

[Anonymous Coward]

Insightful? How about Kino or Cinelerra or Lives or Mainactor?

Sounds good

[goodenoughnickname]

Sounds good -- how much does it cost?

Sincerely,
The New Guy

Re:Nice

[Frogbert]

2.6.16! You're crazy. I'm still holding back on 2.4.10 until the dust settles.

"splice" - because Microsoft did it?

[Animats]

The "splice" system call seems to be an answer to one of Microsoft's bad ideas - serving web pages from the kernel. At one point, Microsoft was claiming that an "enterprise operating system" had to be able to do that. So now Linux has a comparable "zero copy" facility.

"Zero copy" tends to be overrated. It makes some benchmarks look good, but it's only useful if your system is mostly doing very dumb I/O bound stuff. In environments where web pages have to be ground through some engine like PHP before they go out, it won't help much.

The usual effect of adding "zero copy" to something is that the performance goes up a little, the complexity goes up a lot, and the number of crashes increases.

Re:"splice" - because Microsoft did it?

[Frater 219]

"Zero copy" tends to be overrated. It makes some benchmarks look good, but it's only useful if your system is mostly doing very dumb I/O bound stuff. In environments where web pages have to be ground through some engine like PHP before they go out, it won't help much.

On the contrary, there are many cases in a dynamic serving system where you can determine that, after some point, the rest of the operation merely involves copying data from a file or buffer out to the network. Or, similarly, that a large portion of the operation involves such copying.

So even though the whole operation can't be reduced to a splice() or sendfile(), a substantial portion of it can. And the speed improvement you take isn't just that you avoid copying -- as "zero copy" implies; you also avoid unnecessary cache dirtiness.

The usual effect of adding "zero copy" to something is that the performance goes up a little, the complexity goes up a lot, and the number of crashes increases.

I wonder what your sample size is for "usual" there. As far as I can tell, you discuss only one case where this is so: Windows. And since we know that Windows has lots of other architectural problems leading to crashes, and in any event has an architecture entirely different from Linux's, we know that case to be irrelevant.

All in all, your analysis is critically, embarrassingly bad. There is no "serving Web pages from the kernel" going on here. There is simply an optimization for a common case, with no degradation to the less-common cases -- that's why it's implemented as a separate system call.

Re:"splice" - because Microsoft did it?

[ookaze]

The "splice" system call seems to be an answer to one of Microsoft's bad ideas - serving web pages from the kernel

WHat is this nonsense ? The khttp in-kernel web server was implemented on Linux first, then copied by MS.
IIRC it isn't even in 2.6 kernels anymore.

So now Linux has a comparable "zero copy" facility

Linux already had a zero-copy facility, splice is just a new improved one.
What are you talking about ?

"Zero copy" tends to be overrated. It makes some benchmarks look good, but it's only useful if your system is mostly doing very dumb I/O bound stuff. In environments where web pages have to be ground through some engine like PHP before they go out, it won't help much.
The usual effect of adding "zero copy" to something is that the performance goes up a little, the complexity goes up a lot, and the number of crashes increases

Not in this case. Linux made it right.

not like that

[r00t]

This is really just a way for app code to manipulate data without needing to have it copied or memory-mapped.

Linus refused the FreeBSD-style zero-copy because it is often a lose on SMP and with modern hardware. Page table and TLB updates have huge costs on modern hardware.

If you do like the Microsoft way, use Red Hat's kernel. The in-kernel server works very well.

Re:not like that

[master_p]

"This is really just a way for app code to manipulate data without needing to have it copied or memory-mapped."

I think you are wrong. Splice'd data are not processed by userland at all: they are piped from one file to the other at kernel level by page copying.

Re:not like that

[cnettel]

Your parent is right. The user mode code can control what happens to data, without ever mapping it to its own memory. You are right in that it's not processed, but that's not what the original post said.

Re:support for the h.323 protocol, quite unlikely

[nick this]

I read that as ip conntracking to allow videoconferencing devices that follow the h.323 standard to be natted.

obtw: your pedant bit is apparently stuck high. just a fyi -- didn't know if you realized it. :)

Re:Had to exchange a motherboard

[Osty]

Why not try an Intel board? Not to be a fanboy or anything but their linux support is good. Intel NICs can't be beat for support and performance either.

One may find it difficult to fit an AMD CPU on an Intel motherboard. Pesky competition.

As the grand-parent said, he had to exchange a motherboard. That means he wasn't intentionally upgrading, thus expecting to continue to use the rest of his hardware (memory, disk, CPU). Were this a system upgrade rather than a replacement of a faulty part then it may be useful to suggest he look at an Intel-based solution. As it is, he obviously already had an AMD CPU and so he needed another motherboard that would work with said CPU.

Re:Missing driver?

[WhodoVoodoo]

try hitting '/' on make menuconfig, type ov511 hit enter. That's a hot tip that's saved me quite a bit of time...
It'll find it if it's there.

Re:Great, how about stable firewire support someda

[Solra Bizna]

The above comment has been marked WORKSFORME, and is now closed.

-:sigma.SB

Re:OK, so where are they?

[tomstdenis]

You listed I/O schedulers. I think the multi-core bit talks about a PROCESS scheduler. Two different things. Linux already has specific support for Intel's HTT bullshit and understands NUMA. Understanding multi-core is a good move up.

If you have a 2P dual-core setup the best performance for two independent tasks would be spread to both chips. Specially in the AMD camp. That means each task gets a full memory bus to themselves. The trick is to pick up when two tasks have shared memory between each other and schedule that for one chip. Specially on the Intel side of things with their massive shared L2 cache.

Tom

infrared for IBM thinkpads

[Rizzer]

I think they've added the driver to support the IR infrared port on IBM (Lenovo) thinkpads, at last!

Re:support for the h.323 protocol, quite unlikely

[nick this]

Unlikely or not, that's what it appears to be. h.323 conntrack nat helper

This patch (or module, actually) comes with an H.323 decoding library that is based on H.225 version 4, H.235 version 2 and H.245 version 7. It is extremely optimized for Linux kernel and decodes only the absolutely necessary objects in a signal. ... The total size of code plus data is less than 20 KB.

Doesn't look like a gatekeeper or anything, that looks like an honest-to god ipconntrack nat implementation.

For the other responder to my initial post. I have taken your offer into consideration but have decided to decline.
lol.

Re:Video Editing?

[Bent Mind]

I've been doing video editing with Avidemux. It's a nice little program for Windows, OSX, and Linux.

Indeed, Airport Extreme support is HUGE

[RedBear]

Woohoo, Broadcom 4300 drivers! I hope they work. ...I wish this had been brought to my attention before 1 A.M.

I'm somewhat shocked that nobody else has pointed out the new Broadcom 43xx/Airport Extreme support. That's the one thing that grabbed my attention in the whole paragraph. Not having support for Apple's built-in wireless hardware has been a showstopper for a lot of people to even consider trying out Linux on a Mac, especially the portables. This driver will open up several million possible new computers for Linux to be installed on, since at this point the wireless hardware was about the last incompatible piece of hardware on the Mac side. This is a very big deal for anyone with Mac hardware or anyone planning to buy a Mac, and for all the geeks who are already running Linux on their Mac.

Very cool.

That and

[Sycraft-fu]

For kernel operations, you want everything pretty efficient. You want it as fast as possible and you don't want a lot of extra code hanging around. Unfortunately, the higher level a language you use, the more inefficency there is. For most programs it doesn't matter. They are either not the sort of thing that needs speed (like a word processor) or one where you can optimize the small part of the code that takes most of the time (like a game). However the kernel is a little different. Everything in there is time critical essentially.

C is the best compramise. While assembly might give you the theoritical best code, it'll big a giant mess to try and totally unmaintainable. Might actually be slower and larger for it. C is pretty good because it's easy enough to generate deceant code in, but it isn't much higher up the abstraction chain so it compiles quite efficient.

You have to remember that object orientation and such are all human creations. Processors don't think in objects, for that matter they don't really even think in functions. They think in memory locations, and jumps to those locations. Doing OO code means a whole messy layer the compiler has to go through to translate that in to something the processor actually understands.

Re:That and

[k98sven]

Unfortunately, the higher level a language you use, the more inefficency there is.

That's complete nonsense. What do you support that on? A higher level language is only as inefficient as the compiler and/or libraries used. Which is just as true for a low-level language.

C is the best compramise. While assembly might give you the theoritical best code

As someone who's actually spent some years coding assembler, I'll tell you this: Hand-coded assembler is rarely ever better. And with the developments in processors today, it's not even theoretically better. If the Linux kernel had been written in assembler, then , besides being completely unportable they'd have to rewrite the whole thing to take advantage of the 486 instruction set. Then the Pentium instruction set, then MMX and so on. Unless you're going to provide hand-coded routines for every single processor out there, hand-coded assembler is never better.

And the whole basis of your argument is just wrong. You're assumimg that a human will always produce better assembler code than a compiler. That is not true. That's not even true most of the time.

Re:module shotguns

[BrainInAJar]

there is still not a true, separate driver interface API.

Sure there is. There's just not a consistent ABI, and that's on purpose.

If you're contributing a driver, GREAT. It'll compile against the currently installed kernel just fine.

If it's closed-source, go die. The kernel's GPL, not lGPL

Broadcom 43xx HOWTO:

[cbhacking]

Haven't tried the release of 2.6.17 yet, but rcX versions required extracting the firmware for your Broadcom card from a binary such as bcmwl5.sys (Windows driver). The tool bcm43xx-fwcutter does this.

I'm not an Ubuntu guy, but this reference might be useful to anybody trying to make the new Broadcom Wifi driver work in Linux. Very easy steps, and most non-Ubuntu users should find it easy to adapt for their specific distros.

Re:support for the h.323 protocol, quite unlikely

[rodac]

Bloody hell!

They managed to squeeze both PER and also H225/235/245 into just 20kbyte of object code?!
(why implement h235? thats crypto and wouldnt work unless you know the keys?)
That is VERY impressive.

My PER decoder alone ( http://anonsvn.wireshark.org/wireshark/trunk/epan/ dissectors/packet-per.c ) is way larger than that, and that is just aligned PER decoding (ok with some unaligned PER additions recently) and that one itself is >>20kbyte. Adding 225/245 into the mix. Impossible!


I am very impressed. Very impressed.

Re:module shotguns

[wertarbyte]

Many a linux distribution I've used (most noticeably Debian) applies the "shotgun" approach to module-loading because the hardware detection and hotplug methods are so convoluted and undependable. Kind of defeats the purpose of loadable modules if the distribution simply loads everything under the sun to see what sticks.

Obviously you haven't used Linux for a long time. Modules are not loaded to detect hardware, instead the hardware acquires the driver module: The kernel identifies hardware via PCI or USB device ids, which are also stored in the modules. So Hotplug (and newer versions of udev) can load the appropiate module once hardware is added to the system.

Worse, many modules aren't smart enough to determine "hey, I'm a driver for [some non-removable component]. If I can't find my hardware, maybe I should print an error to ksyslogd and unload myself."

The driver will not be loaded if there is no hardware, unless you explicitly tell your system to do so.

Re:suggestions for a laptop

[devlp0]

Looks a good laptop. The only trouble I have ever had with laptops/Linux are: 1) internal modem doasnt work (M$ modems - who cares anyway) 2) might need to disable apic at boot time (will be obvious - machine just wont boot. Add "noapic" to you kernel boot parameters if this is the case) Nvidia drivers can be downloaded from their web site - not open source though :(

Re:Video Editing?

[peterpi]

split, cat and sed.

Re:Video Editing?

[zsau]

Using a graphical editor? I edit my videos by converting them to a series of jpeg images (one per frame) and using ed to edit them! Damn efficient, quality results.

DCCP

[caluml]

DCCP - a kind of UDPv2
Any applications out there using it yet?

What is the klibc for?

[Marc Tanner]

Does somebody know for what purposes the klibc which is now in -mm will be used for? I guess this implementation will only provide a subset of the features found in uclibc, but i am not sure about that. Will it even be possible to link userspace programs against it?

I am a little confused right now, have to read up on it.

Re:Diversity in advocacy

[cab15625]

Yeah, everybody knows that it's those lewd Sports Illustrated swimsuite calendars and foul language that keep women from becoming automotive mechanics. It has nothing to do with generations of parents/teachers/preachers telling children "boys do these jobs and girls do those."

If you want to carry out advocacy of linux to women, then it will take more than just "clean" screen backgrounds and coming up with new terms for male and female plugs to clean up the industry language. What you should really be worried about is a society that still tries to raise it's children to think that only boys can do certain things and only girls can do certain others. What are you teaching your children to think of their abilities?

If pictures and language are what you think deeper problems are, you should probably petition slashdot to make the pinkified april fools theme the default and then go do some ladies things.

As for profesionalism, different industries/companies have different standards irrespective of gender. At work, I keep a blank background because that is what is acceptable where I work. At home, I put whatever I feel like. It doesn't have to become a gender issue.

I really must learn to leave these offtopic threads alone.

Re:Video Editing?

[miracle69]

I recently ran across jashaka which is also cross-platform.

Re:suggestions for a laptop

[cab15625]

Wow, looks like a sweet machine.

My advice, if you really want to be sure, take the time to go through the hardware specs and check the kernel source for drivers for each thing you definitely want to have running (be careful of versions)

You could also check out the linux on laptops site

Finally, I've noticed that the biggest problems for me in the past have always come in with ACPI support. This is where the most noticable improvements(*) in the kernel have come for me lately. You might want to check out the ACPI4Linux site to see if there's anything special that's been discovered about your system yet.

* It's actually not a problem with Linux, it's a problem with the way some OEMs do ACPI using tools from M$ that the kernel guys have been doing a better and better job of working around. Who needs standards when you think you pwn the world.

S-ATA hotplug

[Dolda2000]

Looking through the ChangeLog, I still see no S-ATA hotplug. I've been waiting more or less since the day S-ATA support was introduced in Linux to be able to add new drives without rebooting, and I just cannot understand how such a thing can take so long. I mean, I'm sure that the kernel developers have priorities and stuff, but I would think that adding S-ATA hotplug ought to be simple and important enough not to take more than a year to even get started...? I don't mean it as a complaint, I just find it really weird. Is it just much harder than I think, or is there no particular reason for it not having been done?

Re:S-ATA hotplug

[The_Morgan]

I setup a backup system using DAR and SATA hard drives, since they are a fairly cheap backup medium. I wanted hotplug, really just hotswap so I could take out the full drive and put in an empty one. The commands I found after searching went along the lines of:

echo "scsi remove-single-device 3 0 0 0" >/proc/scsi/scsi
echo "scsi add-single-device 3 0 0 0" >/proc/scsi/scsi

It has to be watched through dmesg, sometimes the drive migrates from /dev/sdX to the next unused letter. The other thing I noticed is that the drive has to be online while booting up to be available, it wouldn't find a new drive. It has been a while since I set the system up so I should upgrade to find out if this still applies.

Re:S-ATA hotplug

[labratuk]

http://linux-ata.org/software-status.html#hotplug

It's not just a matter of enabling it, it's about making it reliable.

More here: http://linux-ata.org/features.html

Re:Broadcom 57xx

[LinuxOnEveryDesktop]

what about the Broadcom 57xx?

That's Broadcom's wired gigabit interface. Has been *long* supported by the tg3 driver. Heck, Broadcom even has an alternative GPL'd driver for this interface downloadable from their website.

Re:module shotguns

[FireFury03]

Sure there is. There's just not a consistent ABI, and that's on purpose.

If you're contributing a driver, GREAT. It'll compile against the currently installed kernel just fine.

Untrue I'm afraid. If your modules aren't in-tree then they *will* break every so often because the kernel API is not stable. Especially under the 2.6 development model - under the previous 2.4/2.5 model you were pretty much guaranteed that API breakages would only be happening in the 2.5 tree, now they happen at any point in the 2.6 tree. (Yes, I do know this stuff - I work on out-of-tree kernel code).

There is some arguement that all drivers should be in-tree, and for common hardware it is definately a Good Thing to have the drivers in the tree - as the API changes then the person implementing the API change will fix up all the in-tree code that uses that API.

For very specialist and expensive hardware it poses a problem though: the person who does the API change won't have the hardware to test with, and probably all the people who use that hardware are using enterprise distributions so breakages to the module won't be spotted for a long time. It's hard for the hardware vendor to track these kinds of updates and perform the necessary regression testing.

Re:Video Editing?

[Khuffie]

Spoken like a true mac fan boy. Final Cut Pro is great, but Premiere Pro really cuts the gap. And After Effects still remains one of the better compositing tools out there.

There are valid uses for a GOTO

[Dan Ost]

I agree that using GOTO is a bad idea when another control structure is adaquate,
but, at least in C, there are times when using GOTO is the most natural and,
unequivically, the best choice.

Off the top of my head, I can think of two situations where using a GOTO is
the best solution:

1. breaking out of nested loops. In C, the break command can only break
out of a single loop level. If you need to break out of 2 or more loops, you
can play an ugly game of setting and checking state flags at each level
of looping or you can simply create a label at the exit point and use
GOTO to get there. (sometimes you can wrap your loops as a function call,
but that's often the ugliest solution)

2. shared cleanup code. In a function with multiple exit points, instead
of doing cleanup at each exit point, it is often clearer to set your
return value and then GOTO a label that handles all cleanup before
returning.

Be cautious when using GOTO, but don't be afraid of it. Learn to
recognize when GOTO is appropriate and when it should be avoided.

Re:There are valid uses for a GOTO

[Dan Ost]

1) If you can create a condition where a goto is to be placed, you can add that same condition to the top loop in the nest and let it exit out gracefully.

If that leads to clearer code, then in the cases where you can do that, fine. Do that.

However, there are situations when a condition doesn't make sense until you've already
entered the nested loops at least once (for example, when allocating lots of chuncks of memory,
you can't test to see if you've successfully allocated memory until after you've tried to
allocate memory). Also, if there are several conditions that might require a break, but
they can all be handled the same (at least until after you break out of your loops),
do you really want each one to be tested at every loop test? Think how big and confusing that
would make your continuation test for your outer loops.

2) Use a clean-up function. It will return to the correct place without all the spagetti code.

There's nothing wrong with using cleanup functions if they are convienent for your
particular purpose, but if you have to free 11 objects before returning, then you'll
need to pass all 11 to the cleanup function each time you call it. I don't know about
you, but I usually find functions with 5+ arguments to be ugly. I would rather simply have
a 'goto cleanup' that jumps to a label that does all the cleanup in place. An acceptable
compromise would be to define a macro that does the cleanup in place but hides it from casual
code inspection, thus keeping the code clear, but avoiding the use of GOTO.

Using GOTO in the manners I've described will not lead to speghetti code since the flow of control
will be clear and uni-directional (the antithesis of speghetti code). In case (1), the use
of GOTO is equivalent to raising an exception in Java, C++, or Python from within the loop and
capturing the exception outside the loop (idioms commonly accepted in all three communities).
In case (2), the use of GOTO maps multiple exit points to a single exit point. If you feel
that these techniques qualify as speghetti code, then I would suggest that you've never
seen real speghetti code.

When Djikstra wrote "Goto considered harmful", he was talking about using GOTO to jump outside
the scope of the current function, something not possible in with C's goto (C's goto can only
jump to a label within the current function). See BASIC and PASCAL (I think) for examples of GOTO that
can jump anywhere in the program.

Re:There are valid uses for a GOTO

[Procyon101]

For the record though, these are *BAD* habits in C++. Use the destructor, Luke.