Slashdot Mirror
Should Linux Have a Binary Kernel Driver Layer?
zerojoker writes "The discussion is not new but was heated up by a blog entry from Greg Kroah-Hartman: Three OSDL Japan members, namely Fujitsu, NEC and Hitachi are pushing for a stable Kernel driver layer/API, so that driver developers wouldn't need to put their drivers into the main kernel tree. GKH has several points against such an idea." What do you think?
Linux, with the stability of Windows prior to the advent of certified drivers.
Lets put it in Kernal 2.7.
Oh wait... OOPS!
Having a kernel API for drivers allows developers to stay away from the mainstream kernel. This will enhance the stability of the kernel in general and also allow hardware vendors to support Linux with less effort.
To Terminate, or not to Terminate, that's the question - SCSIROB
No thanks, this is just a great way to promote closed source inside the linux kernel and to make debugging problems totally impossible.
Shadus
Microsoft could license code into the drivers or otherwise maneuver the driver makers to license their IP from MS, the drivers could form a layer between Linux and the hardware, Microsoft could then pull the run from under Linux.
Don't go there, it protects Linux from getting tripped up, and devalues any hardware that doesn't support Linux.
Don't underestimate the important of driver support for Linux, you practically can't make any server component without a good solid Linux driver.
I have been waiting for that for a long time. The lack of a stable interface is hampering adoption of Linux. Not all of the manufacturers are willing to open source drivers or for that matter to continuously change them as the APIs change. This is very welcome but unfortunately, I think they'll fail. There is just too much politics surrounding Linux these days.
http://www.kroah.com/log/2005/11/07/#osdl_gkai2
:-)
Some misunderstandings were made. But of course, if they posted this link, there'd be no point to posting TFA or the arguments that will almost certainly follow.
They'll think I've lost control again and leave it all to evolution. -- Supreme Being, Time Bandits
I gave up Linux mostly because I was tired of getting punished for having new hardware, which is often unsupported. Especially on laptops.
If you don't force the manufacturers to include their driver source in the kernel, you might get them to release actual drivers for their new hardware.
Toronto-area transit rider? Rate your ride.
It's a bad idea because what happens when the driver ABI changes? You have to wait umpteen months for the company to get off thier asses and fix it - like nVidia.
It also precludes anyone else from fixing bugs in the broken, half assed crap most corporates spit out these days.
feh. stuff.
Not just Heresy, but Linus has said directly that he doesn't want a stable binary kernel driver API percisely so that people *can't* write binary drivers for Linux.
I am unamerican, and proud of it!
one of the main problems for getting device manufacturers to support linux is the fact that they either have to release a new version of their driver every time the linux kernel changes some esoteric internal API, or be badmouthed for not having good linux support.
would it really hurt so much to guarantee a stable DKI? doesn't have to freeze the whole kernel, just a subset of functions that will be guaranteed to work as they do now in perpetuity.
backwards compatibility is just as important to driver writers as it is to app writers.
doesn't even have to be binary backwards compatible, source level would be sufficient for most.
The difference between Theory and Practice is greater in Practice than in Theory.
One of Linux's great strengths is the flexibility of changing to meet new needs and not being hobbled by rigid backwards compatibility. This can only be done if all source is open and anyone can update drivers to meet new needs. When someone comes up with a patch to streamline a certain minor part of the kernel, it frequently has repercussions elsewhere in kernel land. It is these small changes which have made linux better and better with breathtaking speed. A "stable" binary API removes the possibility of keeping everything up to date and would dramatically show down the adoption of new features and general improvements.
Continual refactoring is worth far more than some supposed binary API which prevents changes. Get rid of binary drivers! If companies are so paranoid that they want binary drivers, then the hell with them. Linux can advance better without that baggage.
Infuriate left and right
As someone who has tried to install various Linux distributions on RAID cards, and has had difficulty getting installers to use even third-party open-source drivers*, I'd love a binary driver API.
As someone who supports free software, and has struggled with NVIDIA's video drivers (and they're at least trying to meet us halfway by making it as easy as possible to install their closed-source driver under the current system) I can see the negative consequences of encouraging binary-only drivers.
*Example: Promise SX6000. Old cards work with I20, newer ones use their own interface. An open source driver is available, at least for the 2.4 kernel, but good luck if you want to get your installer's kernel to use it. Unless you can create a driver disk, a byzantine task in itself, you're stuck with a few outdated versions of Red Hat, SuSE, and I think TurboLinux.
Actually, all drivers should be outside the kernel, as in QNX, and now Minix 3. But it's probably too late to do that to Linux.
These companies want a binary layer so they can build binary drivers.
What people tend to forget about this is that it's a bad idea- from most every perspective.
The Linux kernel was written as a Free Softwate alternative to the existing *nix systems.
We have thousands of drivers in the kernel from a combination of development efforts. Sometimes a driver is written by an independant kernel developer, and sometimes it's written from the company producing the hardware, working alongside the community.
What these companies want is to be able to have thier cake without giving back to the community. This is a very slippery slope at the least, and illegal at best, since these sorts of links to binary kernel drivers have been long known to be illegal to distribute alongside the kernel (unless special previsions are made, such as a userland driver).
Also, binary drivers have been known to be buggy and essentially removie the kernel developers from a position where they have control over the kernel as a whole project. I won't even go into the issues associated with a possible security hole in a binary driver, or a binary driver with, for example, spyware in it.
The arguement for it is, of course, that this might mean more drivers. This is a test of our strength as a community. Doing the right thing is harder. It means we won't have all the hardware at all times, and certainly not the newest thing. But we retain control over our computers.
It's hard to say no, but this looks like a clear case where we have to.
Please read The Linux Kernel Driver Interface (all of your questions answered and then some) by the same author before commenting...
IANAKH, but couldn't more drivers be moved into userspace (or other lower rings) --- especially for things like USB printers and miscellaneous gizmos? I think it would also be nice to not bundle thousands of drivers and support for architectures I don't have with the kernel. The kernel itself could provide a very minimal layer of hardware protection (like an exokernel?) and there'd be libraries exporting generic abstractions for particular classes of hardware. Is the context-switching penalty really so great these days? Educate me!
In my opinion, binary drivers are worse than no drivers at all because they release the pressure on the manufacturer. They can say they support Linux which in case of binary drivers is simply not true.
Save the bandwidth. Don't use sigs!
One of Linux's biggest problems is the lack of device drivers for common devices, especially newer video cards. Let's face it, companies like ATI and NVIDIA aren't going to release fully open-source drivers. It would be wonderful if they would, but it would also be wonderful if we had flying cars.
Having a stable binary driver interface would make it easier for hardware manufacturers to embrace Linux, give things like wireless chipsets more usability on Linux and drive further adoption of Linux as a viable competitor to more proprietary solutions
The perfect is the enemy of the good, and the more Linux gains a foothold the better it is for open source. Insisting that device manufacturers need to have on-staff kernel hackers in order to keep ahead of a frequently-changing kernel makes it that much harder for manufacturers to support Linux as a viable alternative.
Provided Linux can have a stable binary driver infrastructure that doesn't harm stability, it would greatly help in the adoption of Linux worldwide.
But we retain control over our computers.
Tell me, what BIOS do you run ? Do you have the source to the firmware in your IDE disk drive ? In your CD-ROM/DVD-ROM drive ? Do you have the source to your SCSI controller's firmware ?
If you think you have control over your computer you are suffering under a delusion.
If a company is developing an embedded Linux ap for their own hardware. All of a sudden, all of the communications with the board-specific hardware is being done through binary drivers, resulting in an effectively closed system.
No more hacking WRT54G's for you, chump.
"Eve of Destruction", it's not just for old hippies anymore...
Linux was, is and hopefully will always be "open". I don't want closed drivers in the kernel (even via an API layer) any more than I want a Sony rootkit masquerading as DRM.
It isn't about "politics". It's about policy and philosophy.
If the hardware doesn't work with Linux, don't buy the hardware/pester the vendor for an open driver, or don't run Linux.
This is the problem with the open source movement. Putting the code before the user.
And this is why you fail.
-everphilski-
It is not welcome. Linux is about Open Source, and allowing people to link-in binary closed drivers goes against this.
Bypassing the dogma of the above, there are numerous pragmatic reasons why this would be better for linux, even if you don't include support for binary third-party drivers.
Sure, some of these are extreme cases. You can usually get away with just re-compiling the driver, and occasionally, you can even use the binary from the existing version.
The point is you should *always* be able to do this wihtin the same major kernel version. There is no technical reason, aside form the politicis of not wanting to ever allow binary drivers, to not have a stable driver API.
Imagine if the Mozilla plugin API changed with every new version of Firefox. And look at all the complaints when a new Firefox version doesn't work with all the old extentions. It is the exact same.
If you are going to take the strategy of "We will do things to attempt to force you to do thigns our way," don't be supprised if the response of companies is "Fine, then we will ignore you."
The simple fact of the matter is that most companies are not willing to go open source, for software or drivers. You can argue that's a bad thing, but it is the reality of the situation. So, if open source is out in their book, either because of contractual obligations or mentality or whatever, they are left with two choices:
1) Do Linux drivers, and update them every time the interface changes, which can be as often as every minor kernel revision.
2) Ignore Linux, and let the community write the drivers if they want.
The problem is that Linux is a bit player. They are larger than the other bit players, but they are still tiny, less than 10%. Given that the continous rewrites can get expensive, the choice for many will simply be not to write the driver.
So if you are ok with that, then great, but don't get mad at companies when they won't play by your rules. Are they being unaccomidating? Sure, but so are you.
In the end, it comes down to needing to make a decision of what you want Linux to be. If you want Linux to try and become the next big thing in OSes and start to really make an entrance in the home market, standardisation is needed. Standard APIs, standard UIs, inter-version consistencies, etc. In essence, it needs to become more like OS-X. Now if you are ok with Linux being more of a geek/server OS then that's not necessary, but you can't demand the world change around you.
Assuming that we had a stable kernel source interface for the kernel, a binary interface would naturally happen too, right? Wrong. Please consider the following facts about the Linux kernel:
But see, the thing is... a "stable" binary interface requires that structures used specify padding, alignment, and fields to be fixed! If these can vary, then by definition , it's not stable. Ditto the variations that depend on kernel build options.
Now, if you want to make the case that it's not possible / practical to make an interface that can cover all of these conditions adequately, well, by all means, do so (though I'd say that the hundreds of existing operating systems with binary interfaces show that this isn't the case in the general sense).
But what I see here is a relatively weak technical argument that is being used to justify an ideological decision.
Having to "hunt down drivers" is an artifact of the old third-party binary driver world. When hardware specifications are available to developers, those developers can add the hardware support to the kernel -- which means it ships with the distribution.
If there's one thing you'll guarantee by providing a binary-only driver interface, it's that you'll have to spend a lot of time hunting down drivers.
I am not a Kernel Developer, but I know some. ;) I guess that my open question is how this would benefit kernel users? Yes, I see it would reduce the workload of kernel devs. Yes, I see it would allow driver developers to not have to go through the kernel code vetting process. But, the kernel code vetting process is what is a strong benefit of using Linux, from a user perspective, as I know that the code is well tested by an army of users and developers.
Once you push driver development out of the kernel, yet give access to kernel internals in this way, you introduce a level of uncertainty in so far as stability and robustness is concerned. One must question why these big comapnies are pushing for this, but most human kernel devs are not.
GKH raises good points about how a stable binary driver interface will open the floodgates to both security problems and to update/maintenance problems. As it stands right now, Linux kernel developers can quickly respond to threats because they are able to fix all instances of a given problem, in all drivers, at the same time. If they do not maintain this flexibility, either some drivers stop working unexpectedly when security fixes are made and the interfaces are forced to change (making Linux appear "unstable") or backwards compatibility must be maintained making the Linux kernel grow over time (whenever a new interface has to be written to address flaws in the old interface).
Yes, abstraction is good...but, in this case, stability, the perception of the user and maintainability (where the *real* costs lie) must win over abstraction. Most of the kernel developers are not being compensated; how often do you think that backwards compatibility is going to be maintained? Its not. Right now, fixes are accomplished because it is easy to accomplish - global search and replace, etc. Make it difficult and it just won't be done.
Manufacturers want binary drivers because they want to play for free - they want all of the benefits of open source without any of the costs. Not cool.
Ignoring for a moment, the ideological points, lets consider a frequently raised practical point: the idea that an API would either get out of sync with the kernel, or be a drag on innovation.
/lib/modules/binary/" than "you need to recompile and reinstall the whole kernel with this patch"?. For obscure hardware which will never be in the official kernel, it would be nice to have the ability to easily use it with any linux distro. No need for a slew of precompiled RPMs, DEBs, and a user-unfriendly source tarball, just one driver binary and you're ready to roll.
I agree that when the Linux kernel was young and untried, standardizing a binary API was bound to become a millstone in a short period of time, as the kernel internals were in a constant state of churn and iterative improvement. Nowadays though, surely, the kernel has been "shaken down" enough that it could afford to commit to binary APIs that are stable at least throughout each minor version number?
Returning to ideology, I can see how a stable binary API would be useful even to open source hardware. How much easier is it, to say "drop this file under
In itself, that says nothing, either pro or anti, about the availability of driver source.
What are his reasons for not putting video card drivers in the kernel, like other Unix operating systems?
Why do we still have to have a user program (X) with device drivers in it? (Would anybody think it's a good idea if the Linux kernel didn't have any sound drivers, and required gstreamer to implement its own?)
It seems we have two competing driver models in Linux: some are in the kernel, and provide a consistent interface (sound cards, SCSI/IDE/... cards, network cards), and some aren't in the kernel at all, but expose them at a low level and rely on userspace programs to provide actual drivers (X11 for video cards, CUPS for printers).
I'm against a binary API, not on philosophical grounds (I like gstreamer's binary API), but because it simply never works: I've tried to use binary-only drivers under Linux in the past, and it never works nearly as well as open-source drivers. But whewher or not you agree with Linus' open-source philosophy, can we at least all agree that we need to put drivers in the same, correct abstraction level?
They won't release now, but eventually they will. You don't cave in to "terrorist threats", ever. And that's what this is. If those vendors want in on the open source market that revolves around the linux kernel, then they can play ball, too, end of story.
Linux is at a really important part in its evolution. Caving in to closed source interests would be counter productive in the long term. It is better to force/cajole the vendors to finally "see the light" with open source and the GPL. Maybe eventually another graphics card vendor will appear and become the champion, precisely from having open source drivers. The manufacture of electronics in Asia in particular is exploding, where there is a market, a vendor will appear. Patience.
the question I have for this is "why?" wouldn't a stable binary API likely result in far more third party hardware support for linux? possibly more laptops that are actually compatible with linux?
This seems like a case of open source programmers shooting themselves in the foot because they want everything to be open source... not every application and driver is going to be made open source just to suit the desires of the linux development community. It seems to me that sticking to a hard party line against closed source software instead of trying to co-exist with said software is bound to keep linux in relative obscurity and pretty much ensure that it never becomes a viable competitor in the desktop market.
"In America, first you get the sugar, then you get the power, then you get the women..." -H. Simpson
More and more hardware is being implemented in software. This makes sense because as the CPU has increased in speed and capability it has gained the resources and free-time to pick up the slack for less-expensive hardware implementations. If you look at the new push for multi-core and multi-CPU systems this will be even more true. Plus, as a bonus, if they mess something up then they can more easily get people to install a new driver revison successfully than to flash a device or fix a problem in the silicon.
The problem for the open-source community is that these drivers are increasingly not just the way to talk to autonomous hardware but actually implement alot of the fucntionality of the device. Taking this in mind, the manufacturers are unlikely to give out the source code for these drivers as they will be giving their competitors a more significant view into their playbook. People in the Linux community complain about the lack of certain drivers like those for wireless cards in many notebooks, which in my understanding work as described, and are left hacking together a solution to run the Windows drivers under linux in order to get them to work. If you want things like that to work and work well in Linux then you have to give them a stable subsystem and make it as easy to port their drivers as possible while providing them the means to not give away the source which contains half of their work in creating the hardware.
I know that is not the ideal solution but thems the breaks. It is either Linux steps up with an API and binary subsystem or they will be left with fewer hardware options consisting of what more expensive all-hardware alternatives are left for many of these peripherals.
Firstly, I get paid to do driver development. Secondly, I have never worked for a company that willingly said, ok lets open source the drivers we spend thousands of dollars writing and optimizing.
That said, this whole discussion is as silly as the one about not putting a kernel debugger in the main kernel source code. Frankly, linux desperatly needs both a kernel debugger, and an ABI to be a REAL alternative for many customers. It also needs the ABI for driver developers so that we can write a single driver and expect it to work on the dozens of flavors of linux we are expected to support. Saying that everyone should opensource their drivers is like saying food should be free. It isn't going to to happen and wishing for it, won't make it happen any sooner. In the interm, almost all the hardware out there has better support on windows (Our sysadmin can't even get support for major linux distributions, from major hardware vendors, even when they have little linux logos on their hardware and websites) the windows drivers tend, to accually work, and they almost always have better features sets. This isn't going to change as long as the "opensource" community treats hardware vendors that think they have IP in the driver as second class citizens.
Oh, and for people who don't accually work for hardware companies that ship drivers, driver development is often times an expensive process, not because the software engineers are expensive, but because the hardware and software needs to be tested and certified in particular enviroments. There are orders of magnitude more linux distributions this makes it cost orders of magnitude more to test and support than a half dozen windows enviroments, most of which can be tested at microsoft, or one of the major OEM's if the hardware isn't avialable onsite. Putting 10x the money into a market that may be less than 1/10 of sales is not always a good idea, especially when resources are limited. Creating an proper ABI helps to solve this problem.
That said, if the damn linux kernel accually had a real architecture, it could support an ABI, and even isolate itself from rogue drivers. As it is, the kernel arch is pretty much non existant and just a pile of code that tends to behave like a real kernel, except when you try to do something a little outside of the mainstream desktop or small web server enviroment. This was fine when the whole kernel was just a few hundred thousand lines, but given its current size its getting massivly unmaintainable. This is proven by the fact that linux system stability seems to have gotten really bad over the last few years. Getting to a stable system, takes a lot of vendor testing by the likes of Suse, Redhat, etc.
Lastly, the tainted concept works fine for the kernel developers, why not carry it forwared so that any binary driver simply marks the kernel as having a binary module loaded, and uses the standard abstract interfaces instead of linking against all kinds of unneeded kernel crap that just provides the posibility to screw something up.
Also a lot of people mix up the issue of "binary vs C" with "closed vs open". You can have open source binary drivers, and that's often a convenient thing for end users to have (because the user experience of installing binaries is generally better than that of installing things from source code).
One of Linux's great strengths is the flexibility of changing to meet new needs and not being hobbled by rigid backwards compatibility.
Yeah, let's hobble it with dependency hell instead.
This is necessary if we ever want Linux to be ready for the desktop. The ability to have driver modules is certainly more advanced than having everything compiled into the kernel, but it's severely lacking in many regards. The module has to be custom made for each kernel, making binary distribution useless because there are 2^100 kernels out there. So unless the manufacturer open sources the driver, they can't make a driver for Linux. Or you could go with an open source interface to a closed source driver.
Here's a sample of what I put up with. I downloaded Agnula Demudi 1.2.1 and installed it with the 2.6 kernel. I was ready to install some Nvidia drivers. But after some searching, I couldn't find any binary driver interfaces compatible with my kernel. Fine, I can compile my own. So I download the interface sources and launch module-assistant. It complains that riva driver support in my kernel conflicts with the nvidia driver, and I need to recompile the kernel. (I then went through the joy of trying to find the hidden demudi sources and figuring out how to patche them and configure them, ultimately failing to compile it, but this is getting away from the topic.) Finally, I said screw it.
You might blame the distro, but it's really the kernel at fault here. Recompiling the kernel to support a driver is NOT something that a user should have to do. Windows does not require you to recompile your kernel to install drivers.
First of all, I completely understand that drivers with source are better than binary-only kernel drivers. However, the fact of the matter is that companies do not feel comfortable freely releasing their intellectual property. Given the choice between a piece of hardware in a laptop which doesn't work with Linux at all and one that works with a binary-only driver, I would rather have the binary-only driver.
As long as Linux kernel developers complain that binary-only drivers are "illegal", Linux will have less hardware support. One of the major complaints people have against Linux is that a lot of devices that one can attach to a Windows machine plain simply do not work in Linux (I still think Linux is far behind Windows when it comes to wireless drivers, for example). I want to see a true alternative to Windows on the desktop; GPL fanaticism and an inability to understnad how big corporations work harms this.
What you seem to forget, is that for the majority of users, Linux is a 'free' unix like system, and they could give a rats ass about "Free". If "Free" starts making things difficult to use, or preventing companies from supplying drivers for hardware users buy, then not only do users not care about "Free", but they will actively dislike "Free".
The bottom line is that having code be open is only important to a fraction of *developers*, and an extremely small small fraction of the general populance. Ultimately, "Free" software people want to push their ideoligy on others, they don't care about makeing functional easy to use systems.
So preach all you want. Very few people care.
At the end of the day, there might not be a choice at all. What's to stop them forking the code and developing their own binary driver api? If people (and by people I mean businesses) want to use the hardware of these companies, it might become widespread.
So in other words, Linus has decided to deliberately damage the kernel (by introducing unnecessary instability) in order to make it harder for people to write software that uses it.
I thought that Linus usually put technical correctness above political correctness, but it looks like I was wrong.
The most rabid believers in American Exceptionalism are the exact same people whose policies are destroying it.
The BIOS is indeed an issue, and there are efforts underway to make a Free BIOS.
But why not try our best to have as much control as we can?
Linux should support user-space drivers. Probably through FUSE and some other apis. These can then be binary, just like any other appliation. If they crash they will not take the system down. The API is limited but you will be able to open/read/write them. ioctl can be done with Plan9 style names, ie open "/dev/neato_device/volume" and write the desired volume there, etc.
Drivers that need more elaborate API's or need more speed will be stuck with the mutable binary interface and occasional GPL restrictions. Too bad. A lot of interesting drivers do not need this speed. And those that do may force the interface to user-level drivers to be improved until it is usable, which is a very desirable result.
They don't release the source. There is no source. That's the whole point. They just release the programming docs so anyone can write a driver. There is no IP for them to protect.
For an OS which is continually evolving and was not designed with a lot many future developments in mind, it is very natural to say no to the stable binary API/ABI concept for drivers. But as it matures and there is no longer a need to fix interfaces to support some out-of-world functionality, the driver interfaces are automatically going to be stabilized. (Unless kernel folks decide they get bored with having one function name for more than a year or that they want to keep driver writers continuosly on their toes - all of which is unlikely.)
/ 31/363790.aspx).
API/ABI compatibility obviously has it's own pros and cons - some times it's impossible to break things, take Windows for example. The world is going with LP64 model for 64 bit machines but Windows developers had to stick with LLP64 just because they made some design mistakes and now they cannot break the tons of applications. (See http://blogs.msdn.com/oldnewthing/archive/2005/01
Linux on the other hand can afford to break and fix things until the time where binary and out-of-tree drivers grow to out number the in-tree stuff. By that time I guess there will be a very less need to break things such as driver interfaces and the like.
And I think the mad rush to put everything in the official kernel tree is not a good idea from maintenance and complexity stand point. So if and when the Linux ABI/API stabilizes that will be a good thing for out-of-tree kernel drivers and Linux itself.
The day you can get third party IP licensors (e.g. that nice crossbar architecture used in previous-gen nVidia chipset memory control blocks wasn't developed in house by nVidia and they have contractual obligations not to release interface specifications for it) to agree to have their interfaces open by the licensees is the day you'll have fully open register-level documentation for consumer 3D graphics chips.
"The Devil does not know a lot because He's the Devil, He knows a lot because he's old." -- unknown
For graphics, GGI and KGI would allow direct binary-only drivers to be written that applications can use, again without modifying the kernel.
Not sure whether you could make any use of the ABI/IBCS work for drivers, but they certainly allow "foreign" binaries to run under Linux, without anything foreign being put in the kernel itself.
In other words, a binary-only driver layer would seem unnecessary, given alternatives and mechanisms that already exist. It may be useful in some cases, but I can't see how it would be essential.
You could also use Xen as a reverse microkernel. Have foreign drivers in a "driver-only" mini OS, running as a parallel kernel. Then all Linux would need would be a way to call the other OS across Xen - and that need not be binary-only/closed-source. Companies interested in binary-only Linux work might even jointly fund development of such a capability.
The problem is not with the kernel, or even with the kernel developers. The problem is that corporations have unofficial choices rather than something they can put the blame on if their coding is crap. Officially sanctioned solutions are always preferred when being able to blame someone else is important.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
If you give companies a way to use closed source drivers, that is all they will ever use, because that is the way they've always done it. Linux presents a serious change to the way hardware vendors do business, and in the long run I think it would be better for everyone if they would embrace it.
It is just that once you open the gates, they won't ever be closed again.
This is an inflammatory issue, but a stable interface doesn't necessarily open the kernel up to proprietary drivers. It's a matter of licensing. Any third party could introduce a GPLed abstraction layer. There are big practical advantages to being able to take a GPLed driver's object file and plug it in to any old kernel. In that case there would be no really fundamental reliability or debuggability problems. The remaining problem would be the increasing mismatch between the abstraction presented to the driver and the abstraction supported by the kernel as it develops.
It would be good to separate the discussion into two, one for inflammatory license- and/or ideology-related culs-de-sac, and another technical, to address legitimate needs for stability in drivers that are not (yet) in the kernel tree.
There's a joke- programming is a lot like sex. One mistake, and you're supporting it for the rest of your life. By this measure, writting a device driver for Linux is a mistake. Because even after the feature set and the hardware itself is stable, you are letting yourself into a never-ending task of constantly changing the driver to keep track of the current API changes. At a previous job we spec'd it out at about 1/2 of a full time position to keep up with all the kernel changes. This is your job, from now until eternity, or at least until you're willing to have the driver declared obsolete and abandoned (after all, we all know that there is just three types of software- vapor, beta, and obsolete).
Well, congratulations. You have your religous purity. And guess what- it's comming at a cost. You wonder why Linux isn't more popular on the desktop? Well, here's part of the reason. It's hardware support will always- ALWAYS- be behind that of Windows. Why? Because when the hardware ships, it ships with Windows drivers that the hardware vendor wrote with it. Note that Windows pulls the same sort of API changing crap that Linux does. The difference is that the hardware vendors look at Windows, and the half man-year per year cost of supporting Windows costs, and go "but we have to support Windows if we want to sell more than 3 units." They then look at Linux, and the half man-year per year cost of support Linux, and go "Supporting Linux is not cost-effective at this time." I know, because I've seen this happen. So now the hardware is out there. And now we wait, for someone willing to step up and volunteer the time to write, and maintain indefinately, the driver. Someone less capable of doing it than the hardware manufacturer (this isn't to question the capabilities of the current kernel developers, but the fact of the matter is that there is a huge advantage to being three cubes down from the hardware developers, and capable of wandering over and asking direct questions, instead of having to reverse engineer what is really going on, having worked both ways).
So this is the fundamental question: which is worse. Having binary-only proprietary drivers, or being forever behind in hardware support and not having people contribute simply because they don't feel like having to constantly update the driver once they finish it, they'd like to be able to move on. I come down on one side, Linus and the kernel developers down on the other.
Fine. Their kernel. Their problem.
On network drivers at least, the driver has such a small interface the driver is put through a full formal methods based proof system to prove the driver doesn't have certain classes of bugs. I hear MS has quite a cluster devoted to model checking network drivers. Granted this doesn't work for all classes, but at least some get fairly rigerous verification. I personally wish Linux drivers got this kind of checking.
"Basically, I want people to know that when they use binary-only modules, it's THEIR problem. I want people to know that in their bones, and I want it shouted out from the rooftops. I want people to wake up in a cold sweat every once in a while if they use binary-only modules."
- Linus Torvalds on linux-kernel
And many people forgets that non-gpl drivers may be very well impossible to write at all (at least some lawyers think this), drivers are not at all like an app is WRT to gtk, drivers are more like "plugins". Plus, a closed driver module makes MUCH HARDER to debug bugs if the driver is doing bad things, and you can't know that (which makes harder to stabilize and/or develop the kernel. Several closed drivers can make it a hell or impossible at all.
So I wrote a nice driver for it, with help from the excellent Rubini book since there's no frickin' driver documentation and the Linux sources are uncommented (putting your name at the top doesn't count as commenting on any team I've been on). It worked for a few months, until the next incompatible change in the kernel. Then I poked around in the kernel until I thought I'd figured out what had changed. Got it kind-of working (DMA is unreliable and I don't know why). Then the 2nd edition of the Rubini book came out, so I rewrote the driver and got it semi-OK again. For a few months.
The kernel has had about a zillion incompatible changes in the driver interface since then, ALL UNDOCUMENTED, there's no 3rd edition of Rubini's book to sort it out for me, and I'm just lost digging through the kernel sources. It's always the same pattern: customer complains, I spend a few days fixing it, and it sort-of works for a few months until the next gratuitous interface change.
OK so I'm a for-profit vendor and therefore deserve to die (how dare I want to be paid for being a programmer, it's only people who don't create new IP who should be allowed to choose what they do for a living). But I'm not the vendor. I wrote the driver for their device (and plenty of other open source stuff) because it needed writing and no one else was doing it. The driver is distributed as fully commented source (obviously), and I've given it out to anyone who asked for it (uhh ... both of them -- this hasn't made me a nickel in extra sales for the package itself, I tell users who want reliable Q-bus interfacing to use my DOS version because there's no OS to break my driver there, it's worked unchanged for YEARS). The users haven't touched a single line of code on their own that I know of, all they do is complain to me. So what's wrong with this picture? Why isn't Open Source magically making everything work, at no cost to anyone?
It's just plain arrogant to assume that Groovy Open Source will always fix itself, and therefore it's OK to make the kernel interface be a rapidly moving target with no real manual. I'm sure that when we're talking about some extremely commonplace device that hundreds of thousands of people have, it's not hard at all to raise a posse and go after new bugs when the kernel changes. But this device I'm talking about costs $2575 a pop and is of no interest to anyone who isn't keeping a 30-year-old minicomputer system alive with a brain transplant. Absolutely no one is helping me maintain the driver even though they have everything they need to do it. I'm doing a lousy job of maintaining it because it's 0.0001% of my job description, and it's just not my strength (I know how to program bare metal, keeping up with Linux internals would have to be its own full-time job).
I couldn't care less whether drivers are required by law to be open source (although I really can't believe that's enforcible on a loadable driver, licenses exist to make exceptions to the default copyright restrictions, and interoperation is not copying). If people want to see my I/O code they're welcome to it. Just please stop making it break!!! Especially if you're not going to bother helping me maintain it. I've got my hands full just keeping the user-mode code working (WTF happened to /dev/vcsa7 and up?! and /proc/meminfo?!).
Anyway my point is, if Linux would pick one driver interface, and commit to supporting it (OK, possibly among other
By the way- another advantage to having a stable kernel API layer is the ability to write a dead-tree book about how to write Linux device drivers and not have it be obsolete by the time it gets published. I have half a dozen books on this subject- half of them were obsolete by the time they were published, they're all obsolete now.
A few year ago I wrote and maintained the Aironet driver. The years of maintaining the driver and supporting users (I still support them) has illustrated to me the importance of an ABI. I didn't really notice it at first, but it hit me in stages.
.config. It was hell. It was something that neither I nor they wanted to go through. I figured the problem would go away once I was integrated with kernel source.
1) Experimental driver stage. I distributed the source and had online instructions, but you would be surprised at the large number of people that I had to walk through compiling their kernel with the correct
2) Late experimental stage. I had all the problems of the previous stage, but now the kernel API was changing under me so I had to put #ifdefs in to deal with it. I figured the problem would go away once I was integrated with kernel source.
3) In the kernel. Yeah I made it into the kernel. Okay first it was the pcmcia package and then the kernel. But now I had to strip out all the #ifdefs because we don't want that cruft in the kernel, but I still had to maintain the #ifdefs for other kernels. So now I had all the previous problems, but now I had to make patches with and without the #ifdefs. I figured the problem would go aways once everyone moved to the new kernel.
4) Firmware changes. Oh no! Cisco changed the firmware which changed a bit the I/O interface. Oh and look they are still changing the API in the kernel. So I can patch the new kernel code to support the new firmware, but I can't expect everyone to upgrade kernel just for my driver. (I wouldn't even do that because the XXXX driver doesn't work so well in the latest kernel.) Now I have even more problems to deal with including everything from before.
5) Throwing in the towel. It became just too much of a time sink. Both sides of my driver was changing like mad (the hardware and the kernel API) and the poor users that were trying to make it all work with kernels that they wanted to use. All my time was being sucked up in maintaining the status quo and I couldn't work on anything new, so I turn the driver over to good hands and moved on.
Now imagine how nice would be if in the experimental phase I could release the source and a binary for everyone to use. I wouldn't have to tweak and recompile for every new kernel. Anyone would be able to just grab the binary and use it if they wanted to. (Kinda like Windows... Ironically I use ndiswrapper for my new laptop with a broadcom driver and it rocks! I've used the same windows driver in linux for the past year across many versions of the kernel. It sucks that the windows network driver ABI is the only driver ABI that linux has.) If the firmware changed, I or anyone else could fix it and everyone could use it.
Whether or not we Linux allows closed source drivers is orthogonal to an ABI. Technically you can write closed source drivers now and if you want to, you can prohibit closed source drivers with your new ABI.
Linux is actually much better at this than windows - you can see what the kernel does. Microsoft's test suite means nothing, as explained by a (great) microsoft programmer: http://blogs.msdn.com/oldnewthing/archive/2004/03/ 05/84469.aspx
"In a comment to one of my earlier entries, someone mentioned a driver that bluescreened under normal conditions, but once you enabled the Driver Verifier (to try to catch the driver doing whatever bad thing it was doing), the problem went away. Another commenter bemoaned that WHQL certification didn't seem to improve the quality of the drivers.
Video drivers will do anything to outdo their competition. Everybody knows that they cheat benchmarks, for example. I remember one driver that ran the DirectX "3D Tunnel" demonstration program extremely fast, demonstrating how totally awesome their video card is. Except that if you renamed TUNNEL.EXE to FUNNEL.EXE, it ran slow again.
There was another one that checked if you were printing a specific string used by a popular benchmark program. If so, then it only drew the string a quarter of the time and merely returned without doing anything the other three quarters of the time. Bingo! Their benchmark numbers just quadrupled.
Anyway, similar shenanigans are not unheard of when submitting a driver to WHQL for certification. Some unscrupulous drivers will detect that they are being run by WHQL and disable various features so they pass certification. Of course, they also run dog slow in the WHQL lab, but that's okay, because WHQL is interested in whether the driver contains any bugs, not whether the driver has the fastest triangle fill rate in the industry.
The most common cheat I've seen is drivers which check for a secret "Enable Dubious Optimizations" switch in the registry or some other place external to the driver itself. They take the driver and put it in an installer which does not turn the switch on and submit it to WHQL. When WHQL runs the driver through all its tests, the driver is running in "safe but slow" mode and passes certification with flying colors.
The vendor then takes that driver (now with the WHQL stamp of approval) and puts it inside an installer that enables the secret "Enable Dubious Optimizations" switch. Now the driver sees the switch enabled and performs all sorts of dubious optimizations, none of which were tested by WHQL.
(IOW: it doesn't guarantee stability or quality at all. It's just a false sense of "stability")
In that case, I would prefer that you use some other OS and not try to influence the direction of Linux development. I am quite happy with the driver situation exactly as it is, and I am not interested in giving up any ground to closed-source drivers -- it is far better to have a choice of three wireless cards with open software than thirty with closed software.
Let's see here. Manufacturers want us to create a kernel than allows them to infect and interfere with its integrity, reliability, performance, and security, just so they don't have to keep maintaining that driver as the design of the kernel continues to be improved? They want us to stagnate the design of the kernel so they can let us use their stagnant device drivers? And they want us to have a system that is no longer viably supported by staff or consultants, while they are most likely not ever going to provide system support (if they can't keep the driver maintained, how the hell are they going to provide support for an old driver)?
I'd just stay away from their hardware.
now we need to go OSS in diesel cars
This is a legitimate question I have. Why not support a system like Apple introduced with their kernel in OS 10.4? When it comes to operating systems, I am just a user. I don't hack on them. So, I could be missing something in the whole "the drivers must be open source so that they can be included in the kernel and updated along with it" thing. I would like a clear explanation why doing things the current way is better than implementing a new system that supports binary drivers in a clean way.
/ 4. I think it is a rather neat solution:
If you are not familiar with the "KPI" thing, here is a short summary from http://arstechnica.com/reviews/os/macosx-10.4.ars
"With Tiger, Apple is finally ready to put some kernel interface stakes in the ground. For the first time, there are stable, officially supported kernel programming interfaces (KPIs). Even better, there's an interface versioning mechanism and migration policy in place that will ensure that the pre-Tiger situation never happens again.
From Tiger forward, kernel extensions will link against KPIs, rather than directly against the kernel. The KPIs have been broken down into smaller modules, so kexts can link against only the interfaces that they actually need to use.
Each KPI has a well-defined life cycle made up of the following stages.
* Supported - The KPI is source and binary compatible from release to release.
* Deprecated - The interface may be removed in the following major release. Compiler warnings are generated on use.
* Obsolete - It's no longer possible to build new kernel extensions using this KPI, but binary compatibility for existing kexts that use this KPI is still assured.
* Unsupported - Kexts using this KPI will no longer work, period.
The most significant part of this new system is that the kernel itself can and will change behind the scenes. KPIs will descend towards the "unsupported" end of the life cycle only as kernel changes absolutely demand.
Best of all, multiple versions of a KPI can coexist on the same system. This allows a KPI to move forward with new abilities and a changed interface without breaking kernel extensions that link to the older version of the KPI. The expectation is that the kernel can undergo a heck of a lot of changes while still supporting all of the KPIs."
Look, Linux is an operating system. It's supposed to operate and help people do their WORK. Adding philosophical crap on something as flat as "computers and operating systems", is just laughable. The best operating system is the one that WORKS with the smallest needed effort. Windows does that pretty well. If Linux needs special extra work and purchase of extra hardware, then it's not a good operating system (from a user's point of view). Users don't care about the source code. Users want a working machine and OS.
You make some good points - but having a stable API/ABI doesn't resolve them.
Right now, the Linux market doesn't seem to have enough pressure to force driver manufacturers to spend resources developing drivers for it. That's not going to change if Linux provides a stable driver API/ABI. What it will do is lower the barrier to providing a closed-source driver. Some may think that's a good thing, but there's enough evidence in this conversation & its links to suggest otherwise:
1. If you let manufacturers produce a closed-source, binary driver, that's what they'll do.
2. If they produce a closed-source, binary driver, it'll only work on a specific version of Linux, for a particular architecture, since binary compatibility depends upon the compiler & compiler options being used.
3. Manufacturers still won't put more resources towards driver development, so you'll get a bunch of half-assed drivers with bugs that now live in kernel space, with the potential to crash your system (ala Windows BSOD)
4. You also get a bunch of unknown binary code running on your system in kernel space, meaning security flaws in those products can now compromise your entire system. And then we can't fix kernel security bugs, because that might break binary compatibility.
So, allowing binary drivers gives no real advantages over the long-term. Companies that care about Linux can and do produce Linux drivers. Companies that don't, won't. I see no advantage to making it easier to produce Linux drivers for companies that don't care about Linux - all that will do is give Linux a bad name as companies with faulty drivers pollute users' machines, and make it more difficult for kernel hackers to debug problems due to bad binary drivers.
In other words, it would turn Linux into the same kind of piece of shit that Windows is, and defeat the entire purpose of using it!
"[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz
Initial costs associated with a manufacturer-supported driver:
Ongoing costs associated with a manufacturer-supported driver:
These costs exist even if a version of the driver is merged into the mainline kernel. The only problem solved by such source-level merging is compatibility with the latest kernel version. It is not acceptable to the manufacturers' customers to be required to update to the latest kernel/distribution to be able to use the device.
Here's the key point: If there is no binary interface between the driver and the kernel, all of the above costs skyrocket. You have M kernel versions against N distributions, with the total increasing over the life of the product. If there is a binary interface guarantee from the kernel development team to change only very slowly and only extremely rarely breaking compatibility -- like the guarantee Windows provides -- then the incremental costs are containable. It is reasonable to expect that 95% of their testing on 2.6.5 is valid on 2.6.14.
The perfectly reasonable response from kernel developers is that with closed-source drivers they get stuck debugging problems that are't kernel-related (I don't hold ideology to be economically significant so I'll ignore it here, without insult to people's strong opinions on the subject). Their proposed solution is to require the driver's source before they'll help with the debugging.
From the manufacturers' point of view that's a very draconian requirement. They are justifiably concerned about intellectual property (availability of the source makes it much easier for competitors to reverse-engineer the hardware/firmware). Surely there must be a middle ground. Is there some way to have a relationship between the device manufacturers and the kernel developers that minimizes everyone's costs?
I think there is. Note that all of the above costs and issues are just as valid in the Windows world as in the Linux world. Microsoft doesn't want to deal with bad drivers crashing their systems, costing them both development/debugging time and reduced perceived stability (--> lower sales). Their solution is the Windows Hardware Quality Lab (WHQL).
The WHQL is a separate entity from Microsoft. Device manufacturers are required to submit their driver source (effectively under NDA) along with their device. The WHQL staff runs the driver through a battery of tests, probably mostly automated. If the device and driver meet stability standards set by Microsoft, the driver is signed by WHQL. Windows checks for this signature at installation time and warns the administrator if it is not present. Microsoft can reasonably refuse to support non-WHQL-signed drivers when crashes occur, for exactly the same reasons that Linux kernel developers refuse to support drivers without the source. This system has been the single most important factor in Windows' significan
Bandannarama
Open Source drivers are the cornerstone of OSS... because if you can't see how something works, or even HOW to make it work then you have nothing but a plastic box on your desk. The point is that you paid for the device, printer, video card, cpu, so you should have to have anybody's permission telling you how or when you can use it.
While I'd like proprietary drivers too, some stuff is patented or secret and they don't want to share it, offically it will never happen... Linus's view is that he can't fix things if he can't see them.. and he's not going to sign a bunch of NDAs just for drivers, he's too busy. Nor, can he gaurantee what the kernel is doing if you're running a bunch of "secret" stuff that could be doing "god-knows-what" behind your back!
Then tell us how they should solve IP issues connected with the knowledge resting in the driver. Tell us how ATI and nVidia should handle this. More and more functionality is in the driver nowadays, and the IP in there is littered with patents, royalties, licenses.... and there is also the fact that the drivers often contain company secrets, so better forget about the idea of companies putting their stuff as open source.
This sig does not contain any SCO code.
When the source code is available for modification, *everyone* benefits, whether they're reading it, hacking it, or just using precompiled binaries. When I need something new in software, I may not be able to code it myself, but I can put it in bugzilla, where others in the community can discuss it, assign it, and eventually implement it.
Of course the process is not perfect. It is a human process. That is part of its appeal -- the community supports its own software, rather than relying on a company to pump out binaries... who may not be around next week.
There is nothing elitist about saying that I am perfectly satisfied with what Linux provides, and there is nothing wrong with valuing the freedom that its open nature gives to all users -- remember, those freedoms benefit them whether they read the code or not.
From this user's point of view, the extra freedom, stability, and community that Linux provides are well worth the wide selection of ordinary hardware -- at ordinary prices -- that I choose to run it on. For me, the value of an operating system is that it provides a platform which offers freedom to me and to my community while we do our work. Putting off-brand generic wireless cards with quick-hacked binary-only drivers into a system does not help me do my work -- even on "supported" systems, that has usually just given me a headache and required a trip back to the shop -- and it would probably force some compromises in the freedom we value.
For you, it seems that an operating system is nothing more than a collection of software to be used unmodified and unexamined, while you do other tasks, without the need for a community to support you if you need to change it. Those needs are perfecly valid, and perfectly incompatible with the Free Software movement. I would be happy to help you select and purchase such a system from one of the many vendors who will sell it to you. You'll be able to plug cheap hardware with crashy drivers into it 'til your heart's content. For support, you'll be able to call the company, whom you'll soon find have invested even less in support than they did in the cheap hardware.
And if your needs should change, our community will still be here, and we'll be glad to help you find a system that meets them.
There is one thing you all keep leaving out about certified drivers:
Without them you aren't guaranteed support from Microsoft.
If you are running machines with all certified drivers and WMI/MSI installed applications then Microsoft will be right there with you until the problem is solved. You won't find it written anywhere but Microsoft gurantees that you're machine will not crash (BSOD) if you use certified drivers and MSI installed software. At home this isn't possible, but in some environments it is possible (and a good idea in other places).
In a way you are locked in to what Microsoft has approved, but if they've approved it then the problem is theirs to fix - not yours. Good luck meeting those two requirements, but if you can: hold them to it.
Get your Unix fortune now!
I know you're joking, but how about this for an idea:
... rant away open source crazies. ;)
A hybrid kernel. Open source drivers are compiled into the kernel. There is a API for closed-source drivers to run in user-space.
Does not violate GPL.
Little compromise to stability.
Developers who only want to do closed-source drivers can do so.
Developers have incentive to open source their drivers in order to have better performance and take advantage of newer kernel features (the internal APIs are updated with the kernel, the external APIs stay fixed and fall behind the feature curve).
Win.
Win.
Win.
Unless its just a philosophical question, in which case
Since the Linux community has clearly not provided a system that matches your needs, I will again ask that you do not attempt to interfere in its development by advocating changes that could end up dumping binary drivers on us. We do not want them. We do not want what they will bring to our system.
I am glad your experience with Windows XP has been so positive. Hopefully you will continue to use it rather than attempt to subvert the Open Source movement with your incompatible agenda.
I'm glad I'm not the only one who recognizes that Project UDI would benefit Linux and free software developers in general. Isn't it obvious that every time someone successfully standardizes open interfaces, major leaps in productivity follow?
Without open standards, the Internet would not exist. Various proprietary networking standards (Novell Netware, IBM channel architecture, Banyan Vines, etc.) used to work in their own isolated worlds, rarely speaking to each other. And people generally thought that was okay, because that's how it had always had been, and look how well each one works in its own little world! Similarly, email systems were proprietary and incompatible. Then along comes TCP/IP, SMTP and other open Internet protocols, and the world is transformed. Suddenly, everything can talk to everything, and with the 20/20 benefit of hindsight, it's clear to all how much better it is with the Internet than it was with all those proprietary islands.
Many implementations of the Internet protocols were proprietary and it didn't matter. There were always both free and proprietary implementations of the Internet protocols, but the important thing was that they all agreed on the same standards and (more or less) followed them, which bridged all those little proprietary islands into this wondrous whole we have today, where virtually any networked device is capable of communicating with any other. What mattered was that the standard was open, no matter how many of the implementations were proprietary. (And, of course, natural evolution tends to favor the extinction of most of the proprietary systems in favor of free software whenever such competition occurred, especially since vendor lock-in fails when customers demand conformance with open standards.)
The computer industry is starting to realize that XML, like TCP/IP, can bridge proprietary islands. Look at the number of legacy systems, interfaces, protocols and file formats which are being interfaced with XML to achieve at the application level what TCP/IP achieved at the networking level. Legacy systems, proprietary systems and even free systems, each with its own way of doing things, can suddenly be made to talk to each other in a robust, loosely-coupled fashion which was unfathomable just a decade or two ago. This process appears to be well on the way to revolutionizing the computer industry yet again.
Operating systems and device drivers are full of proprietary islands just waiting to be bridged, and it could revolutionize operating systems as much as TCP/IP revolutionized computer networking. Not all of these proprietary islands are "proprietary" in the closed-source sense -- many are also free-software islands which are "proprietary" in the "only works with this system" sense. Just in the domain of free software, there are countless little proprietary islands between various versions of Linux, FreeBSD, OpenBSD, NetBSD, Dragonfly BSD, Darwin, HURD, etc. These aren't "proprietary" as Stallman uses the term, but just try to take a random device driver from one of these random islands and dump it on another at random and see how likely it is to work without changes. Then, of course, there are also the truly proprietary systems such as Windows.
Bridging all those islands would benefit free software immensely, regardless of whether or not proprietary closed-source vendors jump on the bandwagon. Imagine if every device driver only needed to be implemented once to a common API, and it worked without source code changes on every operating system that supports that API? That's exactly the promise that Project UDI holds for operating systems and device drivers, and it's as revolutionary as the promise of TCP/IP.
The Internet wouldn't be where it is today without free software, yet free software wouldn't be where it is today without the Internet! This seems like a conundrum -- a chicken-and-egg problem. Actually, it's a truly symbiotic relationship, and it
Deven
"Simple things should be simple, and complex things should be possible." - Alan Kay
No, they don't, unless you are putting Windows XP and Windows 95 in the same category. Just about any Windows 2000 driver will work on any version through service pack 4, but with Linux I need to recompile my drivers whenever a minor version number changes. Joy.
Now before I get modded down, I be to remind whoever might read this that what I am saying is FACT. - bogaboga
Why do we still have to have a user program (X) with device drivers in it? (Would anybody think it's a good idea if the Linux kernel didn't have any sound drivers, and required gstreamer to implement its own?)
That's not exactly a legit comparison. gstreamer is an application; X may run in userspace, but it's part of the system in the same way udev is; it's not in the kernel because it doesn't have to be.
It seems we have two competing driver models in Linux: some are in the kernel, and provide a consistent interface (sound cards, SCSI/IDE/... cards, network cards), and some aren't in the kernel at all, but expose them at a low level and rely on userspace programs to provide actual drivers (X11 for video cards, CUPS for printers).
Not exactly. For instance, CUPS may have printer drivers, but it relies on the USB, parport or network communications exported by the printer. There's no compelling reason for it to be in the kernel, since those drivers don't talk directly to the hardware in the same sense that the port drivers do.
Or consider gphoto. The port drivers are part of the kernel, but the camera drivers are in userspace--because there's no compelling reason to put them in the kernel, since there's nothing in there they need.
So that's my understanding of why X or CUPS or gphoto have their drivers in userspace, while sound drivers and port drivers are in the kernel.
Laws do not persuade just because they threaten. --Seneca
In theory, it could be a single port that the driver crams all requests and data into and the GPU does whatever it wants and hands any return data back on a second port, although there are obvious thoroughput problems there.
Oh Lord. Save me from the ignorant. Hardware systems are NOT that easy to design. The GPU doesn't do JACK other than crunch some numbers, and it's already one of the most complicated devices in existence! (Primarily thanks to the complex pipeline designs, the number of parallel pipelines, and the microcode space and execution hardware carried on chip and intertwined with each pipeline.) ALL the real work is in the software. The hardware just executes its instructions. If you tried to cram all the work the software is doing into the hardware, you'd have something more complex than the latest Pentium and NVidia GPU combined. Not to mention that you'd never be able to increase performance through driver updates.
And you need to be careful with your concepts of Ports. Bus design is tricky business and can easily bottle up the entire computing platform if you're not careful. Many commands on one bus is something you see more of in general purpose computing where the raw performance isn't an issue. In the case of a GPU, raw performance IS an issue, and the software directly controls the entire bus + the GPU.
What the hell are you talking about? I wasn't complaining about GPUs or CPUs.
That's because you have no idea what you're talking about. The hardware doesn't do video processing. I don't know how many times I'm going to have to repeat that. It's all software, and it's all trying to compile the instructions, reorder them, JIT microcode, parallelize the data, and do other complex optimizations that will actually feed the computations into the GPU. The GPU spits out an answer that ends up in VRAM. How nice.
And, before you ask, I do indeed program, although admittedly it's been a decade since any assembly language.
No offense, but ASM is way off in left field. We're talking hardware design here. And the only way to truly understand hardware (including the software that drives it and WHY it drives it that way) is to go design some yourself. Unlike yourself, I've actually taken the time to do so, and now understand why things are the way they are. I'm not sure how I can get this through your skull, but complex hardware is HARD and EXPENSIVE. It never makes sense to use hardware when software will do the job just as effectively (or in this case, more so).
When it does make sense is when you're trying to accelerate very specific and abstract mathematical operations. e.g. Matrix math can be easily accelerated in hardware. Drawing a line on the screen? Not so much. If you want to accelerate something like drawing a line, you'll want to use Microcode. Microcode is tiny software that loads up and executes on the CPU itself. Microcode is pretty micro though, so it can only go so far. To do something more complex, (say, draw a shape with those lines) you want to bump it up to software. The on-chip cache will keep the code running smoothly inside the chip so that the software isn't constantly crossing the bus during the operations. Thus you end up with software that drives the hardware.
Now imagine that you had software that waited until you had drawn all the objects with lines, then sorted them to figure out which ones get drawn in what order, removes any lines that are hidden, checks which lines can be drawn in parallel because they don't cross, then does a final reordering of the line drawing commands so that the hardware can draw four lines simultanousously (complicated by the fact that some lines are shorter than others, and will take fewer cycles to complete). That is what your big blob of "useless" software is doing.
If you think there's a better way this stuff can work, then by all means. Go design some hardware and show the world. (It's not that hard to get started. Really.) In the meantime, though, just be quiet. You're making yourself look bad.
Javascript + Nintendo DSi = DSiCade
See the long version of this post for expanded discussion of these points...
[By the way -- AKAImBatman, please send me some email, I'd like to chat with you further about Project UDI...]
Deven
"Simple things should be simple, and complex things should be possible." - Alan Kay