Slashdot Mirror
Should Linux Have a Binary Kernel Driver Layer?
zerojoker writes "The discussion is not new but was heated up by a blog entry from Greg Kroah-Hartman: Three OSDL Japan members, namely Fujitsu, NEC and Hitachi are pushing for a stable Kernel driver layer/API, so that driver developers wouldn't need to put their drivers into the main kernel tree. GKH has several points against such an idea." What do you think?
Having a kernel API for drivers allows developers to stay away from the mainstream kernel. This will enhance the stability of the kernel in general and also allow hardware vendors to support Linux with less effort.
To Terminate, or not to Terminate, that's the question - SCSIROB
Microsoft could license code into the drivers or otherwise maneuver the driver makers to license their IP from MS, the drivers could form a layer between Linux and the hardware, Microsoft could then pull the run from under Linux.
Don't go there, it protects Linux from getting tripped up, and devalues any hardware that doesn't support Linux.
Don't underestimate the important of driver support for Linux, you practically can't make any server component without a good solid Linux driver.
I have been waiting for that for a long time. The lack of a stable interface is hampering adoption of Linux. Not all of the manufacturers are willing to open source drivers or for that matter to continuously change them as the APIs change. This is very welcome but unfortunately, I think they'll fail. There is just too much politics surrounding Linux these days.
It's a bad idea because what happens when the driver ABI changes? You have to wait umpteen months for the company to get off thier asses and fix it - like nVidia.
It also precludes anyone else from fixing bugs in the broken, half assed crap most corporates spit out these days.
feh. stuff.
one of the main problems for getting device manufacturers to support linux is the fact that they either have to release a new version of their driver every time the linux kernel changes some esoteric internal API, or be badmouthed for not having good linux support.
would it really hurt so much to guarantee a stable DKI? doesn't have to freeze the whole kernel, just a subset of functions that will be guaranteed to work as they do now in perpetuity.
backwards compatibility is just as important to driver writers as it is to app writers.
doesn't even have to be binary backwards compatible, source level would be sufficient for most.
The difference between Theory and Practice is greater in Practice than in Theory.
One of Linux's great strengths is the flexibility of changing to meet new needs and not being hobbled by rigid backwards compatibility. This can only be done if all source is open and anyone can update drivers to meet new needs. When someone comes up with a patch to streamline a certain minor part of the kernel, it frequently has repercussions elsewhere in kernel land. It is these small changes which have made linux better and better with breathtaking speed. A "stable" binary API removes the possibility of keeping everything up to date and would dramatically show down the adoption of new features and general improvements.
Continual refactoring is worth far more than some supposed binary API which prevents changes. Get rid of binary drivers! If companies are so paranoid that they want binary drivers, then the hell with them. Linux can advance better without that baggage.
Infuriate left and right
These companies want a binary layer so they can build binary drivers.
What people tend to forget about this is that it's a bad idea- from most every perspective.
The Linux kernel was written as a Free Softwate alternative to the existing *nix systems.
We have thousands of drivers in the kernel from a combination of development efforts. Sometimes a driver is written by an independant kernel developer, and sometimes it's written from the company producing the hardware, working alongside the community.
What these companies want is to be able to have thier cake without giving back to the community. This is a very slippery slope at the least, and illegal at best, since these sorts of links to binary kernel drivers have been long known to be illegal to distribute alongside the kernel (unless special previsions are made, such as a userland driver).
Also, binary drivers have been known to be buggy and essentially removie the kernel developers from a position where they have control over the kernel as a whole project. I won't even go into the issues associated with a possible security hole in a binary driver, or a binary driver with, for example, spyware in it.
The arguement for it is, of course, that this might mean more drivers. This is a test of our strength as a community. Doing the right thing is harder. It means we won't have all the hardware at all times, and certainly not the newest thing. But we retain control over our computers.
It's hard to say no, but this looks like a clear case where we have to.
IANAKH, but couldn't more drivers be moved into userspace (or other lower rings) --- especially for things like USB printers and miscellaneous gizmos? I think it would also be nice to not bundle thousands of drivers and support for architectures I don't have with the kernel. The kernel itself could provide a very minimal layer of hardware protection (like an exokernel?) and there'd be libraries exporting generic abstractions for particular classes of hardware. Is the context-switching penalty really so great these days? Educate me!
The typical comment of a Slashdot user. We should all buy new hardware and spend hours on top of hours getting things to work, rather than just "having" things work. Call this a troll if you want, but this is the exact reason that the masses AREN'T running Linux.
You people need to figure out exactly what you want, Linux for the masses (read: grandma, mom, etc) or an O/S where you have to spend valuable time just getting it to work with regular hardware. You bashed the poster for buying "random hardware" and expecting it to work even though you don't know what the hardware in question was, yet in your own message you bought 3 "bleeding edge" Gateway laptops (a fairly well known manufacturer) and you had to (in your words) "_try_ to make it work."
I was going to post this anonymously, but it would of course be modded a troll at that point. Let my karma burn for all I care.
One of Linux's biggest problems is the lack of device drivers for common devices, especially newer video cards. Let's face it, companies like ATI and NVIDIA aren't going to release fully open-source drivers. It would be wonderful if they would, but it would also be wonderful if we had flying cars.
Having a stable binary driver interface would make it easier for hardware manufacturers to embrace Linux, give things like wireless chipsets more usability on Linux and drive further adoption of Linux as a viable competitor to more proprietary solutions
The perfect is the enemy of the good, and the more Linux gains a foothold the better it is for open source. Insisting that device manufacturers need to have on-staff kernel hackers in order to keep ahead of a frequently-changing kernel makes it that much harder for manufacturers to support Linux as a viable alternative.
Provided Linux can have a stable binary driver infrastructure that doesn't harm stability, it would greatly help in the adoption of Linux worldwide.
But we retain control over our computers.
Tell me, what BIOS do you run ? Do you have the source to the firmware in your IDE disk drive ? In your CD-ROM/DVD-ROM drive ? Do you have the source to your SCSI controller's firmware ?
If you think you have control over your computer you are suffering under a delusion.
If a company is developing an embedded Linux ap for their own hardware. All of a sudden, all of the communications with the board-specific hardware is being done through binary drivers, resulting in an effectively closed system.
No more hacking WRT54G's for you, chump.
"Eve of Destruction", it's not just for old hippies anymore...
The idea is not just a formal API, but a formal API that doesn't change for a long (VERY VERY long if it is to be usefull at all) time.
/. is not the place to do it) or by throwing money at them.
The problem is that you will be stuck with the mistakes for the same time.
You can't improve the kernel anymore because you have to keep supporting those mistakes.
You would have to add huge glue layers to your kernels to keep emulating thos mistakes.
It is a bad idea. It has only advantages to those driver writers who want to stay far away from the kernel and want to keep using their crap for a long time. The kernel and the kernel writers are not going to get better from it (they only get extra work supporting lazy driver writers) why should they support it?
Linux development works very easy: do it yourself or convince someone else to do it.
You can convince other people either by motivating them with proper reasoning (and
Nobody has been throwing enough money yet to get the kernel developpers to do it. (And very likely nobody ever will throw enough)
Jeroen
Secure messaging: http://quickmsg.vreeken.net/
Driver certification means just that Microsoft received the driver, and agreed to confirm that the driver comes from one of its business partners, and not from a suspicious open-source hacker. You don't even have a guarantee that the driver is free from rootkits.
The company isn't really interested in fixing any issues with the drivers -- if you have problems with a bug, you have already paid them all the money they are going to get for that particular piece of hardware.
An example: the !@#$%^&* nVidia proprietary X driver. On some older cards, it will cause kernel oopses, within my usage patterns around once per several hours. Is there anything we can do to fix the problem? I'm not a master kernel hacker, but I do have some rudimentary skills there -- I would have at least some chances to make a fix myself; if I wouldn't succeed, reporting a data-loss bug would make us have it fixed by someone with more knowledge in no time.
On my current desktop, switching to text mode the first time X is run after a boot puts the console into 80x25 mode without even doing a TIOCSWINSZ. Somehow, if I kill the X server, reset the video mode then start X again, everything is fine, until the next boot/resume. What can I do to fix this annoyance? Begging to nVidia does nyet work.
The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
Linux was, is and hopefully will always be "open". I don't want closed drivers in the kernel (even via an API layer) any more than I want a Sony rootkit masquerading as DRM.
It isn't about "politics". It's about policy and philosophy.
If the hardware doesn't work with Linux, don't buy the hardware/pester the vendor for an open driver, or don't run Linux.
It is not welcome. Linux is about Open Source, and allowing people to link-in binary closed drivers goes against this.
Bypassing the dogma of the above, there are numerous pragmatic reasons why this would be better for linux, even if you don't include support for binary third-party drivers.
Sure, some of these are extreme cases. You can usually get away with just re-compiling the driver, and occasionally, you can even use the binary from the existing version.
The point is you should *always* be able to do this wihtin the same major kernel version. There is no technical reason, aside form the politicis of not wanting to ever allow binary drivers, to not have a stable driver API.
Imagine if the Mozilla plugin API changed with every new version of Firefox. And look at all the complaints when a new Firefox version doesn't work with all the old extentions. It is the exact same.
If you are going to take the strategy of "We will do things to attempt to force you to do thigns our way," don't be supprised if the response of companies is "Fine, then we will ignore you."
The simple fact of the matter is that most companies are not willing to go open source, for software or drivers. You can argue that's a bad thing, but it is the reality of the situation. So, if open source is out in their book, either because of contractual obligations or mentality or whatever, they are left with two choices:
1) Do Linux drivers, and update them every time the interface changes, which can be as often as every minor kernel revision.
2) Ignore Linux, and let the community write the drivers if they want.
The problem is that Linux is a bit player. They are larger than the other bit players, but they are still tiny, less than 10%. Given that the continous rewrites can get expensive, the choice for many will simply be not to write the driver.
So if you are ok with that, then great, but don't get mad at companies when they won't play by your rules. Are they being unaccomidating? Sure, but so are you.
In the end, it comes down to needing to make a decision of what you want Linux to be. If you want Linux to try and become the next big thing in OSes and start to really make an entrance in the home market, standardisation is needed. Standard APIs, standard UIs, inter-version consistencies, etc. In essence, it needs to become more like OS-X. Now if you are ok with Linux being more of a geek/server OS then that's not necessary, but you can't demand the world change around you.
Assuming that we had a stable kernel source interface for the kernel, a binary interface would naturally happen too, right? Wrong. Please consider the following facts about the Linux kernel:
But see, the thing is... a "stable" binary interface requires that structures used specify padding, alignment, and fields to be fixed! If these can vary, then by definition , it's not stable. Ditto the variations that depend on kernel build options.
Now, if you want to make the case that it's not possible / practical to make an interface that can cover all of these conditions adequately, well, by all means, do so (though I'd say that the hundreds of existing operating systems with binary interfaces show that this isn't the case in the general sense).
But what I see here is a relatively weak technical argument that is being used to justify an ideological decision.
I am not a Kernel Developer, but I know some. ;) I guess that my open question is how this would benefit kernel users? Yes, I see it would reduce the workload of kernel devs. Yes, I see it would allow driver developers to not have to go through the kernel code vetting process. But, the kernel code vetting process is what is a strong benefit of using Linux, from a user perspective, as I know that the code is well tested by an army of users and developers.
Once you push driver development out of the kernel, yet give access to kernel internals in this way, you introduce a level of uncertainty in so far as stability and robustness is concerned. One must question why these big comapnies are pushing for this, but most human kernel devs are not.
Because of this, I'm 100% not convinced making binary driver developers lives harder changes anything. Are large businesses (the type who make hardware that's difficult to reverse engineer) likely to say, hey, gosh, you know this Greg KH guy kind of doesn't like closed drivers, maybe we should open them up to please him? Nope. They'll just work around the difficulties or not provide drivers at all.
I've been a Wine developer for years and have spent many hours doing this impossible thing of which you speak, and your average copy of MS Word or Steam is a LOT larger than your average driver. Yes, it's hard. No, it's not impossible. I've heard various excuses as to why kernel development is just different!! to userland software development, and don't buy it. Yes, having to reboot when a crash occurs is a royal pain in the ass, but so is not being able to get a backtrace because the game you're investigating treats any attempt at attaching a debugger as an attempt to hax0r its copy protection. Different space, different challenges. It's still possible.
Ignoring for a moment, the ideological points, lets consider a frequently raised practical point: the idea that an API would either get out of sync with the kernel, or be a drag on innovation.
/lib/modules/binary/" than "you need to recompile and reinstall the whole kernel with this patch"?. For obscure hardware which will never be in the official kernel, it would be nice to have the ability to easily use it with any linux distro. No need for a slew of precompiled RPMs, DEBs, and a user-unfriendly source tarball, just one driver binary and you're ready to roll.
I agree that when the Linux kernel was young and untried, standardizing a binary API was bound to become a millstone in a short period of time, as the kernel internals were in a constant state of churn and iterative improvement. Nowadays though, surely, the kernel has been "shaken down" enough that it could afford to commit to binary APIs that are stable at least throughout each minor version number?
Returning to ideology, I can see how a stable binary API would be useful even to open source hardware. How much easier is it, to say "drop this file under
In itself, that says nothing, either pro or anti, about the availability of driver source.
the question I have for this is "why?" wouldn't a stable binary API likely result in far more third party hardware support for linux? possibly more laptops that are actually compatible with linux?
This seems like a case of open source programmers shooting themselves in the foot because they want everything to be open source... not every application and driver is going to be made open source just to suit the desires of the linux development community. It seems to me that sticking to a hard party line against closed source software instead of trying to co-exist with said software is bound to keep linux in relative obscurity and pretty much ensure that it never becomes a viable competitor in the desktop market.
"In America, first you get the sugar, then you get the power, then you get the women..." -H. Simpson
Firstly, I get paid to do driver development. Secondly, I have never worked for a company that willingly said, ok lets open source the drivers we spend thousands of dollars writing and optimizing.
That said, this whole discussion is as silly as the one about not putting a kernel debugger in the main kernel source code. Frankly, linux desperatly needs both a kernel debugger, and an ABI to be a REAL alternative for many customers. It also needs the ABI for driver developers so that we can write a single driver and expect it to work on the dozens of flavors of linux we are expected to support. Saying that everyone should opensource their drivers is like saying food should be free. It isn't going to to happen and wishing for it, won't make it happen any sooner. In the interm, almost all the hardware out there has better support on windows (Our sysadmin can't even get support for major linux distributions, from major hardware vendors, even when they have little linux logos on their hardware and websites) the windows drivers tend, to accually work, and they almost always have better features sets. This isn't going to change as long as the "opensource" community treats hardware vendors that think they have IP in the driver as second class citizens.
Oh, and for people who don't accually work for hardware companies that ship drivers, driver development is often times an expensive process, not because the software engineers are expensive, but because the hardware and software needs to be tested and certified in particular enviroments. There are orders of magnitude more linux distributions this makes it cost orders of magnitude more to test and support than a half dozen windows enviroments, most of which can be tested at microsoft, or one of the major OEM's if the hardware isn't avialable onsite. Putting 10x the money into a market that may be less than 1/10 of sales is not always a good idea, especially when resources are limited. Creating an proper ABI helps to solve this problem.
That said, if the damn linux kernel accually had a real architecture, it could support an ABI, and even isolate itself from rogue drivers. As it is, the kernel arch is pretty much non existant and just a pile of code that tends to behave like a real kernel, except when you try to do something a little outside of the mainstream desktop or small web server enviroment. This was fine when the whole kernel was just a few hundred thousand lines, but given its current size its getting massivly unmaintainable. This is proven by the fact that linux system stability seems to have gotten really bad over the last few years. Getting to a stable system, takes a lot of vendor testing by the likes of Suse, Redhat, etc.
Lastly, the tainted concept works fine for the kernel developers, why not carry it forwared so that any binary driver simply marks the kernel as having a binary module loaded, and uses the standard abstract interfaces instead of linking against all kinds of unneeded kernel crap that just provides the posibility to screw something up.
First of all, I completely understand that drivers with source are better than binary-only kernel drivers. However, the fact of the matter is that companies do not feel comfortable freely releasing their intellectual property. Given the choice between a piece of hardware in a laptop which doesn't work with Linux at all and one that works with a binary-only driver, I would rather have the binary-only driver.
As long as Linux kernel developers complain that binary-only drivers are "illegal", Linux will have less hardware support. One of the major complaints people have against Linux is that a lot of devices that one can attach to a Windows machine plain simply do not work in Linux (I still think Linux is far behind Windows when it comes to wireless drivers, for example). I want to see a true alternative to Windows on the desktop; GPL fanaticism and an inability to understnad how big corporations work harms this.
So in other words, Linus has decided to deliberately damage the kernel (by introducing unnecessary instability) in order to make it harder for people to write software that uses it.
I thought that Linus usually put technical correctness above political correctness, but it looks like I was wrong.
The most rabid believers in American Exceptionalism are the exact same people whose policies are destroying it.
The BIOS is indeed an issue, and there are efforts underway to make a Free BIOS.
But why not try our best to have as much control as we can?
Wrong. It is closed source companies who put the code before the user. They protect the code more than they protect the users. Open source is about protecting the user by allowing unhindered access to code for modification and redistribution.
It's funny how you warp things around.
They don't release the source. There is no source. That's the whole point. They just release the programming docs so anyone can write a driver. There is no IP for them to protect.
There's a joke- programming is a lot like sex. One mistake, and you're supporting it for the rest of your life. By this measure, writting a device driver for Linux is a mistake. Because even after the feature set and the hardware itself is stable, you are letting yourself into a never-ending task of constantly changing the driver to keep track of the current API changes. At a previous job we spec'd it out at about 1/2 of a full time position to keep up with all the kernel changes. This is your job, from now until eternity, or at least until you're willing to have the driver declared obsolete and abandoned (after all, we all know that there is just three types of software- vapor, beta, and obsolete).
Well, congratulations. You have your religous purity. And guess what- it's comming at a cost. You wonder why Linux isn't more popular on the desktop? Well, here's part of the reason. It's hardware support will always- ALWAYS- be behind that of Windows. Why? Because when the hardware ships, it ships with Windows drivers that the hardware vendor wrote with it. Note that Windows pulls the same sort of API changing crap that Linux does. The difference is that the hardware vendors look at Windows, and the half man-year per year cost of supporting Windows costs, and go "but we have to support Windows if we want to sell more than 3 units." They then look at Linux, and the half man-year per year cost of support Linux, and go "Supporting Linux is not cost-effective at this time." I know, because I've seen this happen. So now the hardware is out there. And now we wait, for someone willing to step up and volunteer the time to write, and maintain indefinately, the driver. Someone less capable of doing it than the hardware manufacturer (this isn't to question the capabilities of the current kernel developers, but the fact of the matter is that there is a huge advantage to being three cubes down from the hardware developers, and capable of wandering over and asking direct questions, instead of having to reverse engineer what is really going on, having worked both ways).
So this is the fundamental question: which is worse. Having binary-only proprietary drivers, or being forever behind in hardware support and not having people contribute simply because they don't feel like having to constantly update the driver once they finish it, they'd like to be able to move on. I come down on one side, Linus and the kernel developers down on the other.
Fine. Their kernel. Their problem.
"Basically, I want people to know that when they use binary-only modules, it's THEIR problem. I want people to know that in their bones, and I want it shouted out from the rooftops. I want people to wake up in a cold sweat every once in a while if they use binary-only modules."
- Linus Torvalds on linux-kernel
And many people forgets that non-gpl drivers may be very well impossible to write at all (at least some lawyers think this), drivers are not at all like an app is WRT to gtk, drivers are more like "plugins". Plus, a closed driver module makes MUCH HARDER to debug bugs if the driver is doing bad things, and you can't know that (which makes harder to stabilize and/or develop the kernel. Several closed drivers can make it a hell or impossible at all.
In other words, it would turn Linux into the same kind of piece of shit that Windows is, and defeat the entire purpose of using it!
"[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz
Initial costs associated with a manufacturer-supported driver:
Ongoing costs associated with a manufacturer-supported driver:
These costs exist even if a version of the driver is merged into the mainline kernel. The only problem solved by such source-level merging is compatibility with the latest kernel version. It is not acceptable to the manufacturers' customers to be required to update to the latest kernel/distribution to be able to use the device.
Here's the key point: If there is no binary interface between the driver and the kernel, all of the above costs skyrocket. You have M kernel versions against N distributions, with the total increasing over the life of the product. If there is a binary interface guarantee from the kernel development team to change only very slowly and only extremely rarely breaking compatibility -- like the guarantee Windows provides -- then the incremental costs are containable. It is reasonable to expect that 95% of their testing on 2.6.5 is valid on 2.6.14.
The perfectly reasonable response from kernel developers is that with closed-source drivers they get stuck debugging problems that are't kernel-related (I don't hold ideology to be economically significant so I'll ignore it here, without insult to people's strong opinions on the subject). Their proposed solution is to require the driver's source before they'll help with the debugging.
From the manufacturers' point of view that's a very draconian requirement. They are justifiably concerned about intellectual property (availability of the source makes it much easier for competitors to reverse-engineer the hardware/firmware). Surely there must be a middle ground. Is there some way to have a relationship between the device manufacturers and the kernel developers that minimizes everyone's costs?
I think there is. Note that all of the above costs and issues are just as valid in the Windows world as in the Linux world. Microsoft doesn't want to deal with bad drivers crashing their systems, costing them both development/debugging time and reduced perceived stability (--> lower sales). Their solution is the Windows Hardware Quality Lab (WHQL).
The WHQL is a separate entity from Microsoft. Device manufacturers are required to submit their driver source (effectively under NDA) along with their device. The WHQL staff runs the driver through a battery of tests, probably mostly automated. If the device and driver meet stability standards set by Microsoft, the driver is signed by WHQL. Windows checks for this signature at installation time and warns the administrator if it is not present. Microsoft can reasonably refuse to support non-WHQL-signed drivers when crashes occur, for exactly the same reasons that Linux kernel developers refuse to support drivers without the source. This system has been the single most important factor in Windows' significan
Bandannarama