Devs Discuss Android's Possible Readmission To Linux Kernel

MonsterTrimble writes "At the Linux Collaboration Summit, Google and Linux Kernel Developers are meeting to discuss the issues surrounding the Android fork and how it can be re-admitted to the mainline kernel. From the article: 'James Bottomley, Linux SCSI subsystem maintainer and Novell distinguished engineer, said during the kernel panel that forks are prevalent in embedded systems where companies use the fork once, then "throw it away. Google is not the first to have done something like this by far, just the one that's made the most publicity. Hopefully the function of this collaboration summit is that there is some collaboration over the next two days and we might actually solve it."'"

Comments

Yawn

[Anonymous Coward]

What does this have to do with the iPad? I come to slashdot for iPad stories, not stuff real nerds have never heard of.

Re:Yawn

[ickleberry]

Really? I come to slashdot to read about how Google is taking yet another piece of technology we have taken for granted for many years and turning it into an online, ad-based Clout 2.0 service and tunneling it through HTTP with JSON and SOAP to their servers for a nice intense data-mining session for better targeted ads and predicting future crimes one might commit.

Re:Yawn

[girlintraining]

Really? I come to slashdot to read about how Google is taking yet another piece of technology we have taken for granted for many years and turning it into an online, ad-based Clout 2.0 service and tunneling it through HTTP with JSON and SOAP to their servers for a nice intense data-mining session for better targeted ads and predicting future crimes one might commit.

Unbridled capitalism and the apathetic and ignorant citizens are to blame for that. Your personal data can be aggregated and monetized, and for the foreseeable future, there's very little legislation to prevent this and very little awareness of how pervasive such technology is. My whole generation is living with software riddled with government and corporations that have put back doors into everything, freely share data with each other, and those living in urban areas (the majority of the population) are rarely out of contact with some device or another wired into the global network, tracking their movements, purchases, communications, relationships, and every aspect of your life. Remotely-enabled webcams, cell phones that can be turned on silently to broadcast everything it hears and sees, and laptops and routers that can be readily converted into eavesdropping devices, just to name a few of the many things that are out there right now. And the only reason it's not all interconnected more seamlessly is because the technology is still rapidly evolving and hasn't reached a stable plateau where convergence is possible, although the internet has made a giant leap forward in enabling that future. The NSA spends billions each year trying to keep up with infrastructure changes and only is able to harness a fraction of that potential.

But I mean, comeon -- what do you expect from a world where we find it okay to setup metal fences with razor-tipped wire and cameras everywhere as "official protest zones", where we have passports, credit cards, (and soon ID cards) that can be remotely scanned to identify you... put it all together. Where do you think this all ends?

Re:Yawn

[maxume]

It is entertaining that a tinfoil hat will work quite well at protecting your wallet from remote scans.

Re:Yawn

[Abreu]

I'm not worried! A lot of the info I have posted online is false!

Re:Yawn

[russotto]

It is entertaining that a tinfoil hat will work quite well at protecting your wallet from remote scans.

Only if you keep your wallet on your head.

Re:Bad Marketing

[theshibboleth]

Outdated webpages are the hallmark of a dying product

Backwards?

[Sponge Bath]

Google must now balance any desire to respect the wishes of the Linux community for compatibility with the more diverse, competing - and not always logical - interests of those now adopting Android and its own plans.

I did a double take on this statement.

What I've seen on the kernel mailing list is more a conflict of commercial developer's desire for compatibility (across kernel versions) with the core kernel developer's more diverse (and not always logical) desire to push pet projects and make frequent cosmetic changes that creates a hellish torrent of code churn. The lack of well defined kernel driver interfaces means a lot of time spent chasing the latest changes instead of adding features or fixing bugs.

Re:Backwards?

[Microlith]

The only people I've seen clamoring for a static, unchanging driver interface are those writing proprietary drivers. Last I checked, changes to the interfaces by someone puts the onus on them to fix all the calls to it in the kernel, which is why getting your driver into the tree is considered better than keeping it closed.

That said, if you're keeping your driver closed it's a problem you're bringing upon yourself.

Re:Backwards?

[Daniel Phillips]

The truth is, Google doesn't really get open source even though its livelihood depends on it.

Re:Backwards?

[Anpheus]

Those proprietary drivers still have to be maintained against the rest of the kernel, and that costs time, and consequently money.

Furthermore, many of these devices are protected by patents, and I'm sure you don't want code for a special model of capacitive multi-touch screen that only one phone uses to be added to the general Linux kernel. There's no point in it.

So that's the problem. All these phones have highly specialized devices that may be protected by patents that in Europe have no weight, but in the US do, and could cause problems for the linux kernel potentially even if they were introduced. Add to that the fact that for many of these devices a generic, unifying framework doesn't exist that they can attach themselves too, and you could end up seeing the kernel with ten thousand different phone drivers, each of them so specific that all it does is bloat the kernel.

So what's the answer? Well, the answer is that if Linux doesn't start building a good ABI, they're going to shoot themselves in the foot, or more literally, they'll end up sawing off their own leg because it decided to fork itself. And for all the babbling the kernel devs do about the difficulty of maintaining an ABI and how it constrains them, it also makes it very difficult for the generic, current Linux kernel to gain widespread adoption in markets that resemble embedded ones in all but name. What is HTC supposed to do, keep people on payroll perpetually to maintain their thousand plus phones and their potentially tens of thousands of drivers?

Suddenly, Linux is starting to look a lot more expensive than free.

Re:Backwards?

[Sponge Bath]

That said, if you're keeping your driver closed it's a problem you're bringing upon yourself.

I should have been more clear. I'm talking about drivers in the main kernel source. I know the linux kernel mantra: binary only drivers are evil (I agree), out of tree open source drivers are slightly less evil. I think out of tree open source drivers can be useful when inclusion to the main kernel is denied because some critical functionality is deemed unnecessary by the gatekeepers who require it to be removed before consideration. But I'm not even talking about that.

Last I checked, changes to the interfaces by someone puts the onus on them to fix all the calls to it in the kernel...

That's the theory. Here is how it works in practice: A pet project or cosmetic change that touches a lot of code is implemented and then dependencies are grepped. The dependencies are fixed up in a cut and paste way. Sometimes more important drivers get some review to make sure nothing breaks. Everything else just gets shipped if it compiles. Then when that kernel is used in a distribution, sometimes years later, many drivers are suddenly broken and you have to back track to see which change took it out. If someone has a lot of time and desire to support a "lesser" driver then they can spend all of their time playing catch up, but that wears out volunteers quickly and annoys commercial vendors.

Re:Backwards?

[Sponge Bath]

The last thing Linux needs is a set-in-stone kernel interface...

I can agree with this, but then again I don't see anyone asking for that.

How about something in between, say a well defined interface that is stable for a reasonable period of time with clear points of deprecation and then replacement with improved interfaces? Windows's driver interface is not set in stone with never ending backwards compatibility, you can't use Win 9X drivers on XP. Yet a binary driver that works on Windows 2K has a reasonable chance of running on Vista.

There needs to be a balance between improvements/changes and stability/maintainability.

Re:Backwards?

[HeronBlademaster]

No, but they're wrong for being unwilling to meet them halfway (even something as simple as a clear schedule for ABI changes and deprecation). There's nothing wrong with adding a little method to the madness.

Re:Backwards?

[dgatwood]

Wrong, absolutely wrong. Greg K-H himself has explicitly said that he WANTS people with drivers for even highly obscure devices to merge them into the mainline kernel. It doesn't matter if your capacitive multi-touch screen is only used in one phone; the code is useful to have publicly available in the kernel as a reference. Furthermore, as more drivers for similar devices are merged into the kernel, commonalities between them can be found, and more generic drivers can be created.

Based on what I've seen over the years (as a developer on a project that never made it back into the mainstream kernel), the problems with this approach are threefold:

1. Nobody maintains most of them. Most of the 5% of drivers that everybody uses are already in the kernel tree. Of the remaining 95%, half of the drivers don't build at all, and most of the other half don't work. If they're barely maintained now, you can bet money that they won't be maintained at all when some kernel tree maintainer gets a hair up his/her backside and decides that a particular fix isn't elegant enough and won't take the changes....
2. The tree is already too large. If every driver out there were in the tree, checking out an update to the tree would be horribly painful, the source packages that distributions include would become huge, etc. The bigger it gets, the fewer people are going to be willing to maintain their drivers inside that tree, so in the long run, encouraging people to put their drivers in the tree is just going to cause other drivers to move back out of the tree, eliminating any real benefit.
3. Many such drivers are outside the tree because they require substantial changes to some subsystem in order to build them. Now one could argue that these changes should be made to those subsystems to make them more general, or one could argue that those drivers are so specialized that nothing else will use them, so there's no reason to bother. That's often not an easy question to answer, and tends to result in highly political shouting matches, with the end result being that the driver never goes in, which is usually why those drivers got published outside the kernel tree to begin with.

There are ways to solve these problems, of course; IMHO, they basically amount to:

• Design a kernel build infrastructure that can easily bring in driver sources from third-party sites (like a ports collection, but for kernel drivers). With proper categorization, this can provide all the same benefits as having the drivers in the main tree, but also allows for a richer tagging scheme instead of a simple filesystem hierarchy, which should actually make it significantly easier to spot patterns (for example, seeing that there are now eighty-seven different drivers for capacitive touchscreens, or whatever), all without bloating the tree that everybody has to download.

• Subject all kernel API changes to a formal API review process in which no API change can go in unless the owners of all drivers in that area agree that the design is acceptable and will meet with their needs. Set up a reasonable set of rules of engagement (e.g. A. don't shoot down the idea just because you don't need it, B. don't shoot down an idea without proposing an alternative). And so on.

• Redesign the kernel interfaces in an object-oriented language. Such designs make it more likely that drivers can extend the interfaces without requiring major changes to the core code. The Linux kernel sort of halfway adopts this approach insofar as code reuse is concerned, but does so in ways that aren't particularly clean and neat.

  For example, if I were writing an ATA driver and needed to do almost everything the same way but change the behavior of one function in some other library... say down at the block device layer, I'd either have to make a change to the block device layer with some special case detection code or I'd have to copy entire swaths of code at the ATA device layer and c

Re:Backwards?

[Mad Merlin]

The problem is that the volatility is so high that kernel drivers need 24/7 maintenance, or else they're dropped and then it becomes even harder to re-integrate them. Ask Microsoft about their paravirtualization drivers. They've submitted two or three versions to the kernel, and each time you had to use the specific version of the kernel that they compiled them on, or it didn't work. That's the problem. Linux. Isn't. Free. Microsoft is however eventually going to have to come to a sad realization: it may cost them a salaried employee and benefits just to maintain these drivers. That's ridiculous. If it's difficult for Microsoft to justify targeting Linux, how is a small business going to justify putting 1/10 of its development staff on it? 1/20?

Bzzt! Wrong.

Once code is properly merged to the Linux kernel, it is maintained by the kernel community at large, which need not include the original author. When a kernel developer changes an API, they are required to simultaneously update all in kernel drivers that use the API in question. The only drivers that require 24/7 maintenance are those that are out of tree (regardless of the reason they're out of tree).

Android was never properly merged to the Linux kernel. Google did a big code dump for Android and it was merged as a set of staging drivers with the caveat that it needed a lot of cleanup before being moved into non-staging. Unfortunately, that cleanup never came and Google basically let the code rot. Thus, a few releases later, Android was removed.

Indeed, probably well over half of the code in the Linux kernel is now maintained by someone other than the original author (be it an individual or a corporation), particularly for non-core subsystems and drivers. As a hardware vendor or other similar party, if you want a) your widget to work out of the box on every Linux distro and b) to not worry about maintaining your driver, you should be getting your driver merged to the Linux kernel.

Re:Backwards?

[MostAwesomeDude]

You're entirely right. That's why they fund several thousand students worldwide to join open-source projects and contribute code to those projects every summer, even if the projects in question don't directly benefit Google.

Re:Backwards?

[dgatwood]

The latest linux kernel weighs 60 MB;

Odd, I'm looking at http://www.kernel.org/pub/linux/kernel/v2.6/ and the latest kernel I see is 2.6.33, and that comes in at a whopping 81 megabytes for the compressed tarball. Extracted, it takes almost 434 megabytes. That's over twelve minutes of DVD-quality video. That's two-thirds of a CD-R. That's ten times the size of the Mac OS X kernel. That's two months of bandwidth at the lowest tier of cell phone service.... You get the idea. It's freaking huge. The kernel sources were too big way back in 2.2. Now, they're just pure comedy.

Also, remember that source control systems add a significant performance penalty that is also proportional to the number of files, not just the number of bytes. So although the giant compressed tarball may take only five minutes to download from kernel.org (which is an eternity), I'd expect a source checkout to take a good bit longer.

But doesn't "just overriding one method in a class" mean changing a function pointer?

The point wasn't that there's an underlying difference, but rather that the syntax of a class hierarchy tends to result in design patterns in which the things that need to be part of the class are part of the class and not part of some giant library of functions. The result is that instead of the semi-OO design pattern of using pointers for only the functions that you already know will need to be replaced, you have a true OO design pattern where any of them can be replaced without having to push for changes to thousands of lines of code all over the place that refer to that function.

It's not that if you use a derived class for your driver, all the other drivers will magically use the derived class instead of the one they were designed and compiled for.

I never suggested that they would. Why would they need to? There should be no accidental interaction between drivers. Any instances of variables shared between two unrelated drivers should be deliberate and rare. I should be able to make changes to my copy of the ATA core code without breaking your driver. That said, if you want the ability to do things like that, Objective-C categories would work.... :-D

Also consider that C++ is less supported on embedded systems, and has a much more complex (and changing) ABI than C.

All the more reason to use a limited subset of C++ (no exceptions, no templates, etc.) and to freeze the parts of the ABI that the kernel uses. Apple has managed to write their driver stack in C++ with AFAIK no binary compatibility breakage since they switched from GCC 2.95 to GCC 3 way back in 2003.... (Okay, so the CPU architecture change was something of a binary compatibility breakage, but you know what I mean.)

Re:Backwards?

[10101001 10101001]

it is if you accept the status quo. If you took all drivers out of the main tree and created a new tree specially for driver code, not only would the kernel suddenly get smaller and easier to work with (as you at least wouldn't have to download all that useless-to-you driver code) but the distinction between them would help to keep drivers as separate, truly distinct modules.

Linux is a monolithic kernel. Just because you can load or unload parts of it at runtime doesn't make those parts of it any less monolithic in design. Taking drivers out of the main tree won't make the modules any more distinct, stable, or secure. It would, potentially, better classify what is and is not a driver. That's the only main change I can really see in the move.

Of course this only happens with a stable ABI. Break that every version and all that driver code starts to wither. Keep it and you won't have to keep going back to fix up the interfaces. A stable ABI would be a good thing.

Nothing about a stable ABI moves code outside of kernel space. It would encourage the production of binary drivers, though, which if anything would worsen security and stability.

And, no that doesn't mean the interfaces couldn't ever be changed, you can change them in major versions of the kernel, just that anything built for 2.6.0 should still work with 2.6.100

So, you wish to turn major version changes into an even more political thing than it is now because?

It won't be perfect, but it'll be a lot easier to manage for driver writers. I can't see how it would be too much of a hardship for kernel developers either, unless they only churn the code in an amateur 'just hack it until it works' way.

It's funny. Your argument for a stable API is, if anything, an argument to make it so driver writers *don't* have to manage drivers. That's hardly a surprise, since I think most driver writers are of the mindset that if the hardware doesn't change, they should only have to write code for the hardware once and never have to touch the code again. It's not the mindset of "I wrote this code perfectly with perfect stability and security and consideration on exactly what my hardware does". Often enough, it's "good, it seems to work, that's good enough". Not surprisingly, kernel developers, who are concerned about more than "good enough" want code that's well designed, stable, and secure, with hopefully flexibility to work with many devices in a close to optimal way. This invariably puts them at odds with driver writers who are uninterested in maintaining anything*.

For kernel developers, a stable ABI includes negative consequences like static or binary drivers that hold back redesign and introduce security and stability concerns. It does nothing to address drivers having to reside in kernel space. Yes, this is a byproduct of "amateur 'just hack it until it works'", but that claim can be put on just about C developer. That doesn't mean all or even most developers actually function that way in the kernel development. But, even with simple screening for obvious bad design in submissions, bugs can be very obfuscated in C; Linux nor any modern OS I know of are willing to spend the time and energy to show code is provably correct**, so trying their best and fixing problems later is the best one can reasonably expect in any remotely large, modern OS.

*I don't mean this to be totally lambasting of driver writers. I can understand that trying to support hardware which you don't even have the specs on is frustrating, so one getting the hardware to seemingly work is usually enough to want to provide others a driver, even if one's work is incomplete or potentially sloppy (not the code itself but the design relative to what's possible). And once hardware does appear supported, one's interest are usually in coding something else (like a driver for other hardware), not doing furt

Tricky

[Monkeedude1212]

"{
        '{
                "{
                  }"
                "{
                    }"
          }'
}"

I wasn't sure if 5 quotes at the end of the article was correct or not. I decided to employ brackets to handle the scenario. Taking out the wording, my findings are above. It IS correct.

Is it bad that this was the most exciting part of the article to me?

Cheaper costs

[girlintraining]

It's a real problem -- Android is easily the most hackable phone out there. And that's exactly the kind of thing cell phone manufacturers in this country don't want. It's bundled services that they make their fortunes on -- selling overpriced phones, contract cancellation fees, locking in devices, and more. Android threatens to separate the market into service providers and device providers and up until now, the service provider dictated what the device providers could do.

Imagine if you could just eject your SIM card from your phone, plug it into your computer, and browse the net, take phone calls, etc., then eject it like it's a memory card, slap it back into your phone, and go off to school, work, wherever. Or using bluetooth so that as soon as you get home, it automagically resyncs all your e-mails, text messages, and more. There's so much the technology can do -- and the only reason it's not happening is because service providers want to charge for everything, rather than simply flat-rating everything on a per minute, day, or megabyte use.

My Sidekick recently lost the ability to send files to my computer over bluetooth. Why? Because of an OTA update that disabled that. So now I can't just sit my phone near my laptop and transfer my pictures out of it, I have to open the back up, eject the little card, plug it into my system, copy the files, and then do the reverse. Very cumbersome when before it was 'click icon, drag files'.

It's complete and utter bullshit that cell phones are as powerful now as desktops were ten years ago sitting in the palm of my hand, and yet they have less than a third of the capability. And not a one of them is really interoperable with any other except on the most primitive level. Hell, the dialup days of computing offered more functionality and standardization than the cell phone market does. Why should a 14.4k modem and an antiquidated pentium 133 have more communication functionality than today's devices? Hell... it even cost less.

Re:Cheaper costs

[EvanED]

It's a real problem -- Android is easily the most hackable phone out there.

I'm not so sure... I think the Nokia N900 has got it beat.

Re:Cheaper costs

[girlintraining]

I'm not so sure... I think the Nokia N900 has got it beat.

Yeah, but who's heard of the Nokia N900, or even knows what that means, outside geek circles? On the other hand, billboards and TVs everywhere are blasting out "Droid does". For bringing a hackable system to the masses, Android has it beat.

Re:Cheaper costs

[cynyr]

i second that.

you have hardware level access to the N[789]00 devices. I would like an ipad sized n900. that would be a great device.

Re:Cheaper costs

[drsmithy]

Yeah, but who's heard of the Nokia N900, or even knows what that means, outside geek circles? On the other hand, billboards and TVs everywhere are blasting out "Droid does". For bringing a hackable system to the masses, Android has it beat.

But "the masses" aren't interested in hacking it, thus making said hackability essentially irrelevant to anyone who isn't in "geek circles" anyway.

Re:Cheaper costs

[girlintraining]

But "the masses" aren't interested in hacking it, thus making said hackability essentially irrelevant to anyone who isn't in "geek circles" anyway.

They said the same thing about the internet, twenty years ago. And yet look what the hackers of the world built out of the refuse of wires and chips that the corporations of then said was useless and had no commercial value. Now they're fighting to tax it, control it, and some countries have declared it an inalienable human right to have it.

Maybe it has no value to them, but that's because they don't know the value of it yet. It's our job to find it and tell them. You just haven't been around long enough to realize the purpose of your own learning yet. Your individuality, your knowledge and talents, are not for your own gratification. The purpose of the democratic process, which the internet comes closest in form and function, is not to create a great country, or great works, but to create great people.

Hacking is therefore the highest form of the democratic process; Not because of what we do, but for what we share.

Would you rather have completely unsupported HW?

[tepples]

If hardware makers can't include third-party code or processes that they aren't permitted to sublicense as free software, then perhaps they won't write a driver at all. Instead of proprietary drivers, you'll have completely unsupported hardware.

Re:Would you rather have completely unsupported HW

[cynyr]

yes, if it's enough of a market for them, they will make sure that they get support from upstream, if enough companies ask for linux support for subassembly Y then maybe it will change. If you really feel you need to keep it closed, do like nvidia, or handle it yourself.

Re:Would you rather have completely unsupported HW

[Microlith]

What's your point, that we should encourage closed drivers by setting the APIs in stone for years on end? Allow the non-open to dictate the actions of the open?

That's not -my- problem. It's theirs. They choose to stay closed, so when the APIs change no one else can fix it but them. They have no room to bitch about unstable APIs in an open kernel that is constantly changing, when they won't commit to being open themselves. Others do, and as a result don't have nearly the problems. It's a cost they must accept, or they can do as you suggest and stop.

Re:Would you rather have completely unsupported HW

[tepples]

So I, an end user, am inside a Best Buy store, and I don't have a cell phone with a data plan to check what is in stock against the distro's HCL. How do I find peripherals that are definitely compatible with a free OS?

Re:Would you rather have completely unsupported HW

[Anonymous Coward]

1. Class compatibility.

Prefer to buy the object which says it implements a device class standard. AHCI is a good example. Classes rule so much that in a lot of cases all the non-class compatible products just went away - HID and ATAPI are good examples of that. In a few cases products don't advertise their class compliance but there's a well known sign that you can learn before you start looking. For example, if a webcam has the symbol that means it's designed to work with Vista, that means it'll work with a modern Linux too, because to get that symbol from Microsoft the webcam must be class compliant.

Today class compatibility covers almost all: input devices, USB storage, onboard sound, internal drives including optical drives, Bluetooth, mid-range webcams.

Class compatibility remains spotty for: add-on PCI sound cards, high-end cameras, graphics display (it's limited to VGA, blergh), printers

Class compatibility is non-existent for: most networking, including WiFi, digital TV, fingerprint readers, scanners

2. "No" drivers

When class compatibility doesn't exist, and you have the choice between two products that make no mention of any OS except Windows, but one says it works without drivers, buy that one. Of course it needs drivers, but they must be built in to Windows, which means hardware of this type is common, and it existed at least long enough ago to include drivers for Windows, which means there's an excellent chance drivers for Linux exist or soon will.

3. Look for the logo

It's not everywhere, but in some categories you will see a penguin logo (or other Free OS logos) on products that offer some support. Now, this doesn't mean you're necessarily going to get a nice Free Software driver that works out the box. It may be an x86 Ubuntu only binary driver, or a patch that only works against Linux 2.6.4 or something equally useless. That's why I didn't list this as option 1. But it's usually a better sign than nothing at all.

Re:Would you rather have completely unsupported HW

[HeronBlademaster]

Step one would be: don't shop at Best Buy, as you're probably paying too much.

Step two would be: shop at home, online, where you can compare both prices and compatibility with your OS.

I think these steps are valid whether or not you're a clueless end-user. Clueless end-users are more than capable of comparison shopping online (and if the end-user really wants to buy from Best Buy, they can look at Best Buy's website without leaving home).

I am against this.

[FlyingGuy]

And here is why.

Google has proven to be benevolent, but I am not sure I want their hooks in my Linux kernel. Google exists to make money and do things in their own self interest. The problem is if their fork gets merged that they will become the maintainers for this. I believe as long as it remains in their self interest they will maintain the code but as soon as it is no longer in their self interest it will be abandoned and where will that leave us should we all decide to begin uses that functionality?

I think they should put the parts that are different out there, lets us all examine them and then let us decide if we want their frankencode or not.

Where's the ROI?

[tepples]

if it's enough of a market for them

It isn't. Because GNU/Linux has roughly 1% of the desktop share, a lot of companies don't see the return on investment in getting support from upstream.