Slashdot Mirror
Android and the Linux Kernel Community
An anonymous reader links to Greg Kroah-Hartman's explanation of a rift (hopefully mendable) in the development culture of Google's Linux-based Android OS and the Linux kernel itself. "As the Android kernel code is now gone from the Linux kernel, as of the 2.6.33 kernel release, I'm starting to get a lot of questions about what happened, and what to do next with regards to Android. So here's my opinion on the whole matter ..."
Because Google doesn't have their code merged into the mainline, these companies creating drivers and platform code are locked out from ever contributing it back to the kernel community.
Google shows no sign of working to get their code upstream anymore.
Oh come on, was it really a surprise to anyone that Google does only care about OSS when it suits them and drops out instantly when it doesn't. All of their own sites, business and back-end technology is just as closed as Microsoft's.
I see someone coming along and saying "but they contribute to open source!". Sure, they do, they release little snippets of code and open source those products they base on OSS code because they have to by GPL. One could seriously argue if their open sourcing efforts are making better open source community in general, or not. Like TFA states, their ignorance has caused more turmoil than ever before in Linux land. Companies are obviously going to create support and drivers for Android-branch of Linux kernel, but cant contribute the same code back to real Linux kernel. And possibly never will because it costs them too much work, money and time. Even those companies that previously did develop linux drivers. That's not harming Linux and OSS community?
Get off your lazy ass and see what's really happening.
Well, I didn't say he meant that either. I'm talking about the great fuzz around these issues. Everywhere I go there's someone trying to give his/her side of the "conspiracy against Linux", whereas I think that there is no such thing. So what if it can't be merged back? If it can't, it can't -- get something better and don't stay up all night banging the ceiling and complaining (if you get what I mean).
I would like to see people looking at what Google did and say: "Oh well, it's their choice and I can't do anything about it. However, I can help the Linux kernel with something better and more useful than that will ever be."
Unfortunately, few people think like me...am I just nuts?
Have you heard about SoylentNews?
Nothing is stopping people from porting android to use a standard Linux kernel.
Honestly It's a two way street, and I doubt that google looked at the Linux guys and told them to go fornicate with themselves.
Google does releases on their own schedule. Being someone that is deep into Android guts on the x86 level for a project of my own (android based car stereo) I am frustrated at them not releasing the latest version, but I can either sit and while like a baby, or take what I got and build on it.
Do not look at laser with remaining good eye.
You don't merge new features into a robust low-level code base.
You merge support for abstractions the new features rely upon into the low-level code base, and build on them.
Make a case for kernel support of the "new lock" and "security model" independent of Android's reliance on it, and remove as many of the Android drivers using these facilities OUT of the main tree. IMHO, drivers really don't belong there, but should be available in "driver package sets" aggregated by distro providers.
In Liberty, Rene
You can use Android, but you're now dependent on Google for maintenance, bug-fixes, and trusting that they don't include any evil bits.
Google has frankly set a new standard as far as how companies can become very successful by embracing the open and free software communities. I honestly don't think you can point to many other companies that are doing better, nor could you realistically expect to. In the mobile space, pretty much the next nearest competitor in terms of openness is Apple (Darwin, et al) - in other words, a joke. Meanwhile Google not only has a wonderfully organized system for playing with all the Android code, but a broad commitment across products. Look at Wave, for instance - wide, wide open, and very deliberately (because they know it cannot succeed any other way). Google has probably done more for Linux and its credibility than most other companies in the world.
I think this is something totally different - a disagreement about direction between the mainline maintainers and Google's Android team. Corporate developers, even well-intentioned ones, have a conceptual hurdle to get over when someone Not Their Manager is telling them "you must spend x man-months refactoring your code thusly."
Many, many companies have run into this issue with Linux (and other projects) before, and many will again. It usually goes something like this:
Step 1: Whatever. We're Google. Am I going to rearrange my whole development roadmap to follow the directions of some whiny nerd in his mom's basement? LOL.
Step 2: Oh. Crap. Wow it is kind of a lot work maintaining my own entire fork of the Linux kernel/KHTML/etc. all by myself.
Step 3: Either A) capitulation - the last guy is fired or smacked with the clue stick, and the cooperation restarts, or B) a true fork. These usually stagnate and die, and are also riddled with bugs and security holes btw... unless, the fork is really more interesting than what it forked from, in which case, the community switches to the fork and justice is served.
Often between Step 1 and 2, the maintainer will attempt to play a little corporate politics by embarrassing said middle manager in the media. By the way, this is pretty smart and it often works - especially with companies as large but otherwise savvy as Google, a slashdot story can jumpstart efforts to mend the rift by bringing more senior eyes on the problem. Cooperating is in everyone's interest, and they will realize it.
Tired of Political Trolls? Opt Out!
That's completely disingenuous. Everyone agrees that the problems Android needs to solve should be solved. The kernel community simply disagrees with *how* Google solved the problems. Long ago Google could have asked "we need to optimize power usage by doing agressive suspend" and then worked with the kernel community to get a solution that everyone was happy with.
But instead, Google went off into a corner, created their own solution behind closed doors that nobody in the kernel community likes and now it can't go upstream.
It's not about "how we use the kernel", it's about how you coded things in isolation then expect everyone to be ecstatic with the result despite the fact that the gatekeepers never had any input into the design.
Even if the floor is less round? (Not saying this is the case here. Just pointing out, that ideals can change. Einstein said it best: “Leaving research exclusively in the hands of engineers, we would have perfectly functioning oil lamps, but no electricity.”)
Any sufficiently advanced intelligence is indistinguishable from stupidity.
Has there ever really been a serious option B effort to just pick a time and do a major kernel fork and maintain it forever by any company as large as google, with their level of developers and resources/cash behind them? Any precedent there?
"Redhat Enterprise Linux 5" is essentially a massive kernel fork at 2.6.18. Redhat is the most plausible contender for doing this, since they employ a really significant number of the world's kernel devs, including Alan Cox (until last year). That split was even acrimonious at times IIRC. But then again, you can just call it similar to Canonical's LTS - people choose whether to go with a version of something that's more or less mature - and most distros going with the "more mature" option frequently cheat and backport all kinds of things in the meantime (with greater and lesser success).
Depending on who you ask, RHEL can be more risky than mainline. I've definitely had RHEL panics take down production, only to later discover linux kernel bugs that had been fixed in mainline for a while, but that redhat hadn't backported to their ancient linux fork. But then again, people get burned going the other way all the time too. I don't know if anyone's really independently studied what's ultimately safest. My guess is that you are usually safest inside the biggest crowd - i.e. closer to mainline - but not too near the very latest version.
Tired of Political Trolls? Opt Out!
I think thats the whole issue here. Why does Goggle have to do any work to get "accepted" into the main kernel anyway?
The android is an embedded device that works with a small subset of chips sets and devices. Why does it have to be in the main kernel to begin with? Is somone going to be shoving a PCI card in that thing? Heck, why does Google have to spend the weeks/months talking back and forth with the developers when they can get their own solution, that works just as well for them, in half the time?
Talking about "giving back to the community" is all well and good, but I don't expect Google to sacrifice their profitability or deadlines just to make some members of the community happy.
Lets be honest here though, all the code is in GPL so whats stopping someone else from doing the work?
The kernel community simply disagrees with *how* Google solved the problems. Long ago Google could have asked "we need to optimize power usage by doing agressive suspend" and then worked with the kernel community to get a solution that everyone was happy with.
Unfortunately, this is Google's standard operating procedure. Don't get me wrong, I like Google and enjoy tinkering with Android, but its far from perfect. And don't make suggestions to the core developers as some are more than happy to give you the middle finger for simply not bowing to their brilliance. Some of them are pretty good, but some are brilliant at being assholes. I mean, look at some of their widgets. Eeek! Its as if the people who wrote some of the widgets had never actually used UI widgets before. Saw someone politely make suggestion and the Google engineer rudely ignored them after telling them they were wrong.
Should double clicking of a button, which opens a new window, result in two new windows being opened? No. Shouldn't widgets maintain state? Yes. According to Google's they do yet cancel doesn't actually cancel despite the fact they've already notified the user of the change. Its up to applications to actually track state and cancel it. So on and so on. If a widget's cancel doesn't actually cancel, I don't believe that qualifies as stateful.
Despite Google's assurances everything they do is better than anything you could ever do, sadly the truth is nowhere near. Google very much has a problem with not invented here and in some cases, a complete lack of consultation with people who actually know what the hell is going on. As a result, some things Google does are just truly horrific, lacking any clue, forcing developers to add lots of code to work around their huge ego and seemingly lack of real world experience.
I'm posting anonymously because some of the Google engineers also hold petty grudges.
So maybe they need to ship the fork in the short term, meanwhile get engaged in that lengthy debate on something everybody agrees with, and get that working for the next release. Problem solved?
http://www.codestrom.com/wandering/2009/03/zfs-vs-btrfs-comparison.html
I'm hiding the easy answer here deep in the thread under your question.
Google has Android slates (tablet format ARM-powered machines). If Google took 1000 of them ($400K worth) and just gave them to select Kernel developers (several each) to do with what they would, the problem would solve itself. The Kernel developers would adapt their preferred distro to work on the first slate they had, and they'd keep the rest to work as they were intended until it limited them - and then they'd want to break the limit by improving the Android code, which they would then want back in the Kernel tree because they would want their improvements to persist. It's all about motivation, and you motivate a Kernel geek with cool tech he wants to continue to use.
HP did this. Why do you think Kernel.org runs on HP servers and has for many years? It's because HP donates servers strategically, and yields huge benefits therefrom.
Help stamp out iliturcy.
I'm commenting anonymously because I'm closely involved with Android. I'm not going to comment on whether wakelocks are good or bad.
But mjg59's comment about ARM retention state being as good as suspend is blatantly wrong at a SoC level. There are several good reasons why a suspend details is better than retention but I can't go into the details due to NDAs. But I can vouch that I have personally seen power usage shoot up on well known Android phones when suspend is disabled and only retention is done.
So, mjg59, I would kindly request that you stop making such claims. I doubt you have worked on embedded devices -- I believe you work on server level stuff -- but you most certainly haven't worked extensively on the SoCs being used on Android phones.
Just to make my stance clear, I'm all for having the Android kernel merged into mainline.
Does that mean Android handles OOM conditions gracefully? I've only seen one OS that does it well[1], but the Linux behaviour in these conditions is the absolute worst (pick a process at random, typically the one with the most unsaved data, kill it). This is one of the biggest problems I saw with Maemo. Run low on memory, and suddenly apps die. Make sure you save regularly, because there's only a small chance it will be the web browser that is killed and not the editor that you were writing in...
[1] Recent OS X suspends processes that try to allocate when memory is full, reserves a small amount of memory for root, and lets you quit apps to free up memory or delete some files to make some space available for swap. Processes can also set a flag indicating that they have no unsaved data. When this is set, the system can kill them at any point to reclaim their resources and restart them later.
I am TheRaven on Soylent News
Firstly, thanks for posting this. It's the kind of thing that we never really got out of previous discussion on the topic, which makes it much harder to make reasonable technical decisions.
Secondly, your results are interesting. I'd be fascinated to know whether they're due to the implicit differences between retention and suspend (ie, transitioning to full suspend stops processes that would otherwise be generating wakeups while retention doesn't, or cuts power planes that should otherwise be powered down manually) or because of fundamental silicon level issues. On the other hand, I don't think it fundamentally matters. We have an rtc that's capable of generating wakeup events, so it's acceptable for the lowest level runtime idle state to be system suspend with the rtc set for the next scheduler tick. Just make sure that the latency values are set correctly and it ought to work fine with the existing cpuidle code.
Example?
N900? Heavily marketed as something a lot like the G1. Usually as a 'Mobile Internet Device and Smartphone'. I interpret that as an even more general-purpose mobile platform. They chose (probably because they started before Android was) to not use Android, the alternative was, um, Linux. And I see them not going to Android, I think they mistrust Google even more.
This is an area where there is disagreement. But phones DO have different needs. Obviously. Nokia accomodated those within the mainline (?) kernel. Google drove Android in a very different direction.
Let's not get into the Maemo thing. It's just a little different, not quite as much as Android.
deleting the extra space after periods so i can stay relevant, yeah.
So, mjg59, I would kindly request that you stop making such claims. I doubt you have worked on embedded devices -- I believe you work on server level stuff -- but you most certainly haven't worked extensively on the SoCs being used on Android phones.
I have worked on embedded devices - Linux based and others - for over a decade, and mjg59 is right. If you need something as brutal as hard suspend mode, you're doing it wrong. Any serious low power optimized ARM SoC can run down to very low current when idle, and has peripherals which can be individually clocked down and/or gated. I did work in a periphery way on the G1 at Google, and was very surprised at the way power gating was done. Put simply: the only other embedded OS in the same class as Android which does power gating like this is Windows Mobile. Everyone else learned it was unnecessary a long time ago. I was fairly shocked how few engineers had ever done serious embedded work before, and the result shows.
I know the Qualcomm parts have horrendously stupid design decisions in them which prevent decent idle current, but it's a wash compared to the other sources of battery drain. It's also a wash compared to the damage it does to your code design to support full suspend as part of normal per-second operation.
The Linux maintainers are right: Android is just doing it the wrong way. If there's any one feature I think Android could have done without, it's wake-locks. Learn how to use fine-grained clock switching and gating, not brutally-coarse-grained suspend. This isn't a bloody laptop. And no, I'm not saying this as a back seat driver: I really have done this kind of crap for a decade.