This is correct. Just check the "unknown sources" option under settings and you can install apps from pretty much anywhere (web, email, etc) -- the system will show what permissions the app needs and obtain permission to install it (or not) from the user.
The biggest advantage here is that if the "top side" interfaces change, or something needs to change with how the library is compiled (say against glibc for a stock linux platform vs bionic for android), that's pretty trivial to resolve compared with getting a handful or.so or.ko binaries. And yeah, it doesn't help you fix the firmware if the firmware is busted, but it's still an improvement over where things were before, so hey, progress.
That's my hope. My issue with the purists is that it's not obviously clear why they want to see the microcode running on a proprietary RISC core inside the GPU, but not for example the Verilog. Stallman is one of the few people who has a self-consistent model of what he wants to be able to see, arguing that code which is "equivalent to a circuit" (i.e. in ROM) need not be made visible. Now we don't meet this criterion as our microcode lives on the SD card, but that's largely a cost and flexibility issue and we may yet go there to get the FSF endorsement.
I consider "ability to redistribute the firmware for use on the hardware it belongs to" totally reasonable, but I definitely fall into the more pragmatic camp.
From one point of view the cost to Broadcom to making this open source is not nearly the same as for the other GPU vendors -- I suspect this RPC glue is not among the crown jewels of Broadcom's IP
I should have kept some of my notes from those meetings:)
I'm sure they were highly entertaining. Once upon a time I was involved in arguing with a SoC vendor's lawyers over the ability to open source *GPIO* driver support... progress in these areas can be amazingly hard to achieve...
Yup. It's a common solution for video codec blocks, camera pipelines, etc, on ARM SoCs (open driver/library, closed firmware for the peripheral) -- less common for GPUs. I'm personally not at all offended by the "open drivers/libraries on host, closed firmware on peripheral" model -- it's a reasonable tradeoff and, as you point out, lets the vendors keep control over their secret sauce while still allowing for completely open software stacks on the host/AP side of the world. Apart from purists who want to have source for every programmable block on the SoC, everybody wins.
From one point of view the cost to Broadcom to making this open source is not nearly the same as for the other GPU vendors -- I suspect this RPC glue is not among the crown jewels of Broadcom's IP -- but from the viewpoint of someone who doesn't want to have to muck with closed binaries on the host side that are hard to debug, keep supported, adapt to changing APIs/ABIs, none of that matters -- the important bit is you get all the host side source.
Does this (or will this) support future / higher end parts using the same VideoCore architecture? It definitely increases my interest in the BCM SoC family if so...
To be fair, "open drivers / libraries plus proprietary firmware binary to download to hw block" is a better model than "closed userspace libraries/drivers" -- at least you have control/ownership over all the software running on the host side in this model.
Haven't had a chance to wade through all the code here, but it looks an awful lot like it's basically rpc stubs and glue (allocator, buffer management, etc) to remote the OGLES calls to the media processor, which presumably handles all the actual translation of OGLES API to whatever the internal architecture of the GPU looks like. Which is a perfectly reasonable approach, just not necessarily 1:1 with the other SoCs (which don't have a GPU block capable of speaking remoted OGLES, thus requiring knowledge of the underlying hardware "secret sauce" in the GL libraries or drivers on the host side).
Awesome that they're releasing this as open source rather than insisting on keeping it closed -- much easier to work with this way.
We've worked hard to never ship binary blob kernel drivers or any nonsense like that in Nexus devices, but yeah, proprietary userspace libraries are still needed for some peripherals against our best efforts. For wifi/bt it's generally just firmware that's downloaded to the device and not actual code that runs on the Linux side of the world. For the GPU all the vendors we've worked with on Nexus devices have provided GPL'd drivers for the kernel (resource management, etc), but provide closed opengl userspace libraries. Camera stacks vary -- the Nexus S didn't require any proprietary camera library goop in userspace iirc. I think we've been steadily improving the situation, but it's an uphill battle and I don't see a fully-open and also adequately performant GPU/opengl solution on the horizon yet.
By Google? Highly unlikely that we'd ever do that. We're pretty heavily invested in the Linux kernel and it's been working well for us in Android.
By some other entity? Could happen, but the mid and lower layers of the Android stack depend pretty heavily on running on a Linux kernel. It'd be a bunch of work for questionable gain.
Well, it wasn't easy to develop for until about year after launch, since they had no SDK at launch. Excellent spit and polish though on the software and hardware. Apple always nails that last 10%.
The handful of closed source userspace pieces necessary (some firmware, the hardware opengl libraries, samsung radio library, etc) will end up here: http://code.google.com/android/nexus/drivers.html (please disregard the unfortunate use of "drivers" here -- all the kernel drivers are GPLv2, none are closed source)
What's not included is the Google Mobile Apps (gmail, gcalendar, gtalk, maps, etc). These are proprietary Google applications, not part of the core Android platform (which consists of the lower level libraries, dalvik vm, framework libraries, services, core apps like phone, contacts, launcher, settings, etc, etc).
The closed bits on Galaxy Nexus are pretty similar to Nexus S (though the orientation sensor no longer requires a closed library -- one is provided that "adds extra value" but the system works fine without it).
Nexus S, for example, requires these pieces above and beyond the available open source userspace and kernel code: - opengl userspace libraries (ImaginationTech) - radio interface library (Samsung) -- glue between the Android Telephony stack and the Radio - firmware for bt/wifi chip (Broadcom) - GPS userspace library (Broadcom) - NFC firmware / loader library (NXP) - auto-calibration library for orientation sensor (AKM)
The closed pieces are available here, under a license that allows you to use them in your own builds, and even to redistribute (non-commercially) entire flashable OS images including them.
We continue to work to reduce the number of closed binaries needed on the lead Android devices. We have *never* shipped a lead device that includes any non-gpl/bsd kernel code.
Seems pretty absurd to me. They just launched an Android-based platform with Amazon-customized UI, their apps already run on Android devices (of which there are quite a few out there), and they have their shiny new cloud-assisted browser, built on Android and EC2. What do they need WebOS for? How are they a "fitting home" for it?
There actually are relatively few closed pieces necessary to make something like Nexus S go. For the most part they're firmware, not actual "drivers", though the opengl libraries fall into a middle ground -- the SGX kernel driver is GPLv2, the userspace opengl libraries are closed. We've been working with vendors to make the closed pieces available under a license that allows them to be included in builds and distributed non-commercially (commercial distributors tend to be OEMs which have direct relationships and licenses with these vendors already): http://code.google.com/android/nexus/drivers.html
I'd use the term "proprietary components" rather than "drivers" (the above URL is unfortunately named), personally, since at least for lead devices that Google works on, all the kernel drivers are GPLv2 and readily available: http://android.git.kernel.org/?p=kernel/common.git;a=shortlog;h=refs/heads/android-3.0 (kernel/common is code common to all variants, SoC-specific work is under kernel/omap, kernel/msm, etc)
Keep in mind that the *any* device that supports USB Host mode can be an Accessory. There's a full open source reference implementation for Arduino, but the protocols are documented and open and you can implement it on any hardware you like.
I have often thought we would have avoided a lot of headaches if we started with NetBSD instead of Linux....
The reality is that in 2005, Linux had the most momentum with the silicon vendors, the largest base of developers (if you're looking to hire systems folks with experience with the kernel), and was easier to explain to carriers, OEMs, silicon vendors, investors, etc. The hardware platform was no longer highly constrained: at Danger in 2000, we started with a 24MHz ARM7, with no MMU, 8K unified cache, 16MB ram, 4MB flash -- at Android in 2005, a 200MHz 64MB ARM9 based system was the baseline -- we had a MMU! luxury! -- Linux no longer seemed like overkill for the hardware at hand.
Linux also draws a bright line between kernel and userspace with the explicit carve-out in the COPYING file declaring that userspace code need to be GPL to interface with the kernel. Explaining that "kernel is GPLv2, userspace is Apache2/BSD/MIT" is pretty straightforward. Explaining that everyone benefits from having a better kernel but the kernel is seldom what makes one product win or lose in the market is also pretty straightforward (seriously -- users only know the kernel exists if it's not working, otherwise it's not what they care about at all).
Realistically, it should be under a week's work for somebody to use the NDK (native development kit) to throw together an Xserver that runs inside a SurfaceFlinger (Android's compositor) Surface and then you could run whatever existing X-compatible thing under that. You could build a wrapper/adapter app that would allow "native linux apps" to trivially run this way (basically app xserver/clientlibrary window compositor/windowmanager).
It's not hard to build open source software to run on Android and there are plenty of tools to help you do it, especially on Gingerbread were the NDK provides support for native (C/C++) apps.
Yes, Android is not natively X11/Gtk/whatever, but that certainly didn't stop people from getting this stuff to run on Win32 or OSX...
I'd argue that having an open platform that Google can ship their apps on is extremely important in the huge and quickly growing mobile space.
It would be unfortunate if the only game in town for mobile OSes were a closed platform where only software approved by the platform owner could be installed. How could one defend against such an eventuality? Perhaps one might invest in building a competitive open mobile platform and make it widely available, no strings attached.
The "collecting data" thing is a bit tinfoil hat-ish for me. I see it much more simply as "if it's impossible for users to use your apps because the dominant platforms are controlled by folks who shut the door on you, you're going to have trouble obtaining (or keeping) users."
Android has a HAL for GPU integration (we call it gralloc), Gingerbread brings incremental and concurrent GC in Dalvik, and the Gingerbread NDK provides for all-native development options, among other improvements. You're welcome.
Nexus S will ship with support for "fastboot oem unlock", allowing for reflashing of the system software "out of the box", like Nexus One.
Something that may interest this community is that the NDK (native development kit) for Gingerbread now supports native apps (intended to simplify mobile gaming ports, etc) -- providing: libc, libm, libz, opengl|ES, opensl|ES, input/events/sensors, app lifecycle management, etc. JNI is available to access various higher level Android APIs as necessary.
Platform sources should ship at or shortly after commercial launch of Nexus S. Kernel git repository (2.6.35 + android + s5pc111/nexus-s) will be available at or shortly before launch.
Slashdot Mirror
This is correct. Just check the "unknown sources" option under settings and you can install apps from pretty much anywhere (web, email, etc) -- the system will show what permissions the app needs and obtain permission to install it (or not) from the user.
The biggest advantage here is that if the "top side" interfaces change, or something needs to change with how the library is compiled (say against glibc for a stock linux platform vs bionic for android), that's pretty trivial to resolve compared with getting a handful or .so or .ko binaries. And yeah, it doesn't help you fix the firmware if the firmware is busted, but it's still an improvement over where things were before, so hey, progress.
That's my hope. My issue with the purists is that it's not obviously clear why they want to see the microcode running on a proprietary RISC core inside the GPU, but not for example the Verilog. Stallman is one of the few people who has a self-consistent model of what he wants to be able to see, arguing that code which is "equivalent to a circuit" (i.e. in ROM) need not be made visible. Now we don't meet this criterion as our microcode lives on the SD card, but that's largely a cost and flexibility issue and we may yet go there to get the FSF endorsement.
I consider "ability to redistribute the firmware for use on the hardware it belongs to" totally reasonable, but I definitely fall into the more pragmatic camp.
From one point of view the cost to Broadcom to making this open source is not nearly the same as for the other GPU vendors -- I suspect this RPC glue is not among the crown jewels of Broadcom's IP
I should have kept some of my notes from those meetings :)
I'm sure they were highly entertaining. Once upon a time I was involved in arguing with a SoC vendor's lawyers over the ability to open source *GPIO* driver support... progress in these areas can be amazingly hard to achieve...
Yup. It's a common solution for video codec blocks, camera pipelines, etc, on ARM SoCs (open driver/library, closed firmware for the peripheral) -- less common for GPUs. I'm personally not at all offended by the "open drivers/libraries on host, closed firmware on peripheral" model -- it's a reasonable tradeoff and, as you point out, lets the vendors keep control over their secret sauce while still allowing for completely open software stacks on the host/AP side of the world. Apart from purists who want to have source for every programmable block on the SoC, everybody wins.
From one point of view the cost to Broadcom to making this open source is not nearly the same as for the other GPU vendors -- I suspect this RPC glue is not among the crown jewels of Broadcom's IP -- but from the viewpoint of someone who doesn't want to have to muck with closed binaries on the host side that are hard to debug, keep supported, adapt to changing APIs/ABIs, none of that matters -- the important bit is you get all the host side source.
Does this (or will this) support future / higher end parts using the same VideoCore architecture? It definitely increases my interest in the BCM SoC family if so...
To be fair, "open drivers / libraries plus proprietary firmware binary to download to hw block" is a better model than "closed userspace libraries/drivers" -- at least you have control/ownership over all the software running on the host side in this model.
Haven't had a chance to wade through all the code here, but it looks an awful lot like it's basically rpc stubs and glue (allocator, buffer management, etc) to remote the OGLES calls to the media processor, which presumably handles all the actual translation of OGLES API to whatever the internal architecture of the GPU looks like. Which is a perfectly reasonable approach, just not necessarily 1:1 with the other SoCs (which don't have a GPU block capable of speaking remoted OGLES, thus requiring knowledge of the underlying hardware "secret sauce" in the GL libraries or drivers on the host side).
Awesome that they're releasing this as open source rather than insisting on keeping it closed -- much easier to work with this way.
We've worked hard to never ship binary blob kernel drivers or any nonsense like that in Nexus devices, but yeah, proprietary userspace libraries are still needed for some peripherals against our best efforts. For wifi/bt it's generally just firmware that's downloaded to the device and not actual code that runs on the Linux side of the world. For the GPU all the vendors we've worked with on Nexus devices have provided GPL'd drivers for the kernel (resource management, etc), but provide closed opengl userspace libraries. Camera stacks vary -- the Nexus S didn't require any proprietary camera library goop in userspace iirc. I think we've been steadily improving the situation, but it's an uphill battle and I don't see a fully-open and also adequately performant GPU/opengl solution on the horizon yet.
Nope. The Nexus 7 has an unlockable bootloader (via fastboot oem unlock), like the Galaxy Nexus, and the Nexus S. Happy Hacking!
By Google? Highly unlikely that we'd ever do that. We're pretty heavily invested in the Linux kernel and it's been working well for us in Android.
By some other entity? Could happen, but the mid and lower layers of the Android stack depend pretty heavily on running on a Linux kernel. It'd be a bunch of work for questionable gain.
Non-carrier-operated app store? Danger Hiptop, 2003. Been there, done that, got (more than one) t-shirt.
Well, it wasn't easy to develop for until about year after launch, since they had no SDK at launch. Excellent spit and polish though on the software and hardware. Apple always nails that last 10%.
Yes, the ICS tree that will be released to AOSP is the same code used to do the build for Galaxy Nexus (among other products).
Instructions on building for Galaxy Nexus will likely end up here (alongside Nexus S):
http://source.android.com/source/building-devices.html
The handful of closed source userspace pieces necessary (some firmware, the hardware opengl libraries, samsung radio library, etc) will end up here:
http://code.google.com/android/nexus/drivers.html
(please disregard the unfortunate use of "drivers" here -- all the kernel drivers are GPLv2, none are closed source)
What's not included is the Google Mobile Apps (gmail, gcalendar, gtalk, maps, etc). These are proprietary Google applications, not part of the core Android platform (which consists of the lower level libraries, dalvik vm, framework libraries, services, core apps like phone, contacts, launcher, settings, etc, etc).
The closed bits on Galaxy Nexus are pretty similar to Nexus S (though the orientation sensor no longer requires a closed library -- one is provided that "adds extra value" but the system works fine without it).
There are remarkably few binary blobs in Android lead devices (some OEMs add more proprietary goop to their own phones though).
http://source.android.com/source/building-devices.html
Nexus S, for example, requires these pieces above and beyond the available open source userspace and kernel code:
- opengl userspace libraries (ImaginationTech)
- radio interface library (Samsung) -- glue between the Android Telephony stack and the Radio
- firmware for bt/wifi chip (Broadcom)
- GPS userspace library (Broadcom)
- NFC firmware / loader library (NXP)
- auto-calibration library for orientation sensor (AKM)
The closed pieces are available here, under a license that allows you to use them in your own builds, and even to redistribute (non-commercially) entire flashable OS images including them.
We continue to work to reduce the number of closed binaries needed on the lead Android devices. We have *never* shipped a lead device that includes any non-gpl/bsd kernel code.
Seems pretty absurd to me. They just launched an Android-based platform with Amazon-customized UI, their apps already run on Android devices (of which there are quite a few out there), and they have their shiny new cloud-assisted browser, built on Android and EC2. What do they need WebOS for? How are they a "fitting home" for it?
There actually are relatively few closed pieces necessary to make something like Nexus S go. For the most part they're firmware, not actual "drivers", though the opengl libraries fall into a middle ground -- the SGX kernel driver is GPLv2, the userspace opengl libraries are closed. We've been working with vendors to make the closed pieces available under a license that allows them to be included in builds and distributed non-commercially (commercial distributors tend to be OEMs which have direct relationships and licenses with these vendors already):
http://code.google.com/android/nexus/drivers.html
I'd use the term "proprietary components" rather than "drivers" (the above URL is unfortunately named), personally, since at least for lead devices that Google works on, all the kernel drivers are GPLv2 and readily available:
http://android.git.kernel.org/?p=kernel/common.git;a=shortlog;h=refs/heads/android-3.0
(kernel/common is code common to all variants, SoC-specific work is under kernel/omap, kernel/msm, etc)
Keep in mind that the *any* device that supports USB Host mode can be an Accessory. There's a full open source reference implementation for Arduino, but the protocols are documented and open and you can implement it on any hardware you like.
Docs and Specs: http://accessories.android.com/
Google IO Talk: http://www.youtube.com/watch?v=s7szcpXf2rE
I have often thought we would have avoided a lot of headaches if we started with NetBSD instead of Linux....
The reality is that in 2005, Linux had the most momentum with the silicon vendors, the largest base of developers (if you're looking to hire systems folks with experience with the kernel), and was easier to explain to carriers, OEMs, silicon vendors, investors, etc. The hardware platform was no longer highly constrained: at Danger in 2000, we started with a 24MHz ARM7, with no MMU, 8K unified cache, 16MB ram, 4MB flash -- at Android in 2005, a 200MHz 64MB ARM9 based system was the baseline -- we had a MMU! luxury! -- Linux no longer seemed like overkill for the hardware at hand.
Linux also draws a bright line between kernel and userspace with the explicit carve-out in the COPYING file declaring that userspace code need to be GPL to interface with the kernel. Explaining that "kernel is GPLv2, userspace is Apache2/BSD/MIT" is pretty straightforward. Explaining that everyone benefits from having a better kernel but the kernel is seldom what makes one product win or lose in the market is also pretty straightforward (seriously -- users only know the kernel exists if it's not working, otherwise it's not what they care about at all).
Realistically, it should be under a week's work for somebody to use the NDK (native development kit) to throw together an Xserver that runs inside a SurfaceFlinger (Android's compositor) Surface and then you could run whatever existing X-compatible thing under that. You could build a wrapper/adapter app that would allow "native linux apps" to trivially run this way (basically app xserver/clientlibrary window compositor/windowmanager).
It's not hard to build open source software to run on Android and there are plenty of tools to help you do it, especially on Gingerbread were the NDK provides support for native (C/C++) apps.
Yes, Android is not natively X11/Gtk/whatever, but that certainly didn't stop people from getting this stuff to run on Win32 or OSX...
https://market.android.com/details?id=com.google.android.apps.authenticator
https://market.android.com/details?id=com.google.android.apps.authenticator
I'd argue that having an open platform that Google can ship their apps on is extremely important in the huge and quickly growing mobile space.
It would be unfortunate if the only game in town for mobile OSes were a closed platform where only software approved by the platform owner could be installed. How could one defend against such an eventuality? Perhaps one might invest in building a competitive open mobile platform and make it widely available, no strings attached.
The "collecting data" thing is a bit tinfoil hat-ish for me. I see it much more simply as "if it's impossible for users to use your apps because the dominant platforms are controlled by folks who shut the door on you, you're going to have trouble obtaining (or keeping) users."
Android has a HAL for GPU integration (we call it gralloc), Gingerbread brings incremental and concurrent GC in Dalvik, and the Gingerbread NDK provides for all-native development options, among other improvements. You're welcome.
http://android-developers.blogspot.com/2010/12/android-23-platform-and-updated-sdk.html
Nexus S will ship with support for "fastboot oem unlock", allowing for reflashing of the system software "out of the box", like Nexus One.
Something that may interest this community is that the NDK (native development kit) for Gingerbread now supports native apps (intended to simplify mobile gaming ports, etc) -- providing: libc, libm, libz, opengl|ES, opensl|ES, input/events/sensors, app lifecycle management, etc. JNI is available to access various higher level Android APIs as necessary.
2.3 (platform 9) SDK: http://developer.android.com/sdk/android-2.3.html
2.3 (revision 5) NDK: http://developer.android.com/sdk/ndk/index.html
Platform sources should ship at or shortly after commercial launch of Nexus S.
Kernel git repository (2.6.35 + android + s5pc111/nexus-s) will be available at or shortly before launch.
Enjoy!
Use adb install to install apps via usb (including third party app stores).