AES is fixed by standard. There is no need to "maintain" it - as long as the code compiles properly you're done.
Not if you care about security. What about side channel attacks? What about leaking sensitive data into the heap? What about performance? On various architectures? I could go on.
Merely compiling and generating correct test vector outputs is far from all an implementation has to do to be good. Luckily, there are good implementations out there, easily obtained and under generous licenses.
There's an implicit "unless you *really* know what you're doing" to the sentence, which just tends to not be the case for most people
It's not the case for any people. You won't see professional cryptographers rolling their own crypto and using it, either.
I'm not a cryptographer myself, but I am a very experienced cryptographic security systems engineer, and I work with a bunch of serious cryptographers, who are well-published and extremely well-respected in academic circles -- exactly the sort of people who you'd expect to be most capable of designing and building custom systems. And you know what? They don't.
And I'm not just talking about creating new ciphers. Even when I go to them with novel requirements that seem to demand some sort of new construction using existing algorithms and techniques, the very first thing they do is go to the literature to see what has been done, how long it's been in use, how widely it's been reviewed and analyzed, etc. The less knowledgeable (like me, frankly, though I'm getting better) tend to start by cooking up some new scheme. Real experts avoid that if at all possible, and if they have to do something new they look really hard at how they can prove its security by reducing it to known constructions.
Even the guys who do create new ciphers do it with great care, often spending years designing and attacking and tweaking, and then their next step is to publish it so others can attack it. Only after it has survived lots of other review does anyone, especially the author, begin to trust it for real use. But the most common outcome, when something new is designed, even by serious experts, is that it gets broken shortly after publication. It's quite common for new algorithms and constructions to be broken at the same conference they're initially presented.
I reiterate: No one who knows what they're doing creates new crypto for production work.
Moreover, people who know what they're doing even approach implementation of known and trusted algorithms with trepidation! There are so many very subtle things you can get wrong. Heh, just last week someone pointed out that my implementation of a constant-time memcmp had a subtle bug that caused it to be not quite constant-time on some architectures. Novices have no idea why it even matters in crypto that memcmp always run in the same amount of time for a given buffer size, irrespective of the contents of the buffers, and assume that the C library's memcmp is fine. More knowledgeable engineers know why it matters, but really deep expertise is required to get it right. That's just one tiny example. My primary mistake wasn't the bug in my implementation, it was trying to write memcmp at all. I should have found a well-vetted implementation and used that.
Doing your own crypto is nothing like doing your own science or doing your own music. The thing about security is that it's only as strong as the weakest link; the tiniest crevice can give the attacker a wedge to bust your system wide open. Other fields are forgiving of minor flaws, you can do useful and interesting work even if it has some defects. In security, and crypto is often at the heart of security solutions, one tiny mistake can render the totality of what you did not only useless, but actively dangerous to your users.
If writing a good secure memcmp is too hard for an engineer with 25 years' experience, including 20 years doing cryptographic security, what does that say about trying to write something that doesn't appear to be trivial? Crypto is hard. Really, really hard. The more you learn about it the harder it gets, because you understand more about what can go wrong.
I agree, although I have a more optimistic outlook. The difference is that in the US the people actually do have a say. Our problem of late has been that a majority, or at least a large minority, of people have favored the "tradeoff", because they didn't understand either the costs or the benefits, and many others simply haven't paid attention, leaving only a fairly small minority opposing it.
I think the pendulum has begun swinging back the other way, though. It has a long way to go, but Snowden's revelations helped a lot, and if we can get a few more I think the people will become more cautious about their rights. In addition, the terrorism scare is dying down. Of course, the risk there is that another large and successful attack will bring that back.
In addition, we do have a legal backstop in the constitution. It takes a long time for that to take effect, as first the people who want to challenge the unconstitutional laws have to figure out how to attack them in court, then the challenges have to wend their way through the court process, which is lengthy. But although the wheels grind slowly, they do eventually get there.
Finally, we have another last resort that the Chinese did not have. Americans are armed. The weapons we have are mostly for hunting and sporting purposes, not military arms, but the differences aren't really that large in practice, at least compared to military small arms. In addition, the states are armed with heavier firepower, up to and including fighter aircraft, attack helicopters, tanks, etc. On top of that, our actual military forces consist of people who have taken an oath to support and defend the constitution, not the government, and many of them take it quite seriously. All this adds up to a citizenry that is capable of beginning the resistance, has an opportunity to convince states to bring their firepower into it, and will attract the sympathy of many in the armed forces.
Of course, I really, really hope it doesn't come to that last option. And I don't think it will. Either enough of the people will come around and the system will be changed via political and legal processes, or we'll collectively decide that we really do want our government spying on us. Either way, I'm confident that the will of the people will win out. Mostly, anyway.
Actually, they recommend browsers disable those capabilities unless the user consents to enabling them.
Right, disable user-positive features by default unless the user reconfigures their browser to the contrary.
No, disable potentially user-positive and potentially user-harmful features unless the user approves on a dialog containing suitably-scary text so they will stop and think about whether or not they trust this extension that they're giving control over and visibility into all of their browsing.
It's fine to actively prompt; no need to require users to go searching for the setting. But it's important to give users the control, so drive-by extension installs (like Superfish) don't assume it for them.
Google sez we must remove ad blocker functionality!
I smell an ulterior motive..
Reading comprehension fail. The summary says:
Amongst the report's recommendations to address the problem is the suggestion that browser makers "harden their environments against side-loading extensions or modifying the browser environment without user consent." Some of the most popular extensions for Chrome and Firefox, including ad-blockers, depend on this functionality.
I'd expect that most users who install ad blockers consent to having it modifying the browser environment.
You asked and I answered. FYI, I've been driving a Nissan LEAF for the last three years, in the mountains of Colorado and Utah. It gets cold here. Car works just fine.
(there used to be a company that wrote compilers for Windows shareware developers called Borland
I can see you're a young'un. Borland didn't make "compilers for Windows shareware developers", Borland was the dominant maker of high-quality Pascal, C, and C++ development tools for the PC platform for many years, for DOS and Windows, especially DOS. Borland's overlay manager was fantastic and made large DOS programs (programs that used more than 640K) very easy. Microsoft eventually managed to shove them out, but Borland was by far the better tool set for years, and was the choice of many professional development shops. It's what the software companies I worked for in the early to mid-90s used.
I think what eventually did them in was their decision to try to offer a better Windows API than that mess that was MFC. Borland's Object Windows Library was orders of magnitude cleaner and better thought-out than MFC, but because they had to wait for each release of Windows before they could update it, it lagged behind MFC in functionality. Probably also because the OWL team took the time to think about how to do things right, rather than just slapping a thin pseudo-OO wrapper on the Win32 API, which was MFC's approach. So Windows devs learned that if you wanted to stay current, you needed to use MFC, hence you may as well use Microsoft's whole tool set, since you had to buy it all (several hundred dollars).
These day's Duff's Device is almost always premature optimization, because compilers will unroll most loops for you. You should only use it after profiling to prove that the loop is a hot spot, and after benchmarking to prove that it actually improves performance. Even then you should include both implementations, selecting one by #ifdef, so you can periodically re-test, because eventually a new platform or new compiler or something will result in it no longer making a difference, or even being more expensive, if the compiler is cleverer than you.
They really ought to have done it the other way around: break by default, and use "continue" to fall through. It can indeed be useful sometimes, but in the vast majority of cases you want it to break, and forgetting that statement causes all sorts of trouble.
Depends on the sort of code. There is lots of code that needs to handle, for example, large classes of error codes, and for 50 codes there may be only two or three sensible ways to handle them. Being able to have big lists of cases that nearly all fall through is very nice then.
Your notion of "optimizing for the hardware" is something that isn't real. According to your theory, Linux also shouldn't perform well because it also is hardware-agnostic.
Why would it not perform well? Optimization would make it perform better it isn't a pre-requisite for it to perform adequately.
And of course we optimize for hardware, if you're doing GPU acceleration for example you're going to use the specific features of a particular GPU, if you want to utilize it effectively you don't just use the same path for all the different PowerVR GPUs, different Ardreno GPUs and different Mali GPUs. It would be quite a task to write an optimized implementation for all which is why iOS and Windows Phone do it only for a limited subset while Android goes for a generic approach. It's not going to exploit the hardware capabilities as well but it will work across more hardware.
The solution to that sort of optimization is to delegate it to drivers. And providing and debugging drivers is a big part of what OEMs have to do (actually, the drivers are provided by the hardware vendors -- OEMs don't make the bits and pieces, they put them together -- but generally as source code and OEMs tweak and fix).
Your notion of "optimizing for the hardware" is something that isn't real. According to your theory, Linux also shouldn't perform well because it also is hardware-agnostic.
As for what OEMs have to do, a modern mobile device is immensely complex, consisting of dozens of processors, many on the SoC (system on a chip) but many not. All of them have to be configured, which is a complex and tedious operation, and easy to get wrong -- and every custom board requires a custom configuration. In addition, there are drivers for all of the bits and pieces that have to be assembled and tested together. Plus there's also typically a complex, multi-stage boot process that has to be orchestrated to bring up all the bits and pieces of the hardware in the right way and in the right order. And other stuff that I don't know about because I'm not a hardware systems guy.
Some of the above doesn't depend on the OS, and can be done before it's available. But much of it does depend on OS requirements and has to wait.
And then if the OEM decides to customize Android they have to do that, with whatever skin, and default apps they want, plus whatever changes they need deep in the system to support the hardware and their changes to the software. Finally there is lots and lots of testing, because such complex, custom devices always expose new interactions between components that have to be debugged and fixed. Oh, and lots of hardware testing as well, including endless burn-in tests to validate that the stuff not only works but that some subtle design flaw doesn't stop it from working.
And I'm sure there's still more that I don't know about at that level as well.
Then they have to run Google's compliance tests, to find out what they've broken with all of their changes, or what they missed in configuring their device for proper support (actually, this is something they do throughout, not at the end), and then go back and fix what's broken until it passes... or else negotiate with Google for waivers on things they think should be okay.
Then comes carrier validation and testing, more rounds of fixes, etc.
Little or none of this has anything to do with "optimization". That's mostly the compiler's job, and it does that job well.
Er, nothing actually. TFA mentioned "Microsoft's take on Unix", which I took to mean NT's stab at POSIX support, or maybe something else equally ridiculous. Looking at the article again, it actually says "Xenix, Microsoft's take on Unix". Not being more than vaguely aware of Xenix, I didn't realize it was bought by Microsoft and I took that text as two separate items in the list (should have paid closer attention to commas vs semi-colons).
Also you forgot SCO if you are including commercial Unixes for 386
Indeed. There I claim selective memory, driven by the massive stain on the Unix world left by SCO's successor-in-ownership, The SCO Group.
Also one that gets forgotten about but was quite good in those early days was: Coherent
I heard good things about Coherent back in the day, but never touched it.
It was because Linux more or less worked, and people could use it and add to it because of the GPL. The competitors all had problems:
* Minix was cheap but not free, and couldn't be redistributed with modifications. People worked around that by maintaining patch sets, but that was even more painful then than it is now (we have better tools now).
* The BSDs were in a quagmire of legal uncertainty and competing claims. Nobody knew for sure if BSD was free or not, so everyone assumed it wasn't.
* Xenix: Not free.
* Microsoft: Are you kidding me?
* SYSV: Not free
* HURD: Didn't work, and had such an elegant architecture that it wasn't clear if it could ever work.
That was the space when Linus Torvalds started hacking around (except HURD didn't even exist yet). If he'd been able to hack on Minix, he would have. But the license prevented it, so he took the opportunity to start his own. Lots of other people saw exactly the same situation and joined him in hacking on something that (a) worked, more or less and (b) they could hack on.
It's not that Linux lucked out and the rest of the competition failed. There was no other competition that satisfied the requirements of being free and hackable. It was also important that Linus was an excellent Benevolent Dictator that gave people few reasons to fork. Actually, on that last point it's rather impressive that Linus is still in charge, even after it's become an incredibly valuable property, used and contributed to by lots of megacorps.
They can prioritize all they want, but no one wants to pay for the carrier certification of thee modified SDRs, particularly when using a T-Zone on a Snapdragon chip in order to run the baseband, and the FCC demands that the SDR be certified as a unit (software + hardware). That's a carrier certifiiation per carrier, per country, per device, per version update.
Heh. That isn't the problem. Unfortunately, I can't explain in more detail, because my conversations with carriers are confidential.
Also no carrier using a contract lock-in revenue model is going to provide an update that doesn't lock you into a new contract
Also not the problem, and I also can't explain. I'll just point out that the carriers have so successfully branded Android as their own that many consumers see the failure to upgrade as the carriers' fault. The carriers aren't blind to this, or what it costs them.
You should also be aware that the image that's shipped by the OEM is often not even buildable by Google engineers
Why yes, Terry, as a Google Android engineer I'm quite aware of this:-)
Except you should replace "often" with "never". Or at least "almost never". There may be some exceptions, though I've never heard of one.
apart from the fact that the devices used during development are generally signature neutered
A bigger issue is that the devices used during development are Nexus devices, not OEM devices. We never see those, either the hardware details, or the code that OEMs build after all of their customization.
Seriously, one of the smartest things that Apple did was keep the baseband processor separate from the application processor so that there was no telecom recertification required, unless they were explicitly hacking the baseband for some reason
That does make a lot of sense, but I don't think it's actually relevant to the problem. The carriers validate a lot more than just telecom functionality on devices that carry their brand.
So, is mute still under power button? Cause it does not make sense to move airplane mode from there and leave mute, data network mode and the rest there. Airplane mode is a mute for the radio.
No, mute was moved to the volume button, and enhanced to allow silencing of all but alarms (which is what mute always did) or complete silencing. I think it makes sense to put muting of audio on the audio volume control.
Didn't older Nexus devices get releases at the same time we heard about them? My Nexus 6 and 7 did not. That took months. Hell, I bought these two Nexus devices (in part) because I thought it'd mean I'd get stuff promptly. Though, for perspective, AT&T and Samsung took well OVER A YEAR to release Android updates... which compelled me to switch.
You apparently completely missed my point. No, Google doesn't arrange to upgrade the Nexus devices as soon as the new release is announced, for the reasons I explained.
The state of it all is really quite absurd, either way.
Actually, with all respect, it's your expectations that are absurd. I understand why you have them, because you see other single-player device lines (notably iOS devices) which can delay announcement until the release is ready for devices. But there's really no way that could work in the Android ecosystem. Engineering takes time, and there's quite a lot of work to be done after the core OS version is fully baked. Google can't use its position as the source of the core OS to give the Nexus line too much of a leg up because the Nexus devices compete with the partners' devices.
If Google is going to hold back on pushing updates to appease partners, why bother having them in the first place?
It's a balance. Google walks a fine line with its partners, and the Nexus line is a major potential conflict point, because Google is competing directly with its partners, and doesn't care about making a profit (though the OEM that actually manufactures the device wants one). So, yes, Nexus is Google's tool for pushing the OEMs around, but it has to be applied with sensitivity.
They are *great* tablets (except for the new 9" model which, IMHO, is a dog.)
I quite like my Nexus 9. Though maybe I'd like it less if I had to pay for it, not sure. I still have my N7 around, but I find I use it less and less.
But the development process is closed, the problem is that they develop the software and don't work with the hardware vendors, this is part of the problem.
We do work with the hardware vendors and they have access to the under-development code. But most of them (quite sensibly, IMO) hold off doing very much until quite late in the product cycle, because change is fast and furious and they don't want to spend a lot of time spinning doing work they just have to redo.
If the OEMs were involved in the development process you would have the hardware and software components better matched and optimized and also a more timely release.
I don't know what you mean by "better matched". Android is specifically not tailored to any particular hardware. That's by design, and it's a good thing. Certainly there are some down sides, but its what makes the vibrant, competitive ecosystem possible, and that ecosystem is why Android's market share is what it is, because it serves consumers.
As for a more "timely" release, I think we already do pretty much everything that can be done, short of inviting all of the vendors to participate in design process. Many of my colleagues previously worked for various hardware vendors before joining Google, and they assure me that would be an absolute disaster for various reasons, not least because design by committee doesn't work -- especially when many of the committee members are deeply self-interested and mutually antagonistic.
Google has a lot of smart people who fully understand these problems and want to find solutions. You can be pretty certain that all of the low-hanging fruit was picked long ago.
Slashdot Mirror
AES is fixed by standard. There is no need to "maintain" it - as long as the code compiles properly you're done.
Not if you care about security. What about side channel attacks? What about leaking sensitive data into the heap? What about performance? On various architectures? I could go on.
Merely compiling and generating correct test vector outputs is far from all an implementation has to do to be good. Luckily, there are good implementations out there, easily obtained and under generous licenses.
There's an implicit "unless you *really* know what you're doing" to the sentence, which just tends to not be the case for most people
It's not the case for any people. You won't see professional cryptographers rolling their own crypto and using it, either.
I'm not a cryptographer myself, but I am a very experienced cryptographic security systems engineer, and I work with a bunch of serious cryptographers, who are well-published and extremely well-respected in academic circles -- exactly the sort of people who you'd expect to be most capable of designing and building custom systems. And you know what? They don't.
And I'm not just talking about creating new ciphers. Even when I go to them with novel requirements that seem to demand some sort of new construction using existing algorithms and techniques, the very first thing they do is go to the literature to see what has been done, how long it's been in use, how widely it's been reviewed and analyzed, etc. The less knowledgeable (like me, frankly, though I'm getting better) tend to start by cooking up some new scheme. Real experts avoid that if at all possible, and if they have to do something new they look really hard at how they can prove its security by reducing it to known constructions.
Even the guys who do create new ciphers do it with great care, often spending years designing and attacking and tweaking, and then their next step is to publish it so others can attack it. Only after it has survived lots of other review does anyone, especially the author, begin to trust it for real use. But the most common outcome, when something new is designed, even by serious experts, is that it gets broken shortly after publication. It's quite common for new algorithms and constructions to be broken at the same conference they're initially presented.
I reiterate: No one who knows what they're doing creates new crypto for production work.
Moreover, people who know what they're doing even approach implementation of known and trusted algorithms with trepidation! There are so many very subtle things you can get wrong. Heh, just last week someone pointed out that my implementation of a constant-time memcmp had a subtle bug that caused it to be not quite constant-time on some architectures. Novices have no idea why it even matters in crypto that memcmp always run in the same amount of time for a given buffer size, irrespective of the contents of the buffers, and assume that the C library's memcmp is fine. More knowledgeable engineers know why it matters, but really deep expertise is required to get it right. That's just one tiny example. My primary mistake wasn't the bug in my implementation, it was trying to write memcmp at all. I should have found a well-vetted implementation and used that.
Doing your own crypto is nothing like doing your own science or doing your own music. The thing about security is that it's only as strong as the weakest link; the tiniest crevice can give the attacker a wedge to bust your system wide open. Other fields are forgiving of minor flaws, you can do useful and interesting work even if it has some defects. In security, and crypto is often at the heart of security solutions, one tiny mistake can render the totality of what you did not only useless, but actively dangerous to your users.
If writing a good secure memcmp is too hard for an engineer with 25 years' experience, including 20 years doing cryptographic security, what does that say about trying to write something that doesn't appear to be trivial? Crypto is hard. Really, really hard. The more you learn about it the harder it gets, because you understand more about what can go wrong.
I agree, although I have a more optimistic outlook. The difference is that in the US the people actually do have a say. Our problem of late has been that a majority, or at least a large minority, of people have favored the "tradeoff", because they didn't understand either the costs or the benefits, and many others simply haven't paid attention, leaving only a fairly small minority opposing it.
I think the pendulum has begun swinging back the other way, though. It has a long way to go, but Snowden's revelations helped a lot, and if we can get a few more I think the people will become more cautious about their rights. In addition, the terrorism scare is dying down. Of course, the risk there is that another large and successful attack will bring that back.
In addition, we do have a legal backstop in the constitution. It takes a long time for that to take effect, as first the people who want to challenge the unconstitutional laws have to figure out how to attack them in court, then the challenges have to wend their way through the court process, which is lengthy. But although the wheels grind slowly, they do eventually get there.
Finally, we have another last resort that the Chinese did not have. Americans are armed. The weapons we have are mostly for hunting and sporting purposes, not military arms, but the differences aren't really that large in practice, at least compared to military small arms. In addition, the states are armed with heavier firepower, up to and including fighter aircraft, attack helicopters, tanks, etc. On top of that, our actual military forces consist of people who have taken an oath to support and defend the constitution, not the government, and many of them take it quite seriously. All this adds up to a citizenry that is capable of beginning the resistance, has an opportunity to convince states to bring their firepower into it, and will attract the sympathy of many in the armed forces.
Of course, I really, really hope it doesn't come to that last option. And I don't think it will. Either enough of the people will come around and the system will be changed via political and legal processes, or we'll collectively decide that we really do want our government spying on us. Either way, I'm confident that the will of the people will win out. Mostly, anyway.
In order to trade off some of A to get some of B, you have to actually get some of B. In this case, we've given a lot of A and gotten nothing for it.
Actually, they recommend browsers disable those capabilities unless the user consents to enabling them.
Right, disable user-positive features by default unless the user reconfigures their browser to the contrary.
No, disable potentially user-positive and potentially user-harmful features unless the user approves on a dialog containing suitably-scary text so they will stop and think about whether or not they trust this extension that they're giving control over and visibility into all of their browsing.
It's fine to actively prompt; no need to require users to go searching for the setting. But it's important to give users the control, so drive-by extension installs (like Superfish) don't assume it for them.
So Google, an advertising company, recommends that browser developers disable the capabilities that ad blockers rely on?
Actually, they recommend browsers disable those capabilities unless the user consents to enabling them.
Google sez we must remove ad blocker functionality!
I smell an ulterior motive..
Reading comprehension fail. The summary says:
Amongst the report's recommendations to address the problem is the suggestion that browser makers "harden their environments against side-loading extensions or modifying the browser environment without user consent." Some of the most popular extensions for Chrome and Firefox, including ad-blockers, depend on this functionality.
I'd expect that most users who install ad blockers consent to having it modifying the browser environment.
You asked and I answered. FYI, I've been driving a Nissan LEAF for the last three years, in the mountains of Colorado and Utah. It gets cold here. Car works just fine.
An electric car? Do you want to charge it in the rain?
There's absolutely no problem with charging an electric car outdoors in the rain.
In a lot of the US and Canada we have these things called winters, and batteries and other components don't work so well when it's way below 0 C.
They work just fine. Range suffers a bit because you use part of your charge for heating, but everything functions fine.
What is an NEV? I get the EV part, but not the N.
(there used to be a company that wrote compilers for Windows shareware developers called Borland
I can see you're a young'un. Borland didn't make "compilers for Windows shareware developers", Borland was the dominant maker of high-quality Pascal, C, and C++ development tools for the PC platform for many years, for DOS and Windows, especially DOS. Borland's overlay manager was fantastic and made large DOS programs (programs that used more than 640K) very easy. Microsoft eventually managed to shove them out, but Borland was by far the better tool set for years, and was the choice of many professional development shops. It's what the software companies I worked for in the early to mid-90s used.
I think what eventually did them in was their decision to try to offer a better Windows API than that mess that was MFC. Borland's Object Windows Library was orders of magnitude cleaner and better thought-out than MFC, but because they had to wait for each release of Windows before they could update it, it lagged behind MFC in functionality. Probably also because the OWL team took the time to think about how to do things right, rather than just slapping a thin pseudo-OO wrapper on the Win32 API, which was MFC's approach. So Windows devs learned that if you wanted to stay current, you needed to use MFC, hence you may as well use Microsoft's whole tool set, since you had to buy it all (several hundred dollars).
These day's Duff's Device is almost always premature optimization, because compilers will unroll most loops for you. You should only use it after profiling to prove that the loop is a hot spot, and after benchmarking to prove that it actually improves performance. Even then you should include both implementations, selecting one by #ifdef, so you can periodically re-test, because eventually a new platform or new compiler or something will result in it no longer making a difference, or even being more expensive, if the compiler is cleverer than you.
They really ought to have done it the other way around: break by default, and use "continue" to fall through. It can indeed be useful sometimes, but in the vast majority of cases you want it to break, and forgetting that statement causes all sorts of trouble.
Depends on the sort of code. There is lots of code that needs to handle, for example, large classes of error codes, and for 50 codes there may be only two or three sensible ways to handle them. Being able to have big lists of cases that nearly all fall through is very nice then.
Your notion of "optimizing for the hardware" is something that isn't real. According to your theory, Linux also shouldn't perform well because it also is hardware-agnostic.
Why would it not perform well? Optimization would make it perform better it isn't a pre-requisite for it to perform adequately.
And of course we optimize for hardware, if you're doing GPU acceleration for example you're going to use the specific features of a particular GPU, if you want to utilize it effectively you don't just use the same path for all the different PowerVR GPUs, different Ardreno GPUs and different Mali GPUs. It would be quite a task to write an optimized implementation for all which is why iOS and Windows Phone do it only for a limited subset while Android goes for a generic approach. It's not going to exploit the hardware capabilities as well but it will work across more hardware.
The solution to that sort of optimization is to delegate it to drivers. And providing and debugging drivers is a big part of what OEMs have to do (actually, the drivers are provided by the hardware vendors -- OEMs don't make the bits and pieces, they put them together -- but generally as source code and OEMs tweak and fix).
Your notion of "optimizing for the hardware" is something that isn't real. According to your theory, Linux also shouldn't perform well because it also is hardware-agnostic.
As for what OEMs have to do, a modern mobile device is immensely complex, consisting of dozens of processors, many on the SoC (system on a chip) but many not. All of them have to be configured, which is a complex and tedious operation, and easy to get wrong -- and every custom board requires a custom configuration. In addition, there are drivers for all of the bits and pieces that have to be assembled and tested together. Plus there's also typically a complex, multi-stage boot process that has to be orchestrated to bring up all the bits and pieces of the hardware in the right way and in the right order. And other stuff that I don't know about because I'm not a hardware systems guy.
Some of the above doesn't depend on the OS, and can be done before it's available. But much of it does depend on OS requirements and has to wait.
And then if the OEM decides to customize Android they have to do that, with whatever skin, and default apps they want, plus whatever changes they need deep in the system to support the hardware and their changes to the software. Finally there is lots and lots of testing, because such complex, custom devices always expose new interactions between components that have to be debugged and fixed. Oh, and lots of hardware testing as well, including endless burn-in tests to validate that the stuff not only works but that some subtle design flaw doesn't stop it from working.
And I'm sure there's still more that I don't know about at that level as well.
Then they have to run Google's compliance tests, to find out what they've broken with all of their changes, or what they missed in configuring their device for proper support (actually, this is something they do throughout, not at the end), and then go back and fix what's broken until it passes... or else negotiate with Google for waivers on things they think should be okay.
Then comes carrier validation and testing, more rounds of fixes, etc.
Little or none of this has anything to do with "optimization". That's mostly the compiler's job, and it does that job well.
Per Wikipedia, Microsoft licensed System 7 from AT&T and called it Xenix because AT&T wasn't licensing the UNIX name.
And I didn't "come at you". I thought the second sentence of my second paragraph made that clear. Sorry if it wasn't.
Other than Xenix what do you mean by Microsoft
Er, nothing actually. TFA mentioned "Microsoft's take on Unix", which I took to mean NT's stab at POSIX support, or maybe something else equally ridiculous. Looking at the article again, it actually says "Xenix, Microsoft's take on Unix". Not being more than vaguely aware of Xenix, I didn't realize it was bought by Microsoft and I took that text as two separate items in the list (should have paid closer attention to commas vs semi-colons).
Also you forgot SCO if you are including commercial Unixes for 386
Indeed. There I claim selective memory, driven by the massive stain on the Unix world left by SCO's successor-in-ownership, The SCO Group.
Also one that gets forgotten about but was quite good in those early days was: Coherent
I heard good things about Coherent back in the day, but never touched it.
It was because Linux more or less worked, and people could use it and add to it because of the GPL. The competitors all had problems:
* Minix was cheap but not free, and couldn't be redistributed with modifications. People worked around that by maintaining patch sets, but that was even more painful then than it is now (we have better tools now).
* The BSDs were in a quagmire of legal uncertainty and competing claims. Nobody knew for sure if BSD was free or not, so everyone assumed it wasn't.
* Xenix: Not free.
* Microsoft: Are you kidding me?
* SYSV: Not free
* HURD: Didn't work, and had such an elegant architecture that it wasn't clear if it could ever work.
That was the space when Linus Torvalds started hacking around (except HURD didn't even exist yet). If he'd been able to hack on Minix, he would have. But the license prevented it, so he took the opportunity to start his own. Lots of other people saw exactly the same situation and joined him in hacking on something that (a) worked, more or less and (b) they could hack on.
It's not that Linux lucked out and the rest of the competition failed. There was no other competition that satisfied the requirements of being free and hackable. It was also important that Linus was an excellent Benevolent Dictator that gave people few reasons to fork. Actually, on that last point it's rather impressive that Linus is still in charge, even after it's become an incredibly valuable property, used and contributed to by lots of megacorps.
They can prioritize all they want, but no one wants to pay for the carrier certification of thee modified SDRs, particularly when using a T-Zone on a Snapdragon chip in order to run the baseband, and the FCC demands that the SDR be certified as a unit (software + hardware). That's a carrier certifiiation per carrier, per country, per device, per version update.
Heh. That isn't the problem. Unfortunately, I can't explain in more detail, because my conversations with carriers are confidential.
Also no carrier using a contract lock-in revenue model is going to provide an update that doesn't lock you into a new contract
Also not the problem, and I also can't explain. I'll just point out that the carriers have so successfully branded Android as their own that many consumers see the failure to upgrade as the carriers' fault. The carriers aren't blind to this, or what it costs them.
You should also be aware that the image that's shipped by the OEM is often not even buildable by Google engineers
Why yes, Terry, as a Google Android engineer I'm quite aware of this :-)
Except you should replace "often" with "never". Or at least "almost never". There may be some exceptions, though I've never heard of one.
apart from the fact that the devices used during development are generally signature neutered
A bigger issue is that the devices used during development are Nexus devices, not OEM devices. We never see those, either the hardware details, or the code that OEMs build after all of their customization.
Seriously, one of the smartest things that Apple did was keep the baseband processor separate from the application processor so that there was no telecom recertification required, unless they were explicitly hacking the baseband for some reason
That does make a lot of sense, but I don't think it's actually relevant to the problem. The carriers validate a lot more than just telecom functionality on devices that carry their brand.
So, is mute still under power button? Cause it does not make sense to move airplane mode from there and leave mute, data network mode and the rest there. Airplane mode is a mute for the radio.
No, mute was moved to the volume button, and enhanced to allow silencing of all but alarms (which is what mute always did) or complete silencing. I think it makes sense to put muting of audio on the audio volume control.
Didn't older Nexus devices get releases at the same time we heard about them? My Nexus 6 and 7 did not. That took months. Hell, I bought these two Nexus devices (in part) because I thought it'd mean I'd get stuff promptly. Though, for perspective, AT&T and Samsung took well OVER A YEAR to release Android updates... which compelled me to switch.
You apparently completely missed my point. No, Google doesn't arrange to upgrade the Nexus devices as soon as the new release is announced, for the reasons I explained.
The state of it all is really quite absurd, either way.
Actually, with all respect, it's your expectations that are absurd. I understand why you have them, because you see other single-player device lines (notably iOS devices) which can delay announcement until the release is ready for devices. But there's really no way that could work in the Android ecosystem. Engineering takes time, and there's quite a lot of work to be done after the core OS version is fully baked. Google can't use its position as the source of the core OS to give the Nexus line too much of a leg up because the Nexus devices compete with the partners' devices.
If Google is going to hold back on pushing updates to appease partners, why bother having them in the first place?
It's a balance. Google walks a fine line with its partners, and the Nexus line is a major potential conflict point, because Google is competing directly with its partners, and doesn't care about making a profit (though the OEM that actually manufactures the device wants one). So, yes, Nexus is Google's tool for pushing the OEMs around, but it has to be applied with sensitivity.
They are *great* tablets (except for the new 9" model which, IMHO, is a dog.)
I quite like my Nexus 9. Though maybe I'd like it less if I had to pay for it, not sure. I still have my N7 around, but I find I use it less and less.
But the development process is closed, the problem is that they develop the software and don't work with the hardware vendors, this is part of the problem.
We do work with the hardware vendors and they have access to the under-development code. But most of them (quite sensibly, IMO) hold off doing very much until quite late in the product cycle, because change is fast and furious and they don't want to spend a lot of time spinning doing work they just have to redo.
If the OEMs were involved in the development process you would have the hardware and software components better matched and optimized and also a more timely release.
I don't know what you mean by "better matched". Android is specifically not tailored to any particular hardware. That's by design, and it's a good thing. Certainly there are some down sides, but its what makes the vibrant, competitive ecosystem possible, and that ecosystem is why Android's market share is what it is, because it serves consumers.
As for a more "timely" release, I think we already do pretty much everything that can be done, short of inviting all of the vendors to participate in design process. Many of my colleagues previously worked for various hardware vendors before joining Google, and they assure me that would be an absolute disaster for various reasons, not least because design by committee doesn't work -- especially when many of the committee members are deeply self-interested and mutually antagonistic.
Google has a lot of smart people who fully understand these problems and want to find solutions. You can be pretty certain that all of the low-hanging fruit was picked long ago.