Slashdot Mirror
iOS Vs. Android: Which Has the Crashiest Apps?
First time accepted submitter creativeHavoc writes "Forbes author Tomio Geron takes a look at data accrued by mobile app monitoring startup Crittercism. After looking at normalized data of crashes over the various mobile operating system versions he compares crash rates of apps on the two platforms. He also breaks it down further to look how the top apps compare across the competing mobile operating systems. The results may not be what you expect."
I've decided to opt for a Samsung Galaxy Nexus Android 4.0 device from SaskTel when I eventually get a smart phone (subject to new models coming out from Samsung or HTC and sold by SaskTel), but that's because it's a Java-based system I already have the tools to program, not because I'm concerned about app stability overall.
In fact, the odds are I won't use the thing to run too many apps if what I need is already included: email/web, GPS mapping and routing, and Java applications (including one I'll be working on myself some time in the next year or so.)
In the short term, I'll probably opt for a BASIC voice-and-text flip-phone of some kind, because I can't afford (nor stomach!) spending $600 on a PHONE whose MAIN purpose is to MAKE CALLS when I can get a $70 model that will take care of that primary function just fine for now.
I do not fail; I succeed at finding out what does not work.
iOS crashes more than Android (for those who don't feel like trawling through the (not brilliantly formatted) article.
You can learn a lot about a person if you just take the time to inject them with sodium pentathol
I was expecting android to outdo iOS in the crash department due to all the variables in the android world hat iOS just doesnt suffer from. Namely, android has a wider range of handset support.
Since iOS 5.0.1 I cannot browse (Safari) for more than a few minutes on my iPad before it crashes. I tried other browser Apps: they also crash 5 - 10 times a day. I don't need iPad 3. I need Apple to fix their horrible firmware. Apple is clearly a hardware company. Software is Apple's Achilles Heel. Google is way better at software.
The $600 device's main purpose is NOT to make calls. It's an internet communications device that just happens to make phone calls. The people who insist that basic phones are just fine need to figure out this slight, but important, distinction. Buy an internet device if you want internet, but don't compare it to a phone.
I would be very interest to see the impact of jailbreaking in this analysis. Do apps crash more often on jailbroken devices? How does it compare between Android and iOS?
One of the main argument for closed down system, putting aside the money factor which no one gives as an official reason, is stability. I do not believe stability is considerably affected by jailbreaking or by the subsequent modification one could do to the OS, but it would be nice to have statistics and some analysis on this.
So IOS 4 used cooperative multitasking (http://forums.pcworld.com/index.php?/topic/89287-apple-ios-4-vs-android-multitasking-which-approach-is-better-for-users/page__st__160) while I believe android uses a modern pre-emptive multitasking approach. I know IOS 5 has updated multtasking but it is unclear to me if they have gone to a full pre-emptive multitasking scheme.
It doesn't surprise me that Android is better in this area. To my way of thinking it doesn't make much sense not to have full pre-emptive multitasking and I have an iPhone. It wouldn't significantly affect battery life. The fact that a ton of processes can be running in the background on android is a significant contributor to the android battery woes. While that wouldn't be possible without full pre-emptive multitasking, I think there are better ways to manage that than simply relying on third party apps to manage background activities.
One of the reasons for app crashes is the proliferation of mobile operating systems on iOS and Android. As Apple and Google have released more new operating systems...
Apple has multiple mobile operating systems? Here i thought they just had iOS? Oh wait. According to this "professional" Google has multple mobile operating systems too! I must really be out of the loop. Maybe they should stick with just one OS and update it when they find bugs. Oh that would be cool! They could like call it the same thing but increment a number next to its name to show which updates have been applied.
Wait! What? Why are you marking this as flaimbait? The article was flaimbait based on a bunch of hair splitting conjecture and the assumption that each update of the OS was some kind of new OS. Why shouldn't I get to join the fun?
Having to work for a living is the root of all evil.
When I owned a Motorola Droid, there were several times when I had to remove the battery to recover from the phone completely locking up.
I don't have that option with my iPhone, but fortunately I haven't had the phone crash.
Going to spend $600 just to be cool? Ok. But lets be honest here, the iPhone makes phone calls just fine. Using the default Apps is also just fine. A beautiful user experience. It's only when people go totally gaga with the apps downloading every spammy, game demo, half-assed implemented app, by the dozens (or even hundreds)....then its not wonder things start crashing. As for QUALITY apps, there are arguably more of them available for iphone than android...then again, most major apps have versions for each. In other words, pick your poison and have fun. If you want to just make reliable phone calls, buy a $30 Nokia choco bar phone and be done with it.
Honestly, Sarfari for iPad is the best mobile experience out there, what is causing it to crash?
iPad 1 "crashes" a lot compared to iPad 2.
All testing by Apple is now done on iPad 2, which has more memory. So some of the "crashing" is iOS telling the app to free up memory, and shutting it down too quickly.
This has made the iPad 1 experience much poorer than it used to be.
As many know, it is possible but tought to make C-based apps that have perfect memory management and no chance of crashing. My own applications, and hundred professional ones written in C/C++ do crash. Sometimes. And it is often hard to figure out where they crash. It could be array out of bounds, some NULL pointer de-referencing, some dependent library doing the same, etc.
While Java-based ones never do. They do throw exception, at which point you know what/where to fix.
Three Dead Trolls in a Baggie said it best. Every OS Sucks http://www.youtube.com/watch?v=CPRvc2UMeMI
vi +
I was bored this morning, so for those interested, since the article makes it hard to extract this information:
All iOS versions total 84.36% of crashes; all Android versions total 15.49% of crashes. The worst offenders for iOS are version 5.0.1 at 28.64% and 4.2.10 at 12.64% (with seven other version listed at above 1% of crashes). The worst offenders for Android are versions 2.3.3 at 3.86% and 2.3.4 at 3.65%, with 4 other versions listed at above 1%.
Loose things are easy to lose. You're getting your hair cut. They're going there to see their aunt.
... Or you don't, at which point exception flies up through exception handlers stack all the way to the top level EH and the app gets killed with "Application was forced to close" message (on Android) or something like that.
Java compiler just buggers you more about uncaught and not rethrown exceptions, as this is declared and checked statically.
I would wager 5 quatloos that there are more "coding newbs" on IOS than Android therefore more poorly written apps on IOS.
"If any question why we died, Tell them because our fathers lied."
I notice the complete lack of a "versions in common use" chart, for those looking to know what we'll actually see in the real world instead of a controlled testing environment.
Anyone care to try to eke that out of the pies?
I disagree
You really don't get much more protection with Java then you do with native code on iOS. Exceptions only help you when you know you need to catch them. Null pointer dereferencing is pretty much a non-issue in ObjC/Cocoa because of the way messages are passed(at least no more than java.lang.NullPointerException). When you are multithreaded in Java it's trivial to make the simplest code throw a ArrayIndexOutOfBoundsException or NullPointerException when you hava your logic wrong.
The data we are looking at is from the small subset of apps which use Crittercism rather than TestFlight (why is anyone not using TestFlight?). It's not meant to be representational of the real-world. It would be interesting to know what iOS-based and what Android-based apps use Crittercism.
facebook.com and www.esprit.nl crash daily. I have an iPad 1 with iOS 5.0.1.
Just browse for a while on www.esprit.nl (click on a few links) and Safari (or Atomic browser) will crash.
iOS 4 crashed a lot less.
So is a BB 9810 or 9900 not a smartphone? I went down this route because many of our customers are of an age that they still use voice when I wouldn't, and the telephone functions on BB are still excellent. I have quite bad hearing, but my 9810 works perfectly for me.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
I don't think this has as much to do with Android and iOS as it does with the state of software quality in general. The current state of software quality is abysmal, since the shift to scripting languages and web apps as the primary platform about ten years, the science and art of writing robust and reliable software for OO, event driven, asynchronous platforms like iOS or Android has become an almost lost skill. Unfortunately failure modes for these platforms are more dramatic than for web apps, in that you'll likely get a crash rather than 'error on page' message. The situation has been further exacerbated by management's insistence an always hiring the lowest quality developers they can find, outsourcing, H1 B's etc. If you use low quality and inexperienced devs, you'll likely get an unstable and and unreliable application on these types of platforms. This should be a wake up call to the industry in general in that we need to focus and engineering, quality and reliability, and not just minimizing cost.
Wait, this study shows 3.66% crash on launch rate on iOS? I realize maybe my personal experience will not align with the data and all, but I have had iOS crash on me once and six times I've seen apps crash or screw up to the point I had to relaunch them. That is over a period of several years. I'm guessing there is something weird with the methodology here, perhaps not a representative sample? Am I truly that much of an outlier?
I saw no evidence that crittercism was normalizing their data. What they called "normalizing" was just breaking crashes down by version.
I felt that I knew less about the problem after I read the article than before, and not in a good way.
What i wonder is, how is this data supposed to be "normalized" as the slashdot summary suggests?
It appears to me the number of crashes would be proportional to the number of OSes out there of that version?
Hey don't blame me, IANAB
"Crittercism, which is backed by Google Ventures...." so why should I believe this data is accurate?
For starters, it's not the OS that crashes...it's the apps. Let's look at how many app crashes take out the OS and make it crash...not studied.
As for the apps - Android is Java based - As such, it is afforded the memory protection that garbage collection entails - at the expense of speed. iOS apps tend to be natively compiled code. Memory management is based on reference counter - in the past, that was a manual process. Now, it's a little easier with the new XCode 4, but still and iffy proposition.
iOS throws an exception when a memory allocation fails. Too many developers fail to properly handle that exception and clean up memory where possible. The default behavior is that the app will just terminate.
Bottom line is that is easier to write code that won't crash on these platforms using Java than a native code compiler. What would be more interesting and of value is to examine:
1. Why, exactly, do these apps crash?
2. What tools were used to write them?
3. Did the crash take out the OS also? While an app that just closes pisses me off - one that crashes the OS REALLY pisses me off - like my Blackberry Storm that would just reboot in the middle of a call.
A summary shouldn't be a cliffhanger IMHO. Why do I have to read TFA to get the result of the study? Could the result be included in TFS?
This chart has already been torn apart on Junk charts. Basically their statistics and reporting are so vague as to make it worthless. But yes, you may be surprised ... lies, damn lies and statistics.
Cwm, fjord-bank glyphs vext quiz
For those that don't want to read the article and disclamers...
iOS has a lot of attention and probably has more first-time and low-quality coders than Android. If Android was more popular, iOS apps would be less crashy instead.
It's common sense, really. And says nothing of the platform, only the dev crowds drawn to them.
The article did not clarify if they removed the "Low Memory" and "Active Assertions Beyond Permitted Time" entries from the crash log.
When iOS has memory demands it will kill suspended background processes and this shows up in the crash logs with a low memory reason. When a background process is running (not suspended) to complete some task (like downloading/uploading data, etc) and it exceeds the allowed execution time, iOS will kill it with an assertions beyond permitted time reason.
Neither of these are actual "crashes" as you might think of them and in fact users are often completely unaware the app was killed because when you switch back to the app it just reloads its state where it left off (and well-written apps actually restore your position in the UI).
If these two items weren't excluded then the results for iOS are worthless.
The article also pointed out that iOS 5 is new and there are likely to be crashes generated due to apps not being updated yet and that Android is likely to have a similar problem as ICS actually starts rolling out (or people buy new devices when they are stuck with a non-upgradable device).
Natural != (nontoxic || beneficial)
really? 99% of apps are cheap garbage whipped out as fast as they can to do only 1 thing, make a buck for the author. who gives a fuck if your dancing cat app or hand warmer app crashes or not
My plan's period is 2 years and the monthly fee is about $65. Of course I am grandfathered with respect to unlimited data, I'm not sure what that would add if I were a new customer. At about 1.5 years they will waive the remaining 0.5 if I am upgrading to a new phone and starting a new 2 year contract.
... I can get the same device for under $100 ...
An iPhone 3GS is free. We are past the point where one has to pay for smart phones.
... most people (at least in the US) buy their phone subsidized with a contract renewal, so the price for even a top-tier phone is $200-$300 ...
We are past that point in history where one has to pay for a smartphone. One can get an iPhone 3GS for free with the sort of plan you describe. Its not the latest model but it is still supported by the latest version of iOS and is highly capable. Not top-tier but second-tier is pretty good these days.
Just a few months ago I attended a training by an Apple rep. He insisted the use of iTunes (he called it the walled garden) and Apple's "rigorous" testing meant that all their apps would work on all their devices. No crashing. No incompatibilities. From experience, husband's iPad 2 crashes more than my Toshiba Thrive.
My phone (Motorola, running Android 2.1 as they've not bothered upgrading it despite hints that they would around the time that I bought the thing, luckily I got it dirt cheap so I don't feel too cheated) reboots itself a couple of times per day. It has never happened while I've been actively using it for anything (calls, messaging, ...) but I know when it does it if it is close to me and not hidden in a pocket that muffles the sound, as there is a characteristic beep when it reboots. I can tell when it has done it if I wasn't in earshot as the recently run apps screen is clear next time I look at it.
I don't run anything much significant app wise (Opera Mobile and Mini are on there, as are a game or two to wend away the time when the weather puts me on a bus to work instead of cycling, and a remote access thingy that only runs once in a blue moon when I need it), and I've tried completely removing all of them which makes no difference.
This isn't app instability, it is either OS or hardware instability. Even if an app were to be the root cause then it is still an OS problem as one app should not be able to have that effect. I've seen many people complain of the same thing on this and other phones. There are those that suggest "well, it is your fault for running such an old version of the OS" but as I have absolutely no choice in the matter I don't accept that - if the phone manufacturers using Android want to keep forcing the appliance model on us then they can take the blame when the appliance they have supplied fails in some way.
At this point my next phone may still be an Android device, though it'll be at least six months before I switch so we'll see what happens over that time. What Google do with the bit of Motorola they've bought is something that could be significant. I refuse to join the Apple lock-in club (if I'm paying that much for a general purpose computing device I want control over what I can run on it thanks, if I just wanted a phone I'd buy something far cheaper) but a modern Windows phone might be something I'd consider depending on what I find when I research the matter closer to change time.
...but the crashiest apps on my iPhone have always been the apps included with the OS. The AppStore crashes on me the most. The Mail app is second. It's very rare that any app that I've downloaded actually crashes on me. Maybe I'm just lucky.
Not a trend, just a data point.
- Spryguy
There are three kinds of people in this world: those that can count and those that can't
The thing is, it really should be Android. I mean, under Android you're free to develop and distribute an app designed just for crashing the phone. I'm sure such a thing would never get past the Apple censors.
I'm sorta new to this.
I got my first smartphone in mid-December. It's a T-mobile Galaxy S2. I was not aware that crashing was a problem with smartphones. Mine hasn't ever crashed or locked up. I do have one app that crashes sometimes, Waze. It's a little annoying but they fixed the main problem in the recent update. It's not like I'm not willing to try apps out. I think I have about 52 installed right now.
So I hate to seem smug, but the big news of this article to me was that apparently many smartphones have instability issues. I'm terribly sorry to hear that.
-- QED
It's a shame you didn't trawl through the article either, because if you had, you would have seen that it was about the applications crashing, not iOS or Android. In this particular article, I'm not seeing any evidence of OS unreliability for these platforms.
What's shocking is the trend over time, where lately (the figures are broken up by calendar quarters) applications are crashing about 3% of the time?!? That's hard to believe. If I had any desktop app that crashed that often, it wouldn't be installed for very long.
This suggests that either the top two mobile platforms are experiencing a crisis in unusually shitty software having recently and suddenly appeared, or there's something funny within that data (e.g. are a small handful of recently-popular apps crashing 100% of the time, messing up the numbers for the whole? Is there some very popular beta test going on out there?).
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
washingtonpost.com crashes Safari on my iPad frequently but not always. It's worse in articles with a comment section. It's usable, but annoying.
This apparently difference between reality and expectation can be resolved by realizing that the expectation was itself unreasonable.
If you spend a lot of time maintaining software, there's a pattern you sometimes see. There will be a few different cases that are similar, but not identical, to one another, and they start accumulating quirks and don't share a lot of code. Then one day someone wants to add another case and you, as the programmer, decide "that's the last straw," and unify all the cases into a general solution in order to save time (because saving time, not maximizing the quality or reliability of the software, is what you actual goal always is). Once you do that, as a bonus, you happen to get better reliability, single-spots where you can maybe spend a little effort on optimization, etc.
Generalism pays off big, when it's possible to do it. Back to the topic at hand, a solution for n*10 things probably really does crash less than a solution for n things.
As Windows app development moves from, say, x86-only to x86-and-ARM, its programmers are going to be thinking in terms of "there's two ways" rather than "there's many ways." I don't blame them for mistakenly thinking like that, but it's going to cost them. They're going to grow up a little, but they'll still have a long way to go.
No, informative.
There is no "I disagree" mod for a reason. Flamebait, Troll, and Overrated are not substitutes.
So the Walled Garden App Store doesn't prevent frequently crashing apps from entry, but does make pushing updates harder?
Go figure I guess all the Hype about the Walled Garden App Store Protecting you, don't protect your user experience just Apple's Profits.
iOS is very aggressive in killing misbehaving apps, while Android seems content to let them carry on until it simply can't continue. I wonder how much of the difference can be attributed to Android giving apps more leeway rather than a difference in app quality?
On a side note, that was a horribly written article. It sounded like somebody who wasn't familiar with technology was repeating something they didn't understand. GPS, cameras, and language support can crash apps? What?
Bogtha Bogtha Bogtha
Android is the best. Here u cand find a lot of very interesting andoid apps.
Using both android and iphone apps, I'm not sure I could say one is more stable over the other, but what about recovery. If an android app crashes you can go into applications and force quit, restart and it is fixed! What is the procedure for recovering a crashed app on an iphone? Opera and pandora have crashed on my iphone, I attempted recovery by powering off the iphone and uninstalling and reinstalling the apps only to arrive in the sane non-functional state. After a day or so of ignoring the issue they seem to be "restarted" and work correctly again.
I would've liked to see Windows Phones included in this article. I'm really curious how they measure up. I have yet to have anything crash since I got the phone a month ago.
I don't respond to AC's.
Normally its really highlighted if it were an other sponsored MS investigation....Seems google can get away with it as being one of the major investors in Crittercism.
instead of another piss poor study.
Of course this is the same thing that always happens when people compare MS it OS X.
One has a very specific hard ware upgrade path, as well as a tiny pathway tom get apps to the device, the other is a more dicerese platform, and ahd a very waide path to get apps. Some yeah, it crashes and has more problems...but it's the wrong comparison.
A good comparison doesn't Compare Apples to window, it compares a specific Apple PC with a specific Windows PC.
Same thing with smart device. You don't lump the iPhone against every other device running Android. You compare it against specific devices running android... preferably a similar device.
The Kruger Dunning explains most post on
"You really don't get much more protection with Java then you do with native code on iOS
Up to iOS 5, that statement is blatantly false.
I have looked at 5s new ARC, so I'll leave thatout.
The Kruger Dunning explains most post on
I have just now reached my limit. This is it, I am sick of not speaking up.
Android and iOS are operating systems. Comparing the applications available for each is like comparing which nails hardware stores have to sell. The hardware store is the OS, with its varied offerings. The nails are the applications, the things you buy and then use. All the hardware stores sell nails, the nails come from different manufacturers, all the nails are suitable for a different purpose... but do you blame the hardware store if a nail breaks, or comes with no point on it? Do you blame the hardware store if you buy a box of nails that turns out to be screws? No, you blame... ... wait for it ... ...the people that made the nails!
In the case where the hardware store is the manufacturer of the nails, then you are correct. So bundled applications can be correctly ranted about in this manner. However, I believe that we are talking about applications available for the OS and not just the ones that come on the phone.
Why do people blame the operating system of smart phones when the applications don't work? "This phone is cr*p, > keeps breaking". Every time I hear this, I die a little inside. The only time this is actually the fault of the OS is if you are using an application designed for an older version, and even this is the fault of the user for not checking which versions the application works on.
Moreover, why do people tout the operating system as superior because someone wrote an application for it? /rant.
Its like looking for the highest car crash rates by seatbelt color. Does the color even have a relevant impact?
We're struggling to find relevant information from the statistics provided in this article. I would bet thats because there are other factors with a MUCH bigger influence on crashing than the OS version. Those unknown factors could be determined by statistics which included more details than just OS version. It could be all 2.3.4 OS devices had a particular characteristic which is yet unidentified.
dumb article is dumb.
I don't run a lot of applications on my smartphone; the important features are email, texting (teenage daughter...) and phone, with a small amount of browsing. The BB was perfect for me as it was email first (my most used function) phone second (as good as any dedicated phone I've owned) and texting third (the best keyboard in the business). When BES started to become unreliable, I switched to Android, and it's been just slightly behind the blackberry for usability. I get one or two "forced close" on apps per week, mostly novelty stuff I don't really need.
Compared to Windows Mobile, the BB and Android are both huge wins, in that neither have system resource crashes. (The Windows audio driver would die several times a day, and the screen driver two or three times a week.) I think this is an important distinction -- an app may crash and life goes on, but if parts of the OS crash on a regular basis, you have a problem.
My company issues iphones on request, and I was tempted at first, but on seeing the problems others were having with phone calls, I declined. I'm sure it's a great internet platform, but calls are important to me. I like to think that's the primary reason, but I have to admit that another reason I don't own an iphone is that I didn't want to be associated with the Apple fanatics in my workgroup. I have never camped on the sidewalk to get a new electronic device... or anything, actually, and I didn't want to be associated with those who did this regularly.
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.