Slashdot Mirror


Nexus S Beats iPhone 4 In 'Real World' Web Browsing Tests

bongey writes "In a series of measured real-world web load tests, the Android-based Nexus S phone spanked the iPhone 4. The Android phone and iPhone 4 median load times were 2.144s and 3.254s respectively. The sample size was 45,000 page loads, across 1000 web sites. It also follows rumors that Apple is intentionally slowing down web apps to make their native apps more favorable."

38 of 260 comments (clear)

  1. Or... by The+Grim+Reefer2 · · Score: 4, Funny

    Maybe they weren't holding the iPhone correctly.

    1. Re:Or... by Dragonslicer · · Score: 2

      Maybe they weren't holding the iPhone correctly.

      The correct holding method is to be standing in queue at the Returns counter.

      I would have thought that the Apple-approved holding method is to be standing in the queue to purchase the next iteration.

  2. Bogus by Anonymous Coward · · Score: 5, Interesting

    They were using a custom app. Not the default browser. So what they are saying is that their app runs faster on the Nexus S. Not that the Nexus S is faster then the iPhone.

    1. Re:Bogus by DurendalMac · · Score: 2

      THIS. Bad test is bad. If you want a good measure of performance, use the native browser on each as that's what the vast majority of users are going to use.

    2. Re:Bogus by coldfarnorth · · Score: 4, Informative

      First, read the article written by the folks who did the test: http://www.blaze.io/uncategorized/mobile/iphone-vs-android-45000-tests-prove-whose-browser-is-faster/

      Here, they address this point. First, they compared their app's times with Safari's times, and found no noticeable difference. Second, they point out that javascript performance accounts for a small fraction of the load times (see large yellow box at the top of the page), and if Nitro was not in use, they estimate that using it would improve Safari's load times, but would not dramatically change the results.

      --
      Lets start refering to The War Against Terror by it's initials. . .
    3. Re:Bogus by molnarcs · · Score: 3, Interesting

      Actually, before buying my NEXUS ONE, I looked up quite a few comparison's on youtube. They were pretty much matched, but it some tests the Nexus was faster. In one particular test, by the time the iPhone4 loaded the homepage of the review sites, on the Nexus it was already loaded and a flash video playing. The difference still was just around 1 second, which is not the end of the world of course, but noticeable enough. I concluded that for web browsing, the Nexus is as good or slightly better as the iPhone. And remember, I'm talking about the Nexus One that came out 4 months before the iPhone4. So I do believe there might be something to this... and yeah, I've been a very happy Nexus owner since then. It's longevity is superb - still can't find anything that tops it. I mean yeah, there are better and faster phones out there right now, but I couldn't find a single compelling feature that would prompt me to buy a new phone for the foreseeable future.

    4. Re:Bogus by Anonymous Coward · · Score: 5, Informative

      First, read the article written by the folks who did the test: http://www.blaze.io/uncategorized/mobile/iphone-vs-android-45000-tests-prove-whose-browser-is-faster/

      Here, they address this point. First, they compared their app's times with Safari's times, and found no noticeable difference.

      Nothing in your link supports this. Their update (http://www.blaze.io/business/embeded-browser-vs-native-browser/) basically admits that they ran a flawed test, and blames Apple for optimizing its browser.

      Second, they point out that javascript performance accounts for a small fraction of the load times (see large yellow box at the top of the page), and if Nitro was not in use, they estimate that using it would improve Safari's load times, but would not dramatically change the results.

      JavaScript is not the only difference between safari and an embedded web renderer. Safari has different caching and multithreading as well.

    5. Re:Bogus by oh_my_080980980 · · Score: 2, Informative

      Except that was not the point of the comparison. The test was comparing web page load time.

      Epic fail criticism. Not too mention app developers would have access to the update engine. Good grief.

    6. Re:Bogus by jbezorg · · Score: 2

      They were using a custom app. Not the default browser. So what they are saying is that their app runs faster on the Nexus S. Not that the Nexus S is faster then the iPhone.

      That's a bold assumption AC. How do you know it didn't run slower on the android phones? Have you bench marked each application?

      Still, what do you expect them to do to get accurate results? Use the actual browsers and sit there with a stopwatch?

      How would you approach the problem of getting accurate times?

      Primary Source:
      http://www.blaze.io/uncategorized/mobile/iphone-vs-android-45000-tests-prove-whose-browser-is-faster/

      The measurement itself was done using the custom apps, which use the platform’s embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone. Manual verification showed that page load performance of the embedded browsers, when properly configured, is effectively identical to the stand-alone browsers. The load times are calculated using the “Document Complete” callback from the browser, which is a standard way of measuring a web page’s load time. As mentioned above, the agents are now a part of a free service available at http://blaze.io/mobile/, and we encourage you to try it out.

      Methodology
      http://www.blaze.io/mobile/methodology/

      --
      I've lost all my marbles except one & It's fun to test angular & centripetal acceleration in my skull
    7. Re:Bogus by iluvcapra · · Score: 3, Insightful

      Here, they address this point. First, they compared their app's times with Safari's times, and found no noticeable difference.

      To go to the trouble of testing the thing with their own app, then testing Safari, publishing the numbers for their own app and not publishing the benchmark for Safari seems obtuse in the extreme. Just tell us the numbers you got for the browser.

      Second, they point out that javascript performance accounts for a small fraction of the load times (see large yellow box at the top of the page), and if Nitro was not in use,

      A web browser renders content and loads it as well as executing stuff; javascript is only one part of the whole operation and only pertains to certain use cases.

      they estimate that using it would improve Safari's load times, but would not dramatically change the results.

      Why estimate when they can just run a benchmark on the actual browser, instead of handwaving?

      --
      Don't blame me, I voted for Baltar.
    8. Re:Bogus by iluvcapra · · Score: 4, Informative

      I think you don't understand the problem. The headline is "Android's browser is faster than iPhone's browser," but all they ever tested was:

      The measurement itself was done using the custom apps, which use the platform’s embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone.

      UIWebView is not Safari, and neither WebView nor UIWebView are "browsers."

      --
      Don't blame me, I voted for Baltar.
    9. Re:Bogus by bemymonkey · · Score: 2

      Rendering, OK, but panning content-heavy and complex pages still needs to be improved. Android stutters/gums up and iOS checkerboards like crazy...

      I'll take a full GB (or preferably 1.5 or even 2GB) of RAM for my next device please...

    10. Re:Bogus by node+3 · · Score: 2

      if Nitro was not in use, they estimate that using it would improve Safari's load times, but would not dramatically change the results.

      And that's the best way to run a test. You run a set of well-defined tests and make precise measurements, then you just "estimate" what the real results of a proper test would be...

      There's no way around the fact that this test is flawed. "Estimating" and guessing at the results of a proper test is nonsense. Any Slashdotter who has any respect for scientific methodology should be ashamed to be playing so loose here in order to make their favorite product look better than some other. There's a word for this that gets thrown at anyone who says nice things about Apple. You know the one.

    11. Re:Bogus by beelsebob · · Score: 2

      No big surprise really. The Nexus One's CPU is 25% faster than the iPhone4's (1GHz vs. 800MHz). The iPhone hardware has always been underpowered compared the Android.

      Not at all true –this is one of the first devices to have a more powerful CPU, and the iPhone's GPU is *only just* being beaten by the most recent releases now. Notably, apple is busy replacing that CPU and GPU combination with a CPU and GPU that benchmark about 4 times faster. The iPhone actually has generally had a significant hardware *advantage* compared to android –this is one of many reasons why my company doesn't consider developing games for android –you can't rely on fast hardware like you can on the iPhone.

    12. Re:Bogus by exomondo · · Score: 2

      The Nexus One's CPU is 25% faster than the iPhone4's (1GHz vs. 800MHz).

      Math fail.

      reading comprehension fail.

  3. Really? by Overzeetop · · Score: 2

    Page load speed, that's their metric? And 50% faster is spanked? We're talking about computers, not 100m dash times - I expect an order of magnitude difference. How is the actual browsing experience - how easy is it to read and navigate on a 4" device?

    I will go so far as to quote from TFA:

    "Users don’t always notice the speed gap because websites are sometimes tailored to mobile phones, Blaze said. The difference will become more obvious as users demand richer experiences and move to tablet computers with larger screens.

    So the metric their using to judge the devices isn't very noticeable, and probably won't matter on a device this size ever. Great. Guess if you have to break out a ruler to feel good about yourself...

    --
    Is it just my observation, or are there way too many stupid people in the world?
    1. Re:Really? by DurendalMac · · Score: 2

      Not to mention that they didn't even use the native browser on each platform, but a custom app, which makes this test even more irrelevant. If you want to measure browser performance, then use the bundled browser that the vast majority of users will be using.

    2. Re:Really? by Gadget_Guy · · Score: 2

      I agree. Unless you are going to run two phones side by side, people will not notice the difference.

      My bigger concern is that Safari on the iPhone makes for a poor user experience (at least compared to Opera on my old Nokia Communicator). Opera did some nice reflowing of HTML elements to fit web pages on a small screen. The iPhone makes the virtual screen size default to 920px across and relies on zooming in and out to be able to read things properly. It is particularly bad when reading text on a page that does not fix the screen size and just flows to the native page width. It is ridiculous that you should have to scroll horizontally to be able to read text that should just wrap to the screen width.

      And unlike Opera, there were no configuration settings to change the way it works.

      It gets worse when filling in forms (like I am doing now) because when the huge keyboard is on screen, the zoom level resets to a stupidly large size. This means that you cannot see full field as you type.

      As a web developer, I can change the way the zooming works, but this relies on changing the HTML just to suit one browser. I had hoped that we saw the end of that madness with the demise if IE6!

    3. Re:Really? by Gadget_Guy · · Score: 2

      You would have been using the crappy Opera Mini, which ran on Java. The one on my Nokia was Opera Mobile (I think that was the name) and it was 1000% better than the one you can download from Opera's site as it had a nice mix of a slight zoom and an intelligent reflow that could fit a site in using the same layout that you see on a PC. It helped that the Nokia was a clamshell design that had an internal screen that was 480px wide.

      There was no way of getting the better version except to have it preinstalled by the phone manufacturer. When I went to Opera's page for my model of phone, it only showed Mini to be available (which was bizarre to read on the phone via Opera Mobile).

      Finally, speaking of annoyances about Safari, my biggest one is that it can't hold many pages in memory at the same time. With my old phone, I could load up 10 Slashdot articles (showing all comments) and the go on a plane and read them all. With Safari, I can only load 7 pages, but it can really only hold 1 or 2 large pages without trying to reload them again as you switch between them. I have lost a number of Slashdot posts in the past when I quickly look up a site in another page only to have the posting form reload when I switch back, losing everything I have typed. Very frustrating!

  4. Re:Meh by wsanders · · Score: 2

    Oops meant 1110. That's more serious. Considering it takes me an average of 7865.349 msec to plug in my charger, still a fair trade..

    --
    Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"
  5. Apple/Oranges by beanball75 · · Score: 5, Interesting

    Someone pointed out already that the way they tested is with apps that use the browser engine available to apps. As the second link says in the main story (probably, I'm too lazy to RTFA, I read others already), the iOS browser engine doesn't use the Nitro javascript engine.

    I found one link that discusses it, but I'm sure there are better ones:

    http://www.informationweek.com/news/personal-tech/smart-phones/showArticle.jhtml?articleID=229301178

  6. That's nice. by DWMorse · · Score: 4, Insightful

    That's nice.

    Now, how quickly does it play Netflix movies? What's it's Hulu Plus app like, does it work nicely?

    You don't say.

    Seriously, for shame. I really do want an Android phone. It just isn't as functional yet. Another year or two of maturity and I think I'll finally get to switch.

    --
    There's a spot in User Info for World of Warcraft account names? Really?
    1. Re:That's nice. by MobileTatsu-NJG · · Score: 2

      That's nice.

      Now, how quickly does it play Netflix movies? What's it's Hulu Plus app like, does it work nicely?

      You don't say.

      Seriously, for shame. I really do want an Android phone. It just isn't as functional yet. Another year or two of maturity and I think I'll finally get to switch.

      Software matters. We keep learning this lesson with video game consoles. For some reason, though, people keep going spec happy with tablets and phones.

      --

      "I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)

  7. How about CSS support? by Mad+Merlin · · Score: 2

    Since the beginning, the iPhone has had busted CSS support for position: fixed; elements, which is terribly unfortunate as it makes Game! difficult to play. How does the Nexus S fare?

  8. Re:Meh by SwabTheDeck · · Score: 4, Insightful

    Isn't the iPhone's A4 CPU supposedly some hundred MHz slower than the the one in the Nexus S, giving it better battery life? I don't think this has anything to do with strangling web apps, just different design goals.

    The iPhone 4 is 777 MHz while the Nexus S is 1 GHz. Both are based on the ARM Corext-A8 and both have 512 MB of RAM. Given the difference in CPU speed, the results of the page load tests don't seem far departed from what would be expected. While the Nexus S is still proportionally a little faster, it isn't so wildly so that it can't be attributed to some minor tweaks in the OS or browser software. Using the term "spanked" seems a bit sensationalist in this instance.

  9. There are more than two operating systems. by 91degrees · · Score: 2

    Why not also include WP7? Has it been written off before people even try it?

  10. Re:Android/iPhone UI performance by Digicaf · · Score: 3, Informative

    This has a lot to do with hardware acceleration in the GUI, which for the most part isn't there in any Android below 2.3. I bought my Droid 2 last september and noticed exactly what you mention. In 2.3, that's no longer true. It feels MUCH smoother. In fact, my wife went out a month ago and picked up a low end device (with 2.3) that has a much better response rate and feel despite having a processor only half as fast.

  11. How Do I Moderate an Entire Article as Flamebait? by macs4all · · Score: 4, Insightful

    Really, this is pretty much a new low in comment-baiting for Slashdot.

    This so-called "test" is so utterly and completely unscientific as to be not worth the service space it is stored on.

    Period.

    It's supposed to be NEWS for Nerds, and this hardly qualifies. And, not content to troll on its own, the summary has to link to ANOTHER Flamebait summary to "support" its "point".

    Note to Slashdot: You can do better than this; so DO it already!

  12. Re:Meh by just_another_sean · · Score: 2

    Touch choice

    Pun intended? :-)

    --
    Creationist Textbook Stickers Declared Unconstitutional by CowboyNeal
  13. Why is the reply always "no one cares about" by bongey · · Score: 2
    Why is that every time you show some Android product has better feature or performance, call it X, than an competing Apple product the reply from Apple fans follows this logic..

    People don't care about X

    Eventually Apples popularity will start to fade and people WILL care.
    1 second difference can add up to a lot of time if you read many web pages, or you are searching for something. Just do the math. Say 100 modest amount web pages a day , 365 days a year. So you have (100*365)/3600 = 10.13 extra hours spent a year staring at screen that is doing nothing. In both tests they used the embedded browser for both handsets respectively. From their testing suite I don't see how they could throw off the benchmark that much, 45,000 samples is a pretty significant sample size.
    More on there testing methodology is here http://www.blaze.io/mobile/methodology/ .
    Finally the second link is complaints from Apple iOS developers. iOS 4.3 browser cannot use the new Nitro javascript engine in full screen mode, html 5 caching is missing, and mode in which the page is drawn on the screen has changed such that it is slower than native apps. Bug or not, it currently slower and no one knows why except Apple.

    1. Re:Why is the reply always "no one cares about" by macs4all · · Score: 2

      Why is that every time you show some Android product has better feature or performance, call it X, than an competing Apple product the reply from Apple fans follows this logic..

      People don't care about X

      Eventually Apples popularity will start to fade and people WILL care.

      You are assuming that Apple will not continue to define and re-define whole markets and classes of devices, as it has done repeatedly in its history. Honestly, to take but one example, do you really think we would even HAVE Honeycomb to compare iOS to if Apple hadn't released the iPad?

      1 second difference can add up to a lot of time if you read many web pages, or you are searching for something. Just do the math. Say 100 modest amount web pages a day , 365 days a year. So you have (100*365)/3600 = 10.13 extra hours spent a year staring at screen that is doing nothing.

      The flaw in your logic is that you spend zero time after the page loads doing things like figuring out what part of the page is relevant, scrolling to read desired content, picking your nose, etc. Ya know, the things that humans do, that an automated test does not. Now, if the difference was like 200 or 300%, then I would agree that it MIGHT matter to a HUMAN.

      In both tests they used the embedded browser for both handsets respectively. From their testing suite I don't see how they could throw off the benchmark that much, 45,000 samples is a pretty significant sample size.

      And the differences are not spectacular, and in the next couple of months, the iPhone 5 will whip on the Nexus. So what? Until Apple and Google decide to synchronize their hardware and software releases (not bloody likely), there will continue to be a back-and-forth "winner", as each respective platform updates themselves asynchronously with the other.

      More on there testing methodology is here http://www.blaze.io/mobile/methodology/ . Finally the second link is complaints from Apple iOS developers. iOS 4.3 browser cannot use the new Nitro javascript engine in full screen mode, html 5 caching is missing, and mode in which the page is drawn on the screen has changed such that it is slower than native apps. Bug or not, it currently slower and no one knows why except Apple.

      And no one knows why Android's scrolling/swiping is jumpy except Google, apparently; because I sure haven't seen anyone posting links to FIXES for that issue, and many, many Android owners complain about just that on Slashdot every single day (some even in this thread).

      Could be that a piece or twelve of iOS javascript engine-code could use a little tightening. Cool thing is, when Apple does that, EVERY iOS device that can run the updated code will be updated to do so. How long do you think such a change would take to propagate through the Android multiverse?

  14. Re:I'm probably going to get flamed for this by Locke2005 · · Score: 2

    Amen brother! What good is a smart phone if I can't use it to view porn videos while driving???

    --
    I've abandoned my search for truth; now I'm just looking for some useful delusions.
  15. Well... no. by joh · · Score: 5, Informative

    First, Apple isn't "intentionally slowing down web apps to make their native apps more favorable." They have added a new JS interpreter (actually a just-in-time JS compiler) to Safari, but not to the "normal" web views that other apps can embed. This means only Safari is faster now, others are as fast as before.

    Second, this test is flawed since it does not use Safari. It uses a custom app which uses neither the new JS engine nor the better caching of Safari or asynchronous multithreading.

    1. Re:Well... no. by Anonymous Coward · · Score: 2, Interesting

      And more to the point, there is actually a technical reason not to have Nitro in apps.
      Nitro is a JIT compiler. To use it means that the App needs to be basically allowed to have self modifying code (the power to say "this data memory is now executable"). This capability is an application-wide setting, that is reserved to Apple-developped apps for now, for good reasons.

      This has never been allowed in apps (and is the technical basis to disallow any frameworks like mono), because it exposes the device to a security threat that is not easily detected by Apple's testing (say an app that will, at some future date, download data from a remote server, turn it into code, and execute it. Voila, instant virus).

      This is not something Apple is prepared to simply allow for the sake of performance. This decision does not degrade web apps performance (as you noted, they still perform as good as they used to).

      People on slashdot usually yell when companies do not take security seriously. Well, this is a decision that is firmly rooted in the security camp.
      There might be technical solutions allowing nitro in apps and not compromising the device's security in the future. But anything that is released is a compatibility weight in the future, so I don't blame Apple for not releasing something half baked that they would have to take away later (or leave a gaping security hole).

  16. Re:Android/iPhone UI performance by Timmmm · · Score: 2

    I'm not convinced by this. The gallery app and the app drawer are both hard-ware accelerated, and neither is as smooth as the iphone. I think it is more to do with the slowness of java/dalvik, and the garbage collector, which only recently became concurrent.

  17. Curious by alvinrod · · Score: 2

    Although nowhere near 45,000 tests, Anandtech recently ran a preview of the iPad 2 and did some browsing benchmarks to test the CPU where they loaded the pages for the iPad 2, Xoom, and the original iPad. Obviously the two tablets are different animals than the two phones, but given they run essentially the same OS and have beefier CPUs, we should expect similar results.

    However, the iPad 2 is clearly faster in 7 of the 8 tests and the same speed as the Xoom in the remaining 1. It's possible that the websites used aren't good predictors for general load time though. Given that the two both have dual-core ARM chips running at similar clock rates, we shouldn't be seeing those results, especially if the ones from this study are a valid indicator of performance. The only other conclusions that can be drawn are that performance regressed in Android 3.0 (Or at least Motorola's implementation of it.), the Tegra 2 is a pile of crap, or that Apple is now somehow capable of making a significantly better SoC than many established players.

    I can't speculate regarding the first, but given that the Xoom has a similar SunSpider result and a better BrowserMark result than the iPad 2, it's unlikely that either of the other two conclusions are correct. Would like to have additional data before concluding one way or the other, but it does appear as though some things are not adding up.

  18. Re:Android/iPhone UI performance by Drakino · · Score: 2

    From experience, iOS will skip frames or otherwise cut the eye candy animations if it needs to. They have never slowed down the system, as they only use otherwise idle power. It's much harder to see this happen on the newer phones, but I saw it plenty of times back on the iPhone 3G as the OS became more complex. Apple does put a lot of effort into this, complete with dedicated system frameworks to drive the eye candy. Apple's view on the eye candy is that it is there to bring attention to what is occurring when transitioning states, and helps provide that "smoother" feeling when using their devices. This applies to both the OS X and iOS sides.

    It's also why input latency and audio latency is something Apple pays a lot of attention to, more so then most of the competition. When you touch an iOS device to scroll, it feels like your moving the content on the screen with your finger. Other systems tend to have a larger time difference between touch input and display action, making the experience feel a bit disconnected. While highly technical people might not mind the lack of smoothness, the general populous is going to go with what feels better. It's not a separation between "Apple Users" and the rest of us, it's a separation between those who know and care about the little minute details behind the scenes and those who don't.

    Windows Phone 7 on the other hand did seem to put a priority on the animations, to the point they would slow down and frustrate me when trying to do basic things. IE took over two seconds to recover from an orientation switch, and every time it would always play the animation. Orientation corrections in other areas were much faster.

  19. updated by milkmage · · Score: 2

    Blaze backed away from its conclusion in light of the new data. Chief Technology Officer Guy Podjarny told CNET in a statement:
    This test leveraged the embedded browser which is the only available option for iPhone applications. Blaze was under the assumption that Apple would apply the same updates to their embedded browser as they would their regular browser. If this is not the case and according to Apple's response, it's certainly possible the embedded browser might produce different results. If Apple decides to apply their optimizations across their embedded browser as well, then we would be more than willing to create a new report with the new performance results.

    Read more: http://news.cnet.com/8301-30685_3-20044325-264.html#ixzz1GtaYoees