Slashdot Mirror


Chrome Users Are Best With Numbers, IE Users Worst

New submitter dr_blurb writes "After reading about last year's hoax report 'Intelligence Quotient (IQ) and Browser Usage' I realized I was in fact already running a real live experiment measuring number skills: a site were you can solve Calcudoku number puzzles. I analyzed two years' worth of data, consisting of over 1 million solved puzzles. This included puzzles solved 'against the clock,' of three different sizes. For each size, Chrome users were the fastest solvers, Firefox users came second, and IE users were the slowest. The number of abandoned puzzles (started but never finished) was also significantly higher for IE users. Analysis shows that the differences are statistically significant: in other words, they did not happen by chance. I put up more details and some graphs, and also wrote a paper about it (PDF)."

36 of 203 comments (clear)

  1. Wrong conclusions by lucm · · Score: 5, Funny

    > The number of abandoned puzzles (started but never finished) was also significantly higher for IE users

    As usual, Microsoft products users show more common sense: they are the ones that figure out quickly that the puzzles are a waste of time!

    --
    lucm, indeed.
    1. Re:Wrong conclusions by Barbara,+not+Barbie · · Score: 4, Insightful

      Or that they're wasting time at work, on a computer that they don't have admin rights to, so no installing extra browsers. Or they're kids using their parents computer. Or they now have a "good enough" browser so they don't care any more.

      --
      Let's call it what it is, Anti-Social Media.
    2. Re:Wrong conclusions by Samantha+Wright · · Score: 3, Interesting

      Then to avoid that contradiction, I propose a new hypothesis: IE users are the most likely to have something better to do than sit around all day solving puzzles. I think this really more suggests that Chrome users are the most bored.

      --
      Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
    3. Re:Wrong conclusions by jamesh · · Score: 5, Funny

      > The number of abandoned puzzles (started but never finished) was also significantly higher for IE users

      As usual, Microsoft products users show more common sense: they are the ones that figure out quickly that the puzzles are a waste of time!

      Interesting conclusion. The more likely conclusion is that IE is likely to crash before a puzzle can be completed.

    4. Re:Wrong conclusions by marcello_dl · · Score: 4, Interesting

      Prejudice: An adverse judgment or opinion formed beforehand or without knowledge or examination of the facts.

      Facts, or better, google hits:
      ie.crashes -> 27800000 results
      chrome.crashes -> 35000000 results
      ff.crashes -> 4.740.000 firefox.crashes ->1.810.000

      So, according to Google itself, IE IS crashy, Chrome IS crashier.

      --
      ---- MISSING MISCELLANEOUS DATA SEGMENT --- [sigdash] trolololol
    5. Re:Wrong conclusions by PNutts · · Score: 2

      Prejudice: An adverse judgment or opinion formed beforehand or without knowledge or examination of the facts.

      Facts, or better, google hits: ie.crashes -> 27800000 results chrome.crashes -> 35000000 results ff.crashes -> 4.740.000 firefox.crashes ->1.810.000

      So, according to Google itself, IE IS crashy, Chrome IS crashier.

      Release Year:
      1995 - IE
      2003 - FF
      2008 - Chrome

      Also, the Chrome folks have to be better with numbers to figure out what verion they're using while FF users only need to add 1 to their version number every few months. IE users don't get to use numbers much. When they copy their IE version into a support incident, the only real decision they need to make is yes or no to leave a large amount of data on the clipboard when finished.

    6. Re:Wrong conclusions by jamesh · · Score: 2

      Have you missed your morning coffee or do think I was seriously suggesting that IE isn't stable enough to survive a 30 second puzzle??

      I was responding to a thread that suggested that IE users left more puzzles unfinished because they think the puzzles are a waste of time, which was in turn in response to a FA suggesting that you can draw meaningful conclusions about intelligence from browser use. Why on earth do you think I was being serious?

      I guess a smiley on the end would have made it clearer, but the joke isn't funny if you have to point it out.

  2. Could happen by chance by dwhitaker · · Score: 5, Informative

    Statistical significance just means something is unlikely to occur by random chance. Said another way, it means there is evidence that it didn't happen by random chance, but not definitive proof. (This couching of conclusions is a mainstay of statistics.) Moreover, statistical significance doesn't necessarily translate to practical significance, but I didn't RTFA to find out if that was being claimed.

    1. Re:Could happen by chance by jpate · · Score: 2

      More precisely, it means that difference is probably non-zero in the population being sampled. And what is the population being sampled? It's people who play sudoku on the internet. Does that generalize to the population at large? Well, maybe. For the purposes of making conclusions about the population at large, this "study" has a potentially huge sampling bias.

      There also seem to be potential problems with multiple testing, but the paper doesn't go into enough detail to be sure.

      In short, you shouldn't trust an answer just because you got it by doing some math on a computer.

    2. Re:Could happen by chance by HarrySquatter · · Score: 2

      No, the problem is your entire premise and conclusion are faulty. That's not even getting into your sampling bias and other issues. Getting a statistically significant result is meaningless with poor sampling. You also provide no figures on your sampling error so your claims are even less meaningful. So sorry, but this whole "study" is total bunk just like the hoax study was despite your attempt at "HURR HURR IE users are teh dumb!" conclusion you are attempting to claim.

  3. Inadequacy by sycodon · · Score: 5, Insightful

    What does this seemingly never ending quest by people to formally define and declare who is best or smartest using various proxy measurements say about the people pursuing it?

    Are they afraid they aren't smart enough and are looking for some kind of reassurance?

    Maybe they want to make all the "not smart" people wear some kind of button. More likely, they just want to crow and be admired by other "smart" people.

    Many "smart" people would be end up standing up in their own shit because they don't understand plumbing. Many "dumb" people end up running the company and making gazillions of dollars. "Smart" is what you do with your brains, not your brain itself.

    Some people need to get a life.

    --
    When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
    1. Re:Inadequacy by Anonymous Coward · · Score: 2, Funny

      Of all the inappropriate places to post this :)

      FWIW, I agree with the sentiment completely....

      This is basically just a rehash of a good part of what Phil Greenspun blogged on years ago... who oddly would only be respected here based on personality cult factors alone.

    2. Re:Inadequacy by sycodon · · Score: 2

      Oh the irony!

      --
      When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
    3. Re:Inadequacy by tompaulco · · Score: 3, Insightful

      Despite what people say they don't really want everything to be "equal"..
      People who are below average or at least who perform below average absolutely want everything to be equal. It's the pesky above average people who want to be rewarded based on their skills and/or performance.

      --
      If you are not allowed to question your government then the government has answered your question.
    4. Re:Inadequacy by AthanasiusKircher · · Score: 2

      It's the pesky above average people who want to be rewarded based on their skills and/or performance.

      Yeah, the only problem is choosing what categories we want to be rewarded for if we're above average in them. For engineers and scientists, it's often advanced problem-solving or technical skills. For business majors, it's often being able to get one over on the next guy. As far as I can tell, the only people who want IQ to be the main category for achievement are deadbeats who seemingly have no real-world skills but can manage to take entrance tests for high-IQ societies.

      Everybody's above average at something, even if it's being above average in the number of categories you're below average in. (And, frankly, there are plenty of ways we often reward those people too... often with good reason.)

  4. 7th post! by jc42 · · Score: 3, Interesting

    Oh, wait ... Hmmm; this is a Safari window. I wonder how Safari users rank.

    Maybe I should switch to one of my Chrome or Firefox windows, then I might get it right.

    It might be interesting if we could get data on users that run multiple browsers. I have at least 10 browsers on this MacBook Pro, slightly fewer on my Ubuntu and Debian boxes, though I've previously found some that I didn't know I had, so I'm not sure how many more their might be. Lots of us developers collect browsers for testing against.

    Anyway, it could be interesting if people showed different math abilities when using different browsers. It'd imply that the differences are due to interference from the browsers' UIs, and not inherent in the individual users. I wonder how this study handle such possibilities. We already have good evidence that the programming language you use can help or hinder various sorts of reasoning ability, depending on the way they implement various capabilities. It wouldn't be too surprising if different browsers' UIs affected the ability of users to perform some mental operations. So we don't really know whether this study was comparing the users' math abilities, or the browsers' interference with their users' abilities.

    --
    Those who do study history are doomed to stand helplessly by while everyone else repeats it.
    1. Re:7th post! by Billly+Gates · · Score: 4, Interesting

      Rule number one in science is never to form causation from generaliztion of data. Studies show that rap music makes you a better basketball player. Ice cream can give you heart attacks too. Why?

      Statistically most NBA basketball players who are African American listen to rap music, therefore rap music made them great basketball players. The ice cream study was based on very hot days in New York when the temperature soared over 100 degrees. People tend to eat more ice cream on those days and there was also a rise of heart attacks. Therefore ice cream gives you heart attacks.

      In Korea there are warning labels that fans give you heart attacks and there are settings to make sure they turn off at night as Koreans believe you can die if you leave the fan on at night. This is because when it is hot people have heart attacks and you can guess where the media made the conclusion.

      It is silly and dangerous to make assumptions. You need a full hypothesis and use the standard scientific method to reproduce the results.

      For all we know more old people use IE who are mentally further declined, or people went to that site at work when the boss wasn't looking and quickly alt tabbed and let the game time out when work needed them, etc. These are valid reasons and does not equate stupidity for people who use IE. Until we know more we just do not know. The work thing with IE is a very likely reason why a user would stop the puzzle as corporate America loves IE and users tend to hate work.

    2. Re:7th post! by jc42 · · Score: 2

      Actually, I do have lynx on all my machines, and it's one of the browsers that I test against fairly often. It's one of the useful tools for verifying that pages are accessible to the visually impaired. I've also found it useful in some discussions to mention that on several projects, I've been explicitly order to not test against lynx, or any other tools for the disabled. A lot of management has open contempt for people with physical disabilities, at least here in the US.

      --
      Those who do study history are doomed to stand helplessly by while everyone else repeats it.
  5. Re:Chrome solve time for all sizes by Robadob · · Score: 2

    "Average solving time as a percentage of the Chrome average (so smaller is better)"

  6. Re:Chrome solve time for all sizes by Samantha+Wright · · Score: 2

    Actually, the data were normalized against the Chrome speed for each category. That's not 100 seconds you're looking at, that's 100 percent of the Chrome rate. It's a weird way of displaying a graph, but if the author hadn't done it then the bars for the larger puzzle sizes would have (presumably) dwarfed the smaller ones, resulting in a loss of visible precision. I guess the more standard solution, using a logarithmic scale, either didn't occur to the author or was for some reason infeasible.

    --
    Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
  7. IQ correlates with motivation by Colin+Smith · · Score: 4, Funny

    http://theness.com/neurologicablog/index.php/iq-and-motivation/

    So what the guy is really saying is that Chrome users are obsessive compulsives and I.E. users are normal.

    --
    Deleted
    1. Re:IQ correlates with motivation by Mistlefoot · · Score: 2

      People at work on locked down computers used IE and kept getting "distracted" and going back to work. Sometimes they got busy enough to not finish the puzzle.
      I might play Sudoku at work on a break and get distracted, never coming back (on IE). I play the same game at home on Chrome and quickly finish as my focus is there.

      Statistically, that makes me "stupid" at work and "smart" at home. Don't let my boss find out!

  8. Re:Is this just a measure of browser performance? by mooingyak · · Score: 4, Informative

    Yeah, my first thought was that maybe his site causes IE to crash sometimes, which would look like an abandoned game.

    --
    William of Ockham had no beard. The most likely explanation is that it was chewed off by squirrels every morning.
  9. Re:I'd rather have HIV than use any Google product by bhagwad · · Score: 2

    I'd like to see you actually act on your claims :)

  10. Yawn, who cares. by xaoslaad · · Score: 2

    If I use my Mac I use Safari. If I use my Fedora laptop I use Firefox. If use my Windows PC I use IE.

    Any one of them works fine for me. If I can look at web pages and Bookmark/Favorite things it does 99.99% of what I want. I keep all my systems up to date, and run active AV of Windows. I'm not in the habit of viewing a wide range of shady web sites either. To top it off I can't think of a site I use that is not compatible with all three. And it is enough of a headache keeping 3 different systems up to date (nevermind the add virtual machines) without downloading extra browsers and making sure they're up to date separately.

    Basically, who cares what browser you use. I doubt it defines you, me, or anyone else.

  11. Re:I'd rather have HIV than use any Google product by K.+S.+Kyosuke · · Score: 2

    So, you're still stuck with Altavista, old pal? ;)

    --
    Ezekiel 23:20
  12. Re:I'd rather have HIV than use any Google product by lucm · · Score: 2

    So, you're still stuck with Altavista, old pal? ;)

    Until altavista has proven to be reliable, I'll keep using Excite.

    --
    lucm, indeed.
  13. Re:I'd rather have HIV than use any Google product by Yvan256 · · Score: 3

    Webcrawler forever!

  14. Need more details by ceoyoyo · · Score: 2, Insightful

    If you're going to write a paper, put the relevant details in. What kind of statistical tests did you do? What correction for multiple comparisons did you do? What are the actual p-values you obtained, for each test? Are the distributions of your data normal? Do they meet the assumptions of your test?

  15. Re:Is this just a measure of browser performance? by dr_blurb · · Score: 3, Interesting

    Possibly, but my guess is that I would have had complaints from people.

    Also note that this was data over two years, and I'm only using it from people who've successfully completed at least 10 timed puzzles of each size.

  16. Re:Chrome solve time for all sizes by Samantha+Wright · · Score: 2

    I see you're quite a good conversationalist; you must be quite a blast at parties. No, the claim that Javascript interpreter speed may have a role in browser performance is not easily testable because the author lumped different versions of the browsers together. It's possible that some obscure intermediary version of Firefox, for example, had an exceptionally poor performance, and that this skewed the data. I have in fact determined that Aurora 12.0a2 seems to have no performance difference between IE 8 on my laptop, but this does not necessarily mean, either, that IE 6 performance is entirely ignorable. Given the statistics for overall browser usage it would be exceptionally improbable that these have an overwhelming role in confounding results, but the landscape could still be measurably different as a result. Given that Javascript engine performance has increased dramatically as a priority for browser manufacturers in the past five years, this data incorporates information from a very broad set of configurations.

    At any rate, I find it very disappointing that you chose to focus entirely on one statement about Javascript engines and not consider the rest of my post, or the larger significance of the point I was making. Instead you chose to attack (and violently, I might add) one relatively arbitrary theory when you could have contributed by gently stating any objections, before positing your own.

    Here is an example of how to make a rebuttal correctly, for future reference: "over the span of a million data points from across the planet, it seems unlikely that network latency would have presented a bias towards one browser or another, particularly since the differences are on the order of magnitude of several dozen seconds, the application does not need to talk to the server for the player's experience to continue. To produce the kinds of bias observed given the nature of the application, Firefox users would have to be several times further from the Earth than the moon."

    Perhaps the phenomenal absurdity of your suggestion explains why none of us are capable of thinking of it. Unless you meant to say network bandwidth, as in "IE users are all on dial-up, which is why they haven't spent the time to download a more secure browser, and it actually takes them 30 seconds to download the page," in which case I sincerely hope your entire post was made in jest, and that you have something better to do than make condescending remarks about intellect.

    --
    Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
  17. tired by Mariomario · · Score: 2

    Tired of these "studies" that shows better apples uses a certain orange. Personally, I use Firefox and IE and find no difference between them. Sometimes something don't work on one of them, so I do it on the other. Considering IE comes as default on windows. Studies like this is like saying "Players who play games with default settings are stupid, and players who edit the settings are smart." Just because you have not found a need to change the settings dose not automatically mean your stupid.

  18. In my experience by Sycraft-fu · · Score: 2

    IE is far more stable than Firefox. Now that is a little skewed, since FF is my normal browser. However FF does piss me off a fair bit by blowing up. When FF start to have problems with some content, I fire up IE and it handles everything no troubles. Of course this is all anecdotal, but then I've seen no evidence of IE being super crashy at work (we have some users who like it).

    I think it is more MS haters wanting IE to be bad. They are worried IE might end up being a good browser and so hate on it.

  19. /sigh by Guppy06 · · Score: 2

    a site were you can solve Calcudoku number puzzles.

    Ahem.

    And publishing your "paper" on your own website doesn't make it peer-reviewed either.

  20. So? by hism · · Score: 2

    So am I expected to move to Chrome now so I can be part of some exclusive club to validate my IQ? Or move away from IE so people don't think I'm stupid? Maybe they should have called this study, "Chrome Users Are Most Insecure?"

  21. highly suspect on the MSIE times by ChipMonk · · Score: 2

    MSIE's download progress bars lie 99% of the time, and just make something up the other 17%.

    Can we trust the reported times for MSIE users to work these calcudoku puzzles?