Chrome Users Are Best With Numbers, IE Users Worst

New submitter dr_blurb writes "After reading about last year's hoax report 'Intelligence Quotient (IQ) and Browser Usage' I realized I was in fact already running a real live experiment measuring number skills: a site were you can solve Calcudoku number puzzles. I analyzed two years' worth of data, consisting of over 1 million solved puzzles. This included puzzles solved 'against the clock,' of three different sizes. For each size, Chrome users were the fastest solvers, Firefox users came second, and IE users were the slowest. The number of abandoned puzzles (started but never finished) was also significantly higher for IE users. Analysis shows that the differences are statistically significant: in other words, they did not happen by chance. I put up more details and some graphs, and also wrote a paper about it (PDF)."

Comments

Wrong conclusions

[lucm]

> The number of abandoned puzzles (started but never finished) was also significantly higher for IE users

As usual, Microsoft products users show more common sense: they are the ones that figure out quickly that the puzzles are a waste of time!

Re:Wrong conclusions

[Barbara, not Barbie]

Or that they're wasting time at work, on a computer that they don't have admin rights to, so no installing extra browsers. Or they're kids using their parents computer. Or they now have a "good enough" browser so they don't care any more.

Re:Wrong conclusions

[Samantha Wright]

Then to avoid that contradiction, I propose a new hypothesis: IE users are the most likely to have something better to do than sit around all day solving puzzles. I think this really more suggests that Chrome users are the most bored.

Re:Wrong conclusions

[jamesh]

> The number of abandoned puzzles (started but never finished) was also significantly higher for IE users

As usual, Microsoft products users show more common sense: they are the ones that figure out quickly that the puzzles are a waste of time!

Interesting conclusion. The more likely conclusion is that IE is likely to crash before a puzzle can be completed.

Re:Wrong conclusions

[exploder]

Their browser crashed before they could finish the puzzle.

Re:Wrong conclusions

[Oligonicella]

I use FireFox and I just went to the puzzle page.

You click the highlighted square and the first thing that happens is an immovable pop-up covers most of the puzzle.

I left it unfinished.

Re:Wrong conclusions

[marcello_dl]

Prejudice: An adverse judgment or opinion formed beforehand or without knowledge or examination of the facts.

Facts, or better, google hits:
ie.crashes -> 27800000 results
chrome.crashes -> 35000000 results
ff.crashes -> 4.740.000 firefox.crashes ->1.810.000

So, according to Google itself, IE IS crashy, Chrome IS crashier.

Re:Wrong conclusions

[rykin]

Interesting. Could this be because Google tends to favor results that link to google products? I ask this because I've been using Chrome at home for a while now and have never experienced a single crash, yet I still use IE at work and it tends to have issues/crash frequently enough for me to notice (it is not the latest IE though).

Re:Wrong conclusions

[Anonymous Coward]

The use of "Google Hits" as a metric relies on at least similar numbers of users of each browser using google as the default search engine. By default, IE uses bing, and chrome uses google.

Re:Wrong conclusions

[AlexBirch]

Perhaps IE users just reboot their computer after a crash instead of trying to figure out why. According to Google itself, its users ask why it crashes more

Re:Wrong conclusions

[PNutts]

Prejudice: An adverse judgment or opinion formed beforehand or without knowledge or examination of the facts.

Facts, or better, google hits: ie.crashes -> 27800000 results chrome.crashes -> 35000000 results ff.crashes -> 4.740.000 firefox.crashes ->1.810.000

So, according to Google itself, IE IS crashy, Chrome IS crashier.

Release Year:
1995 - IE
2003 - FF
2008 - Chrome

Also, the Chrome folks have to be better with numbers to figure out what verion they're using while FF users only need to add 1 to their version number every few months. IE users don't get to use numbers much. When they copy their IE version into a support incident, the only real decision they need to make is yes or no to leave a large amount of data on the clipboard when finished.

Re:Wrong conclusions

[jamesh]

Have you missed your morning coffee or do think I was seriously suggesting that IE isn't stable enough to survive a 30 second puzzle??

I was responding to a thread that suggested that IE users left more puzzles unfinished because they think the puzzles are a waste of time, which was in turn in response to a FA suggesting that you can draw meaningful conclusions about intelligence from browser use. Why on earth do you think I was being serious?

I guess a smiley on the end would have made it clearer, but the joke isn't funny if you have to point it out.

Re:Wrong conclusions

[Dragonslicer]

So, care to back that up with anything resembling facts? Or were you just being snarky?

This is Slashdot. Nobody here is ever snarky.

Re:Wrong conclusions

[Kalriath]

Whereas at work, I find that Chrome is the infinitely crashier one than IE (although Firefox beats them both out for sheer crashy shittiness).

Re:Wrong conclusions

[dr_blurb]

> You click the highlighted square and the first thing that
> happens is an immovable pop-up covers most of the puzzle.
> I left it unfinished.

It goes away when you pick a number, or when you click
outside the popup. Also, this would not have counted as
an unfinished puzzle, since this is only tracked for timed puzzles.

Re:Wrong conclusions

[MaXiMiUS]

Google searches are not statistically valuable, unless you're gathering statistics on Google searches specifically I suppose. Very few IE crashes are reported in a manner Google can index, if they're even reported at all. Firefox on the other hand has Bugzilla, a public bug tracker with hundreds of thousands of reports.

Re:Wrong conclusions

[Riceballsan]

That has to be the worse possible way to calculate out crashes. So many random things you can assert from search results. 1. IE users are more likely to fork over $150 to geek squad, before posting on a forum or anywhere google can index. 2. places they post to are less likely to be indexed. or 3. They are so used to crashes they don't even bother to talk about them. Let me try this game

windows.me.crashes -> 45,900,000

windows.7.crashes - > 50,600,000

macos.crashes -> 42,500,000

There we have it, scientific proof that windows ME is more stable than windows 7.

Re:Wrong conclusions

[Speare]

Do you have extra zeros there?

On a hunch I searched for "chrome crashes -bumper -rollbar" and only get 4,600,000 as the result count. Then I searched for plain old "chrome crashes" and got 5,600,000. I know that Google search results depends somewhat on the browsing history of the individual, but these are an order of magnitude lower than your results.

Re:Wrong conclusions

[marcello_dl]

I use . to separate between words, and the FF search box on iceweasel
  9.0.2 on debian sid
Today, same setup
ie.crashes has 27.900.000
and chrome.crashes 6.090.000
and ff.crashes 4.750.000

I am confident enough it was 30000000 when I wrote the comment because I ran it twice (I was surprised by the result myself)
FF lands me to the italian version of google because of my location.

If anything, this glitch shows that for serious things we should revert to usenet ASAP ;)

Re:Wrong conclusions

[marcello_dl]

The point was that ie crashes, by experience, not prejudice. Chrome and google's variable hits are an aside.

Re:Wrong conclusions

[marcello_dl]

Then, win7 sucks. I guess you searched the internet archive, and compensated for the different amount of pages back in that time? I didn't need to do that as IE started first and sold parallel to chrome later.

Re:Wrong conclusions

[Riceballsan]

The point is there are dozens of other reasons why google search results are absolutely no indication of actual crashes. I'm sure there are bugs that thousands have had but few reported, and crashes that only a handful of people have had, but posted to 15 different places to find a solution. Timeframe is just one of dozens of variables that are examples of why search engine hits is a lousy indicator of actual events.

Re:Wrong conclusions

[mcgrew]

I seriously doubt that very many people using Linux or Macs are using IE, and the first thing a Windows user does when the computer has the slightest glitch is reboot. Knowledgeable users know better, but few of them are using IE.

Windows users are used to rebooting, the rest of us aren't.

Re:Wrong conclusions

[NicknameAvailable]

Your all idiots, IE drops sessions more often than other browsers and that is all there is to this story.

Re:Wrong conclusions

[lucm]

Your all idiots, IE drops sessions more often than other browsers and that is all there is to this story.

"Your" very clever (like the 10 other people who made the same comment) but of course "your" the only one that is not an idiot.

Re:Wrong conclusions

[NicknameAvailable]

Your all idiots, IE drops sessions more often than other browsers and that is all there is to this story.

"Your" very clever (like the 10 other people who made the same comment) but of course "your" the only one that is not an idiot.

Sometimes I wonder if there is anything lower than a grammar Nazi - pointing out flaws in a system meant to communicate when there was not issue with the message being communicated as a result of the grammar utilized - yep, definitely more cleverer than you.

Re:Wrong conclusions

[lucm]

Your all idiots, IE drops sessions more often than other browsers and that is all there is to this story.

"Your" very clever (like the 10 other people who made the same comment) but of course "your" the only one that is not an idiot.

Sometimes I wonder if there is anything lower than a grammar Nazi - pointing out flaws in a system meant to communicate when there was not issue with the message being communicated as a result of the grammar utilized - yep, definitely more cleverer than you.

The real problem in your post was not the syntax, it was the fact that you insult people. If you say "your all idiots" you basically put an end to any kind of civilized discussion, at which point putting an emphasis on syntax is fair game.

Quitters

[petteyg359]

Firefox users were the most persistent. Palin obviously doesn't use Firefox.

Could happen by chance

[dwhitaker]

Statistical significance just means something is unlikely to occur by random chance. Said another way, it means there is evidence that it didn't happen by random chance, but not definitive proof. (This couching of conclusions is a mainstay of statistics.) Moreover, statistical significance doesn't necessarily translate to practical significance, but I didn't RTFA to find out if that was being claimed.

Re:Could happen by chance

[dr_blurb]

You're absolutely right, I should have written: "in other words, they were not likely to have happened by chance", or something similar. (the chance of it happening by chance was less than 5%, which for many papers is enough, and people still play the lottery :-)

Re:Could happen by chance

[jpate]

More precisely, it means that difference is probably non-zero in the population being sampled. And what is the population being sampled? It's people who play sudoku on the internet. Does that generalize to the population at large? Well, maybe. For the purposes of making conclusions about the population at large, this "study" has a potentially huge sampling bias.

There also seem to be potential problems with multiple testing, but the paper doesn't go into enough detail to be sure.

In short, you shouldn't trust an answer just because you got it by doing some math on a computer.

Re:Could happen by chance

[HarrySquatter]

No, the problem is your entire premise and conclusion are faulty. That's not even getting into your sampling bias and other issues. Getting a statistically significant result is meaningless with poor sampling. You also provide no figures on your sampling error so your claims are even less meaningful. So sorry, but this whole "study" is total bunk just like the hoax study was despite your attempt at "HURR HURR IE users are teh dumb!" conclusion you are attempting to claim.

Re:Could happen by chance

[HarrySquatter]

No, he's quite right. This study is chock full of errors and despite the claims of "statistical significance" he doesn't provide any effect size statistics to even show that his result is practically significant. It's just fanboi fueled nonsense.

Re:Could happen by chance

[fbjon]

You may want to revise the paper to take into account the different use cases for each browser. If users of a certain browser are more likely to be distracted or interrupted (like at work, for instance), and similarly if users of a certain browser are more likely to be looking for an easy-to-digest diversion and abandoning some harder puzzles (like at work), this would invalidate the conclusions. Your statistics might say more about where different browsers are likely to be found rather than about the users of those browsers.

Re:Could happen by chance

[dr_blurb]

No, the problem is your entire premise and conclusion are faulty. That's not even getting into your sampling bias and other issues. Getting a statistically significant result is meaningless with poor sampling. You also provide no figures on your sampling error so your claims are even less meaningful. So sorry, but this whole "study" is total bunk just like the hoax study was despite your attempt at "HURR HURR IE users are teh dumb!" conclusion you are attempting to claim.

Yes, I should've included sampling error, and will do so in a later version (sampling error was small).

The fact that the average times were lowest for Chrome and highest for IE was just what it was. If it had been the other way around I would have reported that.

An earlier poster wrote:

There also seem to be potential problems with multiple testing [xkcd.com], but the paper doesn't go into enough detail to be sure.

That's a good point, I need to look into that. You always prefer an interesting result to report on, so there's this risk of "hunting" for one :-(

Re:Could happen by chance

[dr_blurb]

You may want to revise the paper to take into account the different use cases for each browser. If users of a certain browser are more likely to be distracted or interrupted (like at work, for instance), and similarly if users of a certain browser are more likely to be looking for an easy-to-digest diversion and abandoning some harder puzzles (like at work), this would invalidate the conclusions. Your statistics might say more about where different browsers are likely to be found rather than about the users of those browsers.

It'll be tricky to evaluate any kind of data by asking users.

On the site's forum, for example, there's been some back and forth about the influence of age. One user has set up a web survey to collect age data. But I can never be sure I'm getting correct numbers. At some point there'll be so much uncertainty in the source data, I can't derive anything anymore.

Re:Could happen by chance

[fbjon]

At some point there'll be so much uncertainty in the source data, I can't derive anything anymore.

Now there's a conclusion! :)

Inadequacy

[sycodon]

What does this seemingly never ending quest by people to formally define and declare who is best or smartest using various proxy measurements say about the people pursuing it?

Are they afraid they aren't smart enough and are looking for some kind of reassurance?

Maybe they want to make all the "not smart" people wear some kind of button. More likely, they just want to crow and be admired by other "smart" people.

Many "smart" people would be end up standing up in their own shit because they don't understand plumbing. Many "dumb" people end up running the company and making gazillions of dollars. "Smart" is what you do with your brains, not your brain itself.

Some people need to get a life.

Re:Inadequacy

[Anonymous Coward]

Of all the inappropriate places to post this :)

FWIW, I agree with the sentiment completely....

This is basically just a rehash of a good part of what Phil Greenspun blogged on years ago... who oddly would only be respected here based on personality cult factors alone.

Re:Inadequacy

[friedmud]

Competition is a basic human need... the want to compete and come out on top is intrinsic in all of us. We want to come out on top of everything... including being associated with a group that comes out on top of another group.

This competition is one of the reasons pure communism can never work. Despite what people say they don't really want everything to be "equal"... what they mean by that is that they don't want others to have more than them (ie they want _more_ than others! ;-)

In the absence of competition you generate bored, unhappy people.... that will eventually tear down their own society...

Re:Inadequacy

[sycodon]

Oh the irony!

Re:Inadequacy

[sycodon]

Cool blog. Thanks for the tip!

Re:Inadequacy

[Anonymous Coward]

Conflict is what you mean. Conflict ideally ends in a resolution where you come out on top; competition is a family-friendly way to describe it. You imply that human on human conflict is a necessity, I contend there is plenty of real conflict in the realm of human vs nature or human vs science or human vs medicine or human vs machine that would serve our species better to approach.

In the circles I deal with, competition exists as a superficial joke. Cooperation is what actually gets things done, while the conflict makes it popular enough for people to care about.

Re:Inadequacy

[tompaulco]

Despite what people say they don't really want everything to be "equal"..
People who are below average or at least who perform below average absolutely want everything to be equal. It's the pesky above average people who want to be rewarded based on their skills and/or performance.

Re:Inadequacy

[AthanasiusKircher]

It's the pesky above average people who want to be rewarded based on their skills and/or performance.

Yeah, the only problem is choosing what categories we want to be rewarded for if we're above average in them. For engineers and scientists, it's often advanced problem-solving or technical skills. For business majors, it's often being able to get one over on the next guy. As far as I can tell, the only people who want IQ to be the main category for achievement are deadbeats who seemingly have no real-world skills but can manage to take entrance tests for high-IQ societies.

Everybody's above average at something, even if it's being above average in the number of categories you're below average in. (And, frankly, there are plenty of ways we often reward those people too... often with good reason.)

Numerical intelligence

[Anonymous Coward]

Hardly "numerical" intelligence. The sudokus don't involve any properties of the numbers, no arithmetic, nothing. All that is relevant is that the symbols are different, and the fact that you've chosen to use "numerical" symbols is neither here nor there. So rather, I'd suggest that this analysis shows a comparison of "logical" intelligence.

Re:Numerical intelligence

[Jmc23]

I suggest you have very low reading intelligence. This has nothing to do with sudoku, it has to do with calcudoku which does in fact require basic math skills.

7th post!

[jc42]

Oh, wait ... Hmmm; this is a Safari window. I wonder how Safari users rank.

Maybe I should switch to one of my Chrome or Firefox windows, then I might get it right.

It might be interesting if we could get data on users that run multiple browsers. I have at least 10 browsers on this MacBook Pro, slightly fewer on my Ubuntu and Debian boxes, though I've previously found some that I didn't know I had, so I'm not sure how many more their might be. Lots of us developers collect browsers for testing against.

Anyway, it could be interesting if people showed different math abilities when using different browsers. It'd imply that the differences are due to interference from the browsers' UIs, and not inherent in the individual users. I wonder how this study handle such possibilities. We already have good evidence that the programming language you use can help or hinder various sorts of reasoning ability, depending on the way they implement various capabilities. It wouldn't be too surprising if different browsers' UIs affected the ability of users to perform some mental operations. So we don't really know whether this study was comparing the users' math abilities, or the browsers' interference with their users' abilities.

Re:7th post!

[wbr1]

This UI of the puzzle is the HTML page, and if coded well has no significant difference in the major browsers, therefore the browsers UI itself should have little impact if any on the results. In addition, only a very small percentage of the population are developers with 20 different browsers on their system. Most users use whatever the default browser it, or install their favorite (or geek nephews recommended) browser and use that.

Re:7th post!

[Billly Gates]

Rule number one in science is never to form causation from generaliztion of data. Studies show that rap music makes you a better basketball player. Ice cream can give you heart attacks too. Why?

Statistically most NBA basketball players who are African American listen to rap music, therefore rap music made them great basketball players. The ice cream study was based on very hot days in New York when the temperature soared over 100 degrees. People tend to eat more ice cream on those days and there was also a rise of heart attacks. Therefore ice cream gives you heart attacks.

In Korea there are warning labels that fans give you heart attacks and there are settings to make sure they turn off at night as Koreans believe you can die if you leave the fan on at night. This is because when it is hot people have heart attacks and you can guess where the media made the conclusion.

It is silly and dangerous to make assumptions. You need a full hypothesis and use the standard scientific method to reproduce the results.

For all we know more old people use IE who are mentally further declined, or people went to that site at work when the boss wasn't looking and quickly alt tabbed and let the game time out when work needed them, etc. These are valid reasons and does not equate stupidity for people who use IE. Until we know more we just do not know. The work thing with IE is a very likely reason why a user would stop the puzzle as corporate America loves IE and users tend to hate work.

Re:7th post!

[lucm]

I have at least 10 browsers on this MacBook Pro, slightly fewer on my Ubuntu and Debian boxes [...] Lots of us developers collect browsers for testing against.

One OSX and two Debians. It appears to me that you are missing a significant browser market share (and I'm not talking about Lynx).

This being said, I'd be curious to see the user agent stats for nambla.org - maybe the FBI or NSA could tell us.

Re:7th post!

[jc42]

Actually, I do have lynx on all my machines, and it's one of the browsers that I test against fairly often. It's one of the useful tools for verifying that pages are accessible to the visually impaired. I've also found it useful in some discussions to mention that on several projects, I've been explicitly order to not test against lynx, or any other tools for the disabled. A lot of management has open contempt for people with physical disabilities, at least here in the US.

Re:7th post!

[tompaulco]

I don't see anybody claiming causation. Nobody says that using Chrome or Firefox instead of IE will MAKE you smarter. It just shows that people who use Firefox or Chrome tend to be smarter. This conclusion is an obvious one if you think about it. The average person, and the below average person, will most likely use the browser that comes with their computer. And of course, the average or below average person is probably going to have a MS based machine. The average or below average person probably is not even aware that there is an alternative to IE. You have to be an engineer, computer professional, or at the very least someone with a desire to know more about computers to even be aware of the existence of alternative browsers. These types of people are generally smarter than your average bear.
When reading this post, please note that I don't intend to make any absolutes. "On average" should be inferred in front of most of the statements herein. The only reason I didn't type it out every time was because we were already talking about averages.

Re:7th post!

[enrgeeman]

Not that it changes your underlying point, but the main reason Koreans don't sleep with the fan on is not because it causes heart attacks, but because it either a) sucks the air out of the room, or b) chops up the air such that they cannot breathe. Both lead to suffocation, and death, but the "reason" ro different. Doesn't make it any less crazy, though.

Re:7th post!

[Sycraft-fu]

Or, like someone else said maybe the IE users have more common sense and just leave the silly test early. Was a joke but still a good example of a real possibility you'd have to test and control for.

I know personally if someone says "Go try this test," I have a very short attention span for it. I'll answer some questions but more than about 10 or so and I'm done, don't care that much (I use FF though).

With something like this all you know is a correlation (presuming the data was statistically significant) and then only for the self selecting group that was willing to do it. Not much to go on.

Re:7th post!

[sessamoid]

I have at least 10 browsers on this MacBook Pro, slightly fewer on my Ubuntu and Debian boxes [...] Lots of us developers collect browsers for testing against.

One OSX and two Debians. It appears to me that you are missing a significant browser market share (and I'm not talking about Lynx).

This being said, I'd be curious to see the user agent stats for nambla.org - maybe the FBI or NSA could tell us.

The MacBook Pro can run Windows natively or in a virtual machine.

Re:7th post!

[lucm]

I have at least 10 browsers on this MacBook Pro, slightly fewer on my Ubuntu and Debian boxes [...] Lots of us developers collect browsers for testing against.

One OSX and two Debians. It appears to me that you are missing a significant browser market share (and I'm not talking about Lynx).

This being said, I'd be curious to see the user agent stats for nambla.org - maybe the FBI or NSA could tell us.

The MacBook Pro can run Windows natively or in a virtual machine.

Makes you wonder why someone would pay that much money for a regular Intel CPU, 4GB of RAM and a cheap 5400 RPM hard disk then, especially when the same specs can be found in a Dell Vostro for less than half the price (including Windows).

Re:7th post!

[HarrySquatter]

It just shows that people who use Firefox or Chrome tend to be smarter.

No, it doesn't show that at all. It provides no effect size statistics to back up such a claim.

This conclusion is an obvious one if you think about it.

No, it's not. The conclusion is only obvious because your a fanboi and nothing more. This is also why you idiots were fooled by the hoax study. The conclusion seemed correct only because it fed into your bias.

Re:7th post!

[tompaulco]

No, it doesn't show that at all. It provides no effect size statistics to back up such a claim.
Well, by all means, lets have a larger study, then. However, I would still bet large sums of money that browsers that people have to go out of their way to find and use will tend to have smarter users than browsers that come by default with almost every computer. You don't have to get offended, just because this small sampling tends to back up what just plain makes sense. If you are an IE user, that doesn't make you dumb. It's just that a lot of dumb people tend to use IE, because that is what their computer came with, and they tend to drag down the average. Heck, I use IE from time to time, when some idiot webmaster has thrown standards to the wind in favor of special whizbangs that only IE supports. That doesn't make me any dumber for using it. Nor am I offended that people who use Chrome are on average smarter than people who use Firefox, like myself. Averages are just averages. Like 99% of people, I consider myself above average,

Re:7th post!

[Jens Egon]

No, it doesn't show that at all. It provides no effect size statistics to back up such a claim. Well, by all means, lets have a larger study, then. However, I would still bet large sums of money that browsers that people have to go out of their way to find and use will tend to have smarter users than browsers that come by default with almost every computer

.

My first thought was "Or maybe those other browsers give their users a better more engaging experience." It's not what I believe, but then again there's a reason people go out of their way to find these browsers, right?

You don't have to get offended, just because this small sampling tends to back up what just plain makes sense.

No, but we should be wary of believing evidence that supports our prejudices.

Chrome solve time for all sizes

[Robadob]

Doesn't anyone else find it suspicious that the chrome solve time for all 3 sizes was 100 seconds while ie and firefox both changed?

Re:Chrome solve time for all sizes

[Robadob]

"Average solving time as a percentage of the Chrome average (so smaller is better)"

Re:Chrome solve time for all sizes

[Samantha Wright]

Actually, the data were normalized against the Chrome speed for each category. That's not 100 seconds you're looking at, that's 100 percent of the Chrome rate. It's a weird way of displaying a graph, but if the author hadn't done it then the bars for the larger puzzle sizes would have (presumably) dwarfed the smaller ones, resulting in a loss of visible precision. I guess the more standard solution, using a logarithmic scale, either didn't occur to the author or was for some reason infeasible.

Re:Chrome solve time for all sizes

[Samantha Wright]

Here we go, found the actual data in the paper:

browser: 4x4 5x5 6x6
Internet Explorer: 30.9, 73.4, 262
Firefox: 29.4, 70.2, 245
Chrome: 22.0, 61.1, 233

Of course, this doesn't disprove the leading theory, which is that the analysis is total garbage. The number of potential mitigating factors (e.g. "Hey look, solving a browser-based puzzle game reflects the speed of the browser's javascript interpreter!") and alternative hypotheses (such as "IE users haven't invested time in changing their browsers, therefore they aren't interested in investing time to do stuff with their computers in general...") are just too numerous. We can write this story off, entirely, as denigrating propaganda. But maybe that should be expected.

Re:Chrome solve time for all sizes

[Jmc23]

Wow, you must be a piss poor biologist since you don't seem to understand how science works. No experiment is complete by itself but every piece of data is useful. Your 'hypothesis' is easily testable, and you could probably just ask the guy and he might even have the answer of whether or not the browser has anything to do with it, e.g. puzzle time starts once ui drawn, or time starts when first input, or before the ui is even created.

As well, unless the guy wrote a very bad program he's not going to start the timer before the puzzle is even drawn. A confounding factor that none of you seem capable of thinking of would actually be network latency.

Re:Chrome solve time for all sizes

[Samantha Wright]

I see you're quite a good conversationalist; you must be quite a blast at parties. No, the claim that Javascript interpreter speed may have a role in browser performance is not easily testable because the author lumped different versions of the browsers together. It's possible that some obscure intermediary version of Firefox, for example, had an exceptionally poor performance, and that this skewed the data. I have in fact determined that Aurora 12.0a2 seems to have no performance difference between IE 8 on my laptop, but this does not necessarily mean, either, that IE 6 performance is entirely ignorable. Given the statistics for overall browser usage it would be exceptionally improbable that these have an overwhelming role in confounding results, but the landscape could still be measurably different as a result. Given that Javascript engine performance has increased dramatically as a priority for browser manufacturers in the past five years, this data incorporates information from a very broad set of configurations.

At any rate, I find it very disappointing that you chose to focus entirely on one statement about Javascript engines and not consider the rest of my post, or the larger significance of the point I was making. Instead you chose to attack (and violently, I might add) one relatively arbitrary theory when you could have contributed by gently stating any objections, before positing your own.

Here is an example of how to make a rebuttal correctly, for future reference: "over the span of a million data points from across the planet, it seems unlikely that network latency would have presented a bias towards one browser or another, particularly since the differences are on the order of magnitude of several dozen seconds, the application does not need to talk to the server for the player's experience to continue. To produce the kinds of bias observed given the nature of the application, Firefox users would have to be several times further from the Earth than the moon."

Perhaps the phenomenal absurdity of your suggestion explains why none of us are capable of thinking of it. Unless you meant to say network bandwidth, as in "IE users are all on dial-up, which is why they haven't spent the time to download a more secure browser, and it actually takes them 30 seconds to download the page," in which case I sincerely hope your entire post was made in jest, and that you have something better to do than make condescending remarks about intellect.

Re:Chrome solve time for all sizes

[HarrySquatter]

Science doesn't work by pushing conclusions based on correlations from a study that has huge sampling bias and no effect size statistics to prove that the conclusion is even practically significant. If what the submitter was doing is what you consider science then... wow... we are more doomed then I thought.

Re:Chrome solve time for all sizes

[Jmc23]

My tone was exactly the same as yours without the haughty pretentiousness behind it. You were the one to say we should just totally ignore anything about this right?

You still aren't thinking about latency correctly. Browser users form communities, and those communities have certain characteristics. IE is default on all windows machines, it is the bottom, the start from which communities branch off because there has to be a stimulus to replace IE, even if that means replacing windows with linux where IE isn't available. One could easily put forth that academic communities have a smaller percentage of IE as well as more intelligent people and fatter bandwidth and smaller latency than your average home user.

If you had played any of the puzzles you would see where that latency effect could possibly have an effect.

So, yes, when you comment on something in a haughty academic tone without even looking at the implementation then yes you are not representing your proffession and science to the best of your ability.

Re:Chrome solve time for all sizes

[Samantha Wright]

A dial-up modem in Britain takes about half a second, at most, to communicate with a server in Korea. (This is a generous over-estimate calculated from my experience with badly-coded Doom multiplayer servers.) No action on the part of the user in the game can cause a cumulative latency effect; the Javascript communicates with the server at regular intervals asynchronously to preserve the game's play state. Here is the code. Note where and how the XMLHttpRequest objects are used.

The maximum effect that latency can have on the game is that half a second, doubled to represent the start and end of the game. This cannot account for the additional 30 second difference in solving time between Chrome and Internet Explorer users, even if the figure of one half-second is underestimated by a factor of five. I have played the puzzles, and no, I cannot see where latency could come into play. Bandwidth, yes; that could slow down the loading time immensely, but not latency.

As for browser users in academia... well, that's a little more disappointing than you might imagine, but not entirely off, and I do agree with that idea as a whole. Gotta go now; will continue this chat later.

Re:Chrome solve time for all sizes

[Samantha Wright]

I do like your theory of IE as a base, from which people must migrate away. There are of course exceptions to user intention: the archetypal benevolent sysadmin "who takes away the dangerous blue 'e' " being one of them.

That being said, intelligent people don't just automatically migrate away from IE. Informed, interested people with a meaningful investment in the security of their computers do. Inherently that includes an age bias, an interest in computers (which is not the same as an interest in solving math puzzles, or even in programming!), and potentially an interest in the aesthetics of one's computing experience (which many, many highly capable mathematicians completely lack.) So I still don't think that numerical intelligence is well-correlated with browser of choice according to this category.

Re:Chrome solve time for all sizes

[Jmc23]

That has got to be some of the worst code i've ever seen. I would have inspected it further but it was making my eyes bleed. Then i got distracted by doing the puzzles. :)

The first time i did a puzzle there was about a 5 - 6 second delay between completion of the puzzle and the pop up saying puzzle was correct. Dial up can actually have a lot better consistency than a cable connection. Whether or not this has an effect on the solve time I can't say because that code is horrible to look at, however, just the fact that IE must be replaced (on windows machines at least) means that automatically non-IE will reflect a smaller subset of characteristics than the base population. There's a fancy math term for it, but I forget it. What the difference is due to is up for grabs and realistically no study will ever find out (like most things involving human behaviour) but i find these things useful for studying human behaviour and thinking patterns more than whatever conclusions the researchers come up with.

Re:Chrome solve time for all sizes

[Samantha Wright]

Well, certainly there are subsections of said market. :)

I'm pretty sure the code has been automatically generated and automatically minified. Hence the illegibility. I'm pretty sure though that when a puzzle is solved, the duration can't be more than a whole HTTP interaction (which, I guess, is a few more packets than just a single ping, so perhaps the combined metric of latency and bandwidth can have a substantial effect.)

The puzzle is poor

[paulatz]

TI be honest, I've just had a go at the "puzzle" (I suspect having it on slashdot was the main post of all this "statistics"). It is just poor, like sudoku, which I already find boring, but worst. It is harder to find out what the rules are that to actually solve it. And the website looks really amateurish. Really, do not waste you time with this puzzle, no matter which browser you use. BTW, I personally use opera.

Re:The puzzle is poor

[Jmc23]

Really, there's written instructions as well as a video explaining how to solve it right there in big letters. They're actually quite fun as well,

Re:The puzzle is poor

[paulatz]

You are probably one of the 5 persons in world who have played that puzzle before this slashdot commercial. I wonder if you are the one using chrome, or one of the 2 using IE.

Re:The puzzle is poor

[Jmc23]

Never played it before in my life. The instructions do require more intelligence than the puzzles but I guess that's really only a problem when you come from a society that has instructions on toothpicks and matches.

IQ correlates with motivation

[Colin Smith]

http://theness.com/neurologicablog/index.php/iq-and-motivation/

So what the guy is really saying is that Chrome users are obsessive compulsives and I.E. users are normal.

Re:IQ correlates with motivation

[Mistlefoot]

People at work on locked down computers used IE and kept getting "distracted" and going back to work. Sometimes they got busy enough to not finish the puzzle.
I might play Sudoku at work on a break and get distracted, never coming back (on IE). I play the same game at home on Chrome and quickly finish as my focus is there.

Statistically, that makes me "stupid" at work and "smart" at home. Don't let my boss find out!

Re:Is this just a measure of browser performance?

[mooingyak]

Yeah, my first thought was that maybe his site causes IE to crash sometimes, which would look like an abandoned game.

Re:Is this just a measure of browser performance?

[RobbieThe1st]

To be honest, it really depends on the page - for me, with a slow internet connection(1.5mbps), FF loads it faster than chrome... if it's /not/ HTTPS. If it's encrypted, Chrome loads it faster, especially if the site's under massive load(Like Ebay when the last HP Firesale happened)
This is on Linux, with a 3.2ghz quad-core, 4GB-of-ram hulk of a system, so it's not system-performance dependant or being starved for ram.

Re:I'd rather have HIV than use any Google product

[bhagwad]

I'd like to see you actually act on your claims :)

Re:Is this just a measure of browser performance?

[dr_blurb]

The clock is started once the page is fully loaded. The page itself is _very_ light on CPU, so any browser on a Pentium 4 or up should work just as well.

Yawn, who cares.

[xaoslaad]

If I use my Mac I use Safari. If I use my Fedora laptop I use Firefox. If use my Windows PC I use IE.

Any one of them works fine for me. If I can look at web pages and Bookmark/Favorite things it does 99.99% of what I want. I keep all my systems up to date, and run active AV of Windows. I'm not in the habit of viewing a wide range of shady web sites either. To top it off I can't think of a site I use that is not compatible with all three. And it is enough of a headache keeping 3 different systems up to date (nevermind the add virtual machines) without downloading extra browsers and making sure they're up to date separately.

Basically, who cares what browser you use. I doubt it defines you, me, or anyone else.

Re:Yawn, who cares.

[travbrad]

Keeping Chrome/IE "up to date" is pretty darn easy. If they are installed and connected to the internet, they are always up to date.

Another take, from a Firefox user

[93 Escort Wagon]

The number of abandoned puzzles (started but never finished) was also significantly higher for IE users.

Or, perhaps, IE users were more likely to have a life away from their computer. Maybe they abandoned the puzzle because they had to get ready for one of those "dates" - something Chrome users may have read about on Wikipedia.

Re:I'd rather have HIV than use any Google product

[K. S. Kyosuke]

So, you're still stuck with Altavista, old pal? ;)

Correlation, casuation, etc...

[slasho81]

"Chrome users are best with numbers." is a phrasing that indicates causation. This is Slashdot, so no need to remind everyone that correlation does not imply causation, right?

Re:Correlation, casuation, etc...

[rrohbeck]

The causation probably is: More educated or intelligent people have learned about Chrome and have switched.
The default browser on the most widespread system is always the one that will have the least sophisticated users.

Re:Correlation, casuation, etc...

[zill]

"Chrome usage is correlated with Calcudoku proficiency"

Re:Correlation, casuation, etc...

[jones_supa]

Yes, this is what I was thinking too.

Re:Correlation, casuation, etc...

[dr_blurb]

The causation probably is: More educated or intelligent people have learned about Chrome and have switched. The default browser on the most widespread system is always the one that will have the least sophisticated users.

Yes, that's what I'm thinking too. Here's the relevant bit from the conclusion of the paper:

In summary, based on the findings that Chrome users solve Calcudoku number puzzles the fastest, and that IE users give up on solving them the most, it appears that Chrome users have the highest numerical intelligence, followed by Firefox users, then by Internet Explorer users. Note that it does not follow that using Chrome makes you smarter, for example ("correlation does not imply causation"). Also, we can only speculate about the causes of the differences: perhaps Chrome is the browser of choice for more technically inclined people, who tend to have better number skills. And maybe because IE is the default browser for Windows, people who do not choose a different browser possibly are less technically skilled.

And the OPERA users....

[Anonymous Coward]

were so far off the scale that they had to be left out, otherwise there would be no apparent difference between all the others....

Re:And the OPERA users....

[Anonymous Coward]

No, the opera users, all 5 of them, were not included in this sample set, because they were doing this back in 2003, then they moved on to much more advanced games.

Re:I'd rather have HIV than use any Google product

[lucm]

So, you're still stuck with Altavista, old pal? ;)

Until altavista has proven to be reliable, I'll keep using Excite.

Proof that IE is stupid.... ??

[Tim12s]

Proof that IE is stupid.... ??

Re:I'd rather have HIV than use any Google product

[Yvan256]

Webcrawler forever!

Re:look at it another way

[dmesg0]

Are members of MSA (Microsoft Shills Anonymous) even allowed on Slashdot?

Re:Yes! Enough pussyfooting!

[zippthorne]

So... Now that you've gotten past the span filter, what do you have to say?

Re:Is this just a measure of browser performance?

[Billly Gates]

I notice variations in performance between win32 and Unix counterparts too. FF is slow as hell on my old laptop in Fedora. It runs much better under Windows 7. Part of me wonders if the Linux version was using the intel compiler? I have not run it recently but that was the case with FF 3.6 and 4.0. Only Chrome was usable in that operating system.

IE 9 seems to perform best on newer systems that can take advantage of GPU acceleration which Chrome is now catching up to in that area. All the major browsers today are good and IE is not that crappy browser it once was if you use Windows 7.

Need more details

[ceoyoyo]

If you're going to write a paper, put the relevant details in. What kind of statistical tests did you do? What correction for multiple comparisons did you do? What are the actual p-values you obtained, for each test? Are the distributions of your data normal? Do they meet the assumptions of your test?

Re:Need more details

[dr_blurb]

Most of this info is in fact in the paper.

The actual p-values I can put in an updated version (they were all less than 5%).

Re:Need more details

[ceoyoyo]

Yes, I see, not reading it on my phone, you do have a little section on the statistical test buried in the fourth part of the first results section.

You don't mention correction for multiple tests, and you did quite a few. If you included p-values (which is why it's important to do so), the reader could estimate whether your results are actually significant or not. If your p-values are not quite a bit smaller than 0.05 then your results are not actually significant.

Re:Need more details

[HarrySquatter]

The problem is that even if his results were statistically significant it doesn't necessarily mean he has found a meaningful result for the population at large. You can get statistically significant results which have no practical importance. The fact that there is no mention of the sampling error rate or any other meaningful data in which one can use to derive the practical significance of his data is quite telling on how this is a poorly-done study.

Re:Need more details

[ceoyoyo]

There are always study design and interpretation issues, especially with convenience samples, but if you don't have significant results to start with, there's nothing more to discuss.

Re:Is this just a measure of browser performance?

[dr_blurb]

Possibly, but my guess is that I would have had complaints from people.

Also note that this was data over two years, and I'm only using it from people who've successfully completed at least 10 timed puzzles of each size.

Re:Is this just a measure of browser performance?

[K. S. Kyosuke]

HP Firesale

That would be a great name for a SAN solution or something. I had to think quite hard to get what that it actually was something else. ;)

Re:look at it another way

[lucm]

Chrome users are the type to have a sufficiently narrow set of abilities that they can solve silly puzzles but they can't conceive of a problem with one large corporation collecting huge amounts of data about people across the world.

It's sorta like seeing that IQ correlates with financial success, where financial success by definition requires a willingness to amass financial wealth and mostly depends on high skill in a very narrow set of abilities.

Could you say "narrow set of abilities" once more, I'm really getting in a kinky mood

Re:Yes! Enough pussyfooting!

[MLease]

Aw, damn it; he must have said something smart that made the spam filter pay attention to him again!

And...

[no1home]

Those of us who use all three (IE 'cause I have to at work) are confused.

Regression to the mean ?

[wrencherd]

Doesn't this more likely mean that there are just a lot more people using IE than Chrome and so their average is going to be closer to the mean of the greater population?

On the other hand, I don't use Chrome, so my maths may not be as high good as my Englishin' and grammarin' is.

Re:Regression to the mean ?

[dr_blurb]

Doesn't this more likely mean that there are just a lot more people using IE than Chrome and so their average is going to be closer to the mean of the greater population?

From the paper, these were the browser shares by the end of 2011:

At the end of 2011, browser shares were 39% for IE, 38% for Firefox, 27% for Chrome,
and 11% for Safari (the version breakdown for IE was 1% for IE 6.0, 13% for 7.0,
53% for 8.0, and 33% for 9.0).

Re:Is this just a measure of browser performance?

[mooingyak]

Fair enough. That certainly reduces the likelihood of my theory.

tired

[Mariomario]

Tired of these "studies" that shows better apples uses a certain orange. Personally, I use Firefox and IE and find no difference between them. Sometimes something don't work on one of them, so I do it on the other. Considering IE comes as default on windows. Studies like this is like saying "Players who play games with default settings are stupid, and players who edit the settings are smart." Just because you have not found a need to change the settings dose not automatically mean your stupid.

In my experience

[Sycraft-fu]

IE is far more stable than Firefox. Now that is a little skewed, since FF is my normal browser. However FF does piss me off a fair bit by blowing up. When FF start to have problems with some content, I fire up IE and it handles everything no troubles. Of course this is all anecdotal, but then I've seen no evidence of IE being super crashy at work (we have some users who like it).

I think it is more MS haters wanting IE to be bad. They are worried IE might end up being a good browser and so hate on it.

Re:In my experience

[kanweg]

"When FF start to have problems with some content, I fire up IE and it handles everything no troubles."

That is self-selecting. When a page doesn't display correctly, it was probably tested for another browser.

Bert
Who uses FF when Safari has trouble displaying stuff, and finds that FF displays it fine.

/sigh

[Guppy06]

a site were you can solve Calcudoku number puzzles.

Ahem.

And publishing your "paper" on your own website doesn't make it peer-reviewed either.

Re:/sigh

[dr_blurb]

Well, I'm reporting on statistics of people solving number puzzles, this was not a test advertised as "please come do Calcudoku puzzles for a scientific test".

And publishing your "paper" on your own website doesn't make it peer-reviewed either.

I thought about submitting it somewhere, but then I'd have a couple of 2-3 paragraph reviews after 6 months, and it maybe would have been published after another 6.

This way the Slashdot crowd are the peer-reviewers :-), and it's been helpful: I need to add actual p-values, sampling errors, look into possible problems with multiple comparisons, etc...

Re:/sigh

[gl4ss]

well.. slashdot is the peer review.

welcome to future, hypnotoad at 11.

Version numbers

[mishu2065]

Firefox: 10
IE: 9
Chrome: 17.0.963.56

I can see where Chrome users get their practice with numbers.

So?

[hism]

So am I expected to move to Chrome now so I can be part of some exclusive club to validate my IQ? Or move away from IE so people don't think I'm stupid? Maybe they should have called this study, "Chrome Users Are Most Insecure?"

highly suspect on the MSIE times

[ChipMonk]

MSIE's download progress bars lie 99% of the time, and just make something up the other 17%.

Can we trust the reported times for MSIE users to work these calcudoku puzzles?

I love non-concluding stats

[holophrastic]

ok, so those who are good with numbers prefer chrome. those who are good in math prefer star wars over women. therefore, prefering star wars over women is better just like prefering chrome over IE is better. wait.

smart people make fewer mistakes, but those mistakes tend to be longer.

...and this is news because?

[Kplx138]

I love the fact slashdot has a related story link about how AptiQuant browser/IQ study was likely a hoax.

Also I want to submit my own dubious logic story:
I use Firefox and I don't have cancer, therefore Firefox cures cancer!

Please add error bars on the graph

[ogrisel]

Would you be so kind and add error bars on your histograms? Either for the standard error of the mean or percentiles or confidence intervals. Boostrapping can make this very easy to implement.

Absolutely Dumb Premise

[alphatel]

Almost half the world is using Chrome. So almost half the world is better at numbers than the other half?

self-selection

[minstrelmike]

Can you say self-selection?

Computers from stores come with I/E or Safari.
Anyone with Chrome or Mozilla had to at least click something and do an install.
I agree it ain't much but it is something and i suspect that is what got measured by the analysis.

Re:Is this just a measure of browser performance?

[RobbieThe1st]

Hm... Interesting you should mention that:
On Linux, FF offers HW acceleration... If you have the Nvidia blob driver installed. Just about everything else is blacklisted by default.
Stock FF seems to be compiled using the Intel compiler, but on /modern/ AMD CPUs that shouldn't cause issues - it's only on older generation AMD that less effecient code is run.
That being said, I've been using Iceweasel - debian's unbranded firefox - which appears to be compiled with GCC, and it appears to be slightly faster... But that may just be subjective.

As far as IE9 goes, I still don't like it on win32. It's most definitely slower, if only because (my) FF has NoScript blocking Flash from running by default, speeding things up a /lot/.

Re:Chrome users also have bigger dicks!

[cupantae]

I would guess that something like uzbl would have users with the biggest dicks on average. One female user pulls down the average as much as one 12-inch user pulls it up. You need to find the browser with the highest male-to-female ratio.

(why the fuck am I writing this)

No that is Opera users

[SmallFurryCreature]

Opera users are not shown in the graph because their superior minds instantly analyzed all the puzzles and their solution in their mind so they had no need to "play" them through a browser interface.

Meanwhile lynx users just outsourced the puzzle solving to their legion of controlled IE user bots.

Or it could be ...

[RivenAleem]

https://chrome.google.com/webstore/detail/bneajlpihgbinpbljjcadddjljghilho

https://addons.mozilla.org/en-US/firefox/addon/sudoku-helper/?src=search

Could be another answer. I don't see any mention as to whether it tracked what addons/extensions were in use at the time, but this is something that could easily be gamed.

Re:Please think about what you just suggested

[Samantha Wright]

Nah, they could easily make up for it in volume. We don't know that the IE users had the same regularity of patronage as the Chrome users. A few dedicated Chrome users might make up for a large number of IE users who drop by once and never return. I don't think it's reasonable to suggest there are people repeatedly visiting a puzzle site to continually not solve puzzles there, regardless of how bored they are!

Biased study...

[kubusja]

Playing on chrome is more convenient - you can play with keyboard ergo experienced players pick Chrome ergo Chrome users results are better... I played it some time ago as FF user but when I noticed Chrome version is better I switched to it.