Any research of this type has extraneous variables. Sadly, it's just a part of research. However, you don't need detailed life histories. You need a reasonable randomization of cases... I'm giving them that.
As for the SD, I can't be certain, but I'm going to guess that if they managed to dichotomize the variables well, that the deviations should be rather small... smaller on the "no caffeine side" would be my guess, but without looking at the paper, who can tell. So, no, I can't do the actual statistical analysis, but I would be greatly surprised if it wasn't... after all, this is the solution the researchers are claiming--they'd be fools to publish a study WITHOUT significant differences.
You're engaging in the typical anti-research fallacy... that because there is another possible explanation, the research at hand is not useful. In that case, we must give up research entirely. I have no doubt that the researchers would claim that this finding first needs to be replicated and further supported... but there is ALWAYS another possible explanation. That's how scientific paradigms evolve. BUT, finding a statistically significant difference DOES merit further research. Not derision from individuls who claim to be well educated peers.
Your "main point" is irrelevant. That's why I ignored it. Why? Although those external elements MAY influence the relationship, it influences BOTH SIDES. Consequently, with reasonably randomized samples, these effects cancel themselves out. Period. So, we can thereby assume that the observed effects are due to the manipulation... or, a complex of variables related to the manipulation (emergent phenomenon, variables intrinsically--one might say confounded with--the manipulated variable).
This is the scientific method, at least as applied to medical and human subjects research.
As for talking down to peers, no, I don't. Of course, you haven't shown that you actually are a peer. Perhaps you are. But, considering the basic elements of human subjects research you missed, I never could have told from your message.
Don't gloat about your statistical knowledge so quickly.
Cohen (1990) points to a number of factors that influence a researcher's ability to detect a statistically significant difference; sample size is simply the easiest to manipulate and, therefore, the only one anyone ever remembers.
What is probably more important in this case is the magnitude of the effect, known simply as the effect size, defined by the means of the two sample groups divided by the total sample standard deviation. Considering the mean differences were 74 and 200mg a piece, we're looking at a fairly significant difference here--likely a quite robust effect size.
More glaring in your analysis the the fact that it's NOT a sample of 54. It's 54 subjects PER GROUP. That's 108 subjects. Running what we know through a simple power analysis program--the sort of program used to determine if a statistically signifcant effect actually means anything, we obtain a power of.95. The threshold for an acceptable power is.80 (higher is better, BTW).
The results actually look quite good, if you know what the hell you're talking about.
Slashdot Mirror
Any research of this type has extraneous variables. Sadly, it's just a part of research. However, you don't need detailed life histories. You need a reasonable randomization of cases... I'm giving them that.
As for the SD, I can't be certain, but I'm going to guess that if they managed to dichotomize the variables well, that the deviations should be rather small... smaller on the "no caffeine side" would be my guess, but without looking at the paper, who can tell. So, no, I can't do the actual statistical analysis, but I would be greatly surprised if it wasn't... after all, this is the solution the researchers are claiming--they'd be fools to publish a study WITHOUT significant differences.
You're engaging in the typical anti-research fallacy... that because there is another possible explanation, the research at hand is not useful. In that case, we must give up research entirely. I have no doubt that the researchers would claim that this finding first needs to be replicated and further supported... but there is ALWAYS another possible explanation. That's how scientific paradigms evolve. BUT, finding a statistically significant difference DOES merit further research. Not derision from individuls who claim to be well educated peers.
Your "main point" is irrelevant. That's why I ignored it. Why? Although those external elements MAY influence the relationship, it influences BOTH SIDES. Consequently, with reasonably randomized samples, these effects cancel themselves out. Period. So, we can thereby assume that the observed effects are due to the manipulation... or, a complex of variables related to the manipulation (emergent phenomenon, variables intrinsically--one might say confounded with--the manipulated variable).
This is the scientific method, at least as applied to medical and human subjects research.
As for talking down to peers, no, I don't. Of course, you haven't shown that you actually are a peer. Perhaps you are. But, considering the basic elements of human subjects research you missed, I never could have told from your message.
My apologies.
Cohen (1990) points to a number of factors that influence a researcher's ability to detect a statistically significant difference; sample size is simply the easiest to manipulate and, therefore, the only one anyone ever remembers.
What is probably more important in this case is the magnitude of the effect, known simply as the effect size, defined by the means of the two sample groups divided by the total sample standard deviation. Considering the mean differences were 74 and 200mg a piece, we're looking at a fairly significant difference here--likely a quite robust effect size.
More glaring in your analysis the the fact that it's NOT a sample of 54. It's 54 subjects PER GROUP. That's 108 subjects. Running what we know through a simple power analysis program--the sort of program used to determine if a statistically signifcant effect actually means anything, we obtain a power of .95. The threshold for an acceptable power is .80 (higher is better, BTW).
The results actually look quite good, if you know what the hell you're talking about.