Slashdot Mirror
Science and the Shortcomings of Statistics
Kilrah_il writes "The linked article provides a short summary of the problems scientists have with statistics. As an intern, I see it many times: Doctors do lots of research but don't have a clue when it comes to statistics — and in the social science area, it's even worse. From the article: 'Even when performed correctly, statistical tests are widely misunderstood and frequently misinterpreted. As a result, countless conclusions in the scientific literature are erroneous, and tests of medical dangers or treatments are often contradictory and confusing.'"
In other news math may not lie but people still can, all the honesty and good statistics in the world doesnt help end-user stupidity, and there are statistically two popes per square kilometer in the vatican.
A bullet may have your name on it but splash damage is addressed "To whom it may concern."
How do you figure that? My latest calculations placed it at 70% [Note: Error +/- 10%].
It's not just statistics that people have a problem with...
-- Braden's law of data: All data spends some of its lifetime in an excel spreadsheet.
She was like a little ball of sunshine.
As for statistics, does this really surprise anyone in a time when net polls are being reported as hard news?
My doctor was explaining to me that my blood sugar readings should not have a standard deviation of more than 1/3rd of the average blood sugar reading. Just to test if he knew what it meant, I asked him what a standard deviation was. Oh the fun when he tried to bullshit his way out of that one! He eventually told me that when I plot my data in Excel I can ask it to give me statistics on the column and it would mention what the standard deviation value was. But when I pressed on and asked him what a standard deviation is, he shooed me off and told me to go look it up. Never did he confess that he had no clue.
Statistics is terrible for proving things, but rather good at disproving them.
How to Lie with Statistics by Darrell Huff. Recommended reading.
Do not mock my vision of impractical footwear
Actually, it's a tough situation. There is no real life experimental data can 100% fit the assumptions of commonly used statistical models. Real life data is messy. There is some degree of simplification. In addition, resorting to whiz-bang fancy methods that "fit" the real data may not be easily interpretable. Ease of result interpretability is what medical scientists want. There are other issues as well, such as computing time, equations derivability, etc.
In addition, many many medical scientists use statistics as a tool to filter things (e.g. candidate genes, target enzymes, treatments, etc). In this case, 100% accuracy is not really important. Once the scientists narrow down the genes, they can test the validity directly in either test animals or real people.
--
Error 500: Internal sig error
Our company six sigma training included two weeks of collecting and analyzing data with a stats package. I got enough experience to even train me how to use the program. I can still do a few things that come up regularly. Probably the best thing to come out of six sigma (for me at least).
The entire article can be summed up by the tiresome cliche "correlation != causation".
That misses a lot of the problem. For example, observer bias through poor statistical design of the experiment or throwing out data can cause the appearance of correlation or causation in data that isn't so.
As a doctor myself, I feel I should add my $0.02...
Throughout med school we had the odd scattered lecture on statistics, and later when reading papers I used to skim over most of the maths just to look for the P value at the end (one representation of how statistically significant a result is).
However, I then took a formal stats course and was amazed at how little I understood - Monte Carlo techniques, Markov models, and even something as trivial yet important as the difference between a parametric versus a non-parametric test.
And then it struck me - most of the research I had read had applied parametric statistical tests to their data - that it, the researchers made an assumption that the underlying distribution of results would fall on a normal curve. Yet this simple assumption may be all it takes to skew the data when they should have chosen a non-parametric test instead.
So yes, stats are vitally important, badly taught, and focus too much on the maths rather than the concepts. Remember that we're doctors, not mathematicians - the last set of sums I did were in high school. If I need to analyse data, I'll probably plug it into SPSS - although now with my eyes open.
-Nano.
Funny Stat correlation: http://www.seanbonner.com/blog/archives/piratesarecool.jpg
Troll is not a replacement for I disagree.
The entire article can be summed up by the tiresome cliche "correlation != causation"...
The logical fallacy is called "post hoc, ergo propter hoc" - "after this, therefore because of this".
Sort of like - I get a headache every time someone turns on the television, therefore headaches are caused by the television.
Oh, hang on...
Do not mock my vision of impractical footwear
When I did my BA in psychology Statistics was the core of the degree. It was the one subject that you could not escape and had to take for the full year every year of the degree. I heard later that the Psychology department at that Uni was sometimes disparagingly described as teaching Rats and Stats psychology.
this is why people now consider master's degrees to be the equivalent of a high school diploma.
if you want real fun, take the average master's degree idiot and start having them manually add fractions without changing them to decimals. such as adding a bunch of measurements off a tape measure together.
hilarity ensues....................
Statistical methods are typically developed for fairly specific mathematical models. A practitioner may error greatly by using a statistical method outside of its intended purview. For example, many statistical tests assume that different groups of observations are independent or correlated in a specific way. If this isn't true then the resulting inferences can be very inaccurate.
Unfortunately the spread of "easy to use" statistical software is making this problem worse. Many scientists just enter their data and select an analysis from a drop-down menu - thinking that just because their data is in the right format that the results will accurate. It would be better if people had to think about what analysis to choose rather than just treating the choice of a test like the choice of a visual effect in photoshop.
IAAS (statistician), for what it's worth...
that there are only 3 kinds of scientists: those that are good at math and those that aren't.
Game: Player 'Donald J Trump' now has AI skill level 'experimental'.
science is not in the bussiness of proof
So what is it in the business of?
I think your example would be more persuasive if it involved algebra, though.
In other news math may not lie but people still can...
Usually (in science at least) it's not even a matter of lying. Part of the problem is that the multi-headed monster that statistics has become has a tendency to lead people to over-use numerical "answers" vomited up by stats packages, without really understanding what they are for, or how to interpret them.
Statistics are very useful for predicting certain things, but all too often they are submitted as "proof" of a given condition, which is dangerous. Sometimes we need to throw away statistics and start applying common sense.
Does that mean that we should send people who know what they're doing to sort through results and draw more meaningful conclusions? Or just rerun the tests?
This seems obvious, so please don't waste mod points here, people who know what they're actually talking about will probably chime in.
-
It's perfectly reasonable that someone use a calculator for sales tax (if an exact answer is desired).
Also, sales tax is multiplication - not algebra.
And what are we supposed to make of your post where your supposed case for people not knowing algebra has nothing to do with algebra?
They're all buncha crap, and I say this with 95% confidence interval, or sum such stat shit that I wish I can remember.
Fuck systemd. Fuck Redhat. Fuck Soylent, too. Wait, scratch the last one.
You are a jerk.
You are insulting your sister because she is bad at mental math? It is a skill; one not required for extensive knowledge of the social sciences. Additionally, maybe if sales tax is simple in your state like 10%, but where I live it is 4.5% which is not always easy to get exactly right in your head.
I had a roommate who was brilliant,funny, a singer and an artist, and yet, he couldn't calculate tip to save his life, but I don't certainly hold that against him.
> countless conclusions in the scientific literature are erroneous
Number of Publications: Finite
Number of Conclusions: Finite
Time taken to count erroneous conclusions: Finite
Countless Conclusions? I don't think so!
A large but unspecified number of conclusions in the scientific literature are erroneous: Not so compelling
One of the best articles I've seen on stats (and their misuse). I'm taking a data analysis course at the moment and I've spent at least a dozen hours simply computing confidence intervals, testing the null hypothesis, and determining significance. It really has changed how I view statistics because it keeps pounding in these very key but oft-ignored principles.
"Everything is linear if plotted log-log with a fat magic marker."
It is not a shortcoming of statistics that other people, like various scientists who aren't statisticians, don't know how to use or properly interpret statistics. It is a shortcoming of their knowledge.
It is not a shortcoming of the Copenhagen interpretation of quantum mechanics or the Chicago school of economics if I don't understand or know how to correctly interpret their results. It is my shortcoming and fault for not knowing enough to connect the dots.
I do statistical research some of that is through interacting with researchers in the biosciences. Often when I go to talk to a researcher and ask them if they could use some statistical or mathematical or computational assistance with their research it has almost always been a fruitful starting point to long conversations and getting into the research. Now sometimes it was simply a matter of looking at their F-test results or ANOVA scores and telling them what it meant (like with a regression model relating proportions of certain characteristics between taxa), more useful interactions for me often mean working on new algorithms or estimators or working with fitting a model from their empirical data because there isn't a reliable standard model to work off of (like intergenic distance between genes in an operon) that kind of challenge makes less engaging work worth the hassle. Maybe I'm odd because I've worked hard to have a good background in both statistics and biology, but I shouldn't be.
Although here is an observation that perhaps supports some of the intent of the article from my own experience. I was speaking with a biology graduate student and it came up that they had a biostatistics course in the department. Of course as a statistician my mind goes towards survival function, failure rate, life tables, censored data, bioassy, epidemiology, microarrays, clincal trials, topics along those lines. It turned out their course focused z tests, t tests, f tests, confidence intervals, point predictions, least squares regression, multiple regression, ANOVA, and things along these lines just with simulated problems in a lab setting. That is not necessarily a bad thing, but much of the core math was under played or missing like model assumptions and alternate formulations or things like dummy variables. The worst part was that even though they were doing well with the class they had no confidence in actually using the statistics and didn't understand how to interpret the meaning of something like a confidence interval, they knew how to calculate one, but it wasn't clear what it actually meant to them.
The corollary to the notion in the summary I'd rant and claim is that scientists overall have less than desirable skills in mathematics, statistics, and computation than those who studied those disciplines principally and that's hurting science. However many in those three disciplines really know little beyond basic results in any of the sciences which hurts the applicability of these mathematical fields to the sciences and likely hurt our ability to develop certain types of discipline specific results that can be generalized from work in application problems.
In either case whether you're a typical scientist or a typical math/stat/comp person in order to become proficient enough in the other areas it requires going an awfully long out of the way compared to any counterpart who simply does not care and goes straight through as many before have. While in some areas of research on either side it is no problem to do as has been done and not further knowledge into those other areas. Increasingly results that have the highest levels of impact are coming more and more from truly interdisciplinary research. In order to further encourage that for those who are interested in such fields (aside from making more clear what areas in any of the fields fringe to such interdisciplinary work) we need more incentive to study more than one field and/or better ways of enabling fruitful cooperation between the camps.
Evidence. Big difference.
The clearest discussion of the logic of probability reasoning I know of is E.T. Jaynes' Probability Theory: The Logic of Science. (Cambridge University Press). Many of Jaynes' excellent papers on statistics are downloadable from http://bayes.wustl.edu/etj/etj.html.
Science is in the business of probably knowledge. So they really need to improve their probability and statistics knowledge.
I don't have to be a statistician to know that the above post is 97% bullshit.
Yes, it's rarely mentioned that causation implies correlation.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
+/- 50%*.
*confidence interval=100%
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Arithmetic is not algebra. Arithmetic is "What's 10% of $24.45?" Algebra would be "On a given day i, John sells n_i apples to Peter at x_i dollars each, and this price includes sales tax which is a constant proportion 0p1. Let x_1= .. x_2= ... ... What is the tax on the apples sold on days 1 to 12 inclusive?"
The difference is 24.45 . 10/100 versus p\sum_{i=1}^{12} n_ix_i. Granted, there isn't much difference there really, but come on, there is a time and a place for everything, calculators included.
In reading a couple of these types of articles recently I've noticed that the articles always talk about this being a problem across all journals, but only seem to mention a couple of different disciplines - medicine usually chief among them. Has anyone heard/read anything naming a hard science (e.g. chemistry or physics) as full of bad stats? My hunch is that this happens most often in medicine because you have the combination of controlling for a lot of variables as well as inadequate mathematics training.
It's a troll because it implies scientists don't know about those things.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
I had a friend who is doing a PhD in maths and he can't calculate basic arithmetic to save his life. It's a redundant skill for pretty much everyone.
Insanity: voting in the same two parties over and over again and expecting different results
You think I'm full of it? Wait till you hear professors at seminars, making up whatever theories they like. I've witnessed professors from household-name schools acting like this.
From TFA:
One has to wonder, though: how much of that is due to misuse of statistics and how much is because it's paid research expected to get certain results in favour of those paying for the research?
And 77.335% of all statistics claim more accuracy than their expected deviation warrants.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Given that sales tax varies based on type of purchase in some states, and is weird numbers like 6.5% in others, it can vary quite a lot. And oh, my dear lord, try dealing with "valua-added-tax" in Europe....
On the other hand, plenty of very smart physicists, mathematicians, etc. have approached medicine spouting much the same rhetoric as you. They very quickly became embarassed when they tried to apply their fanciful theories to medicine. If you have a better idea on how to tell apart correlation from causation in a medical context, let them know.
87.24% of statistics are made up.
[Insert pithy quote here]
No it can't, what?
"These are subtler issues than the true-but-trivial—and tiresome—cliché you refer to."
Actually the subtler issue here has nothing to do with statistics, they are implying peer-review does not work.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
I had taken a stats class in undergrad... did not really pay attention as I thought it had no use. While getting my masters I was obligated to take an advance statistics class. Going in, for the life of me I thought it would be a waste - it was the best class I ever took. I was able to use it in my job almost every week if not more ( most of the other classes were theoretical at best and had no real world application ). Ten years later, I still rely on things I learned in that class. Statistics should be mandatory for all in college regardless of major because it can be used for so many things.
And lots of others. It then suggests Bayesian reasoning as an alternative to traditional statistical tests.
Most post-PhD scientists are aware of the common mistakes, but being aware that we make mistakes doesn't necessarily stop us from making them. If you chose a random set of conference proceedings, it is almost certain you will find at least one paper (and I suspect usually a dozen or more) that have statistical mistakes in them.
Unfortunately, it is hard to break a viscous cycle. The high viscosity makes it easy to get stuck.
"FDA staff reviewers expressed concern about the number of patients who were left out of the study because they died."
Warning Signs in Experimental Design and Interpretation
http://norvig.com/experiment-design.html
He does an excellent job of describing and illustrating common research mistakes, statistical and otherwise.
Build a man a fire, he's warm for one night. Set him on fire, and he's warm for the rest of his life.
It's in the bussiness of providing the best explaination for the available evidence. Proof is confined to axiomatic systems such as maths and generally you can't prove the axioms of axiomatic systems. Science is not an axiomatic system. See epistomology for further details.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
What is missing in this discussion is systematic error, which is often very large and often dwarfs the analyzed random error or even the result itself. Systematic error is frequently a basic problem in biological research and in emerging technologies with crude tools and poorly understood cofactors. The human factor can hugely inflate systematic errors where legal, marketing or politics are involved. The systematic error may not be uncovered for years or decades, if ever.
One can design "tests" that are beautifully reproducible and precise, but absolutely, and deliberately, absurdly wrong. And get away with it, nay, be be rewarded handsomely as a salable skill. It happens behind the scenes. I have direct experience in science and engineering where politics have butted in, but I see this as more common in medicine, pharmaceuticals and the medical journals. Multiple, blatant design and interpretation errors in any single article that are extremely hard to assign to mere stupidity and/or ignorance, that involve authors with clear conflicts of interest to victimize cheap (defenseless) generic drugs and supplements, and to promote their product.
How blatant does it get? I have industrial experience where a big name, big $ university consultant was given free reign to do a "political assignment". On a literature comparison of two materials' figure of merit, even after fair warning, he missed reality by 9+ ORDERS of magnitude, over a billion fold by avoiding data in equal test environments. The results of internally published, correct tests were later deliberately ignored. This did eventually lead to his backers' catastrophic failure and his dismissal. Millions wasted. This is one of dozens of such situations I've seen in intercorporate wars with NoAm and European companies (no names here). The pharmaceutical and medicine situation appears blatantly worse in terms of number of fundamental test errors in a given high profile paper, resultant damage and duration. But big profits are made!
Yes, it's rarely mentioned that causation implies correlation.
Interestingly, I have observed a correlation between people who cite that "correlation != causation" and those who ignore "causation implies correlation" in their arguments.
Lars T.
To the guy who modded me down from perfect to terrible Karma - Apple haters still suck
Statistics is changing slowly (mostly because computers and R make non-classical statistics more practical) but the way it's taught still leads to problems.
... and those that may or may not be good at math. :P
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
Wait, I'm sorry, biologists make the lesser contribution to medicine when compared to physicists and engineers? You do realize that all of medicine is biology, right?
People in the real Sciences would have been forced to take enough Mathematics and/or Statistics to be able to properly interpret Statistics.
You would think so, if you've never worked with Real Scientists. Most biologists and chemists (can't speak to the other ones) know just enough statistics to get by, and make exactly the kinds of mistakes TFA is describing - there's only so much you can "force" people to learn.
Then there's the whole not being able to tell the difference between causation and correlation. I could go on.
You seriously think this is a common problem in biomedical research? I mean the actual research, not the media spin on it.
sic transit gloria mundi
http://www.americanscientist.org/science/pub/everything-is-dangerous-a-controversy
These posts express my own personal views, not those of my employer
Actually the subtler issue here has nothing to do with statistics, they are implying peer-review does not work.
"Peer review" is another of the things that has been over-sold to the public. A science research group spends six months and a hundred thousand dollars conducting a research study using highly specialised equipement. They submit a paper to an academic conference or a small journal. It gets put out to review by three people who each spend about four hours reading it and reviewing it, and who usually do not have access to the equipment or the original data that was used in the study. Do you really think we're likely to catch every mistake at review? We certainly can't check the stats (except for the most egregious errors) because we don't have the full data tables they analyzed.
Scientists actually accept that inevitably some incorrect results will be published. More often in the smaller conferences than in the most prestigious journals, but even the journals have to publish a retraction every now and then. We also accept that most studies are never repeated, and so the "objective repeatable experiment" is rarely really tested for being either objective or repeatable. However, science has long had the "many eyes" effect at work. There are hundreds of thousands of scientists reading papers and using them in our own experiments. If some theorised effect out there is wrong, usually we'll find out eventually.
A cow is actually a rather topologically interesting beast.
Nah, it's just a funny torus.
The ringing of the division bell has begun... -PF
So call them out. If you don't, you're just a part of the problem you describe.
So what is it in the business of?
Disproof.
sic transit gloria mundi
Research shows that 78% of all people who use the term "research shows" are just making s**t up.
Basically, the point of higher-level math is what can be done without a calculator. A basic example is binomial probability. Say someone rolls six dice. A computer could be used easily to find the probability that exactly three dice are 5's by counting how many outcomes there are (46656), and how many outcomes have exactly three 5's (2500). What math does is to create an equation (n C r) p^r q^(n-r) and prove it works for any values of n, r, p and q. In the example, n = 6, r = 3, p = 1/6, q = 5/6, and P = 625/11664. (Thank you, Maple)
Sure, when one has spent enough time working with numbers, they'll have an intuition as to whether an answer makes sense. But not too many can add a 7.25% sales tax to a $64.38 purchase in their head.
You are conflating repeatability with peer-review. Peer-review is the formalised, first cut of the "many eyes effect" and will usually pick up obvious flaws in statistical methodology. To be sure peer-review is not perfect but I do not think it's being "oversold to the public", in fact I think it's been quite the opposite recently, especially in the field of climate science.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
It's in the bussiness of providing the best explaination for the available evidence.
That sounds right, but isn't part of being the best explanation, that such an explanation is true? Establishing the truth of an explanation would fit into one of the commonly used meanings of "proof."
I don't think your restriction of proof to axiomatic systems coincides with the way that that word is usually used. Axiomatic systems allow for a "proof" in the mathematical sense, but other kinds of demonstrations of truth are also possible, and could reasonably be called "proof."
W. V. O. Quine would like a word.
Peer review is not about catching mistakes, although it can on occation. Peer review is about clear communication, such that the experiment can be repeated as identically as possible and that the readers can understand the authors justification for their conclusions. At least that's what every journal article I've read on the topic indicateded was the reason for the peer review processes creation. One of my advisors asked me about it on my written preliminary exam and I needed to do a lot of reading to be prepared for the oral exam. There were several different societies that claimed to have originated the idea, but no one claimed that the purpose was to catch mistakes, fabrications, or data manipulations.
Bureaucracy expands to meet the needs of the expanding bureaucracy.-Oscar Wilde
Yes, as critical intelectuals, we are able to look at ourselves in this critical manner. However, really successful people, e.g. lawyers, politicians, psychologists, & salespeople never have this drawback. I remember Ronald Reagan talking about various issues and being absolutely wrong. However, he said it with such conviction and determination that I had to go back and check the facts. But apparently, he never did.
Another time, I remember reading about using DNA to cross check previous serious crime conviction. Judges and politician refuse to open closed cases, because doing so undermines the fact that maybe the justice system might be quite faulty. Rather than worrying about incarcerating innociant people, the legal profession was more worried about protecting their own future revenue stream.
Now, salespeople, no matter how professional and honest they might seem, are taught to never let a sucker get an even break. Doctors too, are often taught that you should never allow the impression that you might be wrong to be formed in people's minds.
Last year, NOVA had a episode about the practice of performing lobotomies on mentally ill people. One part of the story focused on treating one of the Kennendy girls during the 1960s. The girl had definite problems. However, the real tragedy of the story was how Harvard and Johns Hopkins cream of the cream doctors turned a girl with an IQ of a little girl into that of a vegatable. Although there were no scientific cases of a lobotomy of curing anyone with her problems, the doctors went ahead and preformed the procedures anyway. Well, the biggest irony of this was if these were the best doctors that money can buy, I shutter to think what would happened to people in mental institutes for the indigent and politically unconnected run by doctors graduating from state universities and military institutes.
Yeah... everyone knows sales tax is multivariable calculus!
Interestingly, I have observed a correlation between people who cite that "correlation != causation" and those who ignore "causation implies correlation" in their arguments.
Ah yes, but can you suggest any causal relationship between those two observations?
Better to be despised for too anxious apprehensions, than ruined by too confident a security. --Edmund Burke
I have come to the same conclusion, "the best and the brightest do not go into medical school".
Interestingly, at the medical school I attended, during the graduation ceremony the PhDs are called up prior to the MDs. This of course implies more deference to the PhD degree. I wonder if this is standard everywhere...
The ability to math in your head (rapidly), and the ability to perform statstical analysis that is appropriate and interpret it in a way consistent with the limiations of the model/design used are not even remotely related. I cannot do math in my head quickly. I tend to drop a digit somewhere and come out with the wrong answer. However, I have extensive training in statistical design and interpretation, and can spot errors in both much easier than my peers. The skills involved, while both mathematical, are not necessarily connected.
Bureaucracy expands to meet the needs of the expanding bureaucracy.-Oscar Wilde
this is why people now consider master's degrees to be the equivalent of a high school diploma.
Who are these "people"? I am a research scientist and I don't know anyone who thinks of a MS as an equivalent to a HS diploma. Hell, as biased as I am (and I know I am), I don't even consider a humanities MA as an equivalent to a BS.
As I stated in response to FrozenGeek (the OP), the ability to add fractions and the skills acquired in the pursuit of a research MS degree are not connected. My step father is a carpenter and he can add fractions without even thinking about it. He deals with them every day. I OTOH, need to write out any remotely complicated math that I do if I want it done quickly and acurately. He would be the first to tell you that he doesn't know shit about higher math, while I am considered by my peers to be above par in my statistical knowledge and understanding (as indicated by their frequent visits to my desk for help with their statistical work).
Bureaucracy expands to meet the needs of the expanding bureaucracy.-Oscar Wilde
... and I noticed it is dated March 27. OK I guess it is when the magazine comes out but still it was a little ironic in this instance.
and statistics.
Seriously, the problem with statistics is that they can be manipulated to mean whatever the presenter wants. Taken out of context, which is how a lot of statistics are presented, enhances the problem. I wouldn't trust any statistic unless I can examine the data behind it.
Statistics are not inherently bad, but I think they are over-used in many areas and often present a purposefully distorted view of something. Statistics do not address causality.
Better yet, last night on "Mad Money with Jim Cramer", someone pointed out that Goldman-Sacks research recommended selling all assets of HOG, yet looking at insider activity of their holdings, GS increased its holdings of HOG from .5M to 4M. Cramer attributed this to different divisions of GS advocating different positions. However, I think most viewers thought that GS is trying to get everyone to sell its shares of HOG so that GS can get them on the cheap.
I have to say I agree with the OP. I got my Ph.D. in Animal Sciences from a University with a Vet School. Several of the courses I took were taught over in the Vet School and later I taught in a couple of classes attended by Vet Students. MS and Ph.D. students (ie scientists) are expected (and in turn expect) to learn concepts, apply critical thinking skills, and reason out problems from day one. Vet Students (the AnSci equivalent to Med Students) are expected to memorize facts for 2 to 2.5 years. There is no expectation of critical thinking, no reasoning out complex problems, no application of previously learned concepts to novel situations until year 3 in Vet School.
They are very highly trained, but the training they recieve is very different from the training I recieved. I very quickly started thinking of MD's and DVM's as biological mechanics by default. Nothing I've seen in my interactions with them has led me to believe that I am mistaken to think of the individuals that way until they proove otherwise on a case by case basis.
Bureaucracy expands to meet the needs of the expanding bureaucracy.-Oscar Wilde
Jesus, fucking, christ...I'm not even going to speed tomorrow after reading that.
I was specializing in methodology during my doctorate work and so had to not only have a good grasp on stats as performed but also able to at least estimate how well the analyses I was developing worked. We had a top notch stats professor who'd started in psychology and so ended up not only teaching all graduate courses for our department but also served as top level consultant for any and all of our projects. Since some of my work was in nonlinear phenomena and therefore stats, I spent many an hour trying to absorb everything he could offer.
When I'd gotten on top of the material, some of what I saw going on made me disturbed, angry and/or disgusted.
In EEG research it was common to go through an analysis system wherein one first does a test on all electrodes together to determine if there's a difference between conditions. Fine. But then to localize, one first divided the electrodes in half and tested left vs. right. Then one tested both left and right according to front and rear. And so on, until individual electrodes are compared. I as told this reduced false positives and retained power. I was told to do it in my dissertation. I was told who started this process. I wasn't told it was bullshit; I figured that out on my own. I looked up the reference. There was no mention of this process in the article. As is common when I tracked down such rituals, the article said to do what you could justify doing but to know what you could and could not justify due to your own ability. I also found an article that said such processes did not retain power nor reduce errors.
The stats prof pointed out that each collection of electrodes in each test was arbitrary. There was no reason that every possible combination should not be included in their ritual. A "real" result from the process should require that. I pointed out that our software localized electrical sources in the brain down to 1mm voxels (to work with fMRI data) making surface electrode analysis extraneous. I took these points and the articles back to the department and was told finally to "do what I had to" for my diss. I ended up using a nonlinear running t-test to analyze time series of signals in 2 msec windows and produced a 'movie' of dopamine effects on the frontal lobe across the first 20 msec post stimulus. Nobody on my committee could understand the analysis, but they all loved the movie. I didn't tell them I'd adapted the analysis technique used in fMRI because some of them had done fMRI research and thought they knew what they were doing. Had I had to explain my workings I'd have had to tell them they didn't understand what they were doing, and at that point I wanted to get done and get to my first job offer. NIH. Invited and non-competitive. They understood my work. Besides, by this time I'd already studied at Santa Fe Institute and had learned the difference between learning from people who knew more than I likely ever would and jumping through hoops for people who I'd already passed in ability.
I also saw colleagues doing fMRI work who had no clue they were pushing statistical testing so hard that due to the necessary correction factor they were trying to find individual data points with p values with up to 22 zeroes between dot and data, a certainty they could never realistically achieve, and a cut off level they'd never even consider trying to look at in any study where they knew at least some of what was going on. I've seen entire poster sessions at conferences on brain mapping where maybe 2 out of 200 could accurately and factually explain how their analysis worked (typically they worked with a biophysicist who could, but none of which understood the phenomenon under test well enough to describe it, meaning together they could produce results but not knowledge as they couldn't pass the latter back and forth between them).
And I've seen researchers who did understand fMRI and SPS (statistical probability mapping, the analysis technique used for fMRI). And they refused to use the technique for the reasons given. My boss at
"I may be synthetic, but I'm not stupid." -- Bishop 341-B
My favourite example of puncturing the "Real Scientists (tm)" who think they're above making these sorts of mistakes?
So You Think You Have a Power Law - Well Isn't That Special?
What part of "a well regulated militia" do you not understand?
I'm not talking about original intent, I'm talking about contempory practice, the first peer-review policy I looked at to check your assertion was the journal Nature. It doesn't say anything about clarity or repeatability, it appears to back up what I said, quoth the policy...
....[snip]...
"Nature journals receive many more submissions than they can publish. Therefore, we ask peer-reviewers to keep in mind that every paper that is accepted means that another good paper must be rejected. To be published in a Nature journal, a paper should meet four general criteria:
* Provides strong evidence for its conclusions.
* Novel (we do not consider meeting report abstracts and preprints on community servers to compromise novelty).
* Of extreme importance to scientists in the specific field.
* Ideally, interesting to researchers in other related disciplines."
"The editors then make a decision based on the reviewers' advice, from among several possibilities:
* Accept, with or without editorial revisions
* Invite the authors to revise their manuscript to address specific concerns before a final decision is reached
* Reject, but indicate to the authors that further work might justify a resubmission
* Reject outright, typically on grounds of specialist interest, lack of novelty, insufficient conceptual advance or major technical and/or interpretational problems"
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
I'm interested in learning the essentials of statistics. What would be a good book to start me out?
I got The Manga Guide to Statistics and it did introduce me to the very basics. However, there are many places where it just gives you an equation, without deriving it or even explaining it. After reading this book, I now know how to calculate standard deviation, but I'm still a bit vague on how people actually use it. I would like to see some examples of how people use statistics in (for example) science experiments.
My ideal book would explain the basics, with examples, and show how the math works. Ideally it wouldn't be a thousand pages long, either, but that's a secondary consideration.
Recommendations, please?
P.S. Those of you who know about statistics: how good are the Wikipedia pages on statistics?
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Not rhetoric, fact.
But, telling correlation from causation is an easy one. Pay attention to the details and learn a lot more Maths. That'll give them some actual critical thinking ability instead of following a checklist.
It really is a rarity to see a study that has been designed properly. The only one that I know of is one studying Recurrent Brief Depression. The study used an entire practice of patients and filtered out people with other disorders to get "pure" RBD people. Then they were paired with healthy controls that were the same age and gender. That's a good experiment. Of course, it's still limited as socio-economic differences between the controls and there RBD counter-parts would change things. Then there's the statistics problem because it really isn't a random sample because all the people involved would live in the same area of the city. I could go on.
But, that's the best one I've found. And I've looked at *lots* of studies.
Also, when it comes to applying "their fanciful theories to medicine", what do you call MRIs? How about X-Ray machines? How about medicines? These things are made, by and large, Physicists, Engineers and Chemists respectively. The Biologists come into play more on the "practicing Medicine" side of things because they are actually working directly with biological systems. So, if the Medicine people want to do any good, then they'll use that to start.
However, this is exactly what the Medicine people aren't doing. They are so concerned with the short game of finding out X for Y, that they ignore the long term benefits of having an overall theory of the human body. They'd be a long way beyond where they are if they would spend even a little time on that.
But, then again, they aren't Scientists. So, they don't think that way. They are practitioners and as such, only see what is directly in front of them. That might have benefits, but it also has some serious drawbacks. Some of which I have listed.
Don't shoot the messenger.
.. or at least not the probability of the hypothesis. This is one of the errors that people make. Having 0.95 significance do NOT imply having 95% chance for the hypothesis being true! The significance is the probability of the test outcome assuming the hypothesis is true (in other words it is a likelihood value). You have to multiply it by a prior to obtain real probabilities.
Significance values will not even add up to 1 over the two hypothesises!
The root of the problem is that frequentists can not use probabilities for statements -- only for events. In frequentist terms you have to have a sigma algebra over some Omega state space which is measurable. Bayesians on the other hand can talk about the probabilities of any statements using probability theory as an extension of formal logic. I really recommend reading the books of E. T Jeynes and David McKay.
Other false assumptions people make with statistics:
- Everything is normally distributed
- Everything has a variance
- Everything has an expected value
- Hypothesis testing is without bias (in fact it is equivalent to give 50% prior probability to both hypothesises)
- Variance means average distance from mean
- Empirical variance does not have a variance
*posted as is without editing, worts and all*
There is a difference between Medicine and Biology; they are NOT the same thing. Medicine is the biology of the human body. Period. End of story. Biology concerns itself will ALL life. In short, Medicine is the APPLICATION of Biology to humans. Different. But, if I'm wrong, go ahead and explain to me how those two domains are the same thing and the same size.
When it comes to Biology's contribution to Medicine, why don't you actually look up what the Engineers and Physicists have done compared to the Biologists before commenting. Biology has really only come into play recently.
Furthermore, the more you get away from Maths, obviously, the less will be known. However, if you've look at the modern Chemistry curriculum, and consider what needs to be known to understand the typical *required* Quantum Chemistry course... that's a fair bit of Maths. Btw, there's a reason why I mentioned Biology's relatively limited contribution to Science. It's because they've really only come into there own, as a Science, recently. Another couple decades or so, and they might be where Chemistry was a couple decades ago. Most of Chemistry today is actually quite good.
When it comes to the causation/correlation problem, yes it is a BIG problem. Just look through PubMed if you don't believe me. It is *very* common to have papers on there that calculate CIs with 20-30 patients (or less) like it means something. Sorry, but if they think that, they're clueless. It takes a statistically significant number of patients studied to make a CI meaningful. That's why I only really pay attention to survey studies (and view others with extreme scrutiny). They are the ones that have the highest possibility of being worth reading.
Finally, I have worked with Scientists. Physicists in particular. I also have payed attention to what the other disciplines have put out. Chemistry is meh, Biology is lesser (to one degree or another depending on the specific field within it) and Medicine is a joke. It might be politically incorrect to say such things. But, it is the honest truth. There's not really any shame in it as the more applied one goes, the more complicated things get. But, to ignore ones place is inviting disaster. That's really the point. To get them to know there place. Enough people have died due to there god complexes, overconfidence and not really understanding things (and not knowing it). They really need to acknowledge the limitations of what they do and who they are.
When it comes to the MDs that I get along with and respect. It's those that explicitly state what they are comfortable doing and what they aren't. It's those that are willing to work /with/ me not the ones who think its OK to tell me what to do when it's something that I care to be involved in. Etc. Guess which type is more rare and the average age of the ones that are more humble.
science is not in the bussiness of proof
So what is it in the business of?
Excluding mathematics, science is generally in the business of disproof.
I'm actually at a scientific meeting and saw 7 presentations in which they "double dipped" on their statisitics before we broke for lunch.
Double-dipping is bad enough, but the medical field is rife with multiple-dipping. Each dataset is plumbed to test dozens of hypotheses, without appropriately adjusting the acceptance criteria. Even with separate datasets, if you test 20 hypotheses and discover that each one is just valid at the 95% confidence level, then there is a very good chance that there are some false positives. In the medical alleged-sciences, however, all 20 would be blindly proclaimed as truth.
And then there are the social nonsenses^W sciences... If practitioners of some discipline do not understand how to use quantitative methods, they should limit themselves to qualitative argument only. Unfortunately, in statistics as in other fields, those who are ignorant or incompetent are generally unaware of the extent of their ignorance and incompetence.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
.. a place where the sun doesn't shine (often - statistically), does that mean 100% of those are stinky?
--- I am known for the ones who want to find me on the net. Is that a privacy risk or a privilege? One might wonder..
I think hippies tried to warn us too in the seventies of avoiding bad trips (LSD) .. Didn't know there was any math involved in that ...
--- I am known for the ones who want to find me on the net. Is that a privacy risk or a privilege? One might wonder..
That would mean your 100% is truely only valid for 99% at most?
Why to people ask .. are you 100% sure? while the answer is mostly "I think so" ?
To stretch this ... are you correct about your statement, even if it is statistically only for 99% correct?
--- I am known for the ones who want to find me on the net. Is that a privacy risk or a privilege? One might wonder..
So, work out 5% and subtract 10% of that from your answer. Not too difficult to do mentally if you want a rough approximate. 5% is merely half of 10%, so if the amount is X, then 4.5% of X is 1/2(X/10) - 1/2(1/2(X/20)).
For example X=234, 10% is 23.4, 5% is roughly 11.7, 10% of that is 1.17 so 4.5% of 234 will be 11.7 - 1.17 which is roughly 10.6. Yes, that was done mentally before I actually wrote it down. Some general rules to use for rough mental arithmetic:
finding a multiple of 5% is easier if you first find 10% (eg 35% = 10% * 3 + 10%/2)
multiplying by 10 is easy
multiplying by 2 is easy
So, break everything into sums of multiples of 2 and 10, followed by an addition
(eg. multiplying by 5 = multiply by ten, divide by two,
multiplying by 6 = multiply by 10, divide by two then add one (original value)
multiplying by 7 = add (multiply by 5 and multiply by 2)
multiply by 8 = multiply by 2, 3 times
etc...)
Well, you get the idea
I'm a minority race. Save your vitriol for white people.
Irrationality by Stuart Sutherland. Talks about irrationality in general, with a focus on how statistics are generally misunderstood and misused by the public, and particularly health officials. He also recommends Innumeracy by John Allen Paulos. As a good start to learn about statistics and probability theory.
I've always thought teaching a good understanding of statistics should be a requirement for high schools, since statistics are so often (mis)used to justify public policies and legislation. We need a citizenry that can see through the bullshit, or at least think a bit critically on the subject.
I think a firm understanding of statistics is more useful than the entry level calculus and the entry-level science courses like chemistry and biology(not that those aren't good too, just not as relevant to citizenship).
Here's a nice book on statistics called "How To Lie With Statistics" that covers a lot of the ways statistics are misused. (not a referrer link or anything like that)
http://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728
>You seriously think this is a common problem in biomedical research?
Of course it is. Medical studies are often condensed to a flashy headline in a newspaper. "Scientists said X is true so it must be now." Then the talking heads run off with it for the next three days. Nobody - certainly not the journalist - reads the paper itself, and generally it's behind a paywall so there's virtually no point in ponying up the $30 to be the only person with an accurate assessment.
If anything, the media "spin" makes it drop-dead easy to have a medical paper say whatever you want, since nobody is going to check it. Peer review? Here's what happened when the FDA looked at the Vioxx (COX2) data. It turns out the "peer review process" omitted the 12 and 15 month data points:
So your saying if you were to measure the popes and antipopes by weight there is just barely not enough antipope-tons to cause a significant reaction?
A bullet may have your name on it but splash damage is addressed "To whom it may concern."
> And then it struck me - most of the research I had read had applied parametric statistical tests to their data - that it, the researchers made an assumption that the underlying distribution of results would fall on a normal curve. Which in cases with lots of samples is a perfectly valid assumption. See http://en.wikipedia.org/wiki/Central_limit_theorem
And why would they? They can make more money on Wall Street
Think you are missing the point dude.
We (mostly!) didn't become doctors / scientists to make money.
If people are only motivated by money.... then have you ever wondered why kids climb trees ?
Anyone quoted by a reporter knows how little they understand
Don't believe what you read is the truth.
>>if you want real fun, take the average master's degree idiot and start having them manually add fractions without changing them to decimals. such as adding a bunch of measurements off a tape measure together.
My question is, what kind of idiot uses fractions these days?
Just use decimals for everything.
I tend to keep running totals in my head when doing stats or budgets, just to make sure the excel spreadsheet hasn't auto-adjusted itself to miss a row. But adding 13/16 + 78/11 + 4 3/2 without converting to decimal? Pfft. You never do that once you leave elementary school. (Mixed fractions, lol.)
And oh, my dear lord, try dealing with "valua-added-tax" in Europe....
500 million people try and succeed every day. The secret: By law, it is included in retail prices, so it does not matter whether there are 50 different rates across the EU, you pay what the sticker says. (If you are a business, your accounting software will apply the right rate and calculate the right amount for you. If you are a retailer, it is not too hard remembering the 1 or 2 rates that you have to add on the sticker.)
Maybe it proves his point more than he intended.
Brain surgery - it's not rocket science!
"That sounds right, but isn't part of being the best explanation, that such an explanation is true? Establishing the truth of an explanation would fit into one of the commonly used meanings of "proof.""
Sure, but then you fall into circular reasoning since you need proof to assert truth. I would go as far as to call well established science "beyond reasonable doubt" but that is neither proof nor truth.
The strength of scientific philosophy is that it is never 100% certain about anything and is willing to change it's explaination if provided with compelling evidence that an alternative explaination is a better fit for the observations. This usually doesn't mean the first explanation was wrong, mearly incomplete (see: Asimov's insightfull essay The relativity of wrong).
Most other philosophies (especially religious ones) view uncertainty and imperfection as weaknesses and hold up dogma and blind faith as virtues, my pet theory on that is that those philosophies seek to control people rather than inform them.
"I don't think your restriction of proof to axiomatic systems coincides with the way that that word is usually used."
Maybe, but that would be because few people are ever taught the basics of epistomology and science itself is generally taught as a grab-bag of usefull factoids rather than a coherent worldview. Also I would love to take the credit for that idea, but I am not that bright.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
I'm waiting for some clever clogs to take the 77.28% and perform a baysian analysis based on your 70% observation and tell us what the modified expectation level should be.
And 77.335% of all statistics claim more accuracy than their expected deviation warrants.
Luckily, only 34.48% of the public ever pays attention to statistics. Only 54.13% of which can properly understand what they mean.
The world of the average Joe is mean.
And the result is only relevant to 27.765% of those.
Apologies that I can't remember the exact details but I read about the case of a university professor in the US who lost his job for allegedly saying there were more men in science because men were more intelligent than women. The issue revolved around the press not understanding standard deviations. What the professor had actually said (in fewer words) was that the standard bell curve for intelligence is slightly difference by gender. For men it is shorter and fatter but the tails don't extend very far while for women the curve is taller but with very long tails. It boils down to there being more intelligent men but equally, more stupid men while women have the potential to be both significantly more intelligent but also significantly less intelligent than the bulk of the male population.
All the details are in the book Super Crunchers which is incidentally a fantastic read for anyone interested in the application of statistics in a very general, non-mathematical sense (it covers the use of statistics by baseball scouts, medical computers, predicting changes in flight prices and predicting wine vintages to name a few scenarios that are covered). Unfortunately the professor lost his job because of the furore generated by the misinterpretation in the press.
I have the same problem. In school they were considering putting me in remedial classes because I had trouble doing basic arithmetic with even single digit numbers(I still have trouble with anything above 6). I can and could do a reasonably accurate estimate, but not the real result (possibly has something to do with me also having a bad short term memory). As soon as we got to the abstract bit (i.e. real math) I had no trouble. I can do integration with coordinatesystem shifts(e.g. cartesian->polar) in my head, but I will have to check my constants with a calculator.
Scientists should start working with statisticians.
How do you prove that isotopes stay in the same place in the mud for millions of years?
Shortcomings of statistics? More like shortcomings of humans *attempting* to use statistics.
Misuse of statistics is well-represented in scientific articles. Other things that are well-represented are poor knowledge and reasoning in the area of the subject discipline, inept writing, misleading or unhelpful graphics, poor scholarship, etc. Sturgeon's Law applies across the board.
Having read a fair number of sky-is-falling articles about statistics in science, and having worked with my share of researchers (MDs and PhDs in a variety of fields) who think everything is rosy, I'm pretty sure that the truth is somewhere in between. A minor saving grace is the fact that getting the statistics wrong is not the same as getting the answer wrong. Although it's certainly quite common to find published articles that make claims with no support whatsoever, in my experience it's much more common to find articles where the inappropriate statistics just mean the support isn't nearly as strong as claimed. Spurious results tend, though not as reliably as we'd like, to get weeded out by the literature. I rarely read an article that isn't specifically about methodology in which the methods/statistics are really solid, but I also rarely read an article in which unsound statistics undermines the entire contribution.
I do stat mech. Most of the papers I read pay very little attention to assigning a level statistical significance to their "measurements". When they do, assumptions of uncorrelated measurements are always made - and probably incorrectly. I struggle with the statistics myself. I find myself working out of my undergraduate stats text mostly. I feel I'm more concerned with understanding how statistically meaningful my measurements are than most of my colleagues. And I worry about my understanding of the statistical methods I use.
46 & 2
Indeed... If you can find causation, you'll find correlation.
There statistics will help confirm you're on the right path...after a fashion.
But so many people misuse the tool in question to go the other way around- and try to prove causation with correlations. You might be able to do that, through dumb luck. As often as not, you'll get all sorts of wild assumptions come up as theory due to that attempt as as likely as not you've missed something. That's why statistical analysis and meta-analysis used solely to validate a premise should be viewed as the hokum it typically ends up panning out to be (Just look at the past- it's replete with people thinking the most ludicrous things based off of "the statistics"...).
I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
That's irrelevant, because isotope studies in geology are not done on "mud". You don't take the lid off a mountain and find liquid mud that's been sitting there for millions of years (in which case the isotopes would have circulated as you suggest).
The "mud" hardens into a rock. If it's really mud, then it might be mudstone or shale. Anything that's in the mud, like isotopes, is trapped in place in the chemistry of the rock. And believe it or not, geologists do realize that things "leak out" or are otherwise mixed up over time, and this is taken into account.
That said, as a geologist myself I do often think results based on isotope studies are bullshit, but not because the science of isotopes is bullshit. It's because of the problems described in TFA - misunderstanding of statistics - and misapplication of isotope-related techniques.
Your disagreement with public health bullshit is understandable, and I agree to some extent with that. However, I really don't think you understand the chemistry of isotope studies and the principles of geology that make these things valid (when used properly).
It sounds like you are a Truth Seeker who has become jaded because of the basic assumptions underlying science and because our broken human nature does not always treat the scientific results properly. You are pointing out the mess we are in and how Naturalism does not solve the problems. I would encourage you to continue to seek to understand Reality/Truth. It is important. I found that the Christian Faith fits reality best. Consider it. There are presuppositions/assumptions also with Christianity but I believe it does explain reality best and science can fit into that Christian framework.
The largest demographic in american prisons are black americans. Real statistic but is it true?
Given a particular sample that indicates blacks are 60% of the prison population this would appear to be true.
But what if I said: "The largest demographic in prison is minority, non-whites." Suddenly the % jumps from 60% (black) to 80% (minority). Which is more right? This is the problem with statistics. Context.
Now I can say readily that the largest demographic in prison is actually right-handed people. The % now jumps to 90%.
But wait! There is more! The largest demographic is prison is actually people who prior to arrest were below the poverty line which jumps to 99% of the population. Again, all of the above are accurate based on a sample but which is MORE correct? Linear Algebra is coming into play here quickly....
When that kind of issue comes into play, it is the classic "Correlation != Causation" confusion. The majority of people in prison are in there because of "Being black? Being a minority? being right handed? or being poor?" None of the above. The majority of them are in there because they were convicted of a crime and sentenced. That is the causation of their imprisonment, the rest is correlation which may have a direct causation on the conviction or sentencing, but no direct causation on being in prison. (e.g. You cannot be thrown into prison for being poor, black, minority, right handed)
Same with medical research, politics, economics, etc. The price of oil rising 10% and a subsequent 5% drop in shipping orders. Measuring the significance of regessors is important but oddly never reported most of the time. Many factors get masked or shadowed by higher level regressors (e.g. being a minority masks a variety of other social and economic factors. In addition it can distort statistical work by being too broad. Asians have a variety of different economic and social factors as north american blacks versus even african immigrants.)
Back to the orignal subject:
We can take 100 prisoners and 100 non-prisoners and figure out rather quickly if being black is statistically significant in prison population. Non-prison population blacks would account for 25%-45% of the population (Depending on location). We can see that 60% of prisoners are black. There is a 20+% deviation from the norm. We can test to see the significance of that. Same with minorities. Now we find something quickly that right handed is insignificant because it doesn't deviate from the norm. We can test left-handed and right-handed populations and rule out the handed-ness of a convict being significant.
We can find the economic status is considerable MORE significant then minority or black as a status. We can determine that the reason minorities or blacks are disporotinally more prevelant in prison is that blacks and minorities have higher rates of poverty. We can extract and determine the statistical weight of POVERTY in regards to imprisonment (Since we find a high % of white in prison that are poor compared to the normal population.) Once we figure that out we can remove that and continue an investigation and figure out what weight minority and black has once we have removed POVERTY from the model (Residual analysis).
The problem in reporting is without providing the whole, comprehensive analysis you can miss important things. For instance to correct the injustice in sentencing, without reporting the weight POVERTY has in contrast to BLACK or MINORITY you may lose sight that you may have better success addressing POVERTY to normalize sentencing rather then MINORITY or BLACK (or not).
The same happens in medical reasearch. Given a cocktail of drugs wirthout having the whole analysis you may end up providing more of Medicine A versus B but lose sight that A & B are limited by the dosage of Medicine C.
Satistics are not bullshit, rather mearly observations with no intrinsic agenda or even implication of truth. Purely amoral, like a hand gun.. useful to both the good and evil.
Statistics don't lie, nor do they tell the truth. They simple show the relationship of the data as it stands. The Truth or Thruthiness of it is subjective and vulnerable to context.
-=[ Who Is John Galt? ]=-
of doctors and researchers who deal with statistics on a regular basis. My aunt and uncle are both oncologists. My grandfather is an orthopedist. Last year, my grandfather discussed this very issue with me: for the majority of his career, he did not understand statistics well enough to truly gain anything from scientific journals. He could understand things like means, standard deviation, median, etc. But when the literature begins to lean toward more esoteric statistics, he can no longer discern the meaning. He then handed me a book titled The Lady Tasting Tea, which he claims made a great difference in his understanding of statistics and their meanings. I graduated with a BS in computer science, and have taken enough statistics courses that the idea of reading one more word about chi square tests would melt my brain. But I digress. The point is that there is accessible literature out there for people who are not versed in statistics.
Lots of statistical problems seem to be ignored. Papers which blithely present meta-analyses as if they had the power of a single large study. Far too much significance attributed to case-control studies (which magnify small effects and can't, by nature, show causation). And statistical tests which simply don't have the power to show what they purport to be showing.
One example: A study purporting to demonstrate the effect of an event E on a particular variable X. The study took the average of the variable 12 months prior to E (high), and 12 months following E (much lower), and determined that event E reduced variable X. Only problem is that variable X had been declining, and about the time event E happened, that decline reversed and X started going up, though more slowly than it had been declining.
Yeah, the mathematical statistics courses were just chock full of what we called "meds keeners" or "hoovers" ie those seeking admission to med school. Even those majoring in alleged sciences like biology were often shockingly ignorant of hard sciences and tended to fulfill only the minimum requirements in things like chemistry.
I did not dispute the properties of isotopes themselves. When studied in the laboratory, isotopes appear to have predictable properties of exponential decay. (Though, I would add that the more stable isotopes which last for millions of years can only be assumed on faith to have the same properties of exponential decay as more short-lived isotopes. Could there be emergent properties that make them deviate from predictions? We know from the world of survival analysis that longer-lived entities often do deviate; the "proportional hazards" model is inappropriate.)
What I question, as you have, are the untestable postulates involving mixtures of isotopes in the crust. The only way to test this is to make hundreds of planets and wait millions of years. We don't have the technology to do so, so we only have postulates that appear logical. As we know, plenty of ideas sound logically elegant, but fail to work in the real world.
he couldn't calculate tip to save his life, but I don't certainly hold that against him.
That is, unless he's like my old roommate and stuck you with the tip each time.
I embrace Christianity as a moral system. I am a Christian in this sense. I'm not here to promote intelligent design or to oppose the theory of evolution as a whole, as the slashdot crowd may wish to label me. My position is that any number of theories can be cooked up about evolution, cosmology, etc., and one can find any amount of data to support their theory. Every theory I've come across relies on a faith in untestable or non-falsifiable postulates. Our physical reality is defined by what we look for; there are any number of legitimate theories based on data that isn't available yet, because we didn't look in the right places, or ask the right questions.
Thou shalt not worship the .05 level.
Correlation does not imply causation -- you need to have some idea of HOW the values are correlated.
Linear regression is only valid when the relationship is in fact linear.
The more variables added to a multivariate statistical model, the greater the likelihood that there will be a spurious correlation.
SPSS will always find something when you tell it to look hard enough.
Definitely a cute trick,
While I thank you for your explanation, I have very little trouble doing mental math, estimations, and basically anything from addition through calculus. The point is that there are those who can do math, but can't do it in their head, even though they are otherwise intelligent, an quick-witted.
The problem you don't see is that while shifting decimals by */10 is easy, as soon as numerals have carries from the *2 or *3 the mental math becomes harder.
A lot don't. I work in biological science, and even with my mediocre maths education (I had four lectures on stats during my undergrad, plus one afternoon at the start of my PhD; everything else I've had to teach myself) I see a lot of people talking about statistical tests that they clearly don't understand.
:).
It's sad but true that a lof of people end up in biology because they love science but can't handle the maths required by physics or even advanced chemistry. While there are plenty of exceptions, there's a very strong tendency to treat statistical tests as black-box tools: plug in the numbers, get an answer and don't worry too much about whether it's an appropriate test or what the answer actually means. The article's example of people misunderstanding the meaning of a p value from Student's T-test is actually distressingly common. Other things -- like designing and drawing conclusions from experiments without ever considering power calculations -- crop up a lot too.
The best area I've encountered so far is bioinformatics, which tends to be the realm of programmers and statisticians who've become interested in biology, rather than the other way around. I'm not in a position to give an informed assessment of their work, but the sheer pain on their faces when advising maths-impaired biologists on study design is a pretty solid sign that they're used to a much higher standard
Definitely a cute trick, While I thank you for your explanation, I have very little trouble doing mental math, estimations, and basically anything from addition through calculus. The point is that there are those who can do math, but can't do it in their head, even though they are otherwise intelligent, an quick-witted.
Well, that's the reason for the "trick" - it enables mental slow-motion-actors like myself to calculate in a reasonable amount of time. I taught this (and a few other basic rules) to a friend of mine who thought my approximations were an intrinsic quality, and he did just fine as soon as he learned the methods.
My only contention is that the mental magic done on numbers is not magic, and can be taught, not that some people can do it and some people cannot (which, feel free to correct me, is your point - that some people can and some cannot (and we should not judge their intelligence on that point)). I figure that *most* can do it if given a few simple rules. Regardless, I believe we both agree that people who cannot do this are not inferior in any intelligence/intellectual way.
The problem you don't see is that while shifting decimals by */10 is easy, as soon as numerals have carries from the *2 or *3 the mental math becomes harder.
Umm ... perhaps more tricks can be employed[1]? After all, if you're not after precision, then you can get pretty close to the answer very quickly.
:-)
[1] I prefer to work with fractions than decimals, as then my normal bag of "tricks" can be used
PS - forgive typo in previous post - typing fast with no coffee
I'm a minority race. Save your vitriol for white people.
What you describe is a matched case-control study. There are better methods such as double-blind randomized clinical trials. An intro to epidemiology course will teach you all of this. If the high and mighty physicist can think of an even better method than clinical trials, go ahead and state them.
As for MRI machines, they produce a lot of data, but they are just statistical associations. Just because a depressed patient's brain looks different on an MRI machine from a normal person's brain does not prove any causal relationship. Again, if the high-and-mighty physicist thinks they have a better answer, they are welcome to state them.
A cow is actually a rather topologically interesting beast.
Nah, it's just a funny torus.
You're udderly correct. Now if you had said it was just a funny taurus, on the other hand...
Every theory I've come across relies on a faith in untestable or non-falsifiable postulates. Our physical reality is defined by what we look for; there are any number of legitimate theories based on data that isn't available yet, because we didn't look in the right places, or ask the right questions.
Yes, we don't know Truth/Reality in the full. We only know bits and pieces and we have some concepts that are just wrong. I agree that all of our theories are incomplete and based "untestable or non-falsifiable postulates". And Kurt Godel showed that to be the case even with math with his incompleteness theorems. And yet I believe we can successfully strive to better understand reality, not the many different realities we perceive, but the one true Reality we live in.
Though science has given us a better understanding of reality, it is good to recognize the limitations of science and to question the assumptions, presuppositions and axioms that make up the theories and our beliefs. Ask yourself how well does this theory/belief match reality? If it does a poor job, if possible replace it with one that does a better job. And recognize that because humans can be biased and blind to reality, there are and will continue to be theories promoted that fall far short of reality and/or are based on bad assumptions. Don't let that get you down. Strive to better understand reality. That is the journey I am on and I believe the Christian worldview gives me the best framework to understand reality.
People who deal with raw physical measurements (radar engineers, astronomers, the guy who makes airspeed sensor of the B2--er,um...) have had this problem figured out for a while.
It sounds easy to you cos you have an easy job. You only have a single, easy to measure parameter.
In other fields there can be dozens, hundreds or thousands of parameters, each with it's own signal. Determining which of the signals (if any) are meaningful is a lot harder than what you're doing. What I'm saying is, you're an engineer, not a scientist.
Deleted
Brought gales of laughter to me
I must change it now.
Do not mock my vision of impractical footwear
This was very profoundly not my experience when helping a corporate partner price and sell equipment. Prices for hardware are advertised _without_ VAT. Slipping in and out portions of VAT depending on the upstream vendor's behavior was insane. And oh, dear lord, if you had a sale, the VAT numbers got even more insane.
This was very profoundly not my experience when helping a corporate partner price and sell equipment. Prices for hardware are advertised _without_ VAT. Slipping in and out portions of VAT depending on the upstream vendor's behavior was insane. And oh, dear lord, if you had a sale, the VAT numbers got even more insane.
Yes, but you were doing B2B/wholesale sales - only a very small percentage of the population will ever be in that situation. And if you are in a business, you usually know the VAT rate and have a dedicated accountant to deal with any problem that a pocket calculator can't solve...
It's a troll because it implies scientists don't know about those things.
The implication has to be incorrect for almost all scientists for that to be a problem. As the other replier noted, there is widespread ignorance among scientists of this sort of thing.
That people are trying to use peer-review as a method to detect fraud, does not make it a good method for doing so. I've mentioned this before on /., although not in this thread, but I have no way of telling if the numbers in a table were generated by the experiment described, some other experiment, a random number generator, or the PR department at the company who's product is being evaluated. As long as the numbers are internally consistent, I have to "trust" that what they describe, happened. I can catch obvious errors, such as the SEM not supporting claims of statistical significance made by the authors. However, if during the review process, they claim that the SEM was a typo (numbers were actually SD and not SEM for example) and change it, I have no way of verifying that their explanation was true.
Also, in your quote you highlighted 2 different lines. The first has to do with the soundness of the conclusions. This is most definitely a role of peer review, but not related to accuracy. It doesn't mean that they verify that your conclusions are correct. Conclusions are not objective. The data gives you objective facts from which to draw subjective conclusions. This line indicates that your discussion will be evaluated for how well the data (yours and previous literature) supports your conclusions. If you extrapolate, or ignore important results then your paper will be rejected.
The second bolded section just indicates that if serious errors are found (using insufficiently large sample size, extrapolating results, etc.) then the paper will be rejected. That's totally understandable to reject, but obviously serious errors of this sort are uncommon. Most errors are much harder to detect, and are not picked up by the peer review process in my experience.
Bureaucracy expands to meet the needs of the expanding bureaucracy.-Oscar Wilde