Slashdot Mirror

(BTW: Isn't p=0.05 only a 2-sigma result? Ick.)

Its a bit less than 2-sigma. It should be more like 3-sigma (about p=0.01) which would make p-hacking much more difficult as it would take 100 variations to see a probable null hypothesis. Although the exact methods of conversion are complex.

That doesn't stop the individual from looking at Cochran reviews and so forth to get a more balanced picture and making up their own minds. I do this whenever the media bring up a new study making some claim or other.

But it won't cause them to look either. A lot of people want to hear that sucrose is healthier than High fructose corn sugar, and that diet drinks cause heart attacks. Now they slurp their Healthy sugar Big Gulps with a clear conscience.

Peeps is peeps, and for some reason they want to think that if they only eat this, or avoid that, they will live forever - at least in their minds.

Moderation has always and remains the best advice. They won't take that advice either.

We should replicate studies and confirm analysis independently before we get too excited about any results..

That's true, but the hitch is there will be financial interests working and braying about whatever study gives them better profit opportunities.

That doesn't stop the individual from looking at Cochran reviews and so forth to get a more balanced picture and making up their own minds. I do this whenever the media bring up a new study making some claim or other.

And a Cochrane review of speed cameras concludes:

The quality of the included studies in this review was judged as being of overall moderate quality at best, however, the consistency of reported positive reductions in speed and crash results across all studies show that speed cameras are a worthwhile intervention for reducing the number of road traffic injuries and deaths. To affirm this finding, higher quality studies, using well designed controlled trials where possible, and studies conducted over adequate length of time (including lengthy follow-up periods) with sufficient data collection points, both before and after the implementation of speed cameras, are needed.

http://summaries.cochrane.org/...

And you should be driving defensively, instead of assuming everyone on the road is driving in the correct frame of mind.

Sounds like he is.

But he was talking about tailgating, which one of the stupidest things you can do on the road and no fault of the car in front.

Red light cameras work in theory. They cause more accidents in reality.

They work in reality too.

Red light cameras have reduced the fatalities and severity of injuries from crashes at red lights. Also the number of right angle (T-Bone) crashes were reduced.
http://summaries.cochrane.org/CD003862/red-light-cameras-cut-casualty-crashes-at-junctions-with-traffic-lights

Good meta-studies...suggest there is no significant overall injury/death mitigation benefit to cyclists from wearing helmets.

According to the excellent Cochrane meta-analyis and review:

'Wearing a helmet dramatically reduces the risk of head and facial injuries for bicyclists involved in a crash, even if it involves a motor vehicle...Head injuries are responsible for around three-quarters of deaths among bicyclists involved in crashes. Facial injuries are also common. The review found that wearing a helmet reduced the risk of head or brain injury by approximately two-thirds or more, regardless of whether the crash involved a motor vehicle. Injuries to the mid and upper face were also markedly reduced, although helmets did not prevent lower facial injuries.'

http://summaries.cochrane.org/CD001855/wearing-a-helmet-dramatically-reduces-the-risk-of-head-and-facial-injuries-for-bicyclists-involved-in-a-crash-even-if-it-involves-a-motor-vehicle

In another review from the Cochrane Collaboration:

'Although the results of the review support bicycle helmet legislation for reducing head injuries, the evidence is currently insufficient to either support or negate the claims of bicycle helmet opponents that helmet laws may discourage cycling.'

http://summaries.cochrane.org/CD005401/bicycle-helmet-legislation-for-the-uptake-of-helmet-use-and-prevention-of-head-injuries

Together, these reviews suggest that an individual would be well-advised to wear a helmet, but the jury's still out on whether mandating helmet use discourages cycling (with its potential health benefits for the population).

Good meta-studies...suggest there is no significant overall injury/death mitigation benefit to cyclists from wearing helmets.

According to the excellent Cochrane meta-analyis and review:

'Wearing a helmet dramatically reduces the risk of head and facial injuries for bicyclists involved in a crash, even if it involves a motor vehicle...Head injuries are responsible for around three-quarters of deaths among bicyclists involved in crashes. Facial injuries are also common. The review found that wearing a helmet reduced the risk of head or brain injury by approximately two-thirds or more, regardless of whether the crash involved a motor vehicle. Injuries to the mid and upper face were also markedly reduced, although helmets did not prevent lower facial injuries.'

http://summaries.cochrane.org/CD001855/wearing-a-helmet-dramatically-reduces-the-risk-of-head-and-facial-injuries-for-bicyclists-involved-in-a-crash-even-if-it-involves-a-motor-vehicle

In another review from the Cochrane Collaboration:

'Although the results of the review support bicycle helmet legislation for reducing head injuries, the evidence is currently insufficient to either support or negate the claims of bicycle helmet opponents that helmet laws may discourage cycling.'

http://summaries.cochrane.org/CD005401/bicycle-helmet-legislation-for-the-uptake-of-helmet-use-and-prevention-of-head-injuries

Together, these reviews suggest that an individual would be well-advised to wear a helmet, but the jury's still out on whether mandating helmet use discourages cycling (with its potential health benefits for the population).

If you wanted to get an idea about why accupunture is so popular, you'd do well to join in on their conversation. Maybe you'd plant some seeds of the value of evidence based medicine, and you could hope to get an insight into how their minds work.

http://www.cochrane.org/reviews/en/ab003281.html

I was just getting info from wikipedia, which cites these sources:
http://www.cochrane.org/reviews/en/ab003281.html
http://www.dieutridau.com/thongtin/detai/acupuncture-does-it.pdf

So, I suppose we should personally research every scientific finding?

Yes, this is the entire foundation of modern science. Believing something, even if it is universally accepted by experts, is simply an appeal to authority. Of course, all the experts in a field agreeing to something is certainly a strong indicator that it's worth looking at. On the other hand, it is difficult for a layman to determine that all the experts in a field are in agreement, as opposed to a vocal minority.

As for quantum theory, you probably don't make many daily life decisions that hinge on the accuracy of quantum theory. Thus there isn't much need to research it, unless you are simply curious (which is always a good thing). As for obesity, there are plenty of studies out there. Since it's a medical thing the Cochrane Collaboration is a good place to start looking. Just keep in mind that a single study doesn't have much weight alone. Look for a through meta study with a large sample and sound methods. Reading studies isn't that difficult, and you don't need to read and understand every single word anyway. Sure you may need to invest a day to determine your stance on a claim, but at the end you will be able to make your own decision.

The alternative isn't even really just trusting the experts either. It's unlikely you are personally interviewing a wide range of experts. Instead, you have to rely on how the media interprets the experts (and probably just one expert). One doesn't even have to assume malice on the media's part, simple ignorance will do. I highly recommend you get into the habit of personally verifying any claims you intend to make life decisions based on.

I can forgive the Atlantic for hyping the story -- it's the only way to get attention! :)

Obviously, Ionnidis' work is well known, since he's published in JAMA and PLOS. It's worth repeating in a popular magazine.

Everybody who keeps up with the medical literature knows about evidence-based medicine and meta-analysis.

They also know that once you apply rigorous standards to conventional wisdom, a lot of the wisdom doesn't hold up. It's humbling.

It's frustrating to look at http://www.cochrane.org/ and see how many of the treatments have a modest effect, or just don't work at all.

But some of it does. It just takes a few good treatments to make a big difference. People live a lot longer now. There's a lot more 80-year-olds. We can control heart disease, high blood pressure and kidney failure, and diabetes a lot better now (good thing, because we've got a lot more of it). Childhood leukemia used to be a death sentence, now the cure rate is up to 95%. I know somebody with chronic myeloid leukemia would would have died young if she didn't have imatinib (Gleevic).

A lot of the research today is directed not at finding new wonder drugs, but on figuring out how to best use the old drugs, like all the drugs for heart disease. A lot of that research is by guys like Ionnidis.

Just remember the next time you see a medical news story: Association isn't causation.

It's worth mentioning that one of the reasons hormone replacement therapy seemed to reduce heart attacks in retrospective studies was that the women who were most concerned about their health were most likely to have good habits like diet and exercise -- and also most likely to take HRT, since they thought it was good. Then the evidence-based medicine guys did a prospective, randomized controlled study, and it did more harm than good. There was actually a measurable, significant increase in breast cancer as a result.

What I hear you saying is, "All these studies are BS to begin with, and metastudies make it worse.

That's pretty much it. Doing statistical manipulations of bad data doesn't get you good data. Meta-analysis is in a technique that is increasingly in vogue these days in medicine. It's nice because the potential author doesn't have to get up off of the chair to do the studies and thus can be used to pad your CV without a whole lot of trouble. Done well, it's been quite useful. For example the Cochrane Collaboration has used meta-analysis to try to make sense out of large numbers of studies with often conflicting conclusions by incorporating it into a standard review structure: (from Cochrane Reviews).

A structured format [that] helps the reader to find his/her way around the review easily.
A detailed methods section allows the reader to assess whether the review was done in such a way as to justify its conclusions.
The quality of clinical studies to be incorporated into a review is carefully considered, using predefined criteria.
A thorough and systematic search strategy, which includes searches for unpublished and non-English records, aims to provide as complete a picture as possible to try to answer the question considered.
If the data collected in a review are of sufficient quality and similar enough, they are summarised statistically in a meta-analysis, which generally provides a better overall estimate of a clinical effect than the results from individual studies. A meta-analysis also allows the author to explore the effect of specific characteristics of given studies (for example, study quality) on the reported results (for example, does exclusion of non-randomized studies change the overall result?). It also allows an exploration of the effects of an intervention on sub-groups of patients (for example, does the treatment have a different effect on smokers compared with non-smokers?).

(Emphasis mine). My major point is summarized in the last bullet. Meta-analysis is only useful if the data is of sufficient quantity and quality. We have little evidence that either aspect of the video game studies is true, especially the quality part. I have also never seen an author of such a study put such a forceful edge on their conclusions. The people that do this for a living realize that meta-analysis is complex and has numerous caveats.

It may well be that video games have some negative effects - most things do. Articles like this don't do much to convince me, however. At a very minimum, the reporting sucks because we are not even given a hint that the analysis was appropriately performed. Given the poor statistical quality of the psychiatric literature, I'm not willing to suspend my disbelief.

I don't know how you can be so definitive about acupuncture being nothing more than an invasive placebo -- it doesn't tally with the literature. Have a look at the excellent reviews on the fabulous Cochrane website. http://www.cochrane.org/

See the review of trials looking at the P6 point and post-operative nausea, for example.

It's currently in the category of "seems to work for certain things, but we don't really know why".

Quite different from chiropractic, which has bugger all evidence of efficacy.

I do not want to join a fight about all this.

Then maybe you shouldn't have posted.

I am convinced, that the drug industry is doing a lot of evil things, and that most bodies like the FDA are actually ment to protect the interest of drug companies (e.g. drugs with same ingredients cannot be sold if they are from India, Canada, etc..) and not the end users'.

So you're a conspiracy theorist. Do you also believe that the government is covering up the evidence for alien abductions and the the UN is secretly plotting to take over the US? What about water fluoridation, is that a secret plot by the communists to implement mind control or steal our "precious bodily fluids?" All of those conspiracy theorist beliefs are equally absurd.

So I recommend making a search on your favourite torrent site or even youtube for "Are Vaccines safe"

Right, because when I want accurate and timely medical information the first thing that comes to my mind is "Hey, I'll see what a bunch of random non-experts on Youtube have to say." If you want to actually learn something useful, try searching on pubmed. You can also read the Cochrane Reviews on the subject if you want the predigested non-definitive summary (being a scientist and not a clinician, I prefer the original literature to the reviews, but of course I'm not busy seeing patients).

By the way they want to make H1N1 shots obligatory in Costa Rica - where I live - and there will be a huge resistance to it as everyone is scared of the shots' side effects, and the fact that it had very little - if any - testing.

Sheesh, the H1N1 vaccine is no different from every seasonal flu vaccine ever made, with the exception of this one likely being more accurately targeted (and thus having higher efficacy). We've been using these things for decades, and you can search pubmed to see studies looking at their safety and efficacy.

You can also make a search for flu shots and alcheimers, shots and tumors and find a scary amount of hype and facts....

You can also search for timecube to learn the true theory of everything. Try searching the actual literature and see what science finds. You do remember science right? Its the thing that gave us the internet, drastically increased our lifespans and largely eradicated previous scourges like polio and measles. Funny how useful that science thing has turned out to be...

"Yes, because being outside the physics of this universe means that there is no possibility of interaction. If there were the possibility of interaction, then god would NOT be outside the laws of this universe." Interesting - but that reiterates what you already wrote. I can't see how you know whether the possibility of a one way interaction is possible or not.

"As for prayer - controlled studies showed that there was no difference..." there are studies where it does make a difference - both positively and negatively. This Cochrane review will point you to some http://www.cochrane.org/reviews/en/ab000368.html the overall result is neutral.

"...the latest battle-front between religion and the masses - gays, lesbians, and transsexuals." Between what particular religions and the masses do you refer to? Unless you can elaborate you are stereotyping.

"Are you going to stand up and condemn ......... is your belief hollow, of no practical effect?" My belief? I haven't said or hinted at what my beliefs are. Just because I argue against your sweeping statements doesn't mean squat. I do profess to not knowing everything and that my knowledge is and always will be limited and tainted. How about you?

I would seriously investigate literature about taking high dose supplements such as hourly vitamin C (2 grams/hr, up to bowel tolerance perhaps 12+ g/hr), high doses of vitamin D (e.g. 10,000 - 20,000 iu for several days).
See: Cochrane review on C and pneumonia http://www.cochrane.org/reviews/en/ab005532.html
See also various papers by Fred R Klenner and Robert Cathcart, championed by later physicians like Tom Levy and orthomolecular MDs.

Modern research in the med schools shows the need for much higher levels of vitamin D, at vitaminDcouncil.org

I find it interesting that people have concentrated on vaccines and missed the part on antivirals.

I see your ancedote and raise you a Cochrane review, which strongly suggests that neuraminidase inhibitors such as Tamiflu really don't do very much.

I think I can see where we are disagreeing here. I think you are generalizing and I am speaking to Meta/SR of RCTs in medicine. I am not sure what you do for a living, but in terms of medicine and medical education and medical research, which is what I do, the pyramid I provided from the center for evidence based medicine at Duke is the real deal. There are similar concepts from McGill in Canada who were really some of the pioneers in this. Here is a link to the pertinent paper. I don't know enough about the methodology of meta-analysis in other fields to comment on that. I can comment that, again in medicine, the Cochrane Reviews are pretty much the gold standard in medical research. You are right, it is some of the most complicated research to partake in, and you are right again, especially in the field of cancer research most trials still go unpublished. But when done well, and there are many, many, well done reviews, they are without peer.

The reason meta-analyses are often used as a gold standard is that they can (usually) command a sample size that is far beyond that of the usual primary study.

Correct, and therein lies the power....... Come on...you've gotta laugh at that one!

Here are some interesting articles that discuss your concerns.

No, he actually does. TFA is talking about arthroscopy for osteoarthritis, not knee surgery in general - although that little point isn't very clear. Arthroscopic surgeries for some things seems to be useful, although formal comparative studies haven't been done. Same for knee replacements. The problem is that it's dammned hard to do the experiment. For osteoarthritis ("bone on bone"), the VA docs did a wonderful thing. They managed to get a protocol whereby the "control" patients underwent anesthesia and had a sham surgery, complete with scars. That's pretty gutsy - you have to show that the risk of anesthesia to the control patients is essentially zero, since presumeably they would not be getting any benefit from the surgery. That turned out not to be true, but from the point of the Investigative Review Board, that had to be a serious consideration.

OK, now you have to do this for the half dozen other reasons you would typically do arthroscopic surgery of the knee for. Then figure out how to do a sham knee replacement.

Now, do you see why we don't know jack.....

And a shameless plug, even though I have nothing in particular to do with them: The Cochrane Collaboration - an attempt to answers these questions by studying the extant literature. Read it and weep.

No, he actually does. TFA is talking about arthroscopy for osteoarthritis, not knee surgery in general - although that little point isn't very clear. Arthroscopic surgeries for some things seems to be useful, although formal comparative studies haven't been done. Same for knee replacements. The problem is that it's dammned hard to do the experiment. For osteoarthritis ("bone on bone"), the VA docs did a wonderful thing. They managed to get a protocol whereby the "control" patients underwent anesthesia and had a sham surgery, complete with scars. That's pretty gutsy - you have to show that the risk of anesthesia to the control patients is essentially zero, since presumeably they would not be getting any benefit from the surgery. That turned out not to be true, but from the point of the Investigative Review Board, that had to be a serious consideration.

OK, now you have to do this for the half dozen other reasons you would typically do arthroscopic surgery of the knee for. Then figure out how to do a sham knee replacement.

Now, do you see why we don't know jack.....

And a shameless plug, even though I have nothing in particular to do with them: The Cochrane Collaboration - an attempt to answers these questions by studying the extant literature. Read it and weep.

The FA does imply that 'cough medications' (Robatussin being singled out as a brand) doesn't work in adults. AFAIK, the author is probably right, but pushing the envelope on "evidence based medicine". Looking at the Cochrane Collaboration data on the subject, we find, as usual, that we don't really know much of anything.

And, before the grammer nazi's pummel me, I will stop using so, many, damned commas.

Instead of banking the cord, I would like to suggest you delay clamping of the cord and let the newborn have the blood themselves, when they're born. There are many benefits to this:
Prevention of anemia, Jundice, and many others.

The above articles all state that there is no scientific reason for early clamping of the cord, and imply that there are significant benefits to waiting until the cord has finished pulsing to clamp and cut.

Anyone doing this should get put in jail for a long, long time. It may not be fraud in the sense that they're publishing fraudulent results, but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.

That's a bit simplistic - often nobody is directly responsible for not publishing results, it comes about because major medical journals are not interested in pubishing negative findings. There are of course exceptions, when the negative findings come as a big surprise or there is a lot of anticipation about them, like the reports of the Alzheimer's disease vaccination not working earlier this year.

The medical journals themselves know this is a problem, but they aren't sure what to do about it. One suggestion has been the journals should make a 'commitment to publish' for a trial based on its design and importance, and then publish the findings whatever they are, but that would of course mean they'd be publishing a lot less interesting stuff.

Also scientists can't be bothered to write up negative findings, because there are proportionally so many more of them than positive ones.

There is a problem of pharma companies suppressing bad findings or writing misleading journal articles, but that's a wholly independant issue and should be dealt with by the schemes in develpment for enforcing results to be left in public databases after the trials are finishsed. The problem with that is that not many GPs will be searching through these databases

The real solution is probably for doctors not to rely on scientific journals for information (which they often don't really understand), but to wait for advice from bodies like the National Institute for Clinical Excellence or the Cochrane Collaboration who do very thorough searches and synthesis.

Oh no! Systematic reviews aren't possible! Quick, somebody tell the Cochrane Collaboration!

OK I feel I have to nit-pick a little.

1) Physicians are mostly trained to be healers (practitioners) not scientists.

      I think that most physicians start out as "scientists", after all, most of us come from science majors. However the biggest mistake that medical students make is thinking in absolutes. Yes, anyone can memorize anatomy, or physiology, or pharmacology, etc. After all, there are only so many anatomic variations in the population, a person's nerves will always transmit a signal the same way, and warfarin/coumadin will always interact with pretty much any other medication you prescribe.

      But something happens during the career. When you actually get a patient in front of you, suddenly all those absolutes go out the window. Everyone is different. Rarely will you see the "classic" or "textbook" or "tv medical show" presentation of a disease. Many patients come with very vague or mysterious complaints. An example, which happened in my practice a month or so ago- who would have guessed that a patient who was treated successfully 2 times elsewhere for a recurring skin infection actually had diabetes - which was CAUSING the infections? Only a physician. After all, if it was THAT easy who needs doctors - we have wikipedia. Right? WRONG.

      So I must state that medicine IS a science. Filled with facts. You can memorize the pathology. You can study algorithms (they do exist) to reach diagnoses. Patient comes in with X, Y and Z, just follow the algorithm. And you'll be right, a lot of the time. But the SUCCESSFUL practice of medicine - how to APPLY the science, is an art. Just like Engineering is an art. Any fool can build a bridge, given enough scientific knowledge of materials and forces and tensile strengths, etc. But the guy who builds a good bridge (except in Minneapolis) where no one thought he could - he is an artist. Does that make him less of a scientist?

I agree with your second point. We're dealing with the normal curve ALL the time. Most patients will have response "X", but you'll get some with "A" and "Z" and there's always one with "4".

3) Dealing with inaccurate data requies advanced knowledge of matematical statistics and most medical doctors do not have a basic grasp of this field.

      We don't need it. That's what we have peer review for - supposedly. To save us from all the BS and pet theories and make sure that only decent studies get published. It doesn't always work, though. But this is also why things like the Cochrane studies are important. Your average doc in the street doesn't need to battle with statistics. (S)He just needs to know if what (s)he's going to or what (s)he's been taught do WORKS. Studies like Cochrane let us look back and see if what we're doing actually WORKS or not. Despite whatever was published/hyped/etc.

I agree with points 4 and 5. Anyone who is going to publish must consult with a statistician. Medical practice is interdisciplinary. Why should medical research be any different?

I got owned by not previewing. Here's the other article I meant to link in there.

http://www.cochrane.org/reviews/en/ab004022.html

This concept is not relatively new. Randomized clinical trials, involving trying to protect the brain, in surgeries like CABG (Coronary artery bypass graft) are taking place for more than 10 years now. 5 years ago, a review of many such trials found that though stroke related deaths decreased by inducing hypothermia, they faced other non-stroke related mortality in operations and overall there was no difference between hypothermia and normothermia. This article http://www.cochrane.org/cochrane/revabstr/AB002138 .htm goes over it in more detail. However, they don't go into the details of to what temperature was the hypothermia induced. Since CABG is a preplanned procedure as opposed to trauma, I guess inducing hypothermia is different in both settings, and I hope we get some positive results in humans.