Slashdot Mirror
Meta-Research Debunks Medical Study Findings
jenningsthecat writes "From The Atlantic comes the story of John Ioannidis and his team of meta-researchers, who have studied the overall state of medical research and found it dangerously and widely lacking in trustworthiness. Even after filtering out the journalistic frippery and hyperbole, the story is pretty disturbing. Some points made in the article: even the most respected, widely accepted, peer-reviewed medical studies are all-too-often deeply flawed or outright wrong; when an error is brought to light and the conclusions publicly refuted, the erroneous conclusions often persist and are cited as valid for years, or even decades; scientists and researchers themselves regard peer review as providing 'only a minimal assurance of quality'; and these shortcomings apply to medical research across the board, not just to blatantly self-serving pharmaceutical industry studies. The article concludes by saying, 'Science is a noble endeavor, but it's also a low-yield endeavor ... I'm not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life.' I've always been somewhat suspicious of research findings, but before this article I had no idea just how prevalent untrustworthy results were."
If medical research were really as close-to-useless as The Fine Summary claims, we'd be hardly better off with modern Western medicine than with homeopathy and prayer. Clearly, we are, refuting the idea that medical research doesn't do a huge amount of good. I'm not saying it isn't flawed, but give it some credit.
Le français vous intéresse?
The problem I know because I am (retired) RN. I went to my MD for a severe sinus infection and chest infection some years ago. During the visit I asked for Abuterol Inhailer to assist with clearing my chest. This is standard Respiratory Therapy stuff. They gave me a puffer which I took home, used 10 or 15 puffs out of and threw it in the drawer. Ever after that BCBC has me as Diagonsis Asthma. I am not asthmatic. This will screw up my healthcare for the rest of my life! Makeing all of these errored stupid databases cross link will do far worse than this. My daughter (age 23) was emergency taken to the local hospital with what appeared to be an Epileptic Seizure. Consequences included she couldn't drive for 6 months! I will skip the details, her seizure was a cardiac seizure. It took her actually taking her case to the local Fire Department to get a heart monitor strip to make this undeniable. She is now treated well but forever she will be DX Epileptic even though it is completely wrong. Does any sane person want this sort of a system where you cannot go to another doctor and have him/her look at you rather than some record first? Who wants in that trap? If you are an MD in that trap even if you see that the record is wrong, you can go to jail, lose your job etc all if you go against this insane record that is completely in error. Please wake up people this is a prison without walls! You cannot escape! You will have to leave the country to get away from a bad diagnosis or a stupid keystroke error. Remember the computers have a forever memory and no intelligence.
I'd replace item C with regular fucking. Don't see the quacks, just fuck someone. Chiropractors are worthless and evil. Fucking is fucking AWESOME!
Fascism trolls keeping me up every night. When I starts a preachin', he HITS ME WITH HIS REICH!
That's one of my favorites... or the presumption that eating fat makes you fat.
People seriously do not understand nutrition or how diet and exercise work. Lately, I have been doing an kind of experiment for the people around me. First, for about a month or more, I started riding my bicycle to work. I was working it hard. Then, after it was established that I had been riding my bike for at least a month, I started on a low-carb diet. Within two weeks people started to notice the weight loss. Some still wanted to believe it was the bicycle riding. I had to lay it out to them what the deal was. Exercise burns carbs and then fat. Trouble is, the carbs we take in our daily diet still outnumbers that which I burn from riding 10 miles each day. It is only after I limited the intake of carbs that a difference could be made and observed.
Here's why I did it like this:
People don't listen for more than a few sentences and are especially resistant when the information conflicts with what they think they know. Eating fatty meats is contrary to their beliefs about what a weight loss program should contain so they simply refused to accept it. Hell, even many doctors don't yet fully acknowledge that making your body burn fat will reduce cholesterol. (Hello? cholesterol is fat floating in the blood!)
Having lost almost 30 lbs in a 6 week period has been noticed by all and the evidence is right in front of their faces. I lost the weight, and this is what I have been eating.
I don't want to discount the value of the study itself. Clearly it's important to quantify how bad the problem is and try to develop solutions. But at the same time, the article and summary might give one the impression that the errors and biases involved were newly discovered by the researchers. A few examples:
The secondary marker problem (e.g. tracking cholesterol levels instead of real outcomes like deaths)
Comparing new drugs only to placebo or only to drugs that aren't best-in-class or using an intentionally weak dose of the comparison drug
Using meta-analysis of other studies instead of doing new research (and often doing it badly)
Doing retrospective analyses like chart reviews instead of prospective studies (and often doing those badly)
To expand upon that last example: common problems with the methodology of chart review studies were investigated thoroughly by Gilbert and Lowenstein in 1996. Despite their findings and recommendations for how to do a chart review properly, things haven't improved much since.
Many doctors and researchers have been critiquing studies and warning about these problems for years. In the emergency medicine context, for example, Jerome Hoffman, a UCLA medical professor and emergency department physician, is well known as a critic of poorly designed studies in the emergency medicine literature. He has critically reviewed studies since 1977 as part of a continuing medical education program called Emergency Medical Abstracts.
So the problems are well known. The bigger issue is how to fix them.
For a year and a half I tracked every ounce of food I ate, recorded it in software I wrote, meticulously recorded calories burned on exercise machines (this is the largest source of error, but I did my best to compensate) and found the following:
Graphing my actual weight and my projected weight using (start weight - ((2660 - calories eaten + calories exercised) / 3500) will never be more than 3 pounds apart, even over a weight change of over 100 lbs, regardless of whether my caloric intake was from a tub of Crisco or from cucumbers dipped in virgin blood. Conclusion: eat whatever the hell kind of food you want, just keep the calories low, your long-term weight change will be the same. But it's easier to eat fewer calories, in my experience, if you cut out carbs. They make me hungrier.
The +/- 3 pounds fluctuates based on when you last peed, how much you drank and when, and how much meat is sitting in your colon pending expulsion.
Doctors are notoriously bad at doing scientific investigations, and since peer review journals are reviewed by peers, they don't know bad science when they see it.
I've been arguing with my father about this for... what, fifteen years?!? He subscribes to the New England Journal of Medicine and is forever telling me about the latest study telling you to that standing on one leg reduces breast cancer in nuns. These studies are, without fail, trivial to eviscerate as their control groups are either non-existent or very poorly picked.
Doctors just don't know much about science. That shouldn't be surprising. They have terrible educations. After all, they studied premed in college and then went to a vocational school. MD/PhDs really ought to do better, but I suspect they do their doctoral work with faculty who come from the same under-educated group.
The problem is one of prestige. You cannot walk into a group of doctors and tell them why they're dumbasses and what to change. They know, and society confirms for them, that they are the best educated people on the planet and smarter than anyone else. "Heck, just look at our paychecks," they might respond. "That proves it right there, doesn't it? Figure it out from that data point only -- you don't need a control group."
"Reduce the primary source of fuel so that it will use the secondary source of fuel. It's really that simple."
No, it isn't.
The (almost) only direct source of "fuel" for the organism is sugar (glucose), full stop*1.
Anything else the body ingests or stores has to be reduced to sugar (usually by means of the krebs cycle) prior to be "burned".
While this is a very basic simplification, this, and the fact that the blood can carry a limited level of sugar at a time, is what explains, at a whole body level, everything else.
Like...:
* Since you can only burn sugar, sugar-equivalent contents is all that counts for weight imballance (of course, within limits: you can't just stop your ingestion of, say, oligoelements). That's what we really talk about when we talk about food calories.
* If you directly eat sugar (glucose), the sugar will be immediatly burn, but since your blood has limited sugar carriage capacity, you should be continously eating like a hummingbird to sustain that, so you usually just can't eat sugar in excess.
* If you eat carbs, they'll be transformed into sugar and burned. Any carb in excess will be stored as glucogen in your liver. If there's still carbs in excess once your liver can't hold any more glucogen, it will be transformed into fat and stored under your skin.
* If you eat fat, it will be transformed into sugar and burned. Any fat in excess will be stored under your skin unless you are so low in glucogen (which usually won't happen) that part of the fat is transformed into glucogen and stored in the liver.
* If you eat proteins, they'll be used for structural development (like muscle mass). Usually, anything in excess will be trashed away, unless you are very low in sugar, carbs and fat intake (it usually doesn't happen) in which case, it will be uneffitiently transformed into sugar and then, burned.
* To explain for long term weigth, all that's needed is accounting for your ballance between ingested calories and burned calories: if you eat more calories than you burn, your weight increases; if you eat less, you lose weight.
For a practical example:
If you eat less carbs and more fat to the point that daily calories stay the same, You Will Not Lose Weight (but in the long run you will develop cardiovascular illness).
If you eat less carbs and more proteins to the point that daily calories stay the same, You Will Not Lose Weight (but in the long run you will destroy your liver).
Given the ballance between ingress and burn, you can obviously go two (complimentary) routes:
1) Eat less calories. Sustaining a varied and ballanced diet, only eating less, is the way any sensible nutritionist will suggest since it's the easiest to do properly long term and the easiest to lead to you changing your habits. But as long as you stay to the First Principle "eat less calories", and within sensible limits, you will get it right.
2) Burn more calories. That's where exercise and rising your basal metabolism come into account. Aerobic exercise is an obvious recomendation, but other less obvious things like lowering your home thermostat 3~4 degrees in winter will have it's effect too. Again, it's not what you do, but what you achieve with regards of burned calories.
Everything else about diets is about making acceptable for you to eat less calories/burn more calories (like, unless you are a kind of iron-man you won't have the will for strengh training like weight lifting unless you go heavy on sugars; the same with aerobics, like long distance running or bycicling unless you go heavy on carbohidrates, or you'll probably break your diet if you are just told "eat exactly the same as you did, only on third the quantities", so you are offered a diet with much less calories but about the same or even more volume so you feel satiated, or you'll probably will abandon a diet if you don't see fast results at least at the beginning, so you are offered a diet very low in calories for the first weeks so you fastly see your efforts are