Slashdot Mirror
Wikipedia Medical Articles Found To Have High Error Rate
Rambo Tribble (1273454) writes "A group of researchers publishing in the Journal of the American Osteopathic Association found that 90% of the Wikipedia articles they sampled contained errors regarding common medical conditions. Unsurprisingly, they recommend your General Practitioner as a more reliable source, while noting, '47% to 70% of physicians and medical students admitting to using [Wikipedia] as a reference.' At issue in the study is the small sample size the researchers used: 10 medical conditions. There are also ongoing efforts to improve the quality of Wikipedia's articles. According to a Wikipedia spokesman, '... especially in relation to health and medicine.' The BBC has more approachable coverage."
[Citation Needed]
But also, Osteopathy leaps a bit toward the âoealternative medicineâ side of things, it wouldnâ(TM)t surprise me if Osteopaths have some issues with medical articles based on more traditional medicine.
Having said that, how dare these quacks question the accuracy of Wikipedia! Donâ(TM)t they know they can run the gantlet of snooty Uber Editors with âoeownership issuesâ and correct these articles themselves? Of course you can be banned doing thatâ¦
If you want news from today, you have to come back tomorrow.
...Wikipedia has been a massive success but has always had immense flaws, the greatest one being that nothing it publishes can be trusted. This, you might think, is a pretty big flaw. There are over 21million editors with varying degrees of competence and honesty. Rogue editors abound and do not restrict themselves to supposedly controversial topics, as the recently discovered Hillsborough example demonstrates....
I am shocked. Shocked. No one would use a widely accessed platform to push a POV nor would it be adequately vetted by professionals for accuracy and completeness and edits limited to trusted sources. Add in that their are many more people who think they are experts that aren't and it is a wonder that Wikipedia's accuracy is above 0%.
I'm a consultant - I convert gibberish into cash-flow.
Too much time reading so called "books" and "Medical Journals" to realise the upsetting truth that nobody on the internets is ever wrong.
A /. article mentions that Wikipedia has an error in 90% of medical articles /. itself has an error in x% of news items posted here /. has a 0% error rate...
Now, keep in mind that
So, the actual error rate of Wikipedia medical articles should be (1-x/100)*90 % shouldnt it?
Assuming it is actually 90% would lead to the conclusion that
the accuracy of medical journals then.
Like I am going to accept wisdom from a bunch of osteopaths???
Yes, I get that no article on Wikipedia is going to be 100% accurate, but this study is just plain bunk.
First off, they mention that they had "experts" review 10 articles for the most expensive-to-treat medical issues. They have all kinds of mathematical figures, but nowhere do they actually list key things like:
- Who was it that reviewed each article?
- Were they an expert in that field, or an osteopath?
- Which "peer-reviewed sources" were they using?
- How did they determine mistakes?
None of these questions are answered in the "methods" section of their paper. Further, their OWN SOURCES dispute what they found. For instance, they link to http://jop.ascopubs.org/content/7/5/319.abstract?ijkey=428353f0b3eb338fad1bf0f79139dd275c7670fe&keytype2=tf_ipsecsha , a study that looked at cancer information on Wikipedia versus information in a maintained professional database on the same subject. What did they find?
"Conclusion: Although the wiki resource had similar accuracy and depth as the professionally edited database, it was significantly less readable. Further research is required to assess how this influences patients' understanding and retention. "
This sounds like bunk to me.
Cancel the appointment with Dr Otto Didact, M D, University of Wikipedia.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
I am a physician, and I admit that I use it on an irregular basis. But let's keep this in context. I don't look up how to diagnose or treat conditions. I do use it to look up obscure things, as well as review anatomy. Information that either is just for personal knowledge that is not critical to management (example: what is the name of the nerve that innervates the serratus anterior?), or information that is hard to get wrong (example: what are the muscles of the knee called? I once had to look up VMO because I could not remember what the "O" stood for). Even then, if it makes even a small difference, I always look it up further in a medical resource. So I am one of the 47-70% of physicians who look up facts in wikipedia. I don't think that is a bad thing.
The much more interesting output of this study is how often the expert reviewers couldn't agree whether an assertion is or is not supported by literature. Kinda hard to conclude whether the wikipedia article is accurate if two experts disagree on about half of its statements' accuracy. Drawing conclusions on data this noisy is in itself something that puts the journal's name in jeopardy (as if it had any in the first place). Wikipedia is far from perfect, but this "peer reviewed article" is even worse.
And who makes those judgments? I can go to pubmed.com right now and find quite a few contradictory articles, and more than a few that might charitably be described as "fluffy."
If the goddamned medical community is so concerned about this, they can come up with a web site that's peer reviewed by their selected group of experts and pretends to be the last word on medical data.
No word yet of course, on how the esteemed "medical community" missed the problems with Vioxx, post-menopausal hormones, cobalt hip implants or any of that sily stuff. Because they're like, you know, infalible.
I read Wikipedia, knowing it's inaccurate. I cross reference and look at multiple other sources like a big boy. I read *everything* knowing that there's inaccuracy somewhere. Sounds like it's time for everybody to grow up. There's no great, all-knowing source of information *anywhere.* No group of wise thoughtful, beard stroking authorities who know all and see all.
It's just us, doing the best we can with the crappy information we're given.
Please do not read this sig. Thank you.
these criticisms of wikipedia are ignorant and useless to profitable discussion
it shows an inherent misunderstanding of how citations work *AND* how the internet works
not every sentence must be cited by a peer reviewed journal...ever...anywhere...only some law briefs go to that length & a human may or may not ever read it
science is NOT a citation competition, nor is it a pedantry pageant
the 90% figure is bullshit stats conjuring...where are the examples?
what's there threshold for "error"?
they only looked at *10 articles*
and they DO NOT, repeat, DO NOT identify side by side the "bad" knowledge of wikipedia and their correct source
all we get is this admission, which confirms all my criticisms, from TFA:
so no consistent definitions of terms or standards were used, at all...
this is crap science....[citation needed]
Thank you Dave Raggett
Using percentages when speaking of a sample size of...god damn 10 conditions....is just really effing stupid and looks like it was specifically meant as click-baiting. The honest way of saying this would have been simply "The researchers sampled 10 conditions on Wikipedia and found that 9 of them were incorrect." See? No alarmist "90%omgomgworld'sgoingtoburn" bullshit there.
Now, call me back when the sample size is actually worth a damn. 10 conditions out of all the bajillion different ones mentioned on Wikipedia is simply too little to draw any sort of meaningful rule about the quality of them all.
I'd be interested to see a similar review of sites such as WebMD. Is this only a Wikipedia issue?
Sent from my TARDIS
http://en.wikipedia.org/wiki/Osteopathy
As with all forms of complementary and alternative medicine, the practice of osteopathy does not always adhere to evidence-based medicine (EBM). There are few high-quality research studies demonstrating that osteopathy is effective in treating any medical condition other than lower back pain. In the UK, the National Institute for Health and Care Excellence recommends osteopathy for the treatment of persistent lower back pain. However, analysis of peer-reviewed research yields little evidence that osteopathy is effective for non-musculoskeletal conditions, and limited evidence that osteopathy is an effective treatment for some types of neck pain, shoulder pain, or limb pain.
Seriously - how many physicians, even among the specialists, keep themselves up to date on the latest research? Many of them do, many of them are passionate, geeky about what they do, and in their spare time they'll be reading up on the latest research, they'll go to conferences, etc., like a passionate geeky programmer would. But many, and i'd say most, just don't. Their knowledge is whatever they were taught. And that wasn't necessarily the state of the art at the time they graduated - that depends on how up to date their *teachers* were.
So, yeah, wikipedia might be misleading; it might be out of date in certain places - in many places, even. But i don't necessarily think your physician will be more up to date. And i'm not sure how to fix that, either, because they *should* be!
The same type of study has been run on webMD several times in the past with similar results. The only difference is that webMD always seems to think you might have cancer.
HUMANITY has a high error rate.
When they found the errors, did the reseachers click on the "Edit" button and correct?
I don't think anyone claims that Wikipedia is perfect, but it's fairly straight-forward to fix things. (Unless some ass-hat editor gets in the way and keeps reverting one's changes.)
The only useful comparison would be against a print-edition encyclopedia. What percentage of medical articles in a typical encyclopedia contain errors? The other thing is, just because it contains "an error" doesn't mean it isn't useful. We get through most days with a fairly flawed view of reality (most of us anyway).
"I have never let my schooling interfere with my education." - Mark Twain
...if they are using Wikipedia as their reference? (or at least 47-70% of them)
It is an art, better crafted by women. Except OB/GYN which of course is better performed by men (if you ask me).
Amen to that. This is why you ask the appropriate person the question. I ask my pharmacist about drugs, the nurse about practical elements, dietician about food, etc.
It should be common sense that one should not rely on WikiPedia for accurate medical info. But, there are not many alternatives. It would be nice if the AMA or similar institution created a kind of medical wiki that is only author-able by vetted practitioners.
Each topic could have a "regular Joe" section (tab) for us, and a medical-expert section (tab) for medical professionals with all their glorious lingo.
Why scatter and reinvent such knowledge all over the place? I'm sure putting such together is a lot of work, but it's work that is already being done, just in an uncoordinated way.
Table-ized A.I.
I saw it on the internet.
Seriously, Who's surprised that Wikipedia has errors in the medical information it contains? Who is surprised that there are doctors that still use it? We all act like this is somehow a serious problem, or that it's dangerous. Dangerous compared to what? Wikipedia is an excellent place to *start* an investigation, but before you start making life and death choices, you really need to check some other sources. Doctor's are generally NOT stupid enough to just go with something they find on the internet.
If you suspect your Doctor is out running a Google search on "Lower abdominal pain with fever" just because they don't know better, I suggest you get another doctor, one who actually has some training. And if you see some website that tries to tell you that you have malaria because you are running a fever you are stupid if you don't consult a doctor before being treated for it.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
Our family doc is a DO, not an MD. She's accepted by my insurance plan, she prescribes appropriate medicines and treatment (not body manipulation) for our family, and IMHO manages my particular health conditions more sanely than our previous family doc, an AAFP board-certified MD. What's more, my spouse (another AAFP board-certified MD) trusts her judgment.
I mean, I know Slashdotters like to label stuff as 'woo' and then jump all over it, but seriously, not all osteopaths are kooks.
For a few years I maintained a sizeable collection of Wikipedia articles. I was very meticulous in checking all of the data, trying to use only the best sources and citing them all, per section of each sentence if necessary. However, it was a constant battle to keep others from adding anything from dubious information found in newspaper articles ("Somebody printed it, so it must be true!") to subtle attempts at vandalism (e.g. changing 501 mg to 502 mg for no reason). Many poor articles are eventually raised up to a certain level, but over time the good ones are also erroded to a point where they contain many more errors than expected. Other than relying on armies of experts (who often receive little respect) to constantly police their articles, Wikipedia has no mechanisms to prevent this from happening. It's a fundamental problem for them, but one which they can do little about without changing their most basic policies.
>> 47% to 70% of physicians and medical students admitting to using [Wikipedia] as a reference.' At issue in the study is the small sample size the researchers used: 10 medical conditions.
Uh...between 47% and 70% of people means you surveyed what - 3 people? 4? (OK, I looked - it's a range of numbers from OTHER people's surveys.)
Here's just one possible flaw with that conclusion: If I was a doctor, I would look up what Wikipedia says about a condition just to see what my patient is going to read when they get home, so I could arm him or her with the right information (rather than Wiki's).
It's not Unicode, it's that iCode crap from iApple.
IMHO the article written is not of publishable quality. (The journal it *did* get published in has a very low impact factor of 1.3.) It's badly written, poorly supported, and subject to substantial methodological errors.
Each subject comparison is based entirely on the subjective evaluation of a random med student. It doesn't seem like they even provided them with standard protocols. They just assumed that any discrepancies represented factual errors in the wikipedia article. They didn't make comparison to other sources or even internal to the literature.
It would be lovely if they would actually include some of the assertions they evaluated. But frankly I would put infinitely more faith in the Wikipedia articles cited than this particular report. Certainly they represent better and more substantial writing.
When things get complex, multiply by the complex conjugate.
Unfortunately for the premise of this study, Dr Ioannidis' well-known findings suggest that most scientific papers are also inaccurate. So we can't draw reliable conclusions as to the accuracy of Wikipedia articles. Indeed, it is possible - though admittedly quite unlikely - that the Wikipedia articles are correct in each case, and the scientific papers incorrect.
See http://www.plosmedicine.org/ar...
I am sure that there are many other solipsists out there.
It's not Lupus! (for the House fans in the audience)
-
...Many articles published in medical journals are also wrong. What about the never-ending debate about X being good for you, then bad, then good again, where X = coffee, butter, etc.?
Once i knew the condition, the next visit, I knew more than the doctor.
And doctors looked at the symptoms and got them wrong as well.
And many doctors treat only the numbers. This ignores the fact that humans react differently to substances.
But it all fits their practice model.
Train hard. Then mostly stop training and rely on information from drug people. See a patient once or twice a year-- one among several hundred-- so you really have no clue who they are or what is wrong with them other than your notes.
Write hundreds of prescriptions-- so they all sort of start to blur to gether plus your staff often really screws up your prescriptions. I've had multiple prescriptions doubled in strength, halved in strength, increased or decreased in frequency from what the doctor said in our meeting. For the doubling- I often let it stand since the price was the same and cut the meds in half and then built up an emergency supply. Which was good because several times the insurance company got a bureaucratic burp and there was a 10 to 15 day interruption.
But doctors are generally more accurate at detecting when you have something new. It shows up in the numbers first. And they can filter out a better diagnosis.
Each has its place. It is VERY important to be "patient active". Here's the key example. I had chemo and always checked my settings. A patient with me didn't and they misset or misprescribed his chemo by 100:1. This was something that was 99% curable if handled properly but getting that much chemo didn't kill the cancer and they had to send him home because he couldn't take any more chemo of that type. So he probably died since that was the only chemo that worked.
Trust but verify.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
This research was done by comparing the content on Wikipedia to peer reviewed documents. How many of these documents are available to Wikipedia editors without expensive subscriptions? I agree that proper sources should be cited, but I also think that all these sources should be publicly available.
(Maybe one day I should actually create a Slashdot account ...)
in residency, and yes, from time to time I'll look up something on wikipedia on my phone for a quick overview if its a condition I'm not familiar with, or is outside my specialty, and I'm rounding or otherwise away from a computer. However, I don't use it for treatment or diagnostic purposes; there exist much better, peer reviewed sources for that, which I will happily access from a computer. That being said, I'd say a large amount of the wikipedia articles tend to be pretty decent, and at least sound as if they've been written by someone with some sort of formal medical treatment. They get the quick and dirty job done about 75% of the time for me.
SQUORE
...when every medical student and intern carried around a copy of "The Merck Manual" http://www.amazon.com/The-Merc...
They even have an on-line version now. Dear gawd why would an MD/DO, or even a wannabe, use the wiki for such things?
If you really want a "holistic" study then you should include stripper massages versus Osteopathic treatment versus reading a Wikipedia article and measure overall satisfaction as a measure of accuracy. If Wikipedia beats the stripper massage, then we know where to send the $20.
In part the issue is one of authority. Who is to decides the 'true' content and description? Just for example, do a search for 'insulin resistance' using Google scholar. You will get about 1,870,000 hits. It is obvious that no one author is going to even read a fraction of those hits. So how do we decide what is right? Well, we use 'official' sources where it is assumed that very knowledgeable scientist have carefully reviewed the evidence to create an authoritarian description. The problem is that authoritarian sources are not necessarily right. We can find many examples in history that we now know were wrong (e.g., flat earth, earth as center of the solar system) and just statistically speaking I am sure that there is a significant number of current authoritarian sources that are wrong. So what do we need? My prescription is a vigorous debate and a willingness to challenge authority. After all, that is the scientific paradigm: hypothesis are not proved by evidence; they are rejected by evidence. We need a lot more of that. And that is what we get with Wikipedia today.
How is the error rate compared to General Practitioners? Surely you don't believe those guys never say any inaccuracy do you?
In fact, almost any source of information in an ever-changing science and practice will contain statements that can be contested. Even major medical textbooks disagree in details or between editions. So, I don't thing anyone expects a wikipedia article to be absolutely accurate because such an article rarely exists even in the "peer-reviewed" domain. In practice, I used wikipedia as a decent source of information several times. I suppose I would have noticed glaring omissions or errors and I'm not even looking for critical pieces of information (say, chemotherapy protocols). For professionals, wikipedia is a tool like any other. For patients, things can be a little more complicated. In the end, I think wikipedia is much better than wacky sites offering "natural cancer treatments" or other scams. So, overall, nothing to see here.
When the primary sources of the knowledge you are compiling are also rife with errors your articles will be full of errors as well. This isn't unique to wikipedia, all encyclopedias suffer from this.
There is no memory shortage. yes I have heard of XFCE. Go away.
While the results of this particular study may be questionable, it's annoying to see how many comments dismiss the study out of hand just because it was performed by osteopathic physicians. I can't speak for the rest of the world, but in the US, "Doctor of Osteopathic" has a specific meaning - they are legitimate physicians whose training differs from that of allopathic physicians in philosophy rather than in medical knowledge or practice.
The only medical distinction between a "traditional" MD and a DO is that a DO undergoes an 8 week course in osteopathic manipulative treatment (OMT), which is a specific therapy for lower back pain, and which NIH studies have shown to be "mildly to moderately effective." Other than that, medical training between the two branches is indistinguishable.
There's a common quip about quack medicine: "What do you call alternative medicine when it gets scientifically verified? Medicine." I find it comforting that at least one group of (former) quacks in the US actually took that sentiment to heart. Now if only chiropractors and homeopaths would do the same...
Wikipedia NO, but thorough scouring of the Internet for information coming from peer reviewed medical journals - U.S., and foreign - greatly reduces the informational advantage that physicians used to enjoy over patients. Furthermore it puts up to the minute information in patients' hands, vs. docs who are so overloaded with patients they scarcely get time to go to conferences or view lectures to satisfy continuing education requirements. Doctors also tend be highly opinionated people who quickly discount information shared by lay people.
Several years ago, I was diagnosed with Borellia Burgdorferi (Lyme Disease) by a family practice doctor at a walk-in Urgent Care outpatient facility. After 28 days of Doxycycline - the "standard protocol" - my symptoms returned significantly worse.
Had I followed the generally held beliefs of the medical community, I would probably be in a wheel chair right now, unable to work or function. Instead I aggressively scoured the Internet for more information, for weeks. On an out -of-the-way antiquated looking Lyme support website, I found recommendations for a Lyme specialist 3 1/2 hours away in another state that was willing to risk the wrath of insurance companies and even loss of his medical license by treating patients in the manner known to actually WORK - Profoundly high dosages of combination antibiotics over extended periods of time (many months) and radical dietary changes with added nutritional supplements.
I sit here today - now several years later - in perfect health thanks to "Internet Medical Information" and a courageous doctor that flew in the face of conventional wisdom.
Should you listen to your doctor? Yes. Far more often than not, they are going to be correct. However, "trust but verify" should be your approach- particularly if you are faced with life changing, life altering, or potentially life-ending medical care decisions.
Your physician is but one person, while the internet contains the collected wisdom of millions.
THIS SPACE INTENTIONALLY LEFT BLANK.
I have a condition that was caused by an adverse reaction to a medication, specifically Cipro which is a fluoroquinolone antibiotic that cannot distinguish between bacteria and mitochondria and, thus, destroys and mutates the latter. Numerous members of the 'fluoroquinolone toxicity' community have attempted to flesh out the articles on fluoroquinolones to include information about the rate of incidence of side effects, details on the way these adverse reactions occur from medical journals, etc. and the pharmaceutical companies have without fail nearly instantaneously deleted all the information about adverse reactions and links directing to scientific studies. Wikipedia has become a way of sanitizing reality from unfortunate facts and it's a damn shame that it's given any credence as an actual repository of factual information given the complete lack of oversight.
Boo osteapths! They're not real doctors like allopaths (M.D.s). If I'm sick I definitely want to be treated by and M.D.
Never mind that according to the JAMA, allopathics medicine is the third leading cause of death in this country. That's right, add up drug interactions, prescription errors, botched surgeries, hospital-borne infections, missed diagnosis, and other medical mishaps, and they add up to the third leading cause of death.
But thinking that disease can affect the posture of the body, that posture can affect disease, and the by manipulating the body we can effect biochemical changes, well that's just dangerous thought right there.
The fact that wikipedia is about the best opinion and not about facts is your first clue. That is the reason any person with more than 2 brain cells know better than to use wikipedia for anything.
This is the worst Slashdot posting I've ever seen. The title mis-states the conclusion, which is not that Wikipedia medical articles have a high error rate, but that a high number of Wikipedia medical articles have an error rate. The study studied 10 articles. The comments section is a pissing match about Osteopathy rather than the subject to the point that people are concluding that these must be Spirit Practitioners who didn't like Wikipedia's approach to chakra and so were writing something negative. The study itself doesn't actually list the errors it found, or even the types of errors, just that assertions in the articles were not supported by best available evidence, the related links are mostly other garbage submissions, the submission links to the BBC "for more approachable coverage" even though the journal summary itself is written in a very accessible manner.
It's basically everything that could possibly be wrong with a Slashdot article besides being an old dupe.
I've been to a lot of doctors lately... 7 to be exact, and all of them make at least minor mistakes. 100% of them as a matter of fact.
I'm willing to bet Wikipedia is more right overall than the majority of paid medical people.
I refuse to post on wikipedia. I have had almost every edit I have ever made reversed by an asshole editor. IN AREAS THAT I AM AN EXPERT IN, AND THE EDITOR IS NOT. It's like I am Galileo having my posts on Heliocentrism edited by the flat earth believing church. Unless we can out these arrogant editors and replace them with true experts, it will never be resolved.
Mike @ The Geek Pub. Let's Make Stuff!
That the article doesn't actually give any examples of what the errors were, nor attempt to assess whether they were material errors or trivial? The methodology is admirably explicit on the science and completely opaque on the cognitive method used by the not especially eminent physicians to determine whether an assertion was accurate or inaccurate in either the Wikipedia article or the medical literature equivalent.
Seems pretty dubious to me!
they recommend your General Practitioner as a more reliable source
They mean the kind of doctor that is of aware of anything new in medicine since he graduated 20 years ago, except for drugs that big pharma advertised?
Better known as the PDR, anytime your Doctor says excuse me for a bit. The chances are very great they are flipping through a PDR trying to find the "right pill/treatment" for you.
If your Google the PDR you get link after link of how reliable it is on, all but the first hit.
http://www.personalconsult.com... and it nails the problem with the PDR.
"The PDR is merely a drug's package insert. It is a FDA regulated article limited to merely the research submitted to the FDA typically to get a product approved for sale to you. Sometimes the information is from research from after the drug is out and being used by patients--new issues or problems arise. Period. It offers little else!" (edited "FDAÑtypically to FDA typically")
If I gave that to kids with psychosis, I would have vomiting and stuporous patients. Continuing to follow the PDR would be cruelty." ...
"In no way does the PDR describe nor purport to describe the standard of care. Half the prescriptions in the nation are written off label. In other words, doctors think of useful and helpful ways which have not been approved by the massive FDA, you know, the ones who shut down Canadian drug stores in the USA.
If a doctor fails to place patients on a medication for the non-approved PDR indication, but the custom is that most doctors do, the doctor is clearly outside the standard of care. Thus quoting the PDR as authoritative represents the failure to comply with half of the standard of care in the US.
Some doctors would testify that limiting oneself to PDR approved indications and dosage is quackery that should result in the loss of license, as a threat to the health of the public. Half the customary prescribed treatment would be missed by this doctor."
http://www.personalconsult.com...
Ok, Wikipedia articles have errors - according to their analysis.
Take a bunch of doctors and have them write articles on the same topics to the *same level of detail* and have them judged as well.
How many of those articles will be judged as having *no* errors?
You see, if they were really *scientists* interested in the truth of whether your average doctor had better information than Wikipedia, that's what they would have done. A comparison.
But instead, they claim to have found errors in Wikipedia (which doesn't establish that there were errors, only that they thought so), and then said "hey, keep forking over money to our government empowered guild". What a surprise.
http://acutemed.co.uk/
Casteism