Algorithm Finds Thousands of Unknown Drug Interaction Side Effects

ananyo writes "An algorithm designed by U.S. scientists to trawl through a plethora of drug interactions has yielded thousands of previously unknown side effects caused by taking drugs in combination (abstract). The work provides a way to sort through the hundreds of thousands of 'adverse events' reported to the U.S. Food and Drug Administration each year. The researchers developed an algorithm that would match data from each drug-exposed patient to a nonexposed control patient with the same condition. The approach automatically corrected for several known sources of bias, including those linked to gender, age and disease. The team then used this method to compile a database of 1,332 drugs and possible side effects that were not listed on the labels for those drugs. The algorithm came up with an average of 329 previously unknown adverse events for each drug — far surpassing the average of 69 side effects listed on most drug labels."

Comments

I was wondered about something

[zero.kalvin]

What kind of statistical analysis methods they use in these studies? For example we use a lot of Likelihood functions, BDT, and neural networks to get the maximum number of information out of our data. Do they use these kind of methods in there analysis ? ps, my field is astrophysics and astroparticles.

Re:I was wondered about something

[whydavid]

In Biomedicine you tend to see a heavy reliance on T-Tests, Chi-Square variants, Fisher's Exact, regression, McNemar's and Cox Proportional Hazards when temporally rich data is being tested. I don't have access to this article yet, but I would be surprised if they weren't performing a paired T-test in situations where outcome variables were measured on a ratio scale, McNemar's for binary outcomes where temporal data is not provided (maybe rare or nonexistent in this study), and Cox Proportional Hazards if there are any cases where we have a long temporal history of the data. Based on the sheer number of hypotheses tested we would expect to see some sort of correction for multiple testing here, too.

Re:I was wondered about something

[buchner.johannes]

As usual, Science&Nature only provide high-level info, so you'll have to dig deeper than the article ( http://stm.sciencemag.org/content/4/125/125ra31.full )
On the authors website, http://www.tatonetti.com/cv.html there is a paper that describes the machine-learning algorithms used:
Tatonetti, N.P., Fernald, G.H. & Altman, R.B. A novel signal detection algorithm for identifying hidden drug-drug interactions in adverse event reports. J Am Med Inform Assoc (2011) DOI:10.1136/amiajnl-2011-000214

Re:not surprising

[Pieroxy]

Now instead of 60 warnings inside the bow we'll have 329 ? Sounds pretty futile to me. Nobody's going to read that.

Why not just writing: In addition to cure you, this drug might kill you or otherwise trigger another disease of allergy. Be warned.

Re:not surprising

[Anonymous Coward]

Presumably so doctors can better select functionally similar drugs to minimise these interactions...

For example, TFA says that the high-blood-pressure medication class thiazides and SSRIs can interact. Neither of these is available without prescription therefore a doctor could use such data to make better treatment decisions...

Re:not surprising

[zero.kalvin]

You can go even further, by using advanced techiniques, you can even combine several drugs to best treat certain conditions without giving the patient one larger dose of one medicine. For example if medicine X was found to react in a certain way with the insulin, and Y the fat cells in the body, while Z can catalyse the reaction of some hormone in the blood that will help. Instead of giving this person one large does of medicine A, he can be given small doses of these 4 things, and keep the harm at a minimum.

Re:not surprising

[aaaaaaargh!]

There are databases and search applications that can be made more accurate with the new data. For example, Denmark has an online system where citizens can enter the name of two drugs and get a list of possible side effects and warnings. There are also big US and European databases of this kind, although less open to the public (I believe).

I can imagine the commercials already..

[screff]

Does this mean those commercials on TV for prescription drugs will now take up an entire commercial break just reading the side effects? With some of the potential side effects being far worse than the condition that the drug intends to treat, it's going to be pretty intimidating for anyone who might have been interested in trying the newest drugs. It may help if the risk were quantified, but with frivolous lawsuits running rampant these days, the drug company legal team probably can't let them skip over the rarer ones for fear of getting sued.

Re:I can imagine the commercials already..

[WillAdams]

A better solution would be to just ban the placement of ads for prescription drugs anywhere other than medical literature.

Re:I can imagine the commercials already..

[__aaltlg1547]

Instead of consult your doctor they'll say consult our computer.

Re:I can imagine the commercials already..

[nbauman]

Not with this Supreme Court.

Say!

[sixtyeight]

Say! Are there any new prescription drugs out there that I'm not taking, but should be? Those seem pretty safe.

Perhaps they'll soon come out with glossy color catalogs for the new ones each season. They'll be full of loads of bikini-clad women draped over cars, popping pills.

Multiple testing problem?

[FhnuZoag]

I am a statistician.

I've only done a light skim of the paper, but it seems to me that the OP (but not the paper itself) is being way too positive here. Their methodology seems to be very vulnerable to false positives - with a massive database of drugs and potential adverse effects, you'd expect a *lot* of apparent side effects occuring solely by chance. For example:

"We constructed a database of 438,801 off-label side effects for 1332 drugs and 10,097 adverse events."

Supposing you are doing a hypothesis test at the standard 0.05 significance level, for each of the 1332*10097 drug-side effect combinations. Then, with naive assumptions, on a null hypothesis, you'd be picking up an average of 666k+ 'side effects' anyway, purely by chance. With the drug interactions case, this multiple testing problem gets even worse.

Now, there are ways to correct for multiple testing, but for things as large and complicated as this problem, I'm not sure the standard methods are going to cut it. At best, this study should be considered more a *filter* on the set of potential side effects, than really an enumeration of effects that are actually there. This is ignoring other issues like the placebo effect.

Re:Multiple testing problem?

[whydavid]

I don't know why standard methods wouldn't cut it. Testing half a million SNPs and several combinations of those SNPs in a GWAS study is far more convoluted and controlling FDR through permutation is still relatively straightforward (though maybe computationally expensive...thought I doubt that is a huge problem for Stanford researchers). And your numbers are inflated by probably an order of magnitude even if no correction was done. They would not be testing all drug/ADE combinations because the vast majority of ADEs would not be present for a given drug and therefore wouldn't be tested.

Re:Multiple testing problem?

[Veetox]

"Their methodology seems to be very vulnerable to false positives..."

I would agree, and go on to suggest that this is intentional. Even after applying "corrective" measures, one has to pick a preference: false negative or false positive, and then show your work (just like in math class). When it comes to drugs, the control methods are never *really* enough. If you're doing an in silico screen, depending on the algorithms used, you may want false positives, because you're just going to throw everything into a high-throughput screen and let the robots do the rest of the work.

But further on down the pipeline, you want to bias towards false negatives, because you're looking for chemicals that have a strong interaction with their target and a week interaction with other targets. The statistics become a tool for making a decision, but never provide 100% assurance.

This study apparently seeks to show the possibilities of side effects, and then let patients/doctors decide if they apply. It's better than not saying anything at all. ...and serotonin reuptake inhibitors? You really want to know even the false positives for those!

Finally, it's likely that the methods of Tatonetti et al. require further refinement, but the rush to publish is an ugly spectre we all have to deal with in science.

Re:Multiple testing problem?

[FhnuZoag]

I both agree and disagree. In principle people really should stop using frequentist methods. In practice, though... There's still substantial disadvantages with Bayesian methodologies. For example, off the top of my head:

1. Relative slowness of computational algorithms for large datasets
2. Difficulty of presenting results to people with different prior beliefs. (Strictly speaking, in Bayesian terms, the answer you give must always be relative to *someone*.)
3. Ease of 'cheating', even unintentionally, by choosing priors to favour a certain result.
4. Proliferation of methods that pretend to be Bayesian but are in fact probably not. (e.g. Empirical bayes methods)

I'm saying this because this always comes up, but people don't realise the bayesian approach is necessarily a magic bullet either.

Re:Multiple testing problem?

[FhnuZoag]

No, that would be the right calculation for arbitary combinations of a certain length. We're looking for the number of pairs (drug, side effect), of which there are #drugs*#sideeffects total.

Re:Multiple testing problem?

[FhnuZoag]

"Why not just report likelihoods instead and let the reader multiply it with any prior they want? In many cases, the prior won't make much of a difference anyway, I suppose."

Reporting likelihoods (or rather summary statistics of likelihoods, because generally likelihoods are functions, not single spot values) is precisely what frequentist statistics is. The use of significance tests, p values and so on can be simply considered a standardised representation of the likelihood.

In the case of multiplying with any prior they want, in many (even most...) cases, the reader simply does not know what their prior is. And often, they do not even know, without a lot of work and some experience in statistical theory, what a 'sensible prior' is. For example, setting a flat prior for x (all values are equally likely a priori) can be actually *very* informative if later calculations make use of 1/x.

In most interesting cases, I'm afraid that the choice of prior *does* make a great difference. This is even more complicated if the prior is in the form of hyperparameters, in which case your prior choice can have a dramatic and non-linear effect on your result.

"Sure, do away with empirical Bayes. Anyway, I don't think "When using Bayesian methods, you run the risk of using non-Bayesian methods." is an argument for not using Bayesian methods."

But Empirical Bayes represents a huge chunk of how 'bayesian stats' is done in practice. The point here is that speakers don't define or understand clearly what is, or is not bayesian in the way that is proposed.

"Regardless of what practical obstacles there might be for using Bayesian inference, using something else would be wrong, leading to results that make you take the wrong actions!"

It depends on how the results are represented. Frequentist results are not 'wrong'. They represent a real value of the data. They may simply be irrelevant. And on the flip side, merely switching to Bayesian inference *does not prevent you from taking wrong actions*. What Bayesian inference accomplishes is that it shifts and makes explicit the mistake you are making.

For example, the false positive results I stated can be *exactly duplicated* by using a prior that is excessively permissive of finding positives, and that prior can look extremely reasonable. The real solution to these problems is to *understand what you are doing*, and this can be done in both a bayesian and frequentist way.

Re:Multiple testing problem?

[martas]

"false positives" come out by chance, there is nothing that can be done to prevent that. Now, underestimating the chance of false positives comes from a poor understanding of statistical tools, and there's a lot that can be done to prevent that. Sadly many scientists don't seem to be interested. (And no, I don't think Bayesian methods are the answer to that; if anything they give much more room for incorrect conclusions than frequentist methods do.)

Are these really the result of drug interactions?

[Attila Dimedici]

I definitely see this type of data mining as a useful tool, but to what degree of surety are they that the adverse effects are caused by the drugs in question? What percentage of people taking the drugs in question have to exhibit the effect before they consider it a product of drug interaction? It appears that they consider even one occurrence of the effect that does not appear in someone with the same condition not taking the drug to be an effect of the drug. If that is true, that would reduce the usability of this analysis. However, even with that flaw, this is a very valuable study. My stepfather struggled with a respiratory problem this winter that was caused by one of the medications he was on. His doctor never admitted that the medication was the problem, but it only started to clear up after he was taken off of it and that only happened when my mom insisted. She had found information that said the drug sometimes resulted in the respiratory problem he was experiencing.

Looks like...

[Shazback]

They're going to need bigger boxes.

Re:not surprising

[whydavid]

Not to plug my profession or anything, but this is exactly why the entire field of biomedical informatics exists. If you think this is bad, consider the fact that there are currently over 20 million abstracts in PubMed....do you think even 10% of that has actually been properly synthesized into operational knowledge and applied to patient care? And we won't even go into genomic data, or even the amount of records that one patient might accumulate in their EMR over the span of a lifetime, or the fact that a 320 slice CT generates so many layers of images that they can't all be carefully reviewed (and an abnormality may be so small it only appears in a couple of them), or the overwhelming breadth and depth of surveillance data collected from ERs/pharmacies/drugstores/monitoring stations/schools/etc... by public health practitioners. There is a critical challenge in biomedicine to distill useful knowledge from all of this data...and it's akin to drinking from a firehose. No one is going to read the 329 warnings for the drug, but in an ideal world we'll be able to identify genetic indicators that make you more or less susceptible to certain side effects (pharmacogenomics) and present this information to you/your doctor (and no one has to read the booklet that comes with the prescription).

Actually, think of the upsides

[twisteddk]

This algorithm may be able to idenify sideeffects when combining medicines.

However, sideeffects are by definition only negative. Once "results" rather than side effects are put through the same algorithm, we may be able to identify better and cheaper ways to combat disease. And we may even be able to find cures for known diseases with combinations of drugs that have never been tested before.

It's all about the data.

Re:Actually, think of the upsides

[Forty Two Tenfold]

However, sideeffects are by definition only negative.

Your geek card, please.

Re:Simple

[Anonymous Coward]

People keep telling me to take headache tablets, cold/flu "remedies", painkillers, etc. etc. etc. and I avoid them like the plague. The people who use them use them CONSTANTLY and still get headaches, flu and pain worse than I ever have. If you have a pack of pills in your bag "just in case" of headache, cold, etc. then you should be made to throw them away - they are purely placebo.

Look, somebody should hit your head with a hammer to make sure you know what you're talking about.

You ignore the fact that we are all different from each other. Headaches are a good example: I practically never get headaches.Other people I'm close to get absolutely terrible headaches from time to time that are so bad that they keep them awake and only the strongest Paracetamol can give some remedy for a short time. You either lack empathy (working in management?) or have really no idea how bad headaches or migraine can be.

The Infancy of Medicine

[iliketrash]

OMG. I really do hope medicine outgrows its infancy during my lifetime.

Re:Algorithm tested?

[ananyo]

So the algorithm found 329 adverse effects which were not known previously. Has someone spent the time in checking the validity of the algorithm by doing studies to check these claims? If not, this is mostly paranoia.

RTFS. They've looked at electronic medical records and confirmed the prediction in the case of thiazides and SSRIs. They're planning more tests - eg a clinical trial.

Re:Simple

[ledow]

Migraines are a different matter that I covered separately in that same post BECAUSE they are nothing to do with headaches and because they have unique drugs that can combat them quite effectively if taken at the onset.

Anybody that confuses or merges migraine and headaches is lacking in understanding themselves. My current and previous partners both suffer severe migraine (up to and including visual effects such as not being able to cross a road because they see cars on the road and/or "seeing" people as headless when the migraine attacks).

And of course some things are subjective but you said it yourself - in a case where "only the strongest paracetamol can give some remedy for a short time", they need to be on the proper drug to deal with that (and/or find the cause of the headaches in the first place) or not at all. It's a case of "mild-drug, quite safe, let's take to help ease a serious warning sign because I always have".

It's not a question of lacking empathy. It's a question of being required to show displaced sympathy. Everyone has headaches, everyone has different severity, but if you are ROUTINELY taking paracetamol when they hit and they have little to NO effect, you're trying to self-medicate where medical advice should be sought.

My previous partner has a severe genetic condition which results in the joint's "safety barriers" being worn away because the collagen in her body is faulty. It's incredibly painful, all over the body, and has any number of weird side effects (immune to some anaesthetics, etc.). Their technique to cope was to take some paracetamol occasionally.

Their doctor advised them to take some more later. Once the doctor was given clear statements on the VOLUME of the chronic pain and my partner realised (in her own words) "that other people DON'T hurt all the time", they actually gave a medication which they themselves called "one step away from morphine". AND you could still take 8 paracetamol a day with it (which, again, we weren't told until we said that even on the drug given, there was still some pain). And that actually had some effect whereas the paracetamol, from day one, was next-to-useless.

If you have a headache that a headache tablet gets rid of, or helps, you don't have a need for the tablet, it's just convenience. If you have a headache that *isn't* affected enough by paracetamol, you need to get your doctor to give you something stronger. I'm *not* against prescription drugs. I'm against people thinking that "mild", over-the-counter drugs are harmless and effective for everything. They aren't.

Side effects

[jonpublic]

Ive had some pretty nasty side effects from a drug that weren't on the label. It really sucked. I'm still dealing with them.

Let me tell you that if you goto a doctor and say that you think that side effects are related to the drug he put you on and those side effects aren't yet on the label they will treat you like you are crazy. It was really eye openning to see how doctors can talk down to patients. The drug I was taking for example can cause tendons to snap. That was known. The doctor flat out didn't believe that incrediblely painful tendinitis could be related to taking the drug.

It really sucks. I got lucky though because a month later the FDA released a new list of side effects that for this drug that included my ones. Then I got a doctor to pay attention and help me out.

I knew the side effects were related to the drug thanks to a quick google search that showed thousands of other people had experienced the same things that I had. There were literally 100,000 posts in a forum discussing what I had experienced in the previous weeks.

I can't even imagine the complexity when combining two drugs and what havoc that could cause.

Re:Simple

[zippthorne]

Have the people you're close to tried the other over-the counters, Aspirin or ibuprofen, etc?

I've personally found paracetamol to be pretty ineffective in just about any dosage, for pretty much every type of pain I've ever had (however, I've known people who found it effective for their own pain). This sucked growing up when I used to get terrible headaches and my parents were afraid of reye's syndrome and assumed all NSAIDS were aspirin for some reason (tylenol marketing dept, I assume...), so that's all I got....

Looking at the wikipedia page, I now find it pretty ironic that I was taking liver poison in an effort to avoid getting a liver disease...

Re:not surprising

[Phrogman]

While I go to my doctor to get treatment and any required prescriptions, I *always* double-check with my pharmacist to ensure there are no likely conflicts between the drugs I have been prescribed, my allergies to certain drugs etc. I trust my pharmacist will have read this stuff in detail, even if my doctor missed something.

Re:not surprising

[artfulshrapnel]

>> Medical doctors are going to read that, it's their job.

I think you mean "Medical doctors SHOULD read that...", or under the best cases "Medical doctors are going to TRY to read that..."

Realistically? They won't have the time to do it properly. Doctors are massively overworked, trying to see far too many patients and dealing with a field that is too broad and grows way too rapidly to keep up with even if they *didn't* have the inconvenience of actually applying their knowledge. I mean, this study alone claims to have discovered 438,228 new drug interactions and side effects. (329 side effects per drug x 1332 drugs) You try to do a thorough read-through and analysis of that kind of data without taking any time off from work; and work quick, you probably only have a week at most until something new you need to learn comes along....