Algorithm Finds Thousands of Unknown Drug Interaction Side Effects

ananyo writes "An algorithm designed by U.S. scientists to trawl through a plethora of drug interactions has yielded thousands of previously unknown side effects caused by taking drugs in combination (abstract). The work provides a way to sort through the hundreds of thousands of 'adverse events' reported to the U.S. Food and Drug Administration each year. The researchers developed an algorithm that would match data from each drug-exposed patient to a nonexposed control patient with the same condition. The approach automatically corrected for several known sources of bias, including those linked to gender, age and disease. The team then used this method to compile a database of 1,332 drugs and possible side effects that were not listed on the labels for those drugs. The algorithm came up with an average of 329 previously unknown adverse events for each drug — far surpassing the average of 69 side effects listed on most drug labels."

Comments

Re:not surprising

[Anonymous Coward]

Presumably so doctors can better select functionally similar drugs to minimise these interactions...

For example, TFA says that the high-blood-pressure medication class thiazides and SSRIs can interact. Neither of these is available without prescription therefore a doctor could use such data to make better treatment decisions...

Re:not surprising

[zero.kalvin]

You can go even further, by using advanced techiniques, you can even combine several drugs to best treat certain conditions without giving the patient one larger dose of one medicine. For example if medicine X was found to react in a certain way with the insulin, and Y the fat cells in the body, while Z can catalyse the reaction of some hormone in the blood that will help. Instead of giving this person one large does of medicine A, he can be given small doses of these 4 things, and keep the harm at a minimum.

Re:not surprising

[aaaaaaargh!]

There are databases and search applications that can be made more accurate with the new data. For example, Denmark has an online system where citizens can enter the name of two drugs and get a list of possible side effects and warnings. There are also big US and European databases of this kind, although less open to the public (I believe).

Say!

[sixtyeight]

Say! Are there any new prescription drugs out there that I'm not taking, but should be? Those seem pretty safe.

Perhaps they'll soon come out with glossy color catalogs for the new ones each season. They'll be full of loads of bikini-clad women draped over cars, popping pills.

Multiple testing problem?

[FhnuZoag]

I am a statistician.

I've only done a light skim of the paper, but it seems to me that the OP (but not the paper itself) is being way too positive here. Their methodology seems to be very vulnerable to false positives - with a massive database of drugs and potential adverse effects, you'd expect a *lot* of apparent side effects occuring solely by chance. For example:

"We constructed a database of 438,801 off-label side effects for 1332 drugs and 10,097 adverse events."

Supposing you are doing a hypothesis test at the standard 0.05 significance level, for each of the 1332*10097 drug-side effect combinations. Then, with naive assumptions, on a null hypothesis, you'd be picking up an average of 666k+ 'side effects' anyway, purely by chance. With the drug interactions case, this multiple testing problem gets even worse.

Now, there are ways to correct for multiple testing, but for things as large and complicated as this problem, I'm not sure the standard methods are going to cut it. At best, this study should be considered more a *filter* on the set of potential side effects, than really an enumeration of effects that are actually there. This is ignoring other issues like the placebo effect.

Re:Multiple testing problem?

[whydavid]

I don't know why standard methods wouldn't cut it. Testing half a million SNPs and several combinations of those SNPs in a GWAS study is far more convoluted and controlling FDR through permutation is still relatively straightforward (though maybe computationally expensive...thought I doubt that is a huge problem for Stanford researchers). And your numbers are inflated by probably an order of magnitude even if no correction was done. They would not be testing all drug/ADE combinations because the vast majority of ADEs would not be present for a given drug and therefore wouldn't be tested.

Re:Multiple testing problem?

[Veetox]

"Their methodology seems to be very vulnerable to false positives..."

I would agree, and go on to suggest that this is intentional. Even after applying "corrective" measures, one has to pick a preference: false negative or false positive, and then show your work (just like in math class). When it comes to drugs, the control methods are never *really* enough. If you're doing an in silico screen, depending on the algorithms used, you may want false positives, because you're just going to throw everything into a high-throughput screen and let the robots do the rest of the work.

But further on down the pipeline, you want to bias towards false negatives, because you're looking for chemicals that have a strong interaction with their target and a week interaction with other targets. The statistics become a tool for making a decision, but never provide 100% assurance.

This study apparently seeks to show the possibilities of side effects, and then let patients/doctors decide if they apply. It's better than not saying anything at all. ...and serotonin reuptake inhibitors? You really want to know even the false positives for those!

Finally, it's likely that the methods of Tatonetti et al. require further refinement, but the rush to publish is an ugly spectre we all have to deal with in science.

Re:Multiple testing problem?

[FhnuZoag]

I both agree and disagree. In principle people really should stop using frequentist methods. In practice, though... There's still substantial disadvantages with Bayesian methodologies. For example, off the top of my head:

1. Relative slowness of computational algorithms for large datasets
2. Difficulty of presenting results to people with different prior beliefs. (Strictly speaking, in Bayesian terms, the answer you give must always be relative to *someone*.)
3. Ease of 'cheating', even unintentionally, by choosing priors to favour a certain result.
4. Proliferation of methods that pretend to be Bayesian but are in fact probably not. (e.g. Empirical bayes methods)

I'm saying this because this always comes up, but people don't realise the bayesian approach is necessarily a magic bullet either.

Re:Multiple testing problem?

[FhnuZoag]

"Why not just report likelihoods instead and let the reader multiply it with any prior they want? In many cases, the prior won't make much of a difference anyway, I suppose."

Reporting likelihoods (or rather summary statistics of likelihoods, because generally likelihoods are functions, not single spot values) is precisely what frequentist statistics is. The use of significance tests, p values and so on can be simply considered a standardised representation of the likelihood.

In the case of multiplying with any prior they want, in many (even most...) cases, the reader simply does not know what their prior is. And often, they do not even know, without a lot of work and some experience in statistical theory, what a 'sensible prior' is. For example, setting a flat prior for x (all values are equally likely a priori) can be actually *very* informative if later calculations make use of 1/x.

In most interesting cases, I'm afraid that the choice of prior *does* make a great difference. This is even more complicated if the prior is in the form of hyperparameters, in which case your prior choice can have a dramatic and non-linear effect on your result.

"Sure, do away with empirical Bayes. Anyway, I don't think "When using Bayesian methods, you run the risk of using non-Bayesian methods." is an argument for not using Bayesian methods."

But Empirical Bayes represents a huge chunk of how 'bayesian stats' is done in practice. The point here is that speakers don't define or understand clearly what is, or is not bayesian in the way that is proposed.

"Regardless of what practical obstacles there might be for using Bayesian inference, using something else would be wrong, leading to results that make you take the wrong actions!"

It depends on how the results are represented. Frequentist results are not 'wrong'. They represent a real value of the data. They may simply be irrelevant. And on the flip side, merely switching to Bayesian inference *does not prevent you from taking wrong actions*. What Bayesian inference accomplishes is that it shifts and makes explicit the mistake you are making.

For example, the false positive results I stated can be *exactly duplicated* by using a prior that is excessively permissive of finding positives, and that prior can look extremely reasonable. The real solution to these problems is to *understand what you are doing*, and this can be done in both a bayesian and frequentist way.

Are these really the result of drug interactions?

[Attila Dimedici]

I definitely see this type of data mining as a useful tool, but to what degree of surety are they that the adverse effects are caused by the drugs in question? What percentage of people taking the drugs in question have to exhibit the effect before they consider it a product of drug interaction? It appears that they consider even one occurrence of the effect that does not appear in someone with the same condition not taking the drug to be an effect of the drug. If that is true, that would reduce the usability of this analysis. However, even with that flaw, this is a very valuable study. My stepfather struggled with a respiratory problem this winter that was caused by one of the medications he was on. His doctor never admitted that the medication was the problem, but it only started to clear up after he was taken off of it and that only happened when my mom insisted. She had found information that said the drug sometimes resulted in the respiratory problem he was experiencing.

Looks like...

[Shazback]

They're going to need bigger boxes.

Re:I was wondered about something

[whydavid]

In Biomedicine you tend to see a heavy reliance on T-Tests, Chi-Square variants, Fisher's Exact, regression, McNemar's and Cox Proportional Hazards when temporally rich data is being tested. I don't have access to this article yet, but I would be surprised if they weren't performing a paired T-test in situations where outcome variables were measured on a ratio scale, McNemar's for binary outcomes where temporal data is not provided (maybe rare or nonexistent in this study), and Cox Proportional Hazards if there are any cases where we have a long temporal history of the data. Based on the sheer number of hypotheses tested we would expect to see some sort of correction for multiple testing here, too.

Re:not surprising

[whydavid]

Not to plug my profession or anything, but this is exactly why the entire field of biomedical informatics exists. If you think this is bad, consider the fact that there are currently over 20 million abstracts in PubMed....do you think even 10% of that has actually been properly synthesized into operational knowledge and applied to patient care? And we won't even go into genomic data, or even the amount of records that one patient might accumulate in their EMR over the span of a lifetime, or the fact that a 320 slice CT generates so many layers of images that they can't all be carefully reviewed (and an abnormality may be so small it only appears in a couple of them), or the overwhelming breadth and depth of surveillance data collected from ERs/pharmacies/drugstores/monitoring stations/schools/etc... by public health practitioners. There is a critical challenge in biomedicine to distill useful knowledge from all of this data...and it's akin to drinking from a firehose. No one is going to read the 329 warnings for the drug, but in an ideal world we'll be able to identify genetic indicators that make you more or less susceptible to certain side effects (pharmacogenomics) and present this information to you/your doctor (and no one has to read the booklet that comes with the prescription).

Re:Simple

[Anonymous Coward]

People keep telling me to take headache tablets, cold/flu "remedies", painkillers, etc. etc. etc. and I avoid them like the plague. The people who use them use them CONSTANTLY and still get headaches, flu and pain worse than I ever have. If you have a pack of pills in your bag "just in case" of headache, cold, etc. then you should be made to throw them away - they are purely placebo.

Look, somebody should hit your head with a hammer to make sure you know what you're talking about.

You ignore the fact that we are all different from each other. Headaches are a good example: I practically never get headaches.Other people I'm close to get absolutely terrible headaches from time to time that are so bad that they keep them awake and only the strongest Paracetamol can give some remedy for a short time. You either lack empathy (working in management?) or have really no idea how bad headaches or migraine can be.

Re:I was wondered about something

[buchner.johannes]

As usual, Science&Nature only provide high-level info, so you'll have to dig deeper than the article ( http://stm.sciencemag.org/content/4/125/125ra31.full )
On the authors website, http://www.tatonetti.com/cv.html there is a paper that describes the machine-learning algorithms used:
Tatonetti, N.P., Fernald, G.H. & Altman, R.B. A novel signal detection algorithm for identifying hidden drug-drug interactions in adverse event reports. J Am Med Inform Assoc (2011) DOI:10.1136/amiajnl-2011-000214

Re:I can imagine the commercials already..

[WillAdams]

A better solution would be to just ban the placement of ads for prescription drugs anywhere other than medical literature.

Re:not surprising

[artfulshrapnel]

>> Medical doctors are going to read that, it's their job.

I think you mean "Medical doctors SHOULD read that...", or under the best cases "Medical doctors are going to TRY to read that..."

Realistically? They won't have the time to do it properly. Doctors are massively overworked, trying to see far too many patients and dealing with a field that is too broad and grows way too rapidly to keep up with even if they *didn't* have the inconvenience of actually applying their knowledge. I mean, this study alone claims to have discovered 438,228 new drug interactions and side effects. (329 side effects per drug x 1332 drugs) You try to do a thorough read-through and analysis of that kind of data without taking any time off from work; and work quick, you probably only have a week at most until something new you need to learn comes along....