Slashdot Mirror
AI Can Predict Heart Attacks More Accurately Than Doctors (digitaltrends.com)
An anonymous reader quotes a report from Digital Trends: Scientists from the University of Nottingham in the United Kingdom have managed to develop an algorithm that outperforms medical doctors when it comes to predicting heart attacks. As it stands, around 20 million people fall victim to cardiovascular disease, which includes heart attacks, strokes, and blocked arteries. Today, doctors depend on guidelines similar to those of the American College of Cardiology/American Heart Association (ACC/AHA) in order to predict individuals' risks. These guidelines include factors like age, cholesterol level, and blood pressure. In employing computer science, Stephen Weng, an epidemiologist at the University of Nottingham, took the ACC/AHA guidelines and compared them to four machine-learning algorithms: random forest, logistic regression, gradient boosting, and neural networks. The artificially intelligent algorithms began to train themselves using existing data to look for patterns and create their own "rules." Then, they began testing these guidelines against other records. And as it turns out, all four of these methods "performed significantly better than the ACC/AHA guidelines," Science reports. The most successful algorithm, the neural network, actually was correct 7.6 percent more often than the ACC/AHA method, and resulted in 1.6 percent fewer false positives. That means that in a sample size of around 83,000 patient records, 355 additional lives could have been saved.
the AI took the easy route and was scaring people into having heart attacks. ;)
Anons need not reply. Questions end with a question mark.
these methods "performed significantly better than the ACC/AHA guidelines,"
Yeah outperforming the general written guidelines is totally the same as outperforming a live doctor.
"Helpful tool that may substitute rule-of-thumb guidelines for doctors", maybe.
The biggest problem with these guidelines (the paper calls it an algorithm) is that humans optimize their predictive power using simple statistical analysis based on a training dataset. Then medical professionals are told the guidelines can be reasonably used anywhere. Throw it some data from a vastly different hospital and it will degrade in performance very quickly. For example, create rules using data from academic hospital, but test it on a rural hospital and the results won't be as good. There are different label distributions and often times even feature distributions.
If you use machine learning to train on the same hospital that you test, of course it will be better than a model created by humans on a different hospital.
I see this happen all the time with readmission prediction. There's a very popular system called LACE that was optimized for a specific hospital. It's simple to implement and test so there's a lot of reproduction. LACE has wild swings in performance (from AUC=0.55 to AUC=0.75 depending the dataset). Researchers will often compare LACE to a machine learning method like Random Forest, training and testing on same hospital, and shockingly the results are better.
This exactly!
The baseline method was a diagnostic algorithm created by humans on a training dataset. Using machine learning, these researchers split train/test using a different dataset and compared models using the same test, but different train. YOU CANNOT DO THIS. But many researchers do and it bothers me because it's not sound methodology. The classification label and feature distributions will be different between the original train and new train datasets.
Also, I've noticed a huge increase in AUC as a primary metric and I'm not really convinced it's a good metric in this field. It's difficult to analyze misclassification cost using AUC and in the medical world FP and FN often have vastly different costs. Very few researchers are currently acknowledging this and I see very little cost sensitive classification in this domain. I've also seen several arguments in journals recently arguing that comparing classifiers using AUC is comparing apples to oranges.
In law enforcement you even get feedback loops where the AI is trained on the police arrest data, which guides the police to where to place enforcement for better arrests, which guides the AI which guides the police which guides the AI which guides the police....
Obviously is focusses the police in a crime area and all their arrests are focussed on that area because that is where they are.
The companies hire smart people, who understand the problem, but the corporate interest is to *sell* the AI, and this feedback loop also makes the AI look better.
There does appear to be an awful lot of sloppy science going on in datamining currently.
And one can "meta-train" for the test data group. Like
* Train
* Compare to test set
* Worse than guideline result => Change training parameters
* Train
* Compare to test set
* Still worse than guideline result => Change training parameters more
* Train
* Compare to test set
* Better than guideline result => Publish
I will be impressed, if it is better than a human doctor on new cases.
The AI does not outperform a doctor. Does not outperform any doctor period.
It has managed to perform better than guidelines.
Now let's consider that the AI is basing this on data gathered by medical science. What it's outperforming in actuality is thus old prediction models.
Overall, quite happy about it but let's not pretend that AI had 5 hours of sleep in the last 48, had a shower, drove to work, checked on the kids and yet manage to perform all work duties admirably. -because then it might actually outperform a doctor.
A 'singular oddity' is an event that cannot be explained and only happens when you are alone.
It will not be long before the AI neural network aggregate more data and determine, these 335 lives are not worth saving. What is the real incentive for artificially intelligent to save the naturally stupid?
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Not just on /. It seems to me that these stories about AIs & automation/robots taking human jobs have been all over the place in the past few months. I consciously try to avoid most mainstream U.S. media, but I can't help some incidental exposure. I also listen to NPR ~ 2hrs/week and they've been doing a series of stories on this general topic as well.
It sure feels like a systematic effort at psychological conditioning. Is this simply the media trying to get back at Trump by blaming automation rather than immigration for displacing U.S. born workers? Is it just some current fad in the media which is going to pass when a majority of people get bored with it? Or perhaps it's just some distorted perception/selective attention on my part. It still feels sort of weird.