Predicting the Risk of Suicide By Analyzing the Text of Clinical Notes

First time accepted submitter J05H writes "Soldier and veteran suicide rates are increasing due to various factors. Critically, the rates have jumped in recent years. Now, Bayesian search experts are using anonymous Veteran's Administration notes to predict suicide risks. A related effort by Chris Poulin is the Durkheim Project which uses opt-in social media data for similar purposes."

Comments

False positives

[dala1]

According to the study this is 67% effective. But, once this is applied to the general population you have an issue, because the vast majority of people are not suicidal. In the US, about 122 in 100,000 people attempt suicide a year, and about one in ten are successful. Even with a test that is 99% accurate, you are going to end up with somewhere around seven million false positives every year if you screen everyone.

Re:False positives

[dala1]

Seven million extra doctors' visits are hardly inconsequential, especially considering that only about 1 in 175 would actually be suicidal. Consider the time and money spent, the extra doctors who have to be trained and hired (I'm assuming psychiatrists since a GP is hardly qualified to assess a potentially suicidal patient), and the days missed from work for false alarms. That's all before the psychological trauma and loss of trust from your doctor telling you out of the blue that they think you might be mentally ill when you're not.

Besides, that's not how it would work, because once the tool is out there and used to profile everyone, someone who is suicidal will commit suicide before that 'extra session.' After that, it will be be considered negligent not to 'do something' immediately once someone is flagged, and that something would likely be intrusive and expensive.

Also, their accuracy rate is 67%, not 99%. I used that number to demonstrate a best-case scenario. As it stands, they would flag around 83 million people while only correctly flagging around 200,000. Good luck with 99.75% false positives.

Re:False positives

[Cinnamon Beige]

Seven million extra doctors' visits are hardly inconsequential, especially considering that only about 1 in 175 would actually be suicidal.

An interesting attitude. Compare this to Foxconn, which reduced the suicide rate among its workforce from 1 in 60,000 to 1 in 400,000 in three years.

All things considered, I think they did it by making it harder to commit suicide, and possibly also by improving labor conditions.

The usual process is to place somebody thought suicidal on a suicide watch. This can actually be very intrusive, and a test like this certainly is less than ideal if you're applying it at large--the accuracy here is for this population, and rather close to chance already. In a wider population, of a different makeup, its accuracy will be different, and probably lower.

More importantly, if you read the PLOS one article, they're discussing data mining the clinical notes themselves, and they admit that this is a branch of research that has been rather neglected: certain factors were deemed to have predictive value, without anybody really checking to see if that was true.

Let's say you're sitting in the entry way of an office building, and you notice that most people who come in to the building are men. This does not mean you can necessarily predict that a man walking past is going to enter the building; it might turn out that, in fact, of the people passing by the building, any given woman is more likely to come in--it's just that most of the people passing by right now are men.

It does not follow that if "Most of the people who do x are y" is true that "Most people who are y do x" is also true, for any set of x and y.

65% accuracy is not good, it's a start and it's better than what we currently have. In fact, the paper outright says that currently, they haven't even managed to validate the tool. In fact, I can easily give you the tl;dr version of this paper:

The indications for the future of this path of research are promising. Please fund the next phase so we can get it closer to practical application(s).

In less scientific phrasing:

We haven't reached a dead in, give us money so we can keep going!

It's not as much a breakthrough as a status report on the progress towards a breakthrough...

A positive use of data mining

[SigmoidCurve]

It's refreshing to see predictive data analysis used for positive efforts, rather than simply selling more ads. Here's a call to action for all you data scientists at Twitter, FB, and other SV startups who think they're changing the world when all they're doing is putting money in their advertisers' pockets. News flash: statistics can be used to benefit society for a change.