An Algorithm That Can Predict Human Behavior Better Than Humans (mit.edu)

Quartz describes an MIT study with the surprising conclusion that at least in some circumstances, an algorithm can not only sift numbers faster than humans (after all, that's what computers are best at), but also discern relevant factors within a complex data set more accurately and more quickly than can teams of humans. In a competition involving 905 human teams, a system called the Data Science Machine, designed by MIT master's student Max Kanter and his advisor, Kalyan Veeramachaneni, beat most of the humans for accuracy and speed in three tests of predictive power, including one about "whether a student would drop out during the next ten days, based on student interactions with resources on an online course." Teams might have looked at how late students turned in their problem sets, or whether they spent any time looking at lecture notes. But instead, MIT News reports, the two most important indicators turned out to be how far ahead of a deadline the student began working on their problem set, and how much time the student spent on the course website. ... The Data Science Machine performed well in this competition. It was also successful in two other competitions, one in which participants had to predict whether a crowd-funded project would be considered “exciting” and another if a customer would become a repeat buyer.

Comments

Try predicting violent behavior.

[Ungrounded Lightning]

It might be interesting to try it at predicting future violent behavior of individuals.

Those who study such behavior have come up with the aphorism "The only effective predictor of future violent behavior is past violent behavior." Let's see if the

Shrinks who try to make predictions about individuals come out WORSE than chance - which implies that there may be SOME prediction possible - but the current paradigms have it backward.

This, by the way, is ONE of the reasons the pro-gun crowd pooh-poohs mental health tests for gun ownership or purchase. Another is the observation that people with mental illnesses are, on the average, far LESS likely to cause harm to others than the average of the population. (They may harm THEMSELVES, but suicide rates don't change if guns aren't available: Instead the suicidal switch to less effective and usually more painful means, averaging more tries before they succeed.)

Overblown headline

[binarstu]

The headline of the Quartz article and the Slashdot summary, "An algorithm can predict human behavior better than humans", is, not surprisingly, hugely overblown.

What these researchers actually did was develop a system for automatically taking a massive data set with a huge number of variables, identifying the subset of variables or new combinations of variables that are most likely to be useful for predicting a particular response, and then formulating a predictive model. (This is an extremely simplified summary.) That is really cool, but to present it as some sort of general "algorithm for predicting human behavior" is silly. It's no more an algorithm for predicting human behavior than are automated statistical methods for building a predictive model from a massive dataset.

Re:Overblown headline

[Lennie]

If it worked so well as the headline claimed then they wouldn't be writing articles about it, but using it to buy/sell stocks and options. ;-)

No surprise

[gweihir]

First, most humans are not that smart, but do not know that. It is called the "Dunning-Kruger" effect and it is well-established. Apparently at least the OP is unaware of it, possibly making him a subject of the Dunning-Kruger effect.

Second, a majority of humans do not use what smarts they have effectively, but rather do decide "emotionally" when it comes to important decisions or understanding important situations. That obviously works rather badly, just look at what politicians get voted into office, or what life-choices people make. The problem here is that the whole "emotional decision" apparatus is a rather primitive left-over from caveman-times that cannot handle even situations of moderate complexity well. The second problem is that most humans never find that out, as the skill for self-reflection is also rather scarce and hence cannot actively compensate.

So give an arbitrary group of people an analysis or decision problem that somehow "touches" them (like asking students to predict whether other students fail at being students), and suddenly most of them turn into morons (or rather do not stop being morons in many cases), and even a simplistic statistic predictor does a lot better than they do.