Slashdot Mirror
A.I. Advances Through Deep Learning
An anonymous reader sends this excerpt from the NY Times:
"Advances in an artificial intelligence technology that can recognize patterns offer the possibility of machines that perform human activities like seeing, listening and thinking. ... But what is new in recent months is the growing speed and accuracy of deep-learning programs, often called artificial neural networks or just 'neural nets' for their resemblance to the neural connections in the brain. 'There has been a number of stunning new results with deep-learning methods,' said Yann LeCun, a computer scientist at New York University who did pioneering research in handwriting recognition at Bell Laboratories. 'The kind of jump we are seeing in the accuracy of these systems is very rare indeed.' Artificial intelligence researchers are acutely aware of the dangers of being overly optimistic. ... But recent achievements have impressed a wide spectrum of computer experts. In October, for example, a team of graduate students studying with the University of Toronto computer scientist Geoffrey E. Hinton won the top prize in a contest sponsored by Merck to design software to help find molecules that might lead to new drugs. From a data set describing the chemical structure of 15 different molecules, they used deep-learning software to determine which molecule was most likely to be an effective drug agent."
I wonder how much of these improvements in accuracy are due to fundamental advances, vs. the capacity of available hardware to implement larger models and (especially?) the availability of vastly larger and better training sets...
I wonder how much of these improvements in accuracy are due to fundamental advances
I was wondering the same thing, and just now found this interview on Google. Perhaps someone can fill in the details.
But basically, machine learning is at its heart hill-climbing on a multi-dimensional landscape, with various tricks thrown in to avoid local maxima. Usually, humans detemine the dimensions to search on -- these are called the "features". Well, philosophically, everything is ultimately created by humans because humans built the computers, but the holy grail is to minimize human invovlement -- "unsupervised learning". According to the interview, this one particular team (the one mentioned at the end of the Slashdot summary) actually rode the bicycle with no hands and to demonstrate how strong their neural network was at determining its own features, did not guide it, even though it meant their also-excellent conventional machine learning at the end of the process would be handicapped.
The last time I looked at neural networks was circa 1990, so perhaps someone writing to an audience more technically literate than the New York Times general audience could fill in the details for us on how a neural network can create features.
A lot of vague marketing-speak in this article. "Deep learning"?
Agreed. Why do we need the adjective "deep"? Perhaps it's because a lot of AI jargon uses "learning" when they really just mean "adaptive" (as in, "programmed to respond to novel stimuli in anticipated ways"), whereas normal human "learning" is much more fluid.
The article basically talks about neural networks
Yet another victory for marketing. These things have been around for at least 25-30 years, and the connection to what little we actually have deciphered about how the brain encodes, decodes, and processes information has always been incredibly tenuous. There always seems to be these AI strands of "cognitive science" or "neural modeling," which are often nothing than just somebody's pet algorithm or black box dressed up with words that make it sound like it has some scientific basis in actual neurophysiology or something.
Don't get me wrong -- I'm sure some of the examples in TFA have made great advances, partly due to speed and hardware unthinkable 25-30 years ago. And some of the functionality of the "neural nets" might give significantly better results than previous models.
But I really wish people would lay off the pretend connections to humanity. Why can't we just accept that a machine might just function better with a better program or algorithm or whatever, rather than saying that "our research in cognitive science [i.e., BS philosophy of the mind] has resulted in neural networks [i.e., a mathematical model instantiated into programming constructs] that exhibit deep learning [i.e., work better than the previous crap]."
(Please note: I mean no insult to anyone who works in neuroscience or AI or whatever. But I do question the jargon that seems to make unfounded connections and assumptions that the brain works anything like many algorithmic "models." We may succeed in creating artificial intelligence by developing our own algorithms or we might succeed by imitating the brain, but I don't think we're making progress by pretending that we're imitating the brain when we're really just using marketing jargon for our pet mathematical algorithm.)