Slashdot Mirror
The New AI: Where Neuroscience and Artificial Intelligence Meet
An anonymous reader writes "We're seeing a new revolution in artificial intelligence known as deep learning: algorithms modeled after the brain have made amazing strides and have been consistently winning both industrial and academic data competitions with minimal effort. 'Basically, it involves building neural networks — networks that mimic the behavior of the human brain. Much like the brain, these multi-layered computer networks can gather information and react to it. They can build up an understanding of what objects look or sound like. In an effort to recreate human vision, for example, you might build a basic layer of artificial neurons that can detect simple things like the edges of a particular shape. The next layer could then piece together these edges to identify the larger shape, and then the shapes could be strung together to understand an object. The key here is that the software does all this on its own — a big advantage over older AI models, which required engineers to massage the visual or auditory data so that it could be digested by the machine-learning algorithm.' Are we ready to blur the line between hardware and wetware?"
Are we ready to blur the line between hardware and wetware?
No. You can't ask that every time you find a slightly better algorithm. Ask it when you think you understand how the mind works.
Are we ready to blur the line between hardware and wetware?
We can now almost convincingly partially recreate the wetware functions of Drosophila melanogaster. Whether we're *ready* for this is another question; as is whether this is what folks have in mind by "AI."
He has a course on them at coursera that is pretty good.
https://www.coursera.org/course/neuralnets
Drosophila melanogaster is commonly known as the fruit fly. Its brain has about 100,000 neurons. The human brain avarages 85,000,000,000.
For such a blatant, transparent, promotional, hyperbolic "story", I wish soulskill would at least throw in a sarcastic jab or two to balance out the stench a bit.
Fuck systemd. Fuck Redhat. Fuck Soylent, too. Wait, scratch the last one.
The Hutter Prize for Lossless Compression of Human Knowledge
The last time anyone improved on that benchmark was 2009.
Seastead this.
Andrew Ng is a brilliant teacher who I respect, but I have questions:
1) What is the constructive definition of intelligence? As in, "it's composed of these pieces connected this way" such that the pieces themselves can be further described. Sort of like describing a car as "wheels, body, frame, motor", each of which can be further described. (The Turing Test doesn't count, as it's not constructive.)
2) There are over 180 different types of artificial neurons. Which are you using, and what reasoning implies that your choice is correct and all the others are not?
3) Neural nets in the brain have more back-propagation connections than forward. Do your neural nets have this feature? If not, why not?
4) Neural nets typically have input-layers, hidden-layers, output layers - and indeed, the image in the article implies this architecture. What line of reasoning indicates the correct number of layers to use, and the correct number of nodes to use in each layer? Does this method of reasoning eliminate other choices?
5) Your neural nets have an implicit ordering of input => hidden => output, while the brain has both input and output on one side (ie - both the afferent and efferent neuron enter the brain at the same level, and are both processed in a tree-like fashion). How do you account for this discrepancy? What was the logical argument that led you to depart from the brain's chosen architecture?
Artificial intelligence is 50 years away, and it's been that way for the last 50 years. No one can do proper research or development until there is a constructive definition of what intelligence actually is. Start there, and the rest will fall into place.
Neural Net's were traditionally based off old Hodgkins and Huxley models and then twisted for direct application for specific objectives, such as stock market prediction. In the process they veered from a only very vague notion of real neurons to something increasingly fictitious.
Hopefully, the AI world is on the edge of moving away from continuously beating their heads against the same brick walls in the same ways while giving themselves pats on the heads. Hopefully, we realize that human-like intelligence is not a logic engine and that conventional neural nets are not biologically valid and posses numerous fundamental flaws.
Rather--a neurons draws new correlating axons to itself when it cannot reach threshold (-55mv from a resting state of -70mv) and weakens and destroys them when over threshold. In living systems, neural potential is almost always very close to threshold--it bounces a tiny bit over and under. Furthermore, inhibitory connections are also drawn in from non-correlating axons. For example, if two neural pathways always excite when the other does not, then each will come to inhibit the other. This enables contexts to shut off irrelevant possible perceptions, e.g. If you are in the house, you are not going to get rained on. More likely, somebody is squirting you with a squirt gun.
Also--a neuron perpetually excited for too long shuts itself off for a while. We love a good song but hearing it too often makes us sick of it, at least for a while.. like Michael Jackson in the late 1980's.
And very importantly--signal streams that dissappear but recur after increasing time lapses stay potentiated longer.. their potentiation dissipates slower. After 5 pulses with a pause between a new receptor is brought in from the same axon as an existing one. This causes slower dissipation. It will happen again after another 5 pulses repeatedly, except that the time lapse between them must be increased. It falls in line with the scale found on the Wikipedia page for Graduated Interval Recall--exponentially increasing time lapses 5 times, each... take a look at it. Do the math. It matches what is seen in biology, even though this scale was developed in the 1920's.
I have a C++ neural modal that does this. I am mostly done also with a Javascript modal (employing techniques for vastly better performance), using Nodejs.
Neural networks are certainly not new, or groundbreaking. We already know their strengths and weaknesses, and they aren't a universal solution to every AI problem.
First of all, while they have been inspired by the brain, they don't "mimic" it. Neural networks are based on some neurons having negative weights, reversing the polarity of the signal, which doesn't happen in the brain. They are also linear, which bears similarities to some simple parts of the brain, but are very far from modeling its complex nonlinear processing. Neural networks are useful AI tools, but aren't brain models.
Second neural networks are only good at things when they have to immediately react to an input. Originally, neural networks didn't have memory, and while it's possible to add it, it doesn't fit right into the system and is hard to work with. While neural networks make good reflex machines, even simple stateful tasks like a linear or cyclic multi-step motion are nontrivial to implement in them. Which is why they are most effective in combination with other methods, instead of declared a universal solution.
If the goal is to pass the Turing Test, that is one thing. But clearly they are trying for something more general in some of their contests. I'm just informing them (assuming they are watching) that better tests are available.
Seastead this.
What's actually new in the neural net business? That's a real question - not a sarcastic or rhetorical one.
Artificial neural nets were suggested and tried for AI at least 50 years ago. They were bashed by the old Minsky/McCarthy AI crowd, who didn't like the competition's idea (always better to write another million lines of Lisp). They wrote a paper that showed neural nets couldn't implement an XOR. That's true - for a 2 layer net. A 3 layer net does it just fine. Nevertheless M&M had enough clout to put bury NN research for years. Then in the 80's(?) they became a hot new thing again. One of the few good things about getting older is that you can remember hearing the same hype before.
However, I'm not saying there hasn't been progress. Sometimes a field needs to go through decades of incremental improvement before you can get decent non-trivial applications. It's not all giant breakthroughs. Sometimes just having faster hardware can make a dramatic difference. Loads of things that weren't practical became practical with better hardware. So what's really improved w/ neural nets these days?
You highlight important points, of which AI researchers should take note.
We don't know what intelligence actually is, but we have an example of something that is unarguably intelligent: the mammalian brain. Any proposed mechanism of intelligence should be discounted unless it behaves the same way as a brain. Most AI research fails this test.
I personally think in-depth modeling of individual neurons is too deep of a level - it's like trying to make a CPU by modeling transistors. We might be better off using the fundamental function of a neuron as a basis - sort of like simulating a CPU using logic gates instead of transistors.
But your point is well taken. Lots of research is done under the catch-all phrase AI simply because they do not constrain themselves in any way. What they make doesn't have to pass any criterion for reality, or even reasonableness.
Neural networks? Is it news?
No, it's misrepresentation. This isn't any more akin to neuroscience than any of the other techniques used with artificial neural networks.
It will be a great thing, though, if it lives up to expectations.
Sheesh, evil *and* a jerk. -- Jade
I post as an anonymous coward so as not to harm my career (any further) by stating *truth* which is not politically nor academically correct.
Also, you wisely don't want your name associated with a positive view of Searle's nonsense.
Sheesh, evil *and* a jerk. -- Jade
"I like beaver. Can you tell me where to get some tail?"
"I like cats. Where can I pick one up?"
Let me know when AI can understand the difference between the preceding sentences.
-CF
Of course, if everyone would just stfu until they have a peer reviewed journal article, there would never be any peer reviewed journal articles... Perhaps one reason AI hasn't progressed might be this kind of brutal cynicism toward new ideas.
Granted, every premise I provided in the modal derives from established science, though replicated, peer review journals.. in fact, much through basic text books in Neural Science... But let's stfu about that, too, since these things don't appear to be yet discussed at the same time in any peer reviewed journal article. I suppose we can only read anything if it comes directly from a peer review journal article..... and only what's in one particular such article at a time... perhaps requiring a holy moment of silence between each article, to ensure a clean separation.
I've been working on these models for decades... I've done science (six accepted peer review articles) but am really an engineer, not a scientist. I prefer it that way. I can leave publishing research to others, who require it for their tenure. At one time, slashdot was actually a mostly intellectually stimulating conversational environment...
Deep learning system are not quite simulations biological of neural nets. The breakthrough in DL happened then researcher stopped trying to emulate neurons and instead applied statistical (energy function) approach to simple refined model. Modern "ANN" used in deep learning in fact are mathematical optimization procedures, gradient decent on some hierarchy of convolutional operators, more in common with numerical analysis then biological networks.
Do you consider proper definitions necessary for the advancement of mathematics?
Take, for example, the [mathematics] definition of "group". It's a constructive definition, composed of parts which can be further described by *their* parts. Knowing the definition of a group, I can test if something is a group, I can construct a group from basic elements, and I can modify a non-group so that it becomes a group. I can use a group as a basis to construct objects of more general interest.
Are you suggesting that mathematics should proceed and be developed... without proper definitions?
That a science - any science - can proceed without such a firm basis is an interesting position. Should other areas of science be developed without proper definitions? How about psychology (no proper definition of clinical ailments)? Medicine? Physics?
I'd be interested to hear your views on other sciences. Or if not, why then is AI is different from other sciences?
The view of mathematics as proceeding from clear-cut definitions and axioms is really an artifact of the way we teach it. Over time theorems can become definitions, and we may choose definitions so as to make certain theorems that ought to be true, true.
If you want an example, look at how much real analysis was going on before we had a proper definition of continuity.
An obsession with rigorous definitions right at the start of a field serves only to force our intuitions to be more specific than they are, with no understanding of the
consequences.
Mathematics is not a science, it's just used by science. Science is about studying phenomena in reality. Mathematics is not part of reality - mathematics is entirely apriori. You can't prove anything in mathematics by studying reality. Mathematics is made entirely of definitions and symbol manipulation, so of course you shouldn't be doing mathematics without definitions. Science isn't like that. Proper definitions can often help in Science, that's true, but it's not a prerequisite. The suggestions is not that proper definitions are bad, the suggestion is that intelligence is too slippery to properly define now - possibly too slippery to ever define properly. As long as there are objective tests to compare efforts at intelligence, it's not absolutely necessary to have a proper definition of what it is, so there is no reason to stop all the research until a proper definition is in.