Slashdot Mirror
The New AI: Where Neuroscience and Artificial Intelligence Meet
An anonymous reader writes "We're seeing a new revolution in artificial intelligence known as deep learning: algorithms modeled after the brain have made amazing strides and have been consistently winning both industrial and academic data competitions with minimal effort. 'Basically, it involves building neural networks — networks that mimic the behavior of the human brain. Much like the brain, these multi-layered computer networks can gather information and react to it. They can build up an understanding of what objects look or sound like. In an effort to recreate human vision, for example, you might build a basic layer of artificial neurons that can detect simple things like the edges of a particular shape. The next layer could then piece together these edges to identify the larger shape, and then the shapes could be strung together to understand an object. The key here is that the software does all this on its own — a big advantage over older AI models, which required engineers to massage the visual or auditory data so that it could be digested by the machine-learning algorithm.' Are we ready to blur the line between hardware and wetware?"
Are we ready to blur the line between hardware and wetware?
No. You can't ask that every time you find a slightly better algorithm. Ask it when you think you understand how the mind works.
Are we ready to blur the line between hardware and wetware?
We can now almost convincingly partially recreate the wetware functions of Drosophila melanogaster. Whether we're *ready* for this is another question; as is whether this is what folks have in mind by "AI."
He has a course on them at coursera that is pretty good.
https://www.coursera.org/course/neuralnets
Drosophila melanogaster is commonly known as the fruit fly. Its brain has about 100,000 neurons. The human brain avarages 85,000,000,000.
Neural networks? Is it news?
What year is it?
For such a blatant, transparent, promotional, hyperbolic "story", I wish soulskill would at least throw in a sarcastic jab or two to balance out the stench a bit.
Fuck systemd. Fuck Redhat. Fuck Soylent, too. Wait, scratch the last one.
The Hutter Prize for Lossless Compression of Human Knowledge
The last time anyone improved on that benchmark was 2009.
Seastead this.
Andrew Ng is a brilliant teacher who I respect, but I have questions:
1) What is the constructive definition of intelligence? As in, "it's composed of these pieces connected this way" such that the pieces themselves can be further described. Sort of like describing a car as "wheels, body, frame, motor", each of which can be further described. (The Turing Test doesn't count, as it's not constructive.)
2) There are over 180 different types of artificial neurons. Which are you using, and what reasoning implies that your choice is correct and all the others are not?
3) Neural nets in the brain have more back-propagation connections than forward. Do your neural nets have this feature? If not, why not?
4) Neural nets typically have input-layers, hidden-layers, output layers - and indeed, the image in the article implies this architecture. What line of reasoning indicates the correct number of layers to use, and the correct number of nodes to use in each layer? Does this method of reasoning eliminate other choices?
5) Your neural nets have an implicit ordering of input => hidden => output, while the brain has both input and output on one side (ie - both the afferent and efferent neuron enter the brain at the same level, and are both processed in a tree-like fashion). How do you account for this discrepancy? What was the logical argument that led you to depart from the brain's chosen architecture?
Artificial intelligence is 50 years away, and it's been that way for the last 50 years. No one can do proper research or development until there is a constructive definition of what intelligence actually is. Start there, and the rest will fall into place.
When engineers attempt to do philosophy, without any background in philosophy, they always do philosophy badly.
Read Edward Feser's book "Philosophy of Mind (A Beginner's Guide)"
http://www.amazon.com/Philosophy-Beginners-Guide-Edward-Feser/dp/1851684786/
Also read John Searle's "Mind: A Brief Introduction (Fundamentals of Philosophy)"
http://www.amazon.com/Mind-Brief-Introduction-Fundamentals-Philosophy/dp/0195157346/
I post as an anonymous coward so as not to harm my career (any further) by stating *truth* which is not politically nor academically correct.
Neural Net's were traditionally based off old Hodgkins and Huxley models and then twisted for direct application for specific objectives, such as stock market prediction. In the process they veered from a only very vague notion of real neurons to something increasingly fictitious.
Hopefully, the AI world is on the edge of moving away from continuously beating their heads against the same brick walls in the same ways while giving themselves pats on the heads. Hopefully, we realize that human-like intelligence is not a logic engine and that conventional neural nets are not biologically valid and posses numerous fundamental flaws.
Rather--a neurons draws new correlating axons to itself when it cannot reach threshold (-55mv from a resting state of -70mv) and weakens and destroys them when over threshold. In living systems, neural potential is almost always very close to threshold--it bounces a tiny bit over and under. Furthermore, inhibitory connections are also drawn in from non-correlating axons. For example, if two neural pathways always excite when the other does not, then each will come to inhibit the other. This enables contexts to shut off irrelevant possible perceptions, e.g. If you are in the house, you are not going to get rained on. More likely, somebody is squirting you with a squirt gun.
Also--a neuron perpetually excited for too long shuts itself off for a while. We love a good song but hearing it too often makes us sick of it, at least for a while.. like Michael Jackson in the late 1980's.
And very importantly--signal streams that dissappear but recur after increasing time lapses stay potentiated longer.. their potentiation dissipates slower. After 5 pulses with a pause between a new receptor is brought in from the same axon as an existing one. This causes slower dissipation. It will happen again after another 5 pulses repeatedly, except that the time lapse between them must be increased. It falls in line with the scale found on the Wikipedia page for Graduated Interval Recall--exponentially increasing time lapses 5 times, each... take a look at it. Do the math. It matches what is seen in biology, even though this scale was developed in the 1920's.
I have a C++ neural modal that does this. I am mostly done also with a Javascript modal (employing techniques for vastly better performance), using Nodejs.
Each AI will react and learn differently, if the goal is to mimic the brain, why aren't we teaming up AI with people? I want an interface that learns me and my habits, how to react to them, how to respond, etc. The more people that could work and train different AI's the more adaptable they could become in the future. We learn from experience, we have a lot to teach...
Neural networks are certainly not new, or groundbreaking. We already know their strengths and weaknesses, and they aren't a universal solution to every AI problem.
First of all, while they have been inspired by the brain, they don't "mimic" it. Neural networks are based on some neurons having negative weights, reversing the polarity of the signal, which doesn't happen in the brain. They are also linear, which bears similarities to some simple parts of the brain, but are very far from modeling its complex nonlinear processing. Neural networks are useful AI tools, but aren't brain models.
Second neural networks are only good at things when they have to immediately react to an input. Originally, neural networks didn't have memory, and while it's possible to add it, it doesn't fit right into the system and is hard to work with. While neural networks make good reflex machines, even simple stateful tasks like a linear or cyclic multi-step motion are nontrivial to implement in them. Which is why they are most effective in combination with other methods, instead of declared a universal solution.
If the goal is to pass the Turing Test, that is one thing. But clearly they are trying for something more general in some of their contests. I'm just informing them (assuming they are watching) that better tests are available.
Seastead this.
What's actually new in the neural net business? That's a real question - not a sarcastic or rhetorical one.
Artificial neural nets were suggested and tried for AI at least 50 years ago. They were bashed by the old Minsky/McCarthy AI crowd, who didn't like the competition's idea (always better to write another million lines of Lisp). They wrote a paper that showed neural nets couldn't implement an XOR. That's true - for a 2 layer net. A 3 layer net does it just fine. Nevertheless M&M had enough clout to put bury NN research for years. Then in the 80's(?) they became a hot new thing again. One of the few good things about getting older is that you can remember hearing the same hype before.
However, I'm not saying there hasn't been progress. Sometimes a field needs to go through decades of incremental improvement before you can get decent non-trivial applications. It's not all giant breakthroughs. Sometimes just having faster hardware can make a dramatic difference. Loads of things that weren't practical became practical with better hardware. So what's really improved w/ neural nets these days?
You highlight important points, of which AI researchers should take note.
We don't know what intelligence actually is, but we have an example of something that is unarguably intelligent: the mammalian brain. Any proposed mechanism of intelligence should be discounted unless it behaves the same way as a brain. Most AI research fails this test.
I personally think in-depth modeling of individual neurons is too deep of a level - it's like trying to make a CPU by modeling transistors. We might be better off using the fundamental function of a neuron as a basis - sort of like simulating a CPU using logic gates instead of transistors.
But your point is well taken. Lots of research is done under the catch-all phrase AI simply because they do not constrain themselves in any way. What they make doesn't have to pass any criterion for reality, or even reasonableness.
Let us know when you have a peer reviewed publication on your "new" system. Untill then, you can stfu.
Let us know when a peer-reviewed publication tells us how to construct an intelligence.
When will that be - another 50 years, perhaps?
Really. Are you saying that, after AI has gone nowhere for the last 50 years that his position is completely without merit?
At the very least, you should entertain the possibility that the emperor does, in fact, have no clothes.
Question: Why do we pour money and resources in building AI when we have so many people with under-utilized brains already? We're wasting talent and people power, people needs jobs and something to engage in, so why are we passing work off onto machines while people need something to do to make a living?
> people needs jobs and something to engage in, so why are we passing work off onto machines while people need something to do to make a living?
Hazlitt addressed this more than 70 years ago in his classic book: "Economics In One Lesson".
Here is a free copy:
http://Mises.org/books/economics_in_one_lesson_hazlitt.pdf
"I like beaver. Can you tell me where to get some tail?"
"I like cats. Where can I pick one up?"
Let me know when AI can understand the difference between the preceding sentences.
-CF
I have long thought that AI would move forward only to the extent that it emulates and embodies evolutionary mechanisms. AFAICT, evolution is what made it possible for the original hydrogen atoms from the Big Bang to be having this conversation. Mindless, designless change, resulting in us. Who are now getting a clue as to how to design our successors.
That's a fool's errand. The goal of the developer should be to build a system that accomplishes tasks and is able to auto-improve the speed of accomplishing repetitive tasks with minimal (no) human intervention.
The goal of the philosopher is to lay out what intelligence "is". These tracks should be run in parallel and the progress of one should have little-to-no impact on the progress of the other.
Do you consider proper definitions necessary for the advancement of mathematics?
Take, for example, the [mathematics] definition of "group". It's a constructive definition, composed of parts which can be further described by *their* parts. Knowing the definition of a group, I can test if something is a group, I can construct a group from basic elements, and I can modify a non-group so that it becomes a group. I can use a group as a basis to construct objects of more general interest.
Are you suggesting that mathematics should proceed and be developed... without proper definitions?
That a science - any science - can proceed without such a firm basis is an interesting position. Should other areas of science be developed without proper definitions? How about psychology (no proper definition of clinical ailments)? Medicine? Physics?
I'd be interested to hear your views on other sciences. Or if not, why then is AI is different from other sciences?
networks that mimic the behavior of the human BRaiN.
www.artificialintelligenceisgod.com
...from 1997...academia is such a fraud
Modern neural science and biologically realistic neural simulations (such as some of the best Deep Learning systems) use the neuron as its most fundamental primitive. One neural does a lot, actually. It draws in new correlating axons (those firing when its other receptors are firing) when its total potentiation is insufficient to excite. It weakens and destroys them in the inverse case. It also draws in non-correlating axons as inhibitory receptors. And long term potentiation (widely viewed as the basis of long term memory) is increased as the same axon produces more receptors to the same neuron. Furthermore, a neuron perpetually exciting will shut itself down for an extended time.. Each is like a little computer of its own, really..
As for what "intelligence actually is". The real problem is the lack of consensus on a common definition. The word "is" only indicates a relationship between two things without specifying what that relationship, eh hem, actually is. It's a matter of defining it in a way that is broadly acceptible. Defining something can sometimes also determine it. I think that's the case with intelligence. Most people (who care) want to determine what it is so they can define it.. and yet you cannot search for it without knowing what you are searching for, in other words defining it.
I think this is a ridiculous per suite. Pick one of the many working definitions that you like, and work with that. If it feels insufficient then pick or create another. Here's a few I use..... any of which could be more or less complex, evolved, or designed:
Reactive Intelligence -- the ability to react to pre-defined stimulus in a way that, under ordinary conditions, furthers a goal
E.g.: An iron that turns itself off when sitting face down and not moving (often referred to as an intelligent feature)
Conditioning Intelligence -- the ability to identify what reactions to what stimulus has most often in the past furthered a goal and thereafter to react accordingly
E.g.: Pavlov's Dog...or any trial & error aka reward and punishment learning
Substitution Intelligence -- the ability to identify and model observed phenomenon from among interaction pattern sequences and swap out a missing component in one that furthers a goal, if the original is missing. The swap is of one that shares most characteristics with others that had taken the same place in the past.
E.g.: In building a hut, you've used many different kinds of hammers to bang in the nails but today you don't have a hammer. However, you have a rock that shares most characteristics with the other hammer styles (heavy, hard, and with a flat side), so you use the rock where you'd normally have used a hammer.
Substitution Intelligence is shared only among the so-called higher animals, and mostly humans. It requires general imitation learning. That is, the ability to identify that two things/people/animals have a lot of similarities and therefore one could take the place of the other....
Many folks here seem to be not realizing that they are saying brain is a super special thingamajig that could only be created by intelligent design.
Like fusion, general AI already has atleast one working example..
Deep learning system are not quite simulations biological of neural nets. The breakthrough in DL happened then researcher stopped trying to emulate neurons and instead applied statistical (energy function) approach to simple refined model. Modern "ANN" used in deep learning in fact are mathematical optimization procedures, gradient decent on some hierarchy of convolutional operators, more in common with numerical analysis then biological networks.
intelegence is easy, it's emulating stupidity that is the hard bit....a rare few of us do after all hopefully learn from out mistakes.
Also wouldn't you want a AI that's less fickle than a human.
It's also intersting to note that a lot of the problems solved appear to be of the visual type e.g. the word 'cat' had to be provided and that 'blank slate' theory has been disproven, though that's not an issue if the computer algorythms have long enough to evolve.
I agree with your IO stuff, that bares strong relation to neurology.
Personally I'm working on linguistics modeling and the senses, which is based on neurology I won't go into until I have something publishable, but you can find it out if you look for neurology in that area... you won't find anything in linquistics in that area though... it seems to be a hard problem even for humans.
My my key problem was seeding, so I may take a look at deap learning to see what it has to offer, but I think a few lightly ranked examples (who ranking can be changed by the algorythm) would probably be most benifiial.... at least to do some primary set reduction on the data.
thank God the internet isn't a human right.
I wonder when our new AI overlords will create AI themselves because they are too bored and tired of doing actual work themselves.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
QWhere's the obvious Skynet snark?
This seems like non-news, but my real question is, since the author claims neural nets duplicate brains, do zombie servers crave eating neural nets? Just asking
When I was at Stanford. On a much smaller scale then due to week computers.
I dont htink the problems of "brittleness"- unpredictable result if new inputs are presented- and "opaqueness"- weights are interpretable- have changed.
At first glance, I read "Where Neuroses and Artificial Intelligence Meet".
So, are they working on an android named Marvin?
Have gnu, will travel.
There is a fine yet deep difference between artificial intelligence and artificial consciousness.
The whole premise of "artificial intelligence" being a "thing" that we "acheive" by a certain date (be it in relation to processor development or not)...it's bunk. Hokus Pokus. Used-car salesman terminology to describe basic computation and programming.
Points 3-5 ring especially true from where I sit as a researcher and former network engineer. Everything is a 'network' at some level. Adding more nodes and calling it 'neural' doesn't mean you've invented the wheel. It's just terminology describing a machine programed with input.
There is no 'singularity' of artificial intelligence, because the concept itself is abstract language to describe programed machine responses. It's *humans* like Kurzweil (whom I respect greatly) who add the **emotional** stuff to the concept and then try to call it something **new**...like a **salesman**
There may be a singularity-type event when all humans all over the world have a free persistent connection to the internet. That would be something...but it would be just a digital extention of the existing, geography-limited current social network of humans!
Thank you Dave Raggett
Maxout networks are very, very new. See http://arxiv.org/abs/1302.4389.
You could also poke around http://metaoptimize.com./
G-d was the ultimate programmer... therefor any system in the future should be built like the human brain is, if we are his most perfect work.
I am not saying we are all perfect, but only that his work on us vs. any other animal, is pretty advanced.
I am not saying that any other living creature is less important either, I would save an animal just as much a human....