Slashdot Mirror
New Hardware Needed For Future Computational Brain
schliz writes "Salk Institute director Terrence Sejnowski has called for more power-efficient, parallel computing architecture to support future robots that could keep up with the human brain. While human brains had 100 billion neurons and required only 20 Watts of energy, today's most powerful supercomputer, the 2.57 PFlop Chinese Tianhe-1A, requires four megawatts, and still has trouble with vision, motion, and 'common sense,' he said."
LOL! "can't approach the capabilities of a common honey bee" might be more accurate.
That most powerful supercomputer, I'd assume, has not been tuned to actually work like a brain would.
This is like an emulator. A lot of computational power is probably wasted on trying to translate biological functions into binary procedures. I think if they truly want to compare, they'll need to create an environment that is enhanced for the tasks we want it to process.
Nobody expects the human brain to compute integer and floating point stuff at the same efficiency either, right?
Instead of trying to emulate the human brain, which at the moment is unattainable, we should concentrate on efficiency paradigms of smaller neural ensembles. Once we achieve efficiency we can scale. Why haven't we learned anything from the CPU industry? They didn't start from 19nm manufacture. Why should we?
We shouldn't hurry. AI comparable to a human person can be achieved, but it is still a long way until we reach it.
"Sum Ergo Cogito"
Each pint of beer contains 600 joules of energy, which can power your 20 watt brain for many hours, and give you trouble with vision, motion, and common sense.
The reason this is the case is because current AI simulates a neural network as a program, you would have to produce chips which where actual neural networks the problem however is the interconnects which is in an order of magnitude more complicated compared to anything we can currently create. In fact the brain is quite slow, but its organization is what makes it powerful.
Watt is a unit of power, not energy.
I have known many people who have ~100billion or so neurons that consume 20 watts of power, but they also have plenty of trouble with "Common sense". Actually they might be less sensible in some areas than a 100Kb C code running on a puny little Pentium 4.
The significant number is interconnect. In that area electronics is several orders of magnitude farther behind. Far enough that is seems doubtful something even remotely like the interconnect of a human brain can be reached artificially.
Side note: Comparing neurons and transistors, as is often done in the popular (but not very knowledgeable) press, is completely invalid as well. You need to compare neurons more to a micro-controller each.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Ok , you can do this with a FPGA but this requires something external to the gate array to reset the logic gates - the array can't rewire itself. Biological neural systems can rewire themselves and not only that - they can do it *while they're running*. Obviously you could have this on the fly rewiring in a software simulation but thats orders of magnitudes slower than using hardware so I don't think we'll see computers simulating human brains in real time anytime soon.
Better in specific dedicated tasks. But the efficiency argument still holds true, especially when you consider the brain is both the computational and memory unit. Furthermore there is the issue of what we are still to learn about the brain, especially those processes that occur below immediate perception.
Getting a little ahead of ourselves aren't we?
We're still not entirely sure of how a brain works. Oh sure, it's a neural network of some kind, but how do the neurons in a brain form meaningful connections with each other? How do they get their weightings of activation? etc.
Chances each neuron in the brain might be representable by a simple mathematical function with only a few terms. The way the neurons connect to each other might also be representable in a simplistic way. (btw. look up dynamic markov coding if you want to see a neat way a state can reproduce in a way that gives the newly created state meaningful input/output connections to other states).
So the problem isn't necessarily that our computers aren't powerful enough. The problem is that we still don't know how a brain works.
Ok, I admit this sounds completely absurd at first, but there's an awful lot of similarities between the neural pathways of the brain and the countless number of ways websites link to each other, both directly and indirectly through their contacts, and their contacts' contacts, and all the contacts that eventually show up in an endless cycle of recursion, etc...
Now, google has to wade through all this, and constantly correct and update itself, to ensure it can get a user to the correct web page that best matches the search criteria.
You can't tell me that as more data on the web becomes increasingly more dynamic with all these forums, blogs, news sites and endless amounts of chat/social engineering sites constantly popping up and then dying, that there isn't at least some algorithm they employ that couldn't be applied to nueron connectivity and communication.
You'd think it'd just be a matter of passively connecting to a neuron to sniff it's traffic and then observing which nearby neurons carry the signals to and from it, then start listening to those neurons and so forth, then use machine learning to break down the patterns enough that google's setup could follow it... ie, determine which neuron is responsible for which patterns in what frequency, etc...
8==8 Bones 8==8
It's a software problem.
As awesome as everyone talks up these 'brains' and how incredibly superior they are with only 20 watts, the fastest brain on earth can't even keep up with a 10 dollar pocket calculator that uses a fraction of a watt when it comes to remotely complex arithmetic.
Obviously, we have very two different things here. We created computers to be good at the stuff we are *not* good at, not to match our capabilities (we wouldn't spend so much money to make machines that are good at just the same things we are). That's one fallacy these discussions keep running into, we assume one is simply 'better' than the other rather than distinct.
XML is like violence. If it doesn't solve the problem, use more.
Why is this article written in past tense? It contains funny paragraphs like this:
'While fundamental physics and molecular biology dominated the past century’s innovations, Sejnowski said the years between 2000 and 2050 was the “age of information”.'
2050 isn't really the past, right?
The architecture on which you run the software also determines quite a lot of what you can do and how the software is executed. You need a certain topology of the hardware, otherwise it is impossible to do certain tasks efficiently. There is a huge difference between a slow but massively interconnected network like the brain, and a sequential microprocessor running instructions one by one at high speed.
Know lots of 20 - 70 somethings with no common sense.
The preceding post was not a Slashvertisement.
Is there a human capable of multiplying precisely billions of numbers per second or doing any other similar tasks?
What, you think the computer invented itself?
We are tool makers, the computer is a tool, we want to multiply at a rate of 10^9 a second, we just build the tool using our brains.
Who mentioned efficiency?
We don't have to do it in real time. But even if we had till the heat death of the universe to let the code run, we still don't know how to write the code, which was the OP's point.
That belongs to the Jaguar Cray XT5-HE, not the overstated specs of the system that "claims" the supposed top slot.
Move it down a bit more and you would truthfully be representing its capability. But then you'd just want to modbomb me into oblivion, since that's easier to do.
Twitter supports and protects racists - by smearing their critics with the "Hate Speech" label.
At the risk of some modpoints:
What China really can't do with computers, they make up with dissidents. The Top500 data from 11/2010 would be suspect, even if that wasnt the cause.
Twitter supports and protects racists - by smearing their critics with the "Hate Speech" label.
computer CPU and software processes in a flat 1 dimensional stream nerual structures are emulated taking time to read each ones state one after another and simulate the actions of the interconnects to get the result
"Hardware/softcore" FPGA based neural net would form a flat even 2 dimensional "grid" array
but a DNA based brain is both a 3D structure and also has sub "fractal" patterned interconnected structures within it
to form even a bee style neural structure in a FPGA would still need the logic cells to be arranged in the correct "fractal" patterned structure of interconnects
ether way current processing technology does not lend itself to creation of viable neural net structures that would allow for a fair comparison
"Who's brain did you emulate?"
"Uh, Abby someone..."
"Abbey who?"
"Abby Normal...."
It seems like we already have this in FPGAs. We don't really have good clusters of them though..at least that I know.
I'm a software developer that has dabbed in VHDL and created some basic programs that got ran directly on a chip.
It was a major pain as someone just trying to write something. A higher level language designed for parallel computation on a large FPGA array might be more in line with what he wants...without trying to design hardware specifically to the problem. Although maybe after a while common patterns would arise.
I suspect there's some trickery going on in the meatputer though. The whole system feels kludgy. They seem to only really have the benefit of being massively parallel and heavily optimized through millions of years of them being eaten if they're not optimized. I'm also somewhat surprised that, given how parallel they are, they're not easier to deadlock. I'd expect it to be easier to crash one, and then execute a privilege escalation exploit to gain root access. I guess some of that millions of years of evolution has also added some decent semaphore code, since easily crashed ones probably also get eaten...
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
The human brain is obviously a combination of a logic system (computer) and a probability system that are able to communicate meaningfully with each other, seamlessly. Our ability to use logic AND guess at probable outcomes is what defines our intelligence. We want computers, and hence AI, to arrive at perfect answers for problems and different situations. Simply not possible. Our greatest mental asset is also our greatest mental liability. The ability to guess and use "intuition" and arrive at answers without consciously thinking through the process spurs great achievements, but it also falls short most of the time. Given the system we have, it is possible to recognize when our intuition is obviously wrong using logic (a recursive algorithm). Guessing from our evolutionary state, this combination would also reduce the amount of energy needed to power such a system. I think my brain uses a bit less energy than a Watson, and is obviously more capable. Then again, maybe a machine would be capable of perfect answers or solutions, since it would be able to search and access stored experiences and information uninhibited. Recall is a key component of formulating solutions.
I object to power without constructive purpose. --Spock
It's not 2009 anymore, Chinese Tianhe-1A is indeed the fastest since November 2010
Read it and weep:
http://www.top500.org/
human brains working together (including passing information from one generation to the next) can build tools that can multiply billions of numbers. Human brains get credit for any supercomputer's computation.
It's a software problem.
Well, that's the hypothesis put forward in the 1950's that hasn't yielded results.
In contrast, something like Watson has massive amounts of processing power and storage access, with relatively simple algorithms. Watson is the ENIAC of the 2029 pocket calculator.
I wonder if humans like to think of themselves as needlessly complex.
But as to the main story - "we need more power-efficient, more parallel hardware":
Watson: "What is the main focus of modern computer architecture for the past 10 years?"
Trebek: "Correct"
Sean Connery: "That's what your mother said last night, Trebek!"
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Moore's law says that the 2.5Pfl machine in a 20 watt package is about 25 - 30 years away.
From what I'm hearing, Dr Sejnowski's plaint only partly addresses the problem. To implement cognition using a computational model, we need a neural simulator that:
- is large enough to represent all the neurons and interconnections needed to synthesize human-level cognition
- uses much less power than a supercomputer
But to be more than "a brain in a jar" it also must:
- learn using supervised and unsupervised instruction
- quickly load and unload modules of what it has learned
Without addressing all four goals, you've just recreated what we already have with its inherent limitations:
- an abstraction that is clunky and inefficient -- neural nets (analog) run on von Neuman (digital) architectures
- an implementation that uses too much power (supercomputer)
- knowledge representation that is not modular or decomposable
So even if we can devise a more appropriate implementation than NNs on the Tianhe-1A, we're still a long way from nirvana.
Comment removed based on user account deletion
It just seems like a massive waste of computational resources... I would rather have a well programmed predictable computer program controlling my spacecraft vs a brian modeled after humans which may decide to go on strike or otherwise act unreliably.
Why not just use GAs and NNs in specific context where they make sense... rather than trying to copy brains?
If you want to solve hard math problems who is to say intelligent solvers can't be designed to provide real results for a fraction of the computer time?
If you want to teach a machine to build a personal spacecraft by itself on an assembly line who is to say state of the art algorithms can't just be programmed into the machine to do the work in a predictable way without having to worry about unrealistic amounts of computation?
It seems to me that developing a brain in a bid to have it develop a better brain...ad nauseum until all the secrets of the universe are unlocked or the earth is converted into replicator blocks ...is actually not very useful in the real world.
This would work except for the fact that most people don't want to wait 25 years to get a computer that can just do basic math.
You can't just focus on neurons and their connections. There are 10x more glial cells in the brain and more and more research is discovering that they not only perform their basic role to support metabolism and structure, they also communicate with themselves, communicate with neurons and are an integral part of cognition.
In addition, they are finding that chemical communication between cells is not point to point contained within the synapse only. Cells are swimming in chemical and electrical communication that is most likely far more complex than a neural network represents.
Unfortunately it doesn't work like that. (Static) data uploaded by a previous version of an AI is of no use whatsoever until the AI's are relatively mature. Even then, it's nowhere near as useful as an actual teacher. Copies of (parts of) their minds ... maybe.
The whole point of ANN logic is to learn feedback loops. Now these feedback loops will be massively different for even small modifications of the robot's body (e.g. despite being a massively overused cliche in movies, one human cannot control the body of another, or even a android body similar to his own, until months or years of training pass)
You could probably duplicate AI individuals, and one might be able to extract parts of their brain to solve small tasks. But they'd have to be pretty small tasks for this procedure to have a snowball's chance in hell of succeeding. Still, reading, that might work. Simple, confined tasks that don't require any grasp whatsoever of "the big picture" (whatever that may be in that specific field).
To grow many capable AI consciousnesses, you'd need a full digital society, if you want a lot of capable specialized individuals. Digital versions of things like, oh, nurseries, parents, schools, police, ... the works. And it would take a while to grow a new one (at least, in the perception of the AI's involved it would take a long while, maybe it could be only a little time in the real world, but it would take years of interaction with the previous generation AI's to get them to say basic words, just like with human infants). Creating a new individual would take a huge amount of processing power.
Of course, given what a huge intelligence difference even 2% more neurons means, it seems unlikely we'd even be able to control AI's once they grow significantly beyond our minds. One buffer overflow mistake ... whoops.
The dose of realism injected by a real live neuroscientist ought to be paid attention to. Most CS types know too little about neuroscience and psychology to have a worthwhile opinion about the viability of human-level machine intelligence and what it takes to get there. I used to believe we'd have a strong AI by oh, 2040 or so until I started really looking into the fields I mentioned, and every informed post like the one I'm replying to reaffirms my belief that we have a very, very, very long way to go--if it is in fact possible at all.
I read somewhere that saying we are on our way to strong AI given our current achievements is like saying we are on our way to the moon after climbing a tree. Sure, we've made some progress, but not nearly as much as is commonly portrayed.
Your brain is not a computer.
I did read it.
Given the low quality of China's manufacturing(and their propensity to copy, not create), the current data would be very suspect. Doubly so for where they use knockoff chips.
Twitter supports and protects racists - by smearing their critics with the "Hate Speech" label.
"... and still has trouble with vision, motion, and 'common sense,' ..."
Management material.
Sure enough, the cow costume was hanging up next to the superhero outfit and sailors uniform. (S,Spud)