Slashdot Mirror
Mapping the Brain's Neural Network
Ponca City, We Love You writes "New technologies could soon allow scientists to generate a complete wiring diagram of a piece of brain. With an estimated 100 billion neurons and 100 trillion synapses in the human brain, creating an all-encompassing map of even a small chunk is a daunting task. Only one organism's complete wiring diagram now exists: that of the microscopic worm C. elegans, which contains a mere 302 neurons. The C. elegans mapping effort took more than a decade to complete. Research teams at MIT and at Heidelberg in Germany are experimenting with different approaches to speed up the process of mapping neural connections. The Germans start with a small block of brain tissue and bounce electrons off the top of the block to generate a cross-sectional picture of the nerve fibers. They then take a very thin slice, 30 nanometers, off the top of the block. 'Repeat this [process] thousands of times, and you can make your way through maybe the whole fly brain,' says the lead researcher. They are training an artificial neural network to emulate the human process of tracing neural connections to speed the process about 100- to 1000-fold. They estimate that they need a further factor of a million to analyze useful chunks of the human brain in reasonable times."
They are training an artificial neural network to emulate the human process of tracing neural connections to speed the process about 100- to 1000-fold.
So, they're training a neural network to automate the process of mapping a neural network, in the hopes of creating an intelligence that they can train to automate other processes?
My brain hurts...
Apology to Ubuntu forum.
One reason we could do the aforementioned mapping with C. elegans is that the worm's neurons are always laid out the same way from worm to worm. This is not the case for humans, and probably not the case for any vertebrate.
Not to mention - and I a sure I'll get modded down here - is that neural networks aren't very effective. They get a lot of hype in the media, and people who don't work in optimization like them. They sound appealing and cool, but there are other methods that are much better. If you look at the liturature, there is not that much hard science behind them. Some statistical mechanics people have some results, but they really are just a fad, like fractals or catastrophe theory. But bear in mind LOTS of people in applied areas immediately jump to the conclusion that neural networks are great, especially computer vision and robotics people (where I used to work.)
Sounds interesting, but is a map enough to understand the brain? I know in artificial neural networks, the actual structure isn't as important as the weights on the nodes. Will hitting the brain with electrons be enough to give us an understanding of these "weights", or just the connections between them?
Science is all about the baby steps. You can't talk about determining the weights before you know what the connections are.
Can you be Even More Awesome?!
http://www.ad.com/
An attempt to emulate a brain on a network of computers.
Deleted
If you emulate a human brain in real time, as well as connections to emulated eyes, ears, and mouths, you just have to talk to it for a couple years and it'll learn English. =D
That's what they call the brain's ability to change. By the time they complete a wiring diagram, it'll have changed. Also, knowing the wiring and connections is not enough. Knowing which connections are excitatory and which are inhibitory is necessary, and then tracking down loops of excitatory against excitatory resulting in inhibitory, etc. It's all fine and well to have a map, but that doesn't tell you squat about what anything does. A useful map would have to be dynamic, and the complexity of that is far more than just what they're considering for a wiring diagram.
"I may be synthetic, but I'm not stupid." -- Bishop 341-B
Many engineers have studies feed-forward neural networks and found them to be far inferior to other solutions. Of course, our brains use recurrent neural networks, which, unfortunately for engineers, are very difficult to analyze. There are many secrets yet to be teased out of these neural networks, but much progress has already been made. Researchers in our lab, for example, have demonstrated how introducing random synaptic failures improves not only energy efficiencies, but also the cognitive abilities of simulated neural networks. I'm currently researching the effects of variable activity (as measured in a biological neural network by an EEG), and I dare say there's a lot more that we don't know about these networks than we do.
Ben Hocking
Need a professional organizer?
Perhaps you didn't get much out of neural networks, but my PhD thesis was on the similarities between a kohonen network and relaxation-labelling equations. Part of it is up on my blog (I haven't actually got as far as that bit yet, but the groundwork is there).
A neural network (well, anything more complex than the single-layer perceptron anyway) is an arbitrary classifier. I'm curious as to why other methods are "much better". Unless you do an exhaustive search of the feature-space, all classifier methods are subject to the same limitations - local maxima/minima (depending on the algorithm), noise effects, and data dependencies. All of the various algorithms have strengths and weaknesses - in pattern recognition (my field) NN's are pretty darn good actually.
It's also a bit odd to just say 'neural networks' - there are many many variants of network, from Kohonen nets through multi-layer perceptrons, but focussing on the most common (MLP's), there's a huge amount of variation (Radial-basis function networks, real/imaginary space networks, hyperbolic tangent networks, bulk-synchronous parallel error correction networks, error-diffusion networks to name some off the top of my head), and many ways of training all these (back-prop, quick-prop, hyper-prop, batch-error-update, etc. etc.) I guess my point is that you're tarring a large branch of classification science with a very broad brush, at least IMHO.
Not to mention that this is all the single-network stuff. It gets especially interesting when you start modelling networks of networks, and using secondary feature-spaces rather than primary (direct from the image) features. Another part of my thesis was these "context" features - so you can extract a region of interest, determine the features to use to characterise that region, do the same thing for surrounding regions, and present a (primary) network with the primary region features while simultaneously(*) presenting other (secondary) networks with the features for these surrounding regions and feeding the secondary network results in at the same time as the primary network gets its raw feature data. This is a similar concept (if different implementation) to the eye's centre-surround pattern, and works very well.
If you work through the maths, there's no real difference between a large network and a network of networks, but the training-time is significantly less (and the fitness landscape is smoother), so in practice the results are better, even if in theory they ought to be the same. I was using techniques like these almost 20 years ago, and still (very successfully, I might add) use neural networks today. If it's a fad, it's a relatively long-running one.
Simon.
(*) In practice, you time-offset the secondary network processing from the primary network, so the results of the secondary networks are available when the primary network runs. Since we still run primarily-serial computers, the parallelism isn't there to run all of these simultaneously. This is just an implementation detail though...
Physicists get Hadrons!
and it's only like 302 neurons,so,it's possible to write a simulator of it?
Our computer technologies have yet to achieve the complexity of most biological brains. I'd love to see these new informations derive a new form of super-computer. Of course...We have to watch out for iRobot scenarios...
Don't hold your breath for an iRobot.
If each of the 100 billion neurons managed the 1000 or so synapses, and say a modern day PC with a quad processor could computationally handle say 100 neurons, you would need 1 billion PCs. Since 1 billion PCs would find it difficult to walk, the old adage of: "computers are as dumb as nails" still applies.
Now lets say usable processing power doubles every 5 years, and it shrinks to something small enough it can walk into our living rooms. That would be at least 150 years from now.
You could argue Moore's law is faster, but 2 issues. First is can it continue, not likely. Second is can we has humans program something this complex, not likely. Each generation of computers we get less and less value out of for the increases in CPU. You could say hello in 10-12 bytes in the '70's and early 80's, now it takes well over 1,000,000 bytes and 20 times longer to do the same thing on today's computers. Go figure.
So iRobot, Cherry 2000 and Commander Data are going to have to wait quite some time long after our great-great-great-great-great-great-great-great-great-great-great-great grandchildren have passed on. Say half the time to get a muppet.
The pig go. Go is to the fountain. The pig put foot. Grunt. Foot in what? ketchup. The dove fly. Fly is in sky. The dove drop something. The something on the pig. The pig disgusting. The pig rattle. Rattle with dove. The dove angry. The pig leave. The dove produce. Produce is chicken wing. With wing bark. No Quack.
Farnsworth: Lie down here and we'll do some tests. If Fry is out there, then Leela's brain could be acting as a five-pound Ouija board.
Leela: Is this some sort of brain scanner?
Farnsworth: Some sort, yes. In France, it's called a guillotine.
Leela: Professor! Can't you examine my brain without removing it?
Farnsworth: Yes, easily!
Actually, more recent methods don't have local maxima/minima. Something like a support vector machine optimizes an objective function. Of course, this is somewhat of a tangent, in that the objective function might not be a useful metric for performance, but people have shown that the minimum objective function value of a SVM does relate to its generalization performance. It's a little disconcerting that a NN has an objective function but that it can find it's minimum or that the minimum doesn't give good performance on test data (over-fitting)...
Of course, part of the NN's problem stems from the fact that it is an arbitrary classifier. It's hard to give generalization results for an algorithm that has an infinite VC dimension. (There are techniques to restrict the size of the weights to give some guarantees.) However, this doesn't mean NNs can't perform well in practice. It probably means that the current theoretical analysis is somewhat flawed in relation to the real world.
So have you ever compared your NN algorithms with the popular algorithms of the day such as SVMs with kernels or boosting algorithms. Also, are your NN algorithms generic or do you heavily customize and tweak to get good performance.
Chris Mesterharm