Slashdot Mirror
Mapping the Brain's Neural Network
Ponca City, We Love You writes "New technologies could soon allow scientists to generate a complete wiring diagram of a piece of brain. With an estimated 100 billion neurons and 100 trillion synapses in the human brain, creating an all-encompassing map of even a small chunk is a daunting task. Only one organism's complete wiring diagram now exists: that of the microscopic worm C. elegans, which contains a mere 302 neurons. The C. elegans mapping effort took more than a decade to complete. Research teams at MIT and at Heidelberg in Germany are experimenting with different approaches to speed up the process of mapping neural connections. The Germans start with a small block of brain tissue and bounce electrons off the top of the block to generate a cross-sectional picture of the nerve fibers. They then take a very thin slice, 30 nanometers, off the top of the block. 'Repeat this [process] thousands of times, and you can make your way through maybe the whole fly brain,' says the lead researcher. They are training an artificial neural network to emulate the human process of tracing neural connections to speed the process about 100- to 1000-fold. They estimate that they need a further factor of a million to analyze useful chunks of the human brain in reasonable times."
One reason we could do the aforementioned mapping with C. elegans is that the worm's neurons are always laid out the same way from worm to worm. This is not the case for humans, and probably not the case for any vertebrate.
Science is all about the baby steps. You can't talk about determining the weights before you know what the connections are.
Can you be Even More Awesome?!
Perhaps you didn't get much out of neural networks, but my PhD thesis was on the similarities between a kohonen network and relaxation-labelling equations. Part of it is up on my blog (I haven't actually got as far as that bit yet, but the groundwork is there).
A neural network (well, anything more complex than the single-layer perceptron anyway) is an arbitrary classifier. I'm curious as to why other methods are "much better". Unless you do an exhaustive search of the feature-space, all classifier methods are subject to the same limitations - local maxima/minima (depending on the algorithm), noise effects, and data dependencies. All of the various algorithms have strengths and weaknesses - in pattern recognition (my field) NN's are pretty darn good actually.
It's also a bit odd to just say 'neural networks' - there are many many variants of network, from Kohonen nets through multi-layer perceptrons, but focussing on the most common (MLP's), there's a huge amount of variation (Radial-basis function networks, real/imaginary space networks, hyperbolic tangent networks, bulk-synchronous parallel error correction networks, error-diffusion networks to name some off the top of my head), and many ways of training all these (back-prop, quick-prop, hyper-prop, batch-error-update, etc. etc.) I guess my point is that you're tarring a large branch of classification science with a very broad brush, at least IMHO.
Not to mention that this is all the single-network stuff. It gets especially interesting when you start modelling networks of networks, and using secondary feature-spaces rather than primary (direct from the image) features. Another part of my thesis was these "context" features - so you can extract a region of interest, determine the features to use to characterise that region, do the same thing for surrounding regions, and present a (primary) network with the primary region features while simultaneously(*) presenting other (secondary) networks with the features for these surrounding regions and feeding the secondary network results in at the same time as the primary network gets its raw feature data. This is a similar concept (if different implementation) to the eye's centre-surround pattern, and works very well.
If you work through the maths, there's no real difference between a large network and a network of networks, but the training-time is significantly less (and the fitness landscape is smoother), so in practice the results are better, even if in theory they ought to be the same. I was using techniques like these almost 20 years ago, and still (very successfully, I might add) use neural networks today. If it's a fad, it's a relatively long-running one.
Simon.
(*) In practice, you time-offset the secondary network processing from the primary network, so the results of the secondary networks are available when the primary network runs. Since we still run primarily-serial computers, the parallelism isn't there to run all of these simultaneously. This is just an implementation detail though...
Physicists get Hadrons!
If you can see snapshots, it is because you have shitty lighting, you are waving your hand in front of a monitor or your eyes are tracking the moving object for a split second. The photo receptors in your eye fire a signal when the "sensor is saturated". It's the rate of these signals which determines the brightness. The eye does not measure the amount of light which reaches its sensors in a fixed amount of time. The process does not need to be clocked and is not clocked, but it is discrete. You can think of it as a kind of pulse width modulation with a fixed on-time and variable off-time.
Your brain does indeed fill in a lot of information. You "see" things which are not really there and it's easy to prove. Another reply to your comment already mentioned the blind spot, but that's not all. The resolution of our eyes is indeed impressive, but only in a very small area at the center of your vision. The rest is pretty much a blur and only there to detect change and motion. You think you see the rest of the picture, but you only see it because you've looked there before and it hasn't changed enough to make you look again. Much of what you see, and even more so what you remember to have seen, is simply what you expect to see and not the result of actual sensory input. It works because we've learned to look at change. What simply "is" isn't so important and is filled in from prior information, and that is essentially a guess, an informed guess but still a guess. That's how people can have something in their field of view and literally not see it. It was in their view, but they just didn't look at it because their attention was captured by something else.
When you pull figures from your arse, you don't actually add anything to the discussion. All that we've learned is that your arse is rather large, and you are used to removing things from it.
Even if your (incorrect assumptions) were correct, 36" x 20" at 1000dpi would be 36000 pixels x 20000 pixels = 720M pixels. Clue: dpi is a scalar measure rather than area.
Of course, the human eye does not work anything like that. Rather than farting numbers I spent 10 seconds on Google to find this which looks into the question of Visual Acuity. The "high-res" part of the eye is a very small circle with about 120 "dots" across its diameter.
As we do not resolve entire "frames" in a single go, the concept of a frame-rate is completely ludicrous. Your argument earlier in the thread about observing skipping when seeing a high speed stimuli doesn't show evidence of a *periodic* frame rate. It just shows that there is a *minimum* temporal resolution. One does not imply the other, especially when the eye is processing asychronous input (from rods and cones).
Although you don't believe that the brain fills in the missing images with educated guesswork, we've already established that what you believe is shit. Most (if not all) neuroscientists have accepted that the high resolution continuous visual imagery that we see is mostly an illusion produced by the mind. There are many well reported experiments that provide evidence of this. You should look for anything on Visual Illusions - there are far too many decent results in peer reviewed journals for me to spend time looking for you. Change Blindness is a related phenomena.
Finally you've cooked up some stupid figures for the number of cells in a brain. Why do you feel the need to demonstrate how stupid you are? The actual numbers (which you get wrong by 3 fucking orders of magnitude) are in the summary of the article! How hard is it to read the 100 billion neurons at the top of the page.
So next time you feel the need to pontificate needless about something that you don't know anything about. Don't. You, sir, are a thief of oxygen and your pointless ramblings have made everyone reading this article collectively dumber.
PS Feel free to mod me flamebait, as I am clearly annoyed. But when you do so remember that the everything the parent poster wrote was incorrect and that I have pointed out to him where he is wrong.
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
Well, that's the hope, and a major source of appeal for the humble nematode. Unfortunately, that's still far beyond what we know right now. The physical map of every neuron and their connections has been complete for decades. Still, despite a whole lot of effort, researchers are still working to piece together small functional circuits for the simplest of behaviors. A lot of complexity arises in neural circuits -- one physical circuit can contain several independent functional circuits, depending on the types of inputs.
The best current knowledge of C. elegans neurophysiology involves qualitative descriptions of small circuits, involving a few dozen neurons. Unfortunately, while you can do a lot of good behavioral studies and other experiments, it's impossible to directly record the activity of specific neurons. Also, it turns out that some "neural" functions are actually performed by other cells. For example, one pattern generator in the digestive tract actually resides in intestinal cells instead of neurons -- my lab is working on the genetics involved.
This shit gets complicated, fast.
IAAUCER
I am an (undergrad) C. Elegans researcher
Actually, more recent methods don't have local maxima/minima. Something like a support vector machine optimizes an objective function. Of course, this is somewhat of a tangent, in that the objective function might not be a useful metric for performance, but people have shown that the minimum objective function value of a SVM does relate to its generalization performance. It's a little disconcerting that a NN has an objective function but that it can find it's minimum or that the minimum doesn't give good performance on test data (over-fitting)...
Of course, part of the NN's problem stems from the fact that it is an arbitrary classifier. It's hard to give generalization results for an algorithm that has an infinite VC dimension. (There are techniques to restrict the size of the weights to give some guarantees.) However, this doesn't mean NNs can't perform well in practice. It probably means that the current theoretical analysis is somewhat flawed in relation to the real world.
So have you ever compared your NN algorithms with the popular algorithms of the day such as SVMs with kernels or boosting algorithms. Also, are your NN algorithms generic or do you heavily customize and tweak to get good performance.
Chris Mesterharm