UK Researchers Make Neural Networks Smarter

Small Hairy Troll writes: "EDTN is running this story concerning a researcher in the UK who has come up with a method for getting those pesky Neural Nets to teach themselves to see. Called the 'Product of Experts,' the Neural Net is built using 'Experts.' If "you had one expert that preferred furry animals, whereas another expert preferred domesticated animals and another preferred small animals, their votes ... would light up dogs and cats very nicely." And an Edinburgh professor is quoted in the story as calling it "the first neural-network architecture that is both sensibly implementable and worth implementing."

Comments

Re:I sure hope so -- they could hardly be dumber!

[FigWig]

You can think of a NN as a non-linear function which is modelled after nueral connections. The inputs are weight vectors determining the strength of the connections between nodes, and a test vector with length equal to the number of nodes in the first layer. The output is a vector of length equal to the number of nodes in the final layer. Backpropagation is a technique that will optimize the link weights in order to minimize the error function - usually one half the sum of the square of the difference between network output and intended output (makes the derivative look nicer).

I don't think you can call a NN any dumber than a curve you fit to a graph, since you are just optimizing a function to minimize the error between the observed output and the output of the fit function. A traditional function may give you slightly more insight into a problem, but they can also easily mislead you (eg every function looks like a parabola around a min or max).

My point is that NNs are tools, sometimes effective, sometimes not. They aren't anything close to the AI you read about in scifi books, but nothing is. More interesting actually are support vector machines (svms). There are several papers on the web about them, they were devloped at bell labs by some russian dude whose name escapes me at the moment. They can be more effective than NNs, but the math is a bit harder to understand.