Slashdot Mirror
Implementing Artificial Neural Networks
Floydian Slip wrote to us with the updated story about a company called Axeon that is aiming to use the concept of artificial neural networks in a processor called the "Learning Processor." It's an array of 256 8-bit [RISC] chips in parallel. The company is aiming in a lot of places - mobile communications, inertial navigation and image analysis. The article also gives some of the background of the "neural chips".
Well, there's TD Gammon, which plays a pretty mean game of Backgammon using a neural network (and is rated as a master level player, no less!).. The author, Gerald Tesauro, has written a paper on the subject at:
http://web.cps.msu.edu/rlr/pub/Tesauro2.html
from the abstract:
"TD Gammon is a neural network that is able to teach itself to play backgammon soley by playing against itself and learning from the results, based on the TD(Lambda) reinforcment learning algorithm (Sutton, 1988). Despite starting from random initial weights (and hence random initial strategy), TD Gammon achieves a surprisingly strong level of play. With zero knowledge built in at the start of learning (i.e. given only a "raw" description of the board state), the network learns to play at a strong intermediate level. Furthermore, when a set of hand crafted features is added to the network's input representation, the result is a truly staggering level of performance: the latest version of TD Gammon is now estimated to play at a strong master level that is extremely close to the world's best human players. "
The folks at Cyberlife would have also used neural networks at the core of their 'Creatures' games and other applications. I like Creatures as an example, just because the neural networks clearly show their strengths and weaknesses in that context. The networks quickly reach an uncanny level of 'intelligence', but that 'intelligence' vanishes rapidly if training continues for too long. (In the Creatures fanbase, this loss of intelligence is called the 'One Hour Stupidity Syndrome', since the Norns typically start showing signs of it about an hour after being hatched)
Cyberlife can be found at:
http://www.cyberlife.co.uk/
or
http://www.creatures2.com/
products:
- most OCR programs, such as the ones that now come free with your scanners,
use neural networks for at least some of the steps to recognize
characters. See, for example, Caere OmniPage and Ligature,
which uses them in its "ocr-on-a-chip" that goes into its
handheld "Quicktionary" pen.
- data mining programs used NNW's to analyse
transactions for unusual patterns, e.g. credit card fraud. This is
now a big time business. See, for example, HNC Software, co-founded
by Robert Hecht-Nielson, a famous NNW guru at Univ. of San Diego.
- Sensory Inc. uses them in its voice recognition chips.
They've sold millions of such chips, which recognize just a few words
but with speaker independence, high background noise, and for low cost.
See the recent article at EE Times: "Toys that talk..."
- Synaptics , co-founded by Carver Mead, uses analog hardware
neural network techniques in its Touchpad that is used in many notebooks.
Have I convinced you yet? Most of these applications are at the
infrastructure level and don't get much PR, often for proprietary
reasons. Calera for example, was using NNW's in its OCR already
in the late 80's but didn't say anything about them until Caere started
bragging in ads in 1992 that it was using NNW's.
I remember an experiment in which a neural network was used to distinguish between objects in photographs. Although successful, the researchers discovered that instead of noticing differences in the objects, what was actually being compared were such subtleties as the color of the sky in the background.
One of the points brought up was that although neural nets hold interesting possibilities in the future, we first must find ways of dealing with the infinite number of possible relations that we take for granted in our own minds.
--
The only thing worse than being redundant is being redundant.
There are a fairly large set of questions to be asked about this sort of project, especially in which AI is being utilised as the sole method for optimising signals or navigation. The primary one is an age old question which is just going to get harder: Is it a feature or a bug?
Take a neural network as the sole navigational utility. Sure its been trained through 100,000 generations to work out the opimal path in realtime to fly out to saturn and back, but when it finally coes down to it, do you trust it? There is no algorithm you can check, there is no definite way of predicting what it may do if it encounters some previously unthought of situation.
Imagine, you turn on your ship, give your target, ship starts flying there no problem, then a meteor flies past you which happens to look remarkably like a square. Neural network gets a flashback to its initial training when simple squares were used to indicate planets because it was simpler, and it makes a massive erroneous gavitational adjustment and starts flying towards the sun.
Thats bad enough, but you can't even tell whether its idea of flying towards the sun is a good idea it has suddenly had about a slingshot that it could do to get you to your destination faster, or whether it has just gone nuts and is trying to get you killed.
The same, although less extreme cases apply to most things, if the AI is the only thing doing signal adjustment on your cellphone, maybe it'll flip out for no reason that is discernable. What do you do then? you can't "fix" the bug, its buried deep in such a complex neural network that it'd be like trying to figure out why a mute human with no body language drew a picture of a frog when told to draw a picture of an apple.
At least to start with, I think we are going to find that neural networks will only be good for tweaking certain aspects of a standard algorithmic system, and while these limitations are in place, they won't be able to show such huge advantages in signal tracking etc that is proclaimed for them. It will be some time yet before we can figure out ways of making AI safe.
You can't win a fight.
We did alot of neural network work at the Media Lab (using them with HMMs are really popular now in "intelligence" systems).
I can see this as being particularly useful for some applications, like the cellular network example the article had. However, there are several problems with Neural Nets that don't make them a panacea, or a wiz-bang solution to duplicating the human brain.
The last point is the biggest hinderance to neural net usage - we don't really know how to apply it to generalized (or even many specific) problem areas.
This is not to belittle to accomplishment. There are quite a few well-defined areas that neural nets are extremely useful, and we should find more as time progresses and our knowledge increases.
Just don't expect any kind of general intelligence system to be coming soon. It won't.
-Erik
There are always four sides to every story: your side, their side, the truth, and what really happened.
The question, which others have commented on here, is what neural networks are especially good at. Certainly narrow things like character recognition, face recognition and so forth seem to be a natural. Picking data out of noise also is promising - IIRC IBM was trying to use neural nets for one of their ultra-huge storage technology, where the signal-to-noise ratio on the magnetic heads is way too low for traditional encoding. Communication technology has essentially the same problem and will benefit.
Now, will you trust a neural network to pilot a Boeing you're on? Arguably, you already do - witness the recent disaster with a Korean jet which was ruled a pilot suicide. I doubt that we'll see such a "general AI" application in the field soon. Using dozens of small neural networks in sharply defined functions on an airplane, or on a car, will be more useful and done early... Mercedes' A-Class car's electronic suspension already uses serious heuristics to stabilize the car in dangerous situations, this sort of thing will probably be one of the first applications of neural networks.
Personally I think the term is somewhat of a misnomer. It's based on an early and too reductionistic view of how neurons were hypothesized to work in the brain. Sort of like the steam-era metaphors in Freud's work are being superseded by information-age metaphors in psychology. And there's the possibility that neurons aren't the basic "neural" building blocks at all... Roger Penrose in his books - "The Emperor's New Mind" and "Shadows of the Mind", proposes that each neuron's behaviour is actually defined by millions or more of tiny quantum switches in each neuron's microtubuli. His theory is very well argued but still controversial among orthodox researchers... I like it personally. If he's right, we'd need first to build a million-element quantum processor building block and then build useful neural networks out of millions such blocks - and even a single block wouldn't be mathematically simulatable by a conventional processor, even at very low speeds!
Frankly, I believe any "strong AI" applications for a neural network chip are out for the next decade or so. All this talk about pattern recognition is nice, but as soon as you get into symbolic processing - meaning as soon as "meaning" is involved" - you get into uncharted territory. "Meaning" is an emergent, bottom-up quality rather than a top-down macrofunction in the human brain (or any animal brain for that matter).