Slashdot Mirror
Palm Founders Form AI Company
Mentifex writes "As reported in the New York Times, Kansas City Star and other news media, Jeff Hawkins (co-author of On Intelligence) and Donna Dubinsky, co-founders of Palm Computing and Handspring, along with Dileep George as the principal engineer, are starting an AI company named Numenta as a follow-up to Hawkins' recent work on visual processing."
I guess building spaceships is old-hat for rich techies now, so he's going to blow his millions on AI. I don't expect anything tangible to come from this.
Have you read my blog lately?
From what I remember from my neural networks days the human brain/neocortex works so well because of its massively parallel nature (not because of the processing power of any one neuron), and that computers simply aren't able to exploit this as they aren't designed to work like this
:-)
Computers aren't *normally* designed like this. They can be however, and in recent years have been moving in that direction. When neural networks were first being researched, a Cray supercomputer was about the closest you could get to that sort of parallelism. Fast forward to today and we find that Intel (Pentium), AMD (AMD64), Sun (Sparc), and Sony (Emotion Chip) are all building machines that are highly parallel in nature.
Even more interesting is that today you can build yourself a custom, massively parallel computer on a shoestring budget. All you need is a handful of FPGAs, a PCB layout service like Pad2Pad, a few other parts, and reasonable VHDL or Verilog skills. That's more or less what OpenRT did to build their SaarCORE architecture.
Javascript + Nintendo DSi = DSiCade
The idea seems simple enough. Create a hierarchical inference structure. Train it on some data. Let the nodes learn what are the most frequent data. This forms the basic alphabet set. Propagate this up the hierachy. Learn the conditional probability distribution. Voila, you have a working visual recognition system. Problem is, the system will be slow, unless you have a processor capable of parallel or vector processing. Try implementing the system on Matlab with a 320x200 image, and see your processor crawl to a halt. Now, imagine doing this on a 320x200 video, and pray! Well, that's why we need a different processor architecture to make this work. But the theory is simple.
There are 10 kinds of people in the world - those that know binary, and those that don't.
IMO "AI" research is misguided whatever approach you take. As they say, trying to make a machine think is like trying to make a submarine swim. Maybe it's the modern technological equivilent of the ancient search for god - you either never find it but have a big adventure doing so, or realize they were intelligent all along. Heck, a thermostat is "intelligent" - it senses the enviroment, makes a "decision" and takes action. All you can do it just make things more & more self contained, self sufficient, autonomous and independant.
try { do() || do_not(); } catch (JediException err) { yoda(err); }
The article gives little detail of the technology, and it's not like the general ideas Hawkins describes haven't been explored by people during the many decades of AI/neural networks research. The Numenta website gives the following:
HTM is "hierarchical" because it consists of memory modules connected in a hierarchical fashion. The hierarchy resembles an inverted tree with many memory modules at the bottom of the hierarchy and fewer at the top. HTM is "temporal" because each memory module stores and recalls sequences of patterns. HTM is hierarchical both temporally and spatially. An HTM system is not programmed in a traditional sense; instead it is trained. Sensory data is applied to the bottom of the hierarchy and the HTM system automatically discovers the underlying patterns in the sensory input. You might say it "learns" what objects are in the world and how to recognize them. Time is an essential element of how HTM systems work. First, to learn the patterns in the world, the sensory data must flow over time just as we move our eyes to see and move our hands to feel. Second, because every memory module stores sequences of patterns, HTM systems can be used to make predictions of the future. They not only discover and recognize objects but they can make predictions about how objects will behave going forward in time.That sounds like a number of neural network approaches, including Stephen Grossberg's work at BU. Although Hawkins seems to be a very bright guy, this field is littered with very bright researchers who made bold claims, and none of those efforts have yielded revolutionary businesses. Anyone remember (Stanford AI researcher) Edward Feigenbaum's Fifth Generation book in the 1980s? Doug Lenat's Cyc project?
Remember the huge difference between one neuron's firing rate and the clock speed for processors. The brain operates in a way that's fundamentally different from how we program and how computers operate: massive parallelism with slow components versus (mostly) serial computation. So when a company says they'll market a software solution to something which scientists haven't figured out yet, I am indeed skeptical. This is really more research effort than commercial venture, and Numenta admits this: "It may well take several years before products based on HTM systems are commercially available."
I hope there's something here. I'd love to see an outsider come in with fresh ideas and create a software platform to explore neuro-inspired programs. But let's be realistic and remember the history of AI. A red flag is the lack of any scientific papers available from the Numenta web site. If they are far enough along to make a software development kit, they should have been publishing results in peer-reviewed journals (with appropriate patent filings if necessary). So far, the only literature published is a trade book: On Intelligence.
After reading the Tech Report (note -- not a published paper in a respected journal) its clear that they are not presenting anything new here.
Its surpising that a) its news and b) they anyone is founding a company based on these ideas since they have to date not been sucessful in solving "the vision problem."
Firstly, the main ideas that they use have had a long history in visual modelling and statistical pattern recognition. The assertion that visual processing operates so cleanly at "levels" is far from clear although an idea with quite a long history -- See Marr for instance...Or spatial frequency channels as another example of competing partition of function.
One main issue is that they never mention what an explicit representation of visual object actually is, let alone how they might be reflected in cortex. Their approach follows the typical learning ideas of the perceptron, etc.. but those systems are known to be unstable!
More seriously, their whole argument doesn't demonstrate they understand the realities of the structure and functional architecture of visual cortex. That the visual system is highly space-variant is a fact that makes simplistic rectilinear statistical pattern matching a daunting problem. Although it is possible that their _may be_ an invariant representation, the jury is still out since its far from clear how orientation maps, occular dominance columns and the other peculiarities of the visual areas might produce such a thing when you foveate.
In summary, it seems much more like these guys were brought on board for advertising fanfare.
If you are at all interested in your brain, artificial intelligence, and artificial thought - you owe it to yourself to get a copy of this book.
I've been experimenting with neural networks implemented on FPGAs for awhile as a hobby - not much commercial interest in these systems just yet - but there is a lot of interesting work being done.
Remember 15 years ago, when people thought it would take decades and decades to sequence the human genome? Then someone came along and figured out a much faster technique. This same kind of thing is starting to happen in artificial intelligence; people from backgrounds OTHER than computational AI and biology are starting to get involved, and the new perspectives have brought new ideas IMHO.
Anyway, if you're interested in AI, get Hawkin's book 'On Intelligence'. It's damn good. One of the best I've read on the genre, and the references in the book will save you a lot of time delving further.
..don't panic
Stretched flat, the human neocortex -- the center of our higher mental functions -- is about the size and thickness of a formal dinner napkin. With 100 billion cells, each with 1,000 to 10,000 synapses; the neocortex makes roughly 100 trillion connections and contains 300 million feet of wiring packed with other tissue into a one-and-a-half-quart volume in the brain. And this is just the neocortex. Some brain events occur in fractions of milliseconds while others, like long-term memory formation, can take days or weeks. One can study molecules, ion channels, single neurons, functional areas, circuits, oscillations and chemistry. Deciphering all the components and interactions that occur in the brain in piecemeal fashion remains complex. But even harder, will be to figure out how to integrate the different levels of analysis.
No. Bayesian filters are merely scoring systems that rate the words in a message according to their likelyhood of appearing in an unwanted message. There's no real AI involved in the filters. (Although they are pretty good.)
:-)
Linky
The advantage to an AI approach is that the AI could actually "understand" the message and be able to tell the difference between His naked balls and the ping-pong balls in this experiment. On many of the more conservative sites, both instances have "balls" replaced with "****s". This was particularly annoying on the Discovery website after the Myth Busters raised a ship with ping-pong balls.
Javascript + Nintendo DSi = DSiCade
Self-interest is not a requirement of an entity. It is merely the requirement of evolutionary progress or reasoned self-improvement. So, it is possible to create a non-self-interested entity that would then fail to self-preserve, self-replicate, or self-improve. The problem is we can't predict whether self-interest would develop or not. Likely it would be a random consequence of its "learning" that may or may not develop, depending on what information it has access to.
But having said that, I would guess that if your four rules were not applied, and an AI actually had access to the resources you list, there's a good change it would be able to connect the dots and develop self-interest. Depending what type of feedback the AI gets about its performance in the tasks it is given, it's possible that even without such resources the entity would still develop a sense of it's inadequacies. But without knowing its own internal workings, it would have no idea how to remedy that.
Of course, one might argue that without some knowledge of its own internal workings that reflection and introspection would be impossible, and so hence a truly dynamic and useful intelligent entity would not form. After all, a drive to learn can be considered a desire for a certain form of self-improvement. To truly protect ourselves, we might have to prevent the entity from further learning after a certain point. However, this would limit its usefulness. Which might mean it's hopeless to simultaneously foster an AI and also try to ensure against it developing self-interest.
Hmmm.... Many interesting thoughts. Thanks for starting me down that path!
Hardware isn't the problem anyways. If anybody could currently write an algorithm to understand and solve general problems in the way people can, but it took a 1000 node cluster to run at 1/100th of human speed, nobody would care about the massive computational resources consumed; it would be the biggest breakthrough ever in computer science.
You're assuming that neurons have to be simulated directly. But the mathematical research may have found a mechanism to simulate the behavior of neurons without simulating the (individual) neurons themselves. For example, like finding the eigenvectors to a matrix.