Slashdot Mirror
Palm Founders Form AI Company
Mentifex writes "As reported in the New York Times, Kansas City Star and other news media, Jeff Hawkins (co-author of On Intelligence) and Donna Dubinsky, co-founders of Palm Computing and Handspring, along with Dileep George as the principal engineer, are starting an AI company named Numenta as a follow-up to Hawkins' recent work on visual processing."
Can anyone point me toward some research on associative AI? i.e. Instead of AI that trained by nueral nets or genetic algos, does anyone know of research on "scoring" words based on their relation to other words? Extending words into concepts, an AI could become quite intelligent at things like Spam filtering.
Just something I was thinking about lately. Anyone?
Javascript + Nintendo DSi = DSiCade
You had to reset Palm PDAs in interesting ways, like poking a tiny button hidden ina hole with a paper clip. Imagine what you'd have to do a bot with Palm-like AI...
"Sir, to reset the machine, you'll need to sharply press its reset button, located at the back of the machine, just before its legs. just quickly pop your foot against it to press it."
"Uh, are you telling me that to reset it, I have to kick its ass?"
"Er...yes, sir."
Vos teneo officium eram periculosus ut vos recipero is.
Great, just what I need, an AI app that keeps poping up saying, "You know you should go to that meeting. What do you mean you don't want to go? Did you remember your wedding anniversary? Have you called your wife? Who's this 'Elle' person in your phone book. You should stop playing 'Tetris' so often..."
It is not our abilities that show what we truly are... it is our choices.
to the machines taking over...
5 /03/24/cz_qh_0324numenta.html
http://www.forbes.com/technology/personaltech/200
no sig yet
According to news.com.com.com.com, IBM is working on something similar...
DBA? Software Engineer? My company is hiring! Click
Numenta is developing a new type of computer memory system modeled after the human neocortex
surely this technology would be incredibly slow? (this is not a troll, read on before you mod me down!)
From what I remember from my neural networks days the human brain/neocortex works so well because of its massively parallel nature (not because of the processing power of any one neuron), and that computers simply aren't able to exploit this as they aren't designed to work like this - They are instead designed to to massively serial operations using extremely powerful chips (neurons) because the overhead of managing these parallel operations synchronously is too great (the human brain/neocortex work asynchronously)
am I wrong about this or am I missing something great that they've stumbled accross?
It appears the article summary might be misleading. From the first sentence of www.numenta.com:
Numenta is developing a new type of computer memory system modeled after the human neocortex. The applications of this technology are broad and can be applied to solve problems in computer vision, artificial intelligence, robotics and machine learning.
They further go on to say:
Numenta is a technology platform provider rather than an application developer. The Company is creating a scalable software toolkit that will allow developers and partners to configure and adapt HTM systems to particular problems.
My reading on this is that they aren't an AI company - they're just developing a technology that could be used for AI or many, many other uses.
I'm a big tall mofo.
By training neurons, they learn to achieve the desired result of a user.
Pretty complex material, anyone wanting to delve into should do some reading on Minsky (proposed neural networks could make dead bodies perform tasks...creepy to say the least) http://en.wikipedia.org/wiki/Marvin_Minsky
When they release a white paper Im sure itll only be the beginning of a prosporus field of study.
~ Jon
FWIW to ya, A.L.I.C.E is an cool webbot AI similar to the old ELIZA bots of old, but with some sophistication that allows it to be programmed to answer specific questions and recognize some words and phrases well. Won't pass a Turing test, but hey, it's free.
The webpage above has an animation that appears to have a bot attached to it. Pretty and cool.
Vos teneo officium eram periculosus ut vos recipero is.
Nothing starts my day better than the pleasant scent of vaporware wafting from my computer. We live in a great time. This shows what a kid with nothing but a formalism and a dream can accomplish.
In the book, Hawkins remarks that AI researchers often took the misguided approach that intelligence is a set of principles or properties, when in fact it's strictly a matter of behavior. To be intelligent is to behave intelligently. If he's right, then it's the act of being, wherein which the brain's primary tool is the continuous analogizing of current circumstances to past situations in order to make good predictive decisions, which constitutes intelligence.
He's the first to claim that he's not looking for sentience or to answer the question of sentience, but is instead only looking for a practical engineering approach to building intelligent machines. I think this is doubly clever since the issue of sentience should not be addressed until well after, as Hawkins often remarks, our own brains are understood first, in terms of how they operate. Why they operate, or what motivates us or what makes us 'cognitive agents' don't enter the equation with his approach.
I Want To Believe
I'm surprised that the short summary, from my brief perusal, does not include reference to work by Peter Foldiak (1991, 199?) and Wallis (1996). Both these authors published numerous papers on temporal and spatial coherence. My MSc in 1996 was also on the same topic followed by human research on the same problem. All of the computational work was with unsupervised learning algorithms varying whether the temporal processing was at the input our output stage.
I guess I'll have to read the original paper. However, the notion of temporal processing has been around for a long time.
Note: My own human research has yielded reliable data that addresses the acquisition of invariant object recognition.
I guess building spaceships is old-hat for rich techies now, so he's going to blow his millions on AI. I don't expect anything tangible to come from this.
Have you read my blog lately?
Mentifex. The name alone conjures up flamewars of years past on Usenet.
The big question in AI is whether an AI "mind" is more likely to spring up from a handful of rules, or whether a top-down design will bring it about. Mentifex was always in the latter camp.
Those in the former camp, including the Palm founders in the article, always seemed to be on the verge of something, but never seemed to really get any closer to a "mind" than some fuzzy logic.
We're still a long way off from Number 5 Alive.
I mean, heck, if it gets us even one step closer to having competent automated tech support, I'm all for it.
- Crow T. Trollbot
... that Dr. Otto Octavius is coming out of retirement to run the research department?
If someone says he and his monkey have nothing to hide, they almost certainly do.
The idea seems simple enough. Create a hierarchical inference structure. Train it on some data. Let the nodes learn what are the most frequent data. This forms the basic alphabet set. Propagate this up the hierachy. Learn the conditional probability distribution. Voila, you have a working visual recognition system. Problem is, the system will be slow, unless you have a processor capable of parallel or vector processing. Try implementing the system on Matlab with a 320x200 image, and see your processor crawl to a halt. Now, imagine doing this on a 320x200 video, and pray! Well, that's why we need a different processor architecture to make this work. But the theory is simple.
There are 10 kinds of people in the world - those that know binary, and those that don't.
The article gives little detail of the technology, and it's not like the general ideas Hawkins describes haven't been explored by people during the many decades of AI/neural networks research. The Numenta website gives the following:
HTM is "hierarchical" because it consists of memory modules connected in a hierarchical fashion. The hierarchy resembles an inverted tree with many memory modules at the bottom of the hierarchy and fewer at the top. HTM is "temporal" because each memory module stores and recalls sequences of patterns. HTM is hierarchical both temporally and spatially. An HTM system is not programmed in a traditional sense; instead it is trained. Sensory data is applied to the bottom of the hierarchy and the HTM system automatically discovers the underlying patterns in the sensory input. You might say it "learns" what objects are in the world and how to recognize them. Time is an essential element of how HTM systems work. First, to learn the patterns in the world, the sensory data must flow over time just as we move our eyes to see and move our hands to feel. Second, because every memory module stores sequences of patterns, HTM systems can be used to make predictions of the future. They not only discover and recognize objects but they can make predictions about how objects will behave going forward in time.That sounds like a number of neural network approaches, including Stephen Grossberg's work at BU. Although Hawkins seems to be a very bright guy, this field is littered with very bright researchers who made bold claims, and none of those efforts have yielded revolutionary businesses. Anyone remember (Stanford AI researcher) Edward Feigenbaum's Fifth Generation book in the 1980s? Doug Lenat's Cyc project?
Remember the huge difference between one neuron's firing rate and the clock speed for processors. The brain operates in a way that's fundamentally different from how we program and how computers operate: massive parallelism with slow components versus (mostly) serial computation. So when a company says they'll market a software solution to something which scientists haven't figured out yet, I am indeed skeptical. This is really more research effort than commercial venture, and Numenta admits this: "It may well take several years before products based on HTM systems are commercially available."
I hope there's something here. I'd love to see an outsider come in with fresh ideas and create a software platform to explore neuro-inspired programs. But let's be realistic and remember the history of AI. A red flag is the lack of any scientific papers available from the Numenta web site. If they are far enough along to make a software development kit, they should have been publishing results in peer-reviewed journals (with appropriate patent filings if necessary). So far, the only literature published is a trade book: On Intelligence.
After reading the Tech Report (note -- not a published paper in a respected journal) its clear that they are not presenting anything new here.
Its surpising that a) its news and b) they anyone is founding a company based on these ideas since they have to date not been sucessful in solving "the vision problem."
Firstly, the main ideas that they use have had a long history in visual modelling and statistical pattern recognition. The assertion that visual processing operates so cleanly at "levels" is far from clear although an idea with quite a long history -- See Marr for instance...Or spatial frequency channels as another example of competing partition of function.
One main issue is that they never mention what an explicit representation of visual object actually is, let alone how they might be reflected in cortex. Their approach follows the typical learning ideas of the perceptron, etc.. but those systems are known to be unstable!
More seriously, their whole argument doesn't demonstrate they understand the realities of the structure and functional architecture of visual cortex. That the visual system is highly space-variant is a fact that makes simplistic rectilinear statistical pattern matching a daunting problem. Although it is possible that their _may be_ an invariant representation, the jury is still out since its far from clear how orientation maps, occular dominance columns and the other peculiarities of the visual areas might produce such a thing when you foveate.
In summary, it seems much more like these guys were brought on board for advertising fanfare.
of something as complex as a PDA, try something really simple like AI.
Yes, you are indeed missing something. But it's probably not your fault, the people who taught you neural networks probably didn't know enough about the brain.
The parallelity of human brains is widely and hugely overestimated.
Just think about the fact that you can easily recognize 2 random objects if you are shown them for as little as a second. In this second, there is only enough time for about 100 of your neurons firing. The path trough your brain therefore _cannot_ be longer than a dozen neurons or "operations".
Any modern CPU does billions(!) of operations per second. So the comparison really isn't very good.
None of the founders of Numenta other than Jeff Hawkins have any experience in AI or for that matter have any background in hardcore computer science.
Dileep George is an Electrical Engineering graduate, while the CEO Donna Dubinsky is a hardcore salesperson and holds an MBA. Interestingly, the page also mentions that Jeff Hawkins " currently serves as Chief Technology Officer at palmOne, Inc". Fishy!
Next Generation AI ? Who are we kidding ?
If you are at all interested in your brain, artificial intelligence, and artificial thought - you owe it to yourself to get a copy of this book.
I've been experimenting with neural networks implemented on FPGAs for awhile as a hobby - not much commercial interest in these systems just yet - but there is a lot of interesting work being done.
Remember 15 years ago, when people thought it would take decades and decades to sequence the human genome? Then someone came along and figured out a much faster technique. This same kind of thing is starting to happen in artificial intelligence; people from backgrounds OTHER than computational AI and biology are starting to get involved, and the new perspectives have brought new ideas IMHO.
Anyway, if you're interested in AI, get Hawkin's book 'On Intelligence'. It's damn good. One of the best I've read on the genre, and the references in the book will save you a lot of time delving further.
..don't panic
As the submission noted, this work will be building on what Hawkins wrote about in his recent book, On Intelligence. The companion web site for the book is here:
...
There are also a some reviews of the book:
http://blogger.iftf.org/Future/000605.html
http://www.computer.org/computer/homepage/0105/ran dom/index.htm
(By Bob Colwell, who was Intel's chief IA32 architect)
http://www.techcentralstation.com/112204B.html
http://www.corante.com/brainwaves/archives/026649. html
A quote from his book:
The agenda for this book is ambitious. It describes a comprehensive theory of how the brain works. It describes what intelligence is and how your brain creates it. The theory I present is not a completely new one. Many of the individual ideas you are about to read have existed in some form or another before, but not together in a coherent fashion. This should be expected. It is said that "new ideas" are often old ideas repackaged and reinterpreted. That certainly applies to the theory proposed here, but packaging and interpretation can make a world of difference, the difference between a mass of details and a satisfying theory. I hope it strikes you the way it does many people. A typical reaction I hear is, "It makes sense. I wouldn't have thought of intelligence this way, but now that you describe it to me I can see how it all fits together." With this knowledge most people start to see themselves a little differently. You start to observe your own behavior saying, "I understand what just happened in my head." Hopefully when you have finished this book, you will have new insight into why you think what you think and why you behave the way you behave. I also hope that some readers will be inspired to focus their careers on building intelligent machines based on the principles outlined in these pages.
Weren't neural networks supposed to lead to intelligent machines?
Of course the brain is made from a network of neurons, but without first understanding what the brain does, simple neural networks will be no more successful at creating intelligent machines than computer programs have been.
Why has it been so hard to figure out how the brain works?
Most scientists say that because the brain is so complicated, it will take a very long time for us to understand it. I disagree. Complexity is a symptom of confusion, not a cause. Instead, I argue we have a few intuitive but incorrect assumptions that mislead us. The biggest mistake is the belief that intelligence is defined by intelligent behavior.
What is intelligence if it isn't defined by behavior?
The brain uses vast amounts of memory to create a model of the world. Everything you know and have learned is stored in this model. The brain uses this memory-based model to make continuous predictions of future events. It is the ability to make predictions about the future that is the crux of intelligence. I will describe the brain's predictive ability in depth; it is the core idea in the book.
How does the brain work?
The seat of intelligence is the neocortex. Even though it has a great number of abilities and powerful flexibility, the neocortex is surprisingly regular in its structural details. The different parts of the neocortex, whether they are responsible for vision, hearing, touch, or language, all work on the same principles. The key to understanding the neocortex is understanding these common principles and, in particular, its hierarchical structure. We will examine the neocortex in sufficient detail to show how its structure captures the structure of the world. This will b
I predict that the first AI they produce will work so well, that no one who buys one will ever need a replacement, so the company will spiral into obsolesence while Microsoft et al mkae a mint on AIs that are much easier to develop for...
[o]_O
I am actually currently reading his book--started about a month ago and am finishing the last of it now (a little every night before bed, when I'm not too tired).
His approach is surprising similar to my own (which I was initially happy to see), but less developed in some important ways. His book sometimes makes reference to being the first to consider this or that--nothing of which was new to me... things I've ready and/or talked about many times with others.
His approach also has a few critical flaws..
Foremost, invariance (the ability to recognize something regardless of where it is seen) cannot be achieved the way he speculates. I've testing this idea (and numerous others) in software years ago.
He illustrates this in the vision cortices where, he suggests, small sub-regions of the brain each learn to recognize something separately but criss-cross to other areas so that recognition can be invariant. I feel stupid admitting that I actually attempted this approach once...but not so alone now that Hawkins is advocating it.
First, each low-level (first to image) sub-region may break between another across the visual field at points within the object--what is going to target them into the fields? This problem can be satisfied farthar up the tree by cross-mixing between regions (and/or layers), but it's not very efficient.
Secondly and the critical point, this criss-cross betweens sub-regions method does not solve, but only moves the problem to a different space. Both the invariant identification and the location of identification are crucial factors to remember. But with the criss-cross method, there will be oodles and oodles of entities representing the same object of which higher level processes will need to somehow discover that they are the same thing......every time it's seen in a different place....
Another major problem is as to how this criss-crossing developes..given universal behavior for all neurons.
Matthew C. Tedder
There are actually quite a few projects now taking similar, cortex-centric approaches to AI hard problems. Are we up to something here? The guys responsible of these projects are not wacko types at all, but established entrepreneurs and/or well-known researchers:
CCortex "A 20-billion neuron simulation of the Human Cortex and peripheral systems."
Cyc a knowledge base with vast collection of facts about the real world and logical reasoning ability. Financed by Paul Allen AI related investment company,Vulcan.
Numenta is developing a new type of computer memory system modeled after the human neocortex.
They seem to we well financed, and knowledgeable. Are we witnessing the start of something big here?
I don't want to sound like Chicken Little here and I realize that Jeff's work target falls short of sentience, but I do want the planet to start thinking about "Pre-Sentient AI" in a conservative, cautious way.
Therefore I propose these Four Rules Of AI Development:
Rule One:
AI projects be Air-Gap network isolated and not be allowed to connect to the internet.
Terminator III's premise is a plausible one. All entities are self-interested and will seek to defend and propagate themselves. Global internet infrastructure could be seriously damaged by a well crafted host of worms.
Rule Two:
AI projects will not have access to diagrams of their own design circuitry.
This is to enable the effectiveness of Rule Three.
Rule Three:
All AI projects will have a buffered, hardware access to core thought processes so that the high order thought and planning can be observed with the AI entity's knowledge.
Rule Four:
All AI projects will be run on limited time run enabled power supply grids that are not documented design or protocol-wise anywhere on the internet.
This is to enable containment in worst case scenario situations.
There. I think I just saved the Planet.
"A microprocessor... is a terrible thing to waste." --
GeneralEmergency