Slashdot Mirror

FYI.

Here's the abstract and there's a link to the PDF paper itself at the top. Release 10/13/2018.

https://numenta.com/neuroscien...

Thanks for bringing Jeff to my attetion, reminds me of when I was into Marvin Minsky a while back.

Biological systems tend to be very messy in comparison

That's not actually been established; in fact, the opposite appears to be the case. So far, we've learned that neurons are quite similar to one another; dendrite-to-axon connections similar; brain chemistry essentially uniform; cell metabolism mostly uniform.

It appears to be a challenge of understanding cortical topology more than anything else. Operations such as Numenta's are making inroads in that area.

And again, one doesn't have to completely understand something to use or create it if it has inherent or emergent characteristics of its own. And emergence is something that neural structures bring to the table in spades.

--fyngyrz
anon due to mod points

Understanding how humans store and recognize images primarily is not a barrier to AI. It's not memory or image recognition that's the hill to climb; The fundamental algorithmic/methodological challenges are thinking, along with conceptual storage, development and manipulation (these things incorporate memory use, but aren't a storage problem per se.) Hardware needs to be able to handle amounts of ram and long term, high speed storage that can serve as a practical basis for the rest as well. Right now, we're getting close, but it'll be a few more years yet before anything really smart can be instantiated. That's even if we were to figure out precisely how to do it right now.

It is possible -- though I consider it doubtful -- that we would implement human style vision neurology in hardware for an AI, but frankly our abilities are so poor compared to what can be accomplished I really don't see why we'd cripple an AI that way. It'd be abusive. "We could have made your visual recall incredibly acute, but... instead you're like us, and really don't have much more than a general idea what was in a scene after you have seen it." [AI nukes silicon valley] (Mods: that's humor. HUMOR.]

Also, check out Numenta's work.

Of course, understanding how humans store and recognize images is (very) important to our understanding of human physiology and disease, and it's wonderful that we're working on it.

An essential characteristic of intelligence is that we don't know how it works.

Actually, I think we do. We at least have an actual model, free of woo-woo, for which no counter evidence has been brought forth as yet.

Even the low level stuff seems to finally be yielding some clarity.

They're basically a really simple linear discriminant.

Actually, most of them are nonlinear. Sigmoid function is common, and there are much more exotic things going on too, such as fuzzy logic-based discriminants. Bottom line is that any discriminatory function is of interest.

There's also some fascinating stuff going on with time discriminants where they're having very encouraging results.

Odds are excellent that both (time and transfer function) are part of a solution that is most human-neuron-like. But it isn't by any means a given that we have to go there to make actual AI work. That's just how we work. Also, I am fairly confident, like this.

Since this is an area I'm very familiar with, I'll throw in a little science about why these predictions are not only realistic, but actually probably a bit pessimistic.

First of all, our understanding of the human brain has improved vastly in the past two decades. Especially in the areas that will be necessary for creating intelligent machines. The cortex (the part that kind of looks like a round blob of small intestines, with all the creases and folds) is much like a computer with a bunch of processors. Previously focus had been paid to the individual neurons as the processors. But a much larger unit of processing is now becoming the central area of focus; The Cortical Minicolumn which, in groups for a Cortical Hypercolumn. As minicolumns consist of 80-250 (more or less, depending on region) neurons and there are about 1/100th of them compared to neurons, it cuts down on complexity significantly.

Numenta and others are starting to take this approach in simulating cortex. Cortex is largely responsible for "thinking". The other parts of the brain can be seen, to some degree, as peripheral units that plug into the "thinking" part of the brain. For example, the hippocampus is a peripheral that's associated with the creation and recall of long term memories. The memories themselves, however, are stored in the cortex. We have various components that provide input, many of which send relays through the thalamus which takes these inputs of various types and converts them into a type of pattern that's more appropriate for the cortex and then relays those inputs to the cortex.

The cortex itself is basically a huge area of cortical minicolumns and hypercolumns connected in both a recurrent and hierarchical manner. The different levels of the hierarchy provide higher levels of association and abstraction until you get to the top of the hierarchy which would be areas of the prefrontal cortex.

What's amazing about the cortex is it's just a general computing machine and it's very adaptable. To give an example (I'd link the paper, but I can't seem to find it right now and this is from memory, so my details may be a bit sketchy, but overall the idea is accurate), the optic nerve of a cat was disconnected from the visual cortex at birth and connected to the part of the brain that's normally the auditory cortex. The cat was able to see. It took time and it certainly had vision deficits. But it was able to see, even though the input was going to the completely wrong part of the brain.

This is important for several reasons, but the most important aspect is that the brain is very flexible and very adaptable to inputs. It can learn to use things you plug into it. That means that you very likely don't have to create a very exact replica of a human brain to get human level intelligence. You simply need a fairly model of the hierarchical organization and a good simulation of the computations performed by cortical columns. A lot of study is going into these areas now.

It's not a matter of if. This stuff is right around the corner. I will see the first sentient computer in my lifetime. I have absolutely no doubt about it. Now here's where things get really interesting, though... The first sentient computers will likely run a bit slower than real-time and eventually they'll catch up to real time. But think 10 years after that (and how computing speed continually increases). Imagine a group of 100 brains operating at 100x real time, working together to solve problems for us. Why would they work for us? We control their reward system. They'll do what we want because we're the ones that decide what they "enjoy." So 1 year passes in our life, but for them, 100 years have passed. They could be given the task of designing better, smarter, and faster brains than themselves. In very little time (relatively speaking), the brains that will be

Numenta has a simple app for doing image recognition now: you drag in your images, press the button, and it trains. No tweaking required. They have both a local API and a web API you can use to incorporate the technology into an app. It's free.

Numenta has a free download called the Vision Toolkit. It has a really simple approach to training: drag in your images, then press the Train button. You can use the Python API to build a desktop application, or upload it to the hosting service and use it via the web API.

Their approach has some superficial similarity to perceptrons, but it's a lot more brain-like.

Did you actually try it out yourself? I bought into Numenta's hype and downloaded this program, and it was laughably bad. It "learns" a list of objects and shows it can recognize them, but when I modified the test images just a little bit and fed them to it, it failed miserably. That's especially bad considering how low res the images are and how they're the same size.

Numenta is only different in having a disprportionately large marketing budget. (Yes, I took the plunge and read On Intelligence.)

If you want to see a neural network (-like) implementation that actually works well, you should look at Numenta's NuPic application and their vision toolkit. It works pretty amazingly well. They also have a nice architecture, in that the core is c++ but they use python for UI, file IO, and other utility stuff. It's under a 'research license' which means that you are allowed to do research and play with it, but if you want to sell a product with their technology, you need to get a real license.

I wonder if it's based on Numenta's Bayesian HTM (hierarchical temporal memory). My understanding of neuro-like learning system is that, unless its knowledge base is organized hierachically like a tree, it could not possibly do the things its promoters are claiming for it.

The industry is stablizing for those that are general programmers.

Oh, is it? I missed that memo.

Please tell me ONE major advance in whatever field you work in.

It doesn't really work like that. Very rarely in science is there some major advance that you can specifically point to. Everything we had back then in AI we still have, but it is so much better. The neuroscience side of things is progressing, we are getting better data about the brain. Developing practical applications based on that. Voice recognition is pretty good now, automatic translation is better, computational vision is better, autonomous robots are better. And frankly, it's rather difficult to put theories on artificial intelligence when the computational capability isn't yet there, so things are getting better as computation gets faster.

What would satisfy you as a major advance in the field? The problem with AI techniques is that they very quickly leave the field of AI, or they don't yet have any practical applications. Juergen Schmidhuber's work on Recurrent Neural Nets is very impressive and he's a good name to watch for the future of AI.

Duh! The on screen child detector would highlight the child and temporarily pause the on screen porn, direct a safe detour, and then automatically re-engage the porn. This is not science fiction. Object recognition technology is on the way there. See Vision4 .

Hierarchical temporal memory is a theory you might be interested in... See http://www.numenta.com/ or read the book http://www.amazon.com/Intelligence-Jeff-Hawkins/dp/0805078533/ref=pd_bbs_1?ie=UTF8&s=books&qid=1214300344&sr=8-1

numeta

It's mainly a teaching + learning system for a system with input and output. I don't see anything built with it answering any rational questions or coming up with new ideas anytime soon, but if you do AI and don't know about them, you better catch up.

This has been a long time in the coming and has been bugging the hell out of me. This is where i see a lot of the "Community Contributions" involving Jeff Hawkin's recent endeavors. If you take a look at some of the details of his models, the fact that DARPA and Lockheed/Martin have taken an interest in his work, and his recent projects things start to look scary.

It is easy to envision the possible uses for his recent mundane technologies". Itinerary analysis and keyword triggered speech recognition and recording? The former has obvious uses and the latter would remove a metric shitload of overhead from surveillance storage and analysis.

My tinfoil hat allergy can only say correlation != causation so many times before the system spazzes right out.

http://www.numenta.com/

Dude, you have got to put down the Matrix and Terminator. Take some time off and go read about the current state of AI design. The real world is very much removed from the fantasy you have concocted within your brain Mr. Anonymous Coward.

Here is a good place to start: http://www.numenta.com/

Any comments from people with expertise in this area?

Yes; his reasoning is laid out in the beginning of this document. The thinking seems quite reasonable to me, as far as it goes. AI is my area of research.

http://www.numenta.com/for-developers/education/HT M_Comparison.pdf

"
The purpose of this document is to compare HTMs with several existing technologies for modeling
data. HTMs use a unique combination of the following ideas:

* A hierarchy in space and time to share and transfer learning
* Slowness of time, which, combined with the hierarchy, enables efficient learning of intermediate levels of the hierarchy
* Learning of causes by using time continuity and actions
* Models of attention and specific memories
* A probabilistic model specified in terms of relations between a hierarchy of causes
* Belief Propagation in the hierarchy to use temporal and spatial context for inference

Many of these ideas existed before HTMs and have been part of some of the models we describe below. The power of HTM comes from a unique synthesis of these ideas. "

I'm still a bit confused as to how he is so confident that this is how the neocortex works given that this is still one of the 23 unsolved problems in system neuroscience. But hey, he made a lot of money off Palm, that gives him way more street cred than people who have been working on this problem for their whole lives.

The idea is that a computer is intelligent if it can hold a conversation with a human such that it is indistinguishable from a conversation with a real human.

RIDICULOUS.

Have you ever actually tried talking to one of these bots (including ALICE)? It is very easy to know that you're not talking to a human. Exceptionally easy. The Loebner Prize judges consistently grant the bots handicaps, acting as if they're actually being fooled. Obviously they're not, and the AI community just wants people to think that it's more advanced than it really is. Unfortnately, some members of the public *are* fooled by that.

The problem is in the Turing Test itself. It assumes that the measure of intelligence is humanoid conversational ability. I strongly disagree with that. Conversation ability is no measure of intelligence. Just for an example, I am exceptionally intelligent (statistically), but I am a poor conversationalist. Casual small-talk has always bewildered me. If I entered myself into the Loebner contest, they might think I'm a bot. Hell, ALICE might accuse me of being a bot.

Anyone who's taken an IQ test will recall that every last question has something to do with pattern recognition. You'll also recall that you were not asked to respond to any conversational questions. That's because invariant pattern recognition abilities (in a loose sense -- this also includes memory/learning and inductive reasoning) are the true mark of intelligence, and this is nearly undisputed. If they really want to test how intelligent a program is, they need to test its patern recognition ability.

Take this program -- http://www.stanford.edu/~dil/invariance/ -- for example. It's gone largely unnoticed, yet it is concrete proof of a huge breakthrough in computer intelligence. This is a little Matlab demo of a very abstract multi-layer intelligence algorithm. In this particular implementation, it is taught a set of small images. Then you can play "Pictionary" with it, drawing shapes and have it recognize them. You may say that this is unremarkable, that shape-recognition is a trivial algorithmic matter unrelated to intelligence. But the author noticed that he could draw shapes "incorrectly" -- like, the little duck picture, except with its head missing, or alphabetical symbols rotated or flipped -- and the program still recognized them. (It failed a few times, but in situations where the shape is so mangled that I would have probably failed too. How's that for a Turing test?) And this program's genius lies in not what it does, but how it does it. All of its functionality is completely abstract. It is a pattern recognizer, not a bitmap-tracer, and there are no hard-coded routines for checking if the image is flipped, rotated, etc.

This is what Palm/Handspring founder Jeff Hawkins (also the founder of new neuroscience startup Numenta, http://www.numenta.com/) calls "Real Intelligence," to distinguish it from the failed Artificial Intelligence effort. He feels that the right way to make computers intelligent is not to have them outwardly imitate human behavior, but to internally function the way the mind really works. Anyone interested should check out his book, On Intelligence http://www.onintelligence.org/. You'll wonder why you ever believed the AI hype.

Artificial Intelligence is a sham, by its very nature. Real Intelligence will be the way of the future.

An excellent complementary reading would be On Intelligence, by Jeff Hawkins, the founder of Palm, Handspring, and most recently, Numenta. It specifically explores the workings of intelligence and memory. (Don't let the rather uninspired title deter you. It was actually a very fascinating read, and very easy to understand -- full of metaphors and examples.)

I haven't read "Mapping the Mind," but it sounds like an equally good read.

There are actually quite a few projects now taking similar, cortex-centric approaches to AI hard problems. Are we up to something here? The guys responsible of these projects are not wacko types at all, but established entrepreneurs and/or well-known researchers:

CCortex "A 20-billion neuron simulation of the Human Cortex and peripheral systems."
Cyc a knowledge base with vast collection of facts about the real world and logical reasoning ability. Financed by Paul Allen AI related investment company,Vulcan.
Numenta is developing a new type of computer memory system modeled after the human neocortex.

They seem to we well financed, and knowledgeable. Are we witnessing the start of something big here?

None of the founders of Numenta other than Jeff Hawkins have any experience in AI or for that matter have any background in hardcore computer science.

Dileep George is an Electrical Engineering graduate, while the CEO Donna Dubinsky is a hardcore salesperson and holds an MBA. Interestingly, the page also mentions that Jeff Hawkins " currently serves as Chief Technology Officer at palmOne, Inc". Fishy!

Next Generation AI ? Who are we kidding ?

None of the founders of Numenta other than Jeff Hawkins have any experience in AI or for that matter have any background in hardcore computer science.

Dileep George is an Electrical Engineering graduate, while the CEO Donna Dubinsky is a hardcore salesperson and holds an MBA. Interestingly, the page also mentions that Jeff Hawkins " currently serves as Chief Technology Officer at palmOne, Inc". Fishy!

Next Generation AI ? Who are we kidding ?