Slashdot Mirror
Artificial Intelligence Pioneer Says We Need To Start Over (axios.com)
Steve LeVine, writing for Axios: In 1986, Geoffrey Hinton co-authored a paper that, four decades later, is central to the explosion of artificial intelligence. But Hinton says his breakthrough method should be dispensed with, and a new path to AI found. Speaking with Axios on the sidelines of an AI conference in Toronto on Wednesday, Hinton, a professor emeritus at the University of Toronto and a Google researcher, said he is now "deeply suspicious" of back-propagation, the workhorse method that underlies most of the advances we are seeing in the AI field today, including the capacity to sort through photos and talk to Siri. "My view is throw it all away and start again," he said. Other scientists at the conference said back-propagation still has a core role in AI's future. But Hinton said that, to push materially ahead, entirely new methods will probably have to be invented. "Max Planck said, 'Science progresses one funeral at a time.' The future depends on some graduate student who is deeply suspicious of everything I have said."
Expert systems aren't AI, and pattern-matching algorithms aren't AI. AI is something that can creatively solve problems based on unreliable inputs and abstracting specific experience to general cases.
The problem there is we don't even understand how that works in theory, so modeling and developing an actually AI based on that model is impressively difficult.
Personally, I think we'll get there (understanding intelligence) faster by trying to replicate a mammalian brain in silicon that we will trying to bash out new algorithms.
Also however important back-propagation is, it is hardly the entire foundation of AI. From my perspective AI is proceeding apace. There are many AI methods. Yes some core algorithms should be reexamined, as should anything in science or industry. We see some stuff that seems to lag in how much improvement we expected (general intelligence), and yet others that are leaping ahead of where we thought they would be like machine learning and pattern recognition. Eventually all the threads will start to come together, but progress will remain hard to predict.
Letter To Iran
The future of AI is a dirty word "stereotyping".
The brain works by making associations, and then drawing stereotypes from them. Every time I've seen a dog or hooded man in a dark alley, it has attacked me. I stereotype dogs and hooded men in dark alleys as being scary and run from them. But then one day, I meet a green hooded man with a bow in the alley, and he saves me from the dog. I have to 'learn' by reshaping my stereotype to include men in green hoods.
Stereotypes get a bad name due to people the refuse to update or rely on bad ones, but they are actually a very useful tool humans have developed to deal with the world.
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
>Likely he is not right either, because AI beyond statistical classification ("weak AI") may well be impossible
Nature did it with meat. Meat is not special. We have to learn how to replicate the mechanisms - which involves first understanding the mechanisms. Both of those are daunting tasks, but not fundamentally impossible.
If you think they are, then you must believe intelligence is a product of a supernatural process, and your theories are not appropriate for a science-based discussion site.
Backpropogation is a form of supervised heuristic learning where you have to know the desired output and so it works backwards. In that context it's about perfect. We don't have any algorithmic techniques in an unsupervised learning context that are as good. Expectation maximization and blind signal separation algorithms all generally suck balls. The goal is unsupervised learning that works as efficiently as backpropogation. I suspect this is what he is saying but since this article and his language aren't technical or sufficiently detailed that's guess work on my part. For what it does there is nothing about backpropogation that needs improving, but it only works in specific use cases and it is not AI. It's just a "gradient descent optimization algorithm" (had to google the formal name for what it is).
Algorithm development at this point is just obscenely hard and is going to be obscenely rarely seen. The easy stuff has all been done and the stuff that is still being worked on may quite literally be impossible.
AI is a joke. There has been no real progress in AI since the 60s. What you see now is parlor tricks and a byproduct of Moores Law. Now that Moores Law is over, we need to find some other way to do computing. We will never have AI with digital computing.
The AI community needs to be much more cautious and circumspect. They have been promising the sky, and otherwise hyping things, for decades now, and, as result, they have become something of a laughing stock in academic circles. And do not say it is the press - luminaries like Minsky and others couldn't wait to come out with ever more outlandish forecasts, that were then just disseminated by the press. The final straw is when these days they are still trying to sell ridiculous gimmicks like Alexa, Google Home, Siri, etc. as AI wonders, when the only wonderful thing is how notably inept and limited in their capabilities they are: good for grins and giggles, party games, but little more. Stop the nonsense.
No, its not impossible, but it requires vastly better hardware that actually mimic or model processes we see in neurons and not the simple stuff we can do with silicon today. We are at the level where we can maybe in a decade and with specialized hardware build a roach brain in silicon. We are nowhere close to a human or even a cat brain. The reality is we are more hardware limited than software limited.
Neural nets using back propagation will likely remain a valuable tool forever, just like Newtonian mechanics. Will they be the only go to solution for all similar AI learning going forward? Of course not, they already aren't. When we do achieve strong AI it will likely be from a system incorporating thousands of different algorithms, of which Dr. Hinton's contributions will be just one.
>stop pretending your anti-science quasi-religious fundamentalist beliefs are science
Project much? What the hell is wrong with you? You're the one making supernatural claims, not I.
>"Nature did it with meat" has no scientific basis.
Wow. So the fact that we can observe evolution and our fellow humans, make predictions, test them... 'no scientific basis'.
> All Science has is interface observations. And even a child these days knows that what you can observe on the outside of a box is not necessarily created on the inside.
Right back to YOUR belief of 'magic inside'.
I shall quote you back to yourself: "Seriously, stop pretending your anti-science quasi-religious fundamentalist beliefs are science. They are not."
If you think that, then you have no clue what the limits on software complexity that can still be handled are. Sure, we are hardware-limited and we will be that for the foreseeable future. But the little overlooked fact here is that we have no clue what the software actually should do in order to simulate a brain, so even if we had the hardware, we would not be any closer to the result.
Also, why assume that just scaling the thing up makes it suddenly be intelligent? That is a baseless assumption as that has never been experimentally verified and there is no theory that has been verified and could be applied either.
At this time, the workings of intelligence, consciousness and free will are "magic", i.e. nobody has a clue how they work. Assuming a purely physical apparatus could attain all these is neither supported by our current understanding of Physics nor does it have any scientific base. It is a belief. And, as it turns out, the follower of this belief ("physicalists") use pretty much the same faulty argumentation techniques so common with religious fanatics. A pathetic fail on their part.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
>The reality is we are more hardware limited than software limited.
Well, I'm not sure it's fair to call it 'software' anyway. It's more like 'firmware', in that the organization of the hardware is the basic 'OS'. And there may be some process going on in a brain that is so much more efficient than attempting to model it in a computer that it's effectively beyond us until we do manage to mimic a biological brain in hardware.
A set of known unknowns?
Well, "God" is a transparent pseudo-explanation for those weak of mind, but the physicalists (fundamentalists that believe everything is just matter and energy) are not much better. Both use belief-based strategies of dealing with the unknown and both are anti-science.
When it comes to consciousness, intelligence and free will, the scientific state-of-the art is "nobody has a clue". Anybody actually thinking scientifically is able to live with that, but that approach is beyond a great many people. Hence they invent stupid pseudo-explanations.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
we may be into the fourth decade since 1986 but it's been 31 years since not 40+.
> Assuming a purely physical apparatus could attain all these is neither supported by our current understanding of Physics nor does it have any scientific base. It is a belief. And, as it turns out, the follower of this belief ("physicalists") use pretty much the same faulty argumentation techniques so common with religious fanatics.
There you go again - the third time in this discussion by my rough count. You deride the idea that physical processes could create intelligence as a product of the faith of religious fanatics. This universe runs on the laws of physics. Claiming anything else is... the product of the faith of a religious fanatic.
If you don't believe in physical laws, you should be having this discussion with your preferred religious authority, and not with us here on Slashdot.
>Using silicone semi brute strength to emulate "meat" may be infeasible as we are rapidly reaching silicone's physical limit.
Have a look into memristors, a new toy that could be very useful for making artificial brains.
Then consider that nature 'figured out' how to be more efficient by using more switches with lower thresholds and taking the average, while we tend to juice transistors to ensure a strong '1' or '0'.
And finally... silicon. Silicone is not particularly useful in computers except as a sealant and sometime adhesive or vibration damper. :)
Consider a nueron will not fire 1 time in 10. To simulate forgetting.
The future may be table-oriented AI (TOAI).
It uses tools and/or conventions more typical of a regular office and thus allows AI problems to be split up and analyzed in a modular team-oriented fashion. Tables are easier to relate to than traditional neural nets (without a lot of training, at least). TOAI allows compartmentalizing AI tasks to distribute to staff (tasks, sub-tasks, etc.), and encourages a kit-oriented approach (modularization).
For example, you may have 3 sub-teams: 1) pattern/test makers, 2) results examiners, and 3) coordinators.
The first group focuses on making the individual tests (patterns or "factors"), the second group focuses on determining which tests are most or least useful for given situations, and the 3rd group be the coordinators who split up the problems and/or AI decision flow between groups and/or template sets (filters).
Table-ized A.I.
Deep learning and other related machine learning techniques are proving very useful for a wide range of tasks. We don't need to "start over" to advance useful machine learning techniques.
Hinton seems to mean to get "strong AI". Yes, I read TFA, but the strength of Axios articles is that they are very short, but that is also their weakness. Very little is actually said in TFA.
We are a long, long way from anything that emulates a natural neural system at any level.
Consider Caenorhabditis elegans. Every cell in this simple worm has been mapped, also the development of every cell from a single cell has been mapped (male worms have 1031 cells). We know every cell in its nervous system (there are 302), and every cell that each cell is connected to, and we know the type of connections for all. What's more we have completely sequenced its genome. We know more about this little multi-cell organism than any other multi-cell animal on the planet.
Since we know every cell in its nervous system, and every connection between every cell, we must be able to emulate this worm's "brain"! Heck we must be able to "upload" the worm's brain to a computer! Right? Right?
No.
We are still working on understanding the functioning and capabilities of a single neuron in its brain. That has proven so complex as to defy characterization thus far. We are essentially nowhere in understanding how this 302 cell brain works despite decades of effort.
Meanwhile Kurzweil has changed his prediction of "when computers will have human-level intelligence" from 2020 to 2029. I guess believing it was going to happen in the next 26 and a half months was cutting it a little too close. I have been reading about his predictions about AI for a couple of decades now and have yet to see any explanation of how he imagines this is going to happen - other than his expectations about hardware capabilities, and that there is still an unspecified "software issue" that needs to be solved. Indeed.
Second class citizen of the New Gilded Age
Do you have evidence to suggest that there is something other than what we presently observe in the universe? If not then you not being very scientific.
I'm not ready for 2026 yet!
Downmodding is the refuge of the weak. Don't downmod, make a better argument!
And there you do exactly what is _not_ done in Science. In Science, a question remains _open_ until there is evidence to close it. You are doing the opposite thing and that is pure belief and has nothing at all to do with Science. Fail.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
When it comes to consciousness, intelligence and free will, the scientific state-of-the art is "nobody has a clue". Anybody actually thinking scientifically is able to live with that, but that approach is beyond a great many people. Hence they invent stupid pseudo-explanations.
Accepting that "nobody has a clue" is not scientific at all because the basis of science is questioning in a systematic manner. That some or many people believe in religion, philosophy, etc. is a direct result of the utter failure of the scientific method to produce answers to how and why the world and life exists. Many of these questions fundamentally deal with past events, and many scientific textbook statements cannot be tested by strong scientific methods, leading to dogmatic belief under the nomenclature of religion or science. How is science sometimes dogma? As an engineer, I know enough to ask about confidence in assertions, but the language of science textbooks overwhelmingly dogmatically asserts the textbook view of truth as simply true without uncertainty, i.e., dogma.
Back propagation is used setup a translation between features and results with the least cost. The problem is some features have more importance than other features, this is where the optimization in the learning process can be done. During the learning process if features that have more importance are given higher importance then the learning will be faster and require less resources. This is where cluster analysis comes in, by optimizing the clusters to achieve the desired results self learning can be achieved.
Just imagine what the human mind's distributed representation of the "intelligence" concept would look like. Clever animate entities (and most associations therewith) are way off in their own private corner of vector space compared to just about everything else.
When the gap is this large, the enormous void in between somehow becomes a non-object (to superficial cognition) and so people just begin to presume that we need to jump the gap, rather than slowly filling the gap in.
It's almost like the travelling moon illusion when you're driving in a car and the moon is low in the sky, off to the side (which children find amazing, but adults have learned to ignore).
I was thinking about the sun this morning and about relative illumination at different latitudes. The correct physical model is parallel rays, which immediately suggests that for a perfect sphere, the poles get no direct radiation at all during equinox, the eternal kiss of sunrise=sunset.
Then I looked outside through the window, and realized that the human brain—which knows the sun is far away—still doesn't think it's as far away as the earth is wide (very wide, if you believe in a flat earth model) or even a few multiples (but it's actually thousands), and so the intuition from our eyes never says parallel rays.
We've been nibbling away at the giant AI void quite successfully, but the travelling moon illusion still makes us think we need to jump.
The reason we keep reclassifying our victories as "not really AI" is because we know for a moon fact that the void never actually changes size. But it does, and it has, and it will continue to shrink, and I really don't think we're going to spring generalized intelligence all at once out of scary clown box.
First we must learn to perceive the void as a continuum of many way points, mapped out by many generations of technical improvement, like Vancouver and Cook or Lewis and Clark.
For me, recent results with LSTMs have made the void seem just a little bit smaller than it was before. I'm now at the very beginning of an ability to perceive the moon as being at a great, yet finite distance.
_____
With something so thoroughly hived off in its own corner of distributed-representation hyperspace as intelligence, what's to define, anyway? Definitions are street signs erected in conurbia, which one resorts to after Toronto and Hamilton and Niagara Falls have all become built-structure indistinguishable as you skirt the horseshoe.
There are many conurbations in distributed-representation vector space where definitions are the last gasp at forestalling cognitive Gangs of New York. Definitions are less important under open skies of Boise, Idaho or Butte, Montana; even less important still when you've wandered out into the green grid-lines of the entirely unpainted Matrix.
Here's a quick test: if your frontier town's "population #,### sign" (there is only one sign) and it has at most one comma, definitions are premature.
Sure it's an open question, there could be more than just matter and energy (I never said otherwise). But, claiming or implying that there is something else without evidence is not science. Being highly skeptical of such claims (as I am) is very scientific since it doesn't fit with what we observe. So, either make an argument or provide evidence that something else exists (other than what is currently unknown to modern physics), or only gullible people will take you seriously.
Furthermore, "do dragons exist?" is an open question too and no amount of evidence will ever "close" it.
And bats don't fly by flappingbtheir wings.
Because I Solved Diffie-Helman Exchange For Catalytic Conversion: https://pastebin.com/ZVvLYYiV
moo
Nature did it with meat. Meat is not special. We have to learn how to replicate the mechanisms - which involves first understanding the mechanisms. Both of those are daunting tasks, but not fundamentally impossible.
What is the basis of your statement "meat is not special"? I mean regards to intelligence? Maybe meat is fundamentally special when it comes to producing high-level intelligence?
I'm not implying any supernatural mechanisms here. Just that what "meat" does may not be reproducible in silicon. Has anyone built a computer that grows a destroyed circuit back? Meat is pretty special. It regenerates. It reproduces. It learns. It evolves. What else on Earth does that?
Perhaps the only way to build artificial (human/animal-level) intelligence is to build an artificial biological brain. Maybe computers are a total dead-end approach, and we should be pursuing synthetic biology instead. I mean, I don't know, but there is no basis on which to declare that whatever is done in meat can be replicated using some other substance.
Then consider that nature 'figured out' how to be more efficient by using more switches with lower thresholds and taking the average, while we tend to juice transistors to ensure a strong '1' or '0'.
More importantly nature does not impose any synchronization or avoid feedbacks. Meanwhile most of A.I. work is done in an extremely, pedantically so, synchronized feedback-less framework.
And good thing... these systems wont collectively decide to kill all humans until there is at least some internal feedback.
"His name was James Damore."
Hebb showed us the way forward right from the start, yet we still managed to get stuck with backpropagation and perceptrons time and fucking time again.
"The future depends on some graduate student who is deeply suspicious of everything I have said."
...he will be an aggressively creative male. Oh, wait a minute, he couldn't get a seat. Well, never mind.
E Proelio Veritas.
But, as Vernor Vinge pointed out in one of his stories (True Names - 1981), who says it needs to run in real time?
Maybe we're aiming to high right now. We want to simulate what we're capable of doing, at the same speed that we can do it. Why?
We talk about mapping the neurons in a worm, and replacing the worms brain with silicon to see if it can still act like a worm. Simulate the rest of the darn worm, and it's environment, and see what happens instead.
If it takes weeks or months of processing to give a second or two of worm thought, why would that be a bad thing? Processors and memory will continue to improve, and we'll still have learned something interesting.
Fascinating. The dumbing-down is in full swing when on gets moderated down to -1, Troll for pointing out the scientific state-of-the-art.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Claiming something is "obvious" and hence must be true is not Science. It is wishful thinking. Care to prove your assertion? Oh, right, you cannot.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
You are kidding yourself. You have closed this question with a pseudo-answer. Very likely you are one of the many people that cannot stand an open question. Incidentally, listing options is completely scientific, even if you do not like them.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
And there you have demonstrated again that you do not understand Science at all. Because you just predicted that Dragons do not exist, and you have done so without a shred of proof. Pathetic.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
You are a moron. What you do is circular reasoning. And you do not even recognize that. Incidentally, this level of reasoning is about as sophisticated as what the religious fuckups do.
Also, your last sentence gives you away nicely: The laws of Physics are not something to "believe" in. They are something to verify. And they are incomplete at this time, as anybody that cared to find out knows. You obviously did not.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I suspect that we're just not smart enough to design a machine as smart as we are, and we never will be.
A neural network is, at the end of the day, glorified curve-fitting. But it beat all the other methods (and there are many, with very diverse underlying math) at a few tasks, like machine translation, image and voice recognition, etc. It does well even for robot object manipulation apparently, a totally different application. But it looks likely that it is way too simple a model to solve intelligence. It could still be useful, perhaps in combination with some other methods or models. For example, AlphaGo uses Monte Carlo tree search in combination with neural networks. Maybe the final architecture will be several low-level modules based on neural networks, dealing with things we find hard to express explicitly, eg, vision, voice, locomotion, pattern recognition, and on top of that higher level models that deal with the world at the level of concepts, also models for reasoning and inference, massive databases of real-world knowledge, and extensive training over decades using reinforcement learning and many human teachers in parallel.
So what happened to his own theory about how backprop works in the spike-train networks of the human brain?
They have been investigating other methods though, like Synthetic Gradients https://iamtrask.github.io/201...
It's been said -- 90% seriously -- that AI is computers doing things that require human intelligence, that computers can't do yet.
Once computers can do it, it's "pattern recognition" or "heuristic blah-blah-blah" or whatever.
The recently developed specific ability or method has a name, so it's no longer considered to be "artificial intelligence" when a computer does it.
There's no time like the present. Well, the past used to be.
The AI community needs to be much more cautious and circumspect.
It's not the fault of the AI community. They are always very cautious about the claims they make.
The misconceptions about AI on the part of the laity is all down to the PR and marketing peeps making claims about things they know nothing about.
My experience with G. Hinton, is that he will refute anyone who conflicts with his ideas, not applying a logical/mathematical argument, but by calling them "a CRACKPOT".
Soo,... looks like he's now calling himself a CRACKPOT !
By the way, wonder how those large corporations (Google, Nvidia, etc..) that invested 100s of $Millions into his "Deep Learning" brand of NN's, feel about Hinton's sudden change of heart.
Hope they are A-OK with an investment, where the inventor is now publicly claiming that "it should all be thrown away"... HA !