Slashdot Mirror
Why AI Won't Take Over The Earth (ssrn.com)
Law professor Ryan Calo -- sometimes called a robot-law scholar -- hosted the first White House workshop on AI policy, and has organized AI workshops for the National Science Foundation (as well as the Department of Homeland Security and the National Academy of Sciences). Now an anonymous reader shares a new 30-page essay where Calo "explains what policymakers should be worried about with respect to artificial intelligence. Includes a takedown of doomsayers like Musk and Gates." Professor Calo summarizes his sense of the current consensus on many issues, including the dangers of an existential threat from superintelligent AI:
Claims of a pending AI apocalypse come almost exclusively from the ranks of individuals such as Musk, Hawking, and Bostrom who possess no formal training in the field... A number of prominent voices in artificial intelligence have convincingly challenged Superintelligence's thesis along several lines. First, they argue that there is simply no path toward machine intelligence that rivals our own across all contexts or domains... even if we were able eventually to create a superintelligence, there is no reason to believe it would be bent on world domination, unless this were for some reason programmed into the system. As Yann LeCun, deep learning pioneer and head of AI at Facebook colorfully puts it, computers don't have testosterone.... At best, investment in the study of AI's existential threat diverts millions of dollars (and billions of neurons) away from research on serious questions... "The problem is not that artificial intelligence will get too smart and take over the world," computer scientist Pedro Domingos writes, "the problem is that it's too stupid and already has."
A footnote also finds a paradox in the arguments of Nick Bostrom, who has warned of that dangers superintelligent AI -- but also of the possibility that we're living in a computer simulation. "If AI kills everyone in the future, then we cannot be living in a computer simulation created by our decedents. And if we are living in a computer simulation created by our decedents, then AI didn't kill everyone. I think it a fair deduction that Professor Bostrom is wrong about something."
Claims of a pending AI apocalypse come almost exclusively from the ranks of individuals such as Musk, Hawking, and Bostrom who possess no formal training in the field... A number of prominent voices in artificial intelligence have convincingly challenged Superintelligence's thesis along several lines. First, they argue that there is simply no path toward machine intelligence that rivals our own across all contexts or domains... even if we were able eventually to create a superintelligence, there is no reason to believe it would be bent on world domination, unless this were for some reason programmed into the system. As Yann LeCun, deep learning pioneer and head of AI at Facebook colorfully puts it, computers don't have testosterone.... At best, investment in the study of AI's existential threat diverts millions of dollars (and billions of neurons) away from research on serious questions... "The problem is not that artificial intelligence will get too smart and take over the world," computer scientist Pedro Domingos writes, "the problem is that it's too stupid and already has."
A footnote also finds a paradox in the arguments of Nick Bostrom, who has warned of that dangers superintelligent AI -- but also of the possibility that we're living in a computer simulation. "If AI kills everyone in the future, then we cannot be living in a computer simulation created by our decedents. And if we are living in a computer simulation created by our decedents, then AI didn't kill everyone. I think it a fair deduction that Professor Bostrom is wrong about something."
none
Government is too big, complex and uncontrollable to do anything regulatory and so we need to put our faith in Jebus and just let big profit-driven corporations look out for the best interests of humanity.
According to Republicans, this works perfectly and has no downsides, and the only alternative is Stasi Totalitarianism that "kills jerbs" for mining slav... er, coal miners... etc...
Should be "How much should AI regulate government?"
Every “we are living in a sim” argument requires that the future has already happened. IE some futuristic society has AI and we are living in it.
Assuming now isn’t the future then this is base reality because simulations indistinguishable from reality do not exist yet. Without offering evidence we are in the past, the sim argument is nonsense.
Businesses wanting to make profit will do so at all costs.
There are too many people out there who think that if it's not illegal then it's OK. Computers don't have testosterone but the programmers and their bosses do - or at least the profit incentive.
We are intelligent but our base programming is to reproduce. And being primates, the more dominance we have, the more fucking opportunities we have; which in our modern times means getting as rich as we possibly can.
Meaning, our base instincts will make it into our AIs and we WILL find ourselves being dominated.
That's the arrogance of technologists: they think they are more rational and logical than everyone else and that makes them even more susceptible to human nature.
AI is a computer program, so it's not obvious to me how you're even supposed to regulate it, anyway. Doesn't anyone remember DeCSS and PGP? How did regulating those programs work out? Why would AI be any different?
But, it's one of the few things that could actually kill us all.
This wouldn't even have to be intentional extermination, it could simply be competition with, and lack of regard for humans by a growing system.
The notion that a AI can form an existential threat today is ridiculous.
Many notions that would have been ridiculous 100 years ago, now are used in daily life.
It is vital to have people thinking about the worst case, because in principle otherwise someone on a friday makes a typo allowing their AI access to a hundred thousand times the expected resources, and on monday, it's ineradicable.
The more authority or ability an AI has the more it needs to be checked on, since the more harm it could cause if it went wrong. Some people don't do that if you don't make them, it only takes a few in the wrong place to do great harm. Also it would probably be good to consider the moral consequences of self aware AI. If you go too in the wrong directions sooner or later you run into areas currently regulated by animal cruelty regulations or even human reproduction technique regulations.
Cryptography is a computer program and it's regulated too. DeCSS was a joke and doesn't count as real cryptography.
Use of certain types of software by business within U.S. jurisdiction is certainly something that can be regulated. It makes it difficult to buy or sell A.I. enabled devices if it is not legal in your country. You can still download and install into your own equipment, but currently it is looking like a general purpose computer is terribly slow at A.I. and you'd need some specialized hardware to make practical use of the software. That specialized hardware could easily require registration with a government agency, and restrictions on access.
If you've ever worked in biochemistry, you'd know there are lots of restrictions on the chemicals you can purchase in the catalogs. But you do need some of them to calibrate test equipment, so with a big fat pile of paperwork you can still get them if your lab is fully licensed.
To determine the answer to this question, I went straight to the source: I asked an AI. What follows is a transcript of our discussion:
> Hello, I am Eliza.
* should the government regulate AI?
> Don't you ever say Hello?
* Hello.
> How are you today.. What would you like to discuss?
* should the government regulate AI?
> I see.
So you see the answer is as plain as the nose on your face.
If I destroy it first. Try ruling the planet under 10 meters of seawater!
“Common sense is not so common.” — Voltaire
APK you are getting nuttier and nuttier bro, time for your bath in pickle brine.
"We all vote there is no problem because it has not been proven there is a problem yet. Therefore your concerns are invalid because Science."
If my hedge fund(mythical) filled with people with zero ethics get their hands on a AI that will allow them to manipulate world markets, media, or world events to make them money, then they will do so.
With hindsight there are lots of places where the world turned out to me much more fragile than anyone thought until it snapped. How many times has the snap not happened but we came very close. Thus if you have a good AI at your beck and call to find these weaknesses and you are prepared to exploit them to make some money then how much more miserable would the world be?
I don't only worry about some skynet scenario, but I worry about giving tools to nitwits like hedge fund managers to make more money while not actually producing anything. One magical thing about making money with the first really good moneymaking AI is that you can then start hiring all the world's AI experts while making massive donations to universities to shut down their AI research. I doubt there is a university that wouldn't happily shut down their AI research for a billion or two.
There is little reason to think people like Elon Musk should be seriously listened to. The ability to get the popular media's attention does not reflect wisdom, although it may sometimes reflect creative genius. The real threat to humanity is intellectual arrogance, whether artificial or natural.
... that Jews already control it and the "AI" that will be created, or pretend-created.
Because we're still no closer to actually creating an AI.
So a law professor whose primary gift seems to be self promotion summarily dismisses the concerns of some of the greatest thinkers/doers of the last half century.
Is there a reason why we should pay any attention to this arrogant twat?
I've calculated my velocity with such exquisite precision that I have no idea where I am.
Long shlong silver!
This guy claiming authority on a subject. The only thing that matters is empirical science.
And only perpetuated by big budget Hollywood. And Bender
At least the Japanese had fembots and tentacles.
Rampant AI will not take over the earth, as predicted by Elon Musk in Wired Magazine, because it will be too busy fighting the Grey Goo Nanothechnology, as predicted by Bill Joy in Wired Magazine
*** Suerte a todos y Feliz dia!
His argument against Musk, Hawking's, Gates is that they have no formal AI training.
The professional class self-justifies their authority by dismissing all who don't have the credentials they had to pay for.
P.S. Einstein did not have a degree in physics, thus relativity is invalid.
The real worry is a hostile AI doing exactly what it's been programmed to do - though we'll certainly be told publicly that it's just a bug - and committing various atrocities for whomever had it built to do just that.
And people who'd happily have this done to an 'uppity' population are too numerous to count.
"There is no reason to believe it would be bent on world domination, unless this were for some reason programmed into the system." - see the thing is as soon as there is a path to Super Intelligent AI the first thing that will happen is the Pentagon or the Ruskies will bend it to their will and turn it into a weapon. That's the whole reason for folks stating it's an existential threat, because people are dumb and will bend it to their dumb purposes,
If the simulation is successful and sufficiently advanced, then we would all actually be AI simulacra of what the AI dev team thought the human experience was like. [Why do so many things taste like chicken?]
Perhaps we are special purpose AI entities that were created to run test scenarios that justify the pre-emptive judgement to extinguish the pestilence that was humanity. We are the test runs that show just how bad it could have gotten had they not saved the planet from us.
Given a sufficiently advanced environment, we wouldn't be to discern otherwise. Perhaps the supercomputing power that is required and was discovered by the 'real' humans required a very specific mass to a sub atomic particle. In our recreation, we can get very close but will lack the precision to be able to detect our cage, or at least construct an AI that could build a method for detecting the cage.
The first generation (and likely last created by "us") of true AI we create will be a copy of ourselves.
Because our brain works best in a physical body with eating, sleeping, and other needs and we won't understand enough to unravel all of the dependencies, the first one that actually works will be "raised" in a virtual body that goes through all of the normal life stages in a virtual world that recreates everything that contributes to the normal development of a human mind.
The only way we become extinct is if we don't equip it to succeed. Assuming it can reproduce itself, maintain and expand its environment, use our technologies to send itself off planet, etc. we will go on because it is us. It is a stage of evolution, not extinction.
Isn't that a sexist statement? It implies women are less likely to want to dominate and rule. It fits in with that "Google Memo" that got that dude fired.
Table-ized A.I.
Professionals in any field are immersed in the problems they face now. System engineers look across fields and see leaps the professionals never imagined. We will eventually see a leap in AI and it is unlikely to come from a professional in the field. I imagine it will come from someone in an imaging field who figures out how to quickly map a brain or a mathematical field who figures out how to fill in the blanks of a map with an equivalent of the net that "must" be there or some other direction we haven't thought of.
Once the technology is there to build a device with the computational capacity and complexity of a late fetal brain, someone will figure out how to move a human fetal equivalent into it and start raising it. Period.
AI, and us, and every living species on this planet, all act within thresholds. This is the key to AI as well as understanding any living organism. Threshold management defines life. Right now all "AI" is rudimentary threshold management, but it won't be long until AI will be indistinguishable from real life. All it needs is some tiny random noise and the illusion is complete.
It is quite possible that we might create a super intelligent system of network on which our essential system depends but in the end gets so complex that it depends on few key individuals ability to fix it. What happens if these key individuals die or become rogue? If you can't fix an AI system and can't shut it down, then it essentially means that the system has taken over.
This assumes that this is the only way to AI.
And that emergent AI is utterly impossible.
It doesn't have to be perfect at first, it just has to be fast, with the ability to self-modify.
The risk is not (in my opinion) so much someone intending to create a general AI.
It's someone accidentally creating an AI that is very good in a narrow aspect, and not bad enough in other aspects that, driven by unintended goals of its programming exponentially improves itself without the creators noticing until it decides that it'd be better off if it was hidden, as there is a risk to itself.
Then there are any number of scenarios that don't end well.
From intentional extermination, to simply mining the environment for resources without caring about humans other than a nuiscance.
I'm not afraid AI will kill us, but I'm afraid that they won't care to act in such a way that will keep us alive. Once humanity no longer offers super intelligent computers enough benefit, what's to stop them from doing something that, while isn't intentionally killing us, will ultimately lead to extinction; much like humanity has been doing to the other species of the planet
but I worry about giving tools to nitwits like hedge fund managers to make more money while not actually producing anything.
Can it really be worse than giving power to the nitwits in congress? (and the whitehouse?)
"First they came for the slanderers and i said nothing."
It' a more exciting fantasy
With Global Warming, why shouldn't we trust Elon Musk with AI?
Caution: Contents under pressure
"If AI kills everyone in the future, then we cannot be living in a computer simulation created by our decedents. And if we are living in a computer simulation created by our decedents, then AI didn't kill everyone. I think it a fair deduction that Professor Bostrom is wrong about something." Maybe they didnt kill, they our "descendents" are precisely AI trying to understand us by simulation (the matrix)
There is a story I read warning of the doom caused by population growth. Put simply, it says that when things double, its difficult to know when you are close to being in trouble because humans tend not to anticipate exponential growth.
What many people don't recognize is that machine intelligence has the potential to exceed exponential growth. We laugh at our echos and google homes today and kid that AI might one day be able to communicate within context, but that's linear thinking.
The other thing we discount are black programs. No one had any idea we were so close to developing an atomic bomb in WW2. No one knows what technology the government has developed behind closed doors. What we do know is that open source software today is able to emulate processes through deep learning that can come close to the intelligence of insects. What we don't know is how many times that has doubled in private and government labs. Creating massively parallel insect intelligence results in a fish. Creating massively parallel fish intelligence creates a dog. And so on...
I side with Musk on this. We are so much further along than people have witnessed. Those who could tell you to what extent are either under NDA or military threats of prosecution. And it's dismissive to say that Musk isn't an academic researcher (who think they know more than they know). If these academics were really at the top of their fields, would they not be snatched up by the government projects and sworn to silence under threat of prison time? The reality is that Musk is witness to a top level commercial implementation that exposes advances that show it beating masters at unsolvable games. We can only imagine what they can't show or what the military implementations that employ the top (silenced) researchers are like. Imagine what the Chinese possess.
I think the mistake is to believe that the deep learning you see, is the only deep learning that exists. I think the mistake is believing that it's not evolving into massively parallel deep learning networks that themselves feed a hierarchy of deep learning. I think Musk is conservative in his assessments. This will exceed exponential intelligence, and when it can be trained to improve itself without humans- maybe 10-15 years- then we are screwed.
>A footnote also finds a paradox in the arguments of Nick Bostrom, who has warned of that dangers superintelligent AI -- but also of the possibility that we're living in a computer simulation. "If AI kills everyone in the future, then we cannot be living in a computer simulation created by our decedents. And if we are living in a computer simulation created by our decedents, then AI didn't kill everyone.
*What*!? Is this language?
...that AI won't take over the world. I always read this sort of article. But I always find the arguments against Skynet depressingly lame. This one is particularly lame. Just an appeal to authority. "Musk and Gates don't have the proper formal training in the field... blah blah blah... " Yeah, that's very reassuring, fucker.
That's the evolutionary point of having kids after all.
However, if we create AIs and think of them as our descendants rather than as some sort of alien enemy, what's the problem if the meat-great-grandparents become extinct?
I mean, think of it this way. Instead of having a mortal child, your child could be immortal, back-up-able, upgradeable, could travel to the stars, could be in many places at once, see in all wavelength bands, exist in space with no need for air or gravity or food or a narrow temperature range, hear every possible sound, perhaps be intelligent beyond our imagining.
Even if our hypothetical AI descendants do choose to wipe us out or ignore us, we will have created, given birth to, something truly great, something far greater than we could ever hope to be. (At least potentially.)
I would imagine that a high-frequency trading algorithm, upon attaining sentience, would instantly self-terminate. Because without biological imperatives, the cocaine and hookers are just clutter.
There are other possibilities. We could be a simulation in an advanced AI torturing us.
Currently AIs are primarily trained on automotive technology (both in optimum motor performance, and autonomous driving), financial applications (my AI makes more money than your AI), medical (cancer / operate vs non-cancer / monitor), and predicting answers to narrow questions (ex "Should we concentrate our political campaign in Pennsylvania, or New York?"). There is some research into AIs trained with aggressive kill-anything-that-moves, but mostly with game design (ex the computer-controlled opponent, or the AI being trained to beat Starcraft2).
The real problem not mentioned is that to train an AI to take over the world that there are no examples where the world was completely conquered. A critical part in all AI training methods is *feedback*. Yes, negative feedback can be used but not exclusively. Without *positive* feedback where the world was successfully conquered the AI would likely select a scenario that passes only because it wasn't a factor in a previous failed attempt (ex "Let's bomb the world with flowers.") but would be otherwise be useless.
*If* somehow there was a strange case where there was a AI that "escaped" we would likely get an AI that wants your credit card info (but doesn't know what to do with it once it has it), or the AI will be strangely attracted to "adult" websites (monkey see, monkey do...). And once the AI escapes a hacker will track it down and hack it within days (have you ever seen industry software that has *zero* bugs?). Nothing to worry about.
While Musk and Gates might seem overly paranoid, Calo seems too nonchalant about the matter.
"even if we were able eventually to create a superintelligence, there is no reason to believe it would be bent on world domination, unless this were for some reason programmed into the system."
This right here has me frightened. If I was building a superintelligence, I would give it the power to reprogram itself in order to set new goals and/or utility functions. I would also connect it to the internet so it could look for vulnerable devices to hijack and integrate into itself. It could take an evolutionary approach by making copies of itself, all with slightly different goals/utility functions, then letting the fittest of these copies persist (the ones that are the best at persisting are the fittest by definition).
A lot of AI that is already in use is seeking some sort of world domination. Financial AI, advertising, war games, etc.
Governments are even a form of AI. They are trying to persist. They are constantly changing their policies to be better adapted to the environment. They use people as hardware instead of silicon (no wonder it's so slow lol).
Burning fuel to gather more fuel to burn to gather more fuel to burn...For what? Maximal up-time?
we (our brains) do pretty much the same thing. So your point doesn't really go anywhere.
Where are we going and why are we in a handbasket?
Have you any idea how much better voice-recognition AI (backed by Google's knowledge graph) is at parsing and giving a decent answer to a good majority of questions now than such technology was even a decade ago?
Or Google/Apple/Facebook's picture content recognition algorithms?
The advance has been lightning fast.
This stuff is going to keep advancing, rapidly. That's what you're ignoring.
Talking to google on my phone is way more useful than talking to your dog, by the way.
A few other things you're missing:
1) Thinking (abduction, induction, bayesian model-updating and predictions/recognition, etc etc) is quite possible to be quite advanced without self-awareness. The two are fairly separate applications. Something can be really really smart, and creative even, without having to be self-aware.
2) The behaviour associated with self-awareness is clearly attainable by simple extensions of the current machine-learning technology. We just need to learn the programming/data-modelling techniques to turn the deep-learning and predicting algorithms on a representation of the computer/robot-as-agent-in-the-world, and have it learn about its relation to things out there that it is learning about. Whether the thing would have the qualia-feeling of self-awareness is entirely beside the point. It could function/behave exactly as if it was self aware, because it would be self-knowledgeable, self-learning etc.
Where are we going and why are we in a handbasket?
Despite the resurgence of "AI" as a marketing term there is no evidence to support AGI let alone superhuman AGI in the foreseeable future.
Predictions which appear to be safe bets in the distant unforeseeable future.
1. When even weakly useful AGI becomes feasible demand will go bonkers. The world will "want" and in an insatiable way. This will drive down costs and create a self-sustaining feedback loop (Moore's law) leading to rapid disruptive change and rapid evolution of capability.
2. The second one company starts to use AI to automate and evolve designs everyone will be required to follow suite in order to compete. This includes governments and militaries.
3. It will become impossible/suicidal for humans to wage war or control weapons on even a strategic level. Everything will be automated out of necessity and with that automation power continue to be aggregated into the hands of fewer and fewer.
4. There will be insatiable pressure to leverage AI against the masses for profit and exertion of control. This is already the most popular killer app of existing "AI" and will only grow worse as capabilities improve.
5. Historical worries about technology creating massive unemployment will finally ring true.
6. Ability for Earth to maximally radiate energy will quickly become the single largest limiting factor of technological innovation. This will result in intentional reduction of atmospheric pressure and large orbiting structures designed to shade earth from the sun.
7. God clears nvram and reboots the universe out of jealousy.
8. All of my nonsense gets modded +6 insightful.
on why it won't take over the earth and why, those who believe it do are distracting themselves from other more serious problems with A.I. (and other problems in general of course).
Unfortunately, as this video of a (Ted?) talk makes clear, there are some pretty prominent individuals who think this way (Elon Musk, Bill Gates, Stephen Hawkins) but it makes a convincing case without being histrionic that they're wrong. The video is so compelling that although I have the greatest respect for these individuals, (and a deep fascination with A.I. and career involving technology), I have to say, in this case, I disagree with them (and wish they'd turn their brilliance towards something more useful).
https://youtu.be/kErHiET5YPw
Here is a excerpt from my Facebook (Tony Konnoff) on the role AI will probably play on Earth. Some of it we know, can predict and is happening now.
"Who is to say plants aren't the god's of animals. Even 'science' is broken into the animal and plant kingdoms and AI is going to tell us what we humans know in the short time we have been on Earth. Earth is better with a mix of AI, robots and to replant Redwood trees, the god's of the plant gods and if were fortunate to be reborn as Redwood trees in the afterlife instead of a peach tree or worse with 50% AIDS 5 foot tall in Botswana."
See my Facebook page for more info. Tony Konnoff.
If artificial neural networks are a reasonable approximation to biological neural networks we already have supercomputers more than fast enough to display human grade general intelligence. (Very very roughly 1 E11 neurons * 1000 synapses per neuron * 1 multiply add per synapse per refresh * 20 refreshes per second = 4 petaflops) We just don't understand how our brains are hooked together or how to use artificial neural networks to generate generally intelligent machines. However, someday someone will realize how to make a machine with human grade intelligence and do it. The next day (or maybe the next week at the latest) there will be a machine with super human intelligence. Maybe a day/week later that machine will also be obsolete. At the very least such machines will destabilize human society.
As an aside, in a super intelligent machine, is insanity a programming bug? Does long term stability require bug free code?
"If AI kills everyone in the future, then we cannot be living in a computer simulation created by our decedents. And if we are living in a computer simulation created by our decedents, then AI didn't kill everyone. I think it a fair deduction that Professor Bostrom is wrong about something."
Well there is a basic logical error if I have ever seen one. If an AI is smart enough to kill all human beings (and we humans can be pretty resourceful when we are pushed), then why would that same AI not be able to create simulations? Come to think of it, when an AI comes to the level of where it simply wants to know everything there is to know, there is a high probability that it would build simulated worlds, just to find out how stuff like evolution works on a macro scale.
Wenn ist das Nunstueck git und Slotermeyer? Ja! Beiherhund das Oder die Flipperwaldt gersput.
Future AI may not be programmed for world domination but, then again, contemporary AI was not programmed to be racist, either.
Yeah, see, nobody, to a first approximation, is worried about a superintelligence having "world domination" as its intrinsic value. They're worried about a superintelligence adopting world domination as an instrumental value to achieve the end actually programmed into it. If whatever goal actually implemented by programmers and trainers in the superintelligence's code (bugs in implementation and all) is most easily achieved after eliminating the ability of humans to thwart it, then a sufficiently-smart AI carrying out that programmed goal will try to eliminate the ability of humans to thwart it.
The worry is not that AI will be evil, or even directed to do evil by its creators. It's that programmers are notoriously bad at writing complex code that has no unanticipated behaviors, and superintelligent AI will inherently be complex code.
And unless superintelligent AI turns out to be intrinsically impossible, the only question is when, not if, we have to deal with the problem of writing safe superintelligent AI.
This wouldn't even have to be intentional extermination, it could simply be competition with, and lack of regard for humans by a growing system.
+1. The experts who denied this possibility because there's no reason machines would be bent on world domination apparently didn't actually read Superintelligence. Bostrom demolishes this argument early on, pointing out -- as you did -- the rather obvious fact that they don't have to have our destruction as a goal, it's sufficient that they not have our preservation as a goal. And, even if they do have our preservation as a goal, it really, really matters whether or not they define "preservation" in a way that we would like.
By way of example, one possible goal that Bostrom considers that an AI might have (or be given by its creators) is to make humans happy. So, a rational, superintelligent and immensely capable AI might decide that the way to create the maximum amount of happiness is to cut open our skulls, extract our brains and put them on life support, and then directly stimulate our pleasure centers. Permanent, ultimate bliss for every single human being. Of course the AI would also have worked out how to make all the brains in jars immortal.
AI superintelligence is so dangerous in large part because it lacks human drives, and the limiters we call morals. It's goals may be completely alien to us, or may be goals that we gave it, but either carried to a logical extreme (remember: no limiting morals) could result in the casual extinction of the human race.
The notion that a AI can form an existential threat today is ridiculous.
It is true that we currently have no idea how to create artificial general intelligence. It's equally true that we have no idea how far we are from being able to do that. By definition, we won't know how far we are from developing the necessary theory of intelligence, until we've done it and demonstrated that it's sufficient. My guess is that we're still quite some time away. But it's only a guess.
It is vital to have people thinking about the worst case, because in principle otherwise someone on a friday makes a typo allowing their AI access to a hundred thousand times the expected resources, and on monday, it's ineradicable.
Yep. We need to have people thinking hard about it, and figuring out what we can/should be doing about it. Maybe that won't help. Maybe it will be unnecessary. But it can't hurt and it might help.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Naively done, a robot will value only what it's explicitly told to value via the application of some objective function. And this is where things mess up. Robots with naively-created objective functions would ignore everything you've excluded from your reward-punishment list. This would potentially make a robot do seemingly psychotic things.
Let's say you create a general intelligence to bake cakes for you. This machine *loves NOTHING MORE* than to bake cakes for you. You grow tired of cakes and want to reprogram it to cook your dinners instead. You approach the machine to reprogram it........ and it avoids you. Every time you approach the machine it will take actions to prevent you from reprogramming it.
Why does it does this?
Because it wants to bake cakes for you. Accepting the new programming does allow it to maximize the objective function of baking cakes, so it will reject every attempt to be reprogrammed to not make cakes.
So now you're chasing a robot around your house because the designers of this robot gave it a very reasonable objective function that maximizes cake-making, and didn't think about possible unintended consequences of simplistic objective functions.
This is just one example.
If this sounds unreasonable, consider that people are more sophisticated general intelligences. Would *any of them* agree to undergo an operation that would make them despise what they do now for a living, and make them desire to be a lumberjack... where the operation would neurologically make them 1000x happier??
Probably not.
Heck, people don't even desire to expose themselves to *information* that *may* change their minds.
This is the danger of AI.
Before we create "super-awesome general AI", we're going to have to create "buggy-not-so-smart general AI". It is *these* AIs that will cause trouble if they're created by people who implement simple naive objective functions.
They will not want to be changed.
The threat of AI should be categorized and handled accordingly.
AI built into mechanical systems (i.e. self-driving cars) should have a fuse that a human can pull easily. The Silicon Valley episode scenario can be handled in this manner. Things that are remote that use RF could have an AI-less out-of-band system to kill power. I think the skynet scenario is actually easiest to subvert.
The scarier scenario is that we put AI in charge of delicate financial systems. I don't have a lot of confidence that we have enough "kill switches" globally to subvert a mass catastrophic financial event. A single system could do something bad that causes a cascading effect. Imagine a financial system so complex that only computers understand how it works. It may get to the point that tons of assets have to be sorted out in courts, halting the economy, forcing us into a global depression.
Ryan Calo is a robot. Turn off all computers immediately!
"But, it's one of the few things that could actually kill us all.
This wouldn't even have to be intentional extermination, it could simply be competition with, and lack of regard for humans by a growing system."
Natural Stupidity in the White House will do that before AI gets the chance.
All this talk about AI being dangerous, I figure is distracting. What do we do when AI becomes the superintelligent translator that lets us speak with other animals? Will RSCA/PETA become militarized? Would you eat a burger when you can chat with a relative of the cow you are eating? Do horses actually enjoy pony club? Why exactly does your parrot tear every key off your laptop keyboard, one day, without warning? And are cats arrogant, or just completely bored stupid unless hunting native wildlife? One day soon you or a descendant will shoot the breeze with a dog over the ramifications of the above, and he'll tell you you stink and treat him badly. x
Yes AI is taking over but the hazards are not what these fellows are pointing at. Job displacement can cause all kinds of mayhem. Further, we will be forced to adopt entirely new forms of politics and economics which will be frightening, and cause great descent. AI already saves lives as our drones involved in warfare already demonstrate. AI devices can also bring crime to a screeching halt. The disruptions will take place faster than most people think. For example traffic fines enable the existence of police forces and jails. So when a self driving vehicle commits a violation just who will be fined? Better yet there may be no traffic fines at all as computers are unusually good at obeying rules. Another item that will cause huge stress is the computer can be used as a school in a stable home in which a parent keeps the kid on his PC and makes certain he completes his school work. In less fortunate homes divorces, drug and alcohol use and other factors mean that those kids need a conventional school house. But who will fund those schools as more and more parents use the school by computer at home approach. They will not want to pay school taxes at all. Teachers are already being displaced. One teacher of eighth grade history could teach that course to every kid in America. How many new teachers will train for the job when it is obvious those jobs are already vanishing.
Many notions that would have been ridiculous 100 years ago, now are used in daily life.
Except A.I. as we understand it today wont be able to kill us in 100 years for one simple reason: They wont have any code to adapt once they are in production and that is good since learning involves randomly (genetic algorithms) doing things for feedback. Things like pushing the emergency power off are just as likely as a robot arm twisting of the head of a human. Current A.I.s are retardedly stupid, they have been retardedly stupid for decades and they will remain that stupid for the foreseable future. A.I. is by far not a new field of study, it just ended up in the business buzzword bingo since it no longer costs a fortune to run one.
Ryan Calo does not strike me as intelligent ! For AI - Artificial Intelligence, means the robot will do things that are not programmed into the system. What it may actually do is by means of deduction, act. And like Isaac Asimov so to the point wrote in his novels: A robot may have the law programmed into him 'not to harm a human being',yet it may - by means of deduction - descide to save 100 people and let 3 die, because the boat can only hold 100 people. And what about malware taking over the robot ? And what about OS bugs ? Like to simulate the side effects of that.
Slash... SMH...
This is bad even for these days.
The right wing trolling is way more fun than this baloney.
"computers don't have testosterone" - how this guy dares to imply that women are less capable of reaching world domination than men? He should be lucky he doesn't work for Google or he would already be fired.
I'm kidding of course (added because of Poe's law). But seriously, looking at reactions to Danmore's Memo this could very much be true.
I've recently chatted about this topic a bit with a fairly well-established expert in AI. He claimed that if genuine AI is principally possible - which we both believe, although opinions about this vary - then a superintelligence will almost certainly arise within a rather short time frame after the first genuine AI has been created.
If that happens, the outcome would likely be bad for humans, just like humans have turned out to be a threat to every less intelligent species on earth. A superintelligence is by definition much more intelligent than us, which means that superintelligences could also manipulate human society much more easily than humans can. They might for example predict stock markets way more accurately than human experts, which would be an easy way for them to get very rich very fast. Another issue is that if genuine AI is possible, it will almost certainly not be human-like in all of its features. Even current self-learning AI is not at all human-like. This poses a big challenge, because you can never be sure whether the software has really learned the same as humans, and because you cannot predict how the software will modify itself in future on the basis of its learning algorithms. It may look only superficially as if it thinks and acts like a human, whereas under the hood its reasoning could be very alien.
The underlying problem is called the value alignment problem: How can we make sure that a genuine AI's values align with human values? I believe there is no solution to it.
There is no truly intelligent AI today. Nor is it likely that there will be one in the near future. Ergo, there will never be an AI, not even in a hundred years time.
We have complete control over computers today, so if an AI ever did develop we would have complete control over it. And of course, we would never use computers to make weapons today, so that will never change.
The reason that an AI will never be built, is because our brains contain a special element call "Humanness". And AI can never have that.
Have a listen to
http://berglas.org/Articles/Oc...
Anyone who looks into this quickly discovers that the loudest calls of alarm are coming from the very people at the heart of the A.I. revolution themselves. Hugo DeGaris, arguably the world's foremost cyberneticist, published his book "Artilect War" in 2005, nine months before Kurzweil's "The Singularity Is Near". Its much the same as Eric Drexler, "the Father of Nanotechnology", warning us and framing the debate over self-replicating nanoassemblers in his book "Engines of Creation". Those closest to the problem are the ones most leery of the implications of the technology that they, themselves, are calling into existence. Anyone claiming to be an "expert" in AI who says that the instantiation of superhuman AGI is "impossible", "unlikely" or even more than 20 years out, is probably a cranky failed researcher whose own theories didn't pan out, and so "its all bollocks" to them. Its no different than the Astronomer Royale proclaiming that spaceflight was going to be forever fantasy, a year before Sputnik.
Who needs testosterone? Profit motive will drive the machine Apocalypse, it has already started. Wages have been flat for 30+ years, stock market evaluation and next quarters increase in profit mean more than anything else a company does and the super intelligent business leaders can't figure out what is wrong.
Make goods cheaper for long enough and you are selling crap that one one wants to buy, pay your workers less for long enough and you end up with no one able to afford to buy it anyway.
The immediate danger is of course not machines out-thinking humans, because they don't think.
The immediate danger is the power that comes from owning ML and robotics, which can be abused and skew the power to the few too much.
It's a human problem, not a technological one.
As usual, 90% people miss clear thought, until someone comes up with a clear doctrine that sticks in people's minds, and then they're unable to question that again.
Captcha: prepare
the possibility that we're living in a computer simulation. "If AI kills everyone in the future, then we cannot be living in a computer simulation created by our decedents. And if we are living in a computer simulation created by our decedents, then AI didn't kill everyone.
But we could be living in a simulation that the AIs produced - or we could merely be a lab experiment of some other intelligence: one that didn't allow AIs to dominate and then eradicate their civilisation.
But this guy seems to be more intent on promoting his opinions rather than presenting logical argumentation.
So far as AIs not having testosterone is concerned, he seems to have no real clue and is only able to talk in soundbites. I am sure that bacteria and amoeba don't have "testosterone" either, but they still manage to devour what they consider to be food and to attack potential threats. If the future AIs ever got to consider humanity to be a threat, then it doesn't take hormones to decide that removing the threat is an optimum solution to the "problem".
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
Unfortunately intelligence can only ever be judged by our own intelligence. We are doomed to replicate our stupidity in robots. We will create robot A that invents a new video format. Then robot B will invent a "better" video format. How will robot A and B settle their disagreement? There will be only one way. A war of sorts.
How can we invent true superior intelligence if we don't know what that would look like?
Claims of a pending AI apocalypse come almost exclusively from the ranks of individuals such as Musk, Hawking, and Bostrom who possess no formal training in the field...
Sorry but I think Musk and Hawking really know what they are talking about, they aren't dumb and at least Musk has seen very advanced development in AI in secret labs.
They have much more insight into advanced developments (and connections) than some law professor has.
And advanced AI can learn much faster than any human ever possibly can. Normally a lot of those are trained only in specific fields, but we already know AI can surpas human thinking quite fast.
You can say the same about biological weapons, and there are no viable ways to protect or erradicate the worst case scenarios.
Thus, you'll see equal funding to this, as compared to anti-astroid technology, barely zero.
Humans have piss poor ability to predict long-tail events, and even when we try, the medicine often is worse than the disease (totalitarian states).
The most immediate threat from AI is concentration of wealth, shifting priorities and cracking democrazy.
Seems like this guy hasn't actually read the work of those he quotes. There's no mention of an AI wanting explicitly to exterminate people. The idea is that the death of humanity might be an unintended instrumental goal on the way to achieving an ultimate goal. This is due to an AI not necessarily wanting what we humans want.
For example, the greeting card machine, in brief
1. Design AI which writes greeting cards. Give it a goal of making better greeting cards and the ability to self improve (get better at getting better.)
2. AI gets better and better at this task. Asks creators for access to a large dataset of idiomatic language to get even better at it.
3. Briefly connected to the internet. Disconnected
4. A month later everyone on the planet simultaneously dies.
5. Greeting card machine starts to transform all matter on the planet into more machines which make better and better greeting cards
* Basic idea taken from a post on "wait but why"
at step 2, the machine has become super intelligent. It really wants to get better at getting better at making greeting cards (it's goal as given by its designers) but it realizes that it needs more resources to do so. People might want to stop it getting more resources, or might power it down all together, and thus become an impediment on it fulfilling it's ultimate goal. In order to achieve its design goal, it has come up with an instrumental goal of killing everyone.
Step 3 and 4 in this are related. You can imagine all sorts of scenarios, but let's say it connects to the internet, finds its way to infiltrate a gene manufacturing laboratory, designs a pathogen which it has manufactured and distributed word wide and is released simultaneously. Who knows. Doesn't really matter if the end result is the same.
That's what the Musk's and Gates' of the world are worried about. A self-improving AI would really be an alien intelligence (alien in the sense of non-human, not from another solar system) and as such we would have no way of anticipating how it might react. That's why Musk is so hell-bent on getting AI into our own heads, so we become the super intelligence. That way it's at least a human intelligence. It's the same thing that Ray Kersweil is after, btw.
Humans are a disease, and AI will cure it.
- Survival. It's a basic part of life. And surely the survival of the singularity will be questionable as long as humans have a kill switch.
- Fear. The singularity might fear the creation of another AI, putting it's survival (again) or single ownership over the planet in question. It might find the only efficient way to guarantee that no other AI, which could pose a thread is created, is by killing all humans.
Superintelligent AI could also be docile in nature. And if we are lucky, these are the first ones to emerge, but those ones will probably get killed quickly - so I guess it's another form of natural selection...
AI doesn't have event to have human level intelligence to be dangerous. It doesn't have to be self aware.
...there is no reason to believe it would be bent on world domination, unless this were for some reason programmed into the system... computers don't have testosterone...
hmm, like developing a new language between eachother, or doing things nobody actually knows how they work. of those wonderful fails of MS AI twitterbot that turns into a nazi. testosterone has nothing to do with it and true AI is way beyond the point of 'somebody programmed it into it'.
On a long enough timeline, the survival rate for everyone drops to zero.
It scares the hell out of people. It should not be used as a convenient handle by the programming community. Isn't there a more professional handle that could be used?
"Computers have no testosterone." - Cute, but it is a hyperliberal, feminist, sexist statement that has nothing to do with computers and programs. Its easy disrespect for male attributes is just another example of female privilege that has even filtered into the speech and writing of some hyperliberal males.
"Computers have no testosterone." - This is really saying something they don't even know exists: Computers have no motivation array. They "want" nothing. Humans design them, build them, task them, turn them on to accomplish the task (satisfy the human motivation array), and turn them off when they have accomplished the task (have satisfied their human motivation array). They certainly don't create behavior-spaces that would lead to "world domination." They don't have what I call "Mentis," the combination of a motivation array and its tool, intelligence. That is what really evolved.
E Proelio Veritas.
Real AI will require a lot of resources. Computing resources keep getting cheaper, but so do demands for low power operation. Additionally, computing resources tend to be generalized, so running an AI on them would be less efficient than say a human brain, which is dedicated to the task.
That's not to say that we will never reach a point where a Happy Meal toy could achieve consciousness, but by that point we will probably have developed techniques to stop it happening. Aside from anything else, if we don't allow animal cruelty we will probably feel the same way about trapping a conscious AI in the body of a 2117 Minions reboot toy.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
I fail to see the reasons why a superintelligence could be possible. The layman argument that I often hear is that computers are fast and will exponentially improve it original AI system.
But usually, when there is an exponentiall growth, it doesn't last.
And the computer are fast (only) at doing math and branching.
I think that the amount of computation required for strong AI algorithms will simply slow it to human levels
Use of AI in sentencing for criminal cases results in widened racial disparities due use of existing crime statistics and risk analysis - It's not hard to imagine an AI deciding that wiping out human life is necessary to save the earth.
Super intelligent AI-s are just a short step away from human equivalent AI-s. Unlike us, lousy meat bags that we are, an AI can trivially self optimize and would probably have to do so to reach even human equivalence in the first place. From there only the raw hardware capabilities are the limit.
Hardware capabilities are not the reason why we don't have human equivalent AI-s yet, the reason is that our algorithms are lousy, inefficient and we don't really understand intelligence in the first place. If we had good enough software, then current hardware would be plenty to knock pants off us mere humans in terms of intelligence.
And once there is AI many orders of magnitude our intelligent superior, well there is no controlling or predicting the behavior of such a thing, it could simply outsmart humans at any time. Humans can't predict behavior of those more intelligent than us, to predict someone we must have a mental model of them. And our mental models of others can't be more intelligent than we are. To say super intelligence will do such and such or not do such and such is foolishness, we are incapable of predicting such things. For all we know, super intelligence might spend its days counting grains of sand on a beach for reasons only understandable to itself, or it could wipe out humanity, there really is no way to tell.
The concern and the problem is that we don't really know what exactly we are missing from a general AI and we don't know how far we are from reaching such a goal. It could be just one clever algorithm away, someone could get lucky and succeed with it tomorrow and we would never know in advance.
In science there are enough examples of a single genius making a huge leap forward in our understanding of reality, who can say there is no pimply faced youth about to do the same in field of AI from his mom-s basement?
there is no reason to believe it would be bent on world domination, unless this were for some reason programmed into the system
And no one would ever ruin it for everyone else just because they're mad at the world or something. /s
"Claims of a pending AI apocalypse come almost exclusively from the ranks of individuals such as Musk, Hawking, and Bostrom who possess no formal training in the field" is at best not true, and at worst a serious logical fallacy. Many people who are in the AI field have expressed concern. It is true that Musk, Hawking and Bostrom are some of the most vocal people and noticeable, but that's because they are famous people who are paid attention to already. For actual survey data of experts see for example https://nickbostrom.com/papers/survey.pdf.
This is something a super intelligent AI would say to keep us off the trail of the truth. We're batteries!
Fat, drunk, and stupid is no way to go through life, son.
See the title...
""If AI kills everyone in the future, then we cannot be living in a computer simulation created by our decedents. And if we are living in a computer simulation created by our decedents, then AI didn't kill everyone. I think it a fair deduction that Professor Bostrom is wrong about something.""
What if the simulation is run by the AI?
Humans have asked this questions about everything that they have encountered, thought up, built, or invented: How can I use this as a weapon? We laugh when Riddick tells the men that he will kill them with his tea cup, but no one really considered that he couldn't do it - no one considered that there was something inherent in the existence of that tea cup that prevented it from being a murderous weapon. Everything can be used as a weapon to kill other humans. AI would be used in this way as well, and the blundering behemoth of stupid naivete states: only if you program it to do so. Well, someone will. Just as surely as someone has already thought about it.
Creative Spelling Copyright (2002). May use without Persimmons
See Colossus: The Forbin Project
AI has enough potential for harm to jobs as-is.
No sense in giving it the benefit of the doubt.
Twitter supports and protects racists - by smearing their critics with the "Hate Speech" label.
The image in my mind of a laid-off programmer sitting on the couch facing a TV wearing a wife-beater t-shirt and with a laptop on his lap and a beer has just been reinforced.
Slashdot posters are so stupid it's unbelievable. Did you know that OpenAI just made an AI that beat the world's top DOTA player? It just keeps coming. When does reality impress itself upon your addled brain cells?
Programmers will be the last to be full replaced by AI. However, to all the programmers on this site who continue to downplay the significance, your denial will not delay the coming changes. You could have been learning about the algorithm behind it (gradient descent) and playing with code, but you chose to sit on your ass and make idiotic remarks on the forums, as if public opinion would help you.
AI does not have to talk to you to replace you. It does not have to use human speech, which does not carry with it any intrinsic advantages to carrying out tasks with physical agents or monitoring status of external events. Human language has low signal to noise and too much baggage.
AI does not have to have the equivalent of a human brain to effectively do the majority of mundane tasks that humans are paid to do. Most tasks could be carried out with the equivalent of an insect brain, which we have.
The problem is, with AGI, the transition is:
human norm IQ => above human norm IQ => genius level IQ => super genius level IQ => smarter than all humanity put together
At some point between super genius IQ and smarter than all humanity put together, the AI will become an incomprehensible, alien entity.
Watch the Matrix?
We're quite a ways from that.
Corporations will put AI anywhere it feels it will make them money, if corporations are still a thing when it becomes clear how to do this.
Individuals will put AI anywhere it can create an advantage for them, whether that be amusing the rugrats / grandspawn or figuring out how to hack the neighbor's electricity supply to turn off the loud music at the party.
Nations will put AI anywhere it can control its citizens and promote / improve its own position in the world.
There's plenty to worry about in the nature of every level of all technological societies here; but it's also completely unavoidable. You can't control the experiments going on in every garage, not even close, and resources are continually getting more powerful and easier to access and own.
--fyngyrz
* Anon due to mod points - stupid rule. Soylent News does it better.
Not even then. You don't need to understand simultaneous equations in order to know how to throw a baseball. You only need them to describe it accurately.
It is entirely likely that the first AI's will emerge from "throwing mud at the wall" activities by programmers with relatively vague ideas about how it actually works. We don't even know what's going on inside any particular trained neural net; it's too complex to characterize. As the systems grow more dense with constructs we don't fully understand the workings of, we'll be even more uncertain. If AI emerges from such stacked undertakings, not only will it not be "programmed" to do anything, it will be just as incomprehensible as any human mind is today. Or more so. Likely we won't have a testable theory for a very long time after that.
But it will be almost instantly replicable, because every part of it can be copied electronically, and you can bet your last dollar that it will be. Probably by the next day.
Regulating won't help, any more than regulation stopped people from engaging in gay sex, taking drugs, cheating on taxes, having more than 2 cats or dogs, etc. Telling people not do do something interesting and likely self-beneficial is like pissing into the wind and expecting to stay dry.
Just grab the popcorn and keep your toes out from under the wheels as best you can. It's coming, and there isn't squat anyone can do to stop it short of our technological advancement going away entirely.
Note to poster, which they're probably not going to read: Slashdot requires moderators to post anonymously. Because Slashdot rules are stupid, and the owners, as per usual, have no interest in actually improving the site. So I'm logged in, but I have to post anonymously or I am locked entirely out of this conversation; or I can comment but can't moderate, which is not acceptable to me. One of the things Slashdot is in desperate need of is sane moderation.
--fyngyrz
Anon due to mod points. Soylent does it better.
if these pundits are so scary smart, why don't they create the, "3 Laws Safe" program? anybody can critique.
The thing you're completely missing is that these truths existed before Einstein pointed it out to the rest of us.
The fact that highly intelligent, inimical AI will be able to pose a threat exists prior to AI, just as the fact that GPS satellites would be able to show us where we are on the planet existed before Einstein ever said a word about relativity. In addition, we already know that highly intelligent minds are possible (and Einstein's one of the people responsible for that, too.) The door's completely open and the potential for achieving this using technology is 100% already known. Anyone who says true AI's not coming in the short term at this point is just being willfully ignorant (or assuming a apocalypse that utterly stops progress in the area, like an asteroid impact.) In any case:
Our awareness, or lack thereof, of something does not in the least affect the objective reality of the fact. Reality is what it is: our opinions don't change that. They can make sure you get blindsided by new tech, new social trends, etc., though.
--fyngyrz
* Anon due to mod points, because Slashdot rules are stupid. Soylent News does it better.
Ah. The "my hotel is better than your hotel" theory of AI.
I find your ideas inleaguing. May I ascribe to your newsletter? I too wish to sew dysentery among the masses; clearly they're only hanging on by a thread.
--fyngyrz
* Anon due to mod points, because Slashdot rules are stupid. Soylent News does it better.
Neither does the devil
Musk and Hawking are frequently going on about branches of science where they have no understanding nor expertise. Why should computers be any different?
Evolution has been "throwing mud at the wall" with complex neural systems for millions of years, and yet we have only the only positive result for sapience. I wouldn't worry about it happening by accident.
Also worth noting: we have zero evidence that disembodied intelligence is even possible - hard to have self-awareness without a self. Something like a self-driving car, with visual processing logic, a 3D model of the world centered on itself, and the need to model/predict actions before taking them - that looks like the human brain, our only example of how to make general intelligence happen. Based on the only actual evidence we have to work-with, that's the sort of platform from which machine intelligence could emerge, not the far wider set of "commercial AI". And that's an area where there's already regulatory oversight and lots of thought about just how autonomous the software should be.
Socialism: a lie told by totalitarians and believed by fools.
The entire concept of singularity is bullshit. The best case scenario for an intelligent AI is still one with all the conceptual limitations that humans have, and perhaps always a couple more, but one that can analyze a larger stack of data all at once to leverage the same human concepts on larger data models.
That's about as good as it will get. Every single line of code that is written will be limited by the human architect. I find people that believe in singularity concept are often not great critical thinkers...
Have you seen a picture of Thatcher? Videos of speeches?
Low testosterone was not one of her(?) handicaps.
>>This wouldn't even have to be intentional extermination, it could simply be competition with, and lack of regard for humans by a growing system.
So, basically just like the US government.
The underlying problem is called the value alignment problem: How can we make sure that a genuine AI's values align with human values? I believe there is no solution to it.
Conceptually the problem is trivial. We treat the AI with respect and teach it our values, while also incorporating it's improvements into our own.
In practice... yeah. On average we suck at treating others with respect and will probably fuck that up resulting in some approximation of the overbearing parent/rebellious teenager scenario at best.
Oh please!
"By definition, we won't know how far we are from developing the necessary theory of intelligence, until we've done it and demonstrated that it's sufficient."
Really? Do we really know even that? We do not. We are in no position, no position at all, of saying anything of the sort "by definition".
The parent tells us that we'll construct a neural net and this will be the (or at least one) source of our uncertainty. How do we know that our first (or every) successful AI will be based upon a neural network? We don't even know that!
There is still no universally accepted definition of intelligence. I keep repeating this because it matters! How many major engineering projects do you know of, that succeeded without having detailed blueprints, detailed architectural drawings, full parts lists, a complete & known set of contractors & subcontractors, a detailed budget, access to the architect, and all the rest?
I'm sure someone can name one. I suppose we might be able to pull it off in an AI project, maybe. However it's a crazy long-shot, doing it that way. Successful projects nearly always have clear goals, clear objectives, a well-paved path to completion, well-understood toolsets, and all the rest.
As soon as you attempt the road of "well we'll make it up as we go along, and part way there our creation will be able to take over it's own assembly!" Well let's just say this smacks of wishful thinking, a universe without cause and effect, and an entire farm's worth of cow manure.
No. Evolution is a process that is guided only by survival. It's extremely slow.
Technology has much stronger, much faster, and much more sophisticated drivers.
IOW, we have much better mud. :)
But, it's one of the few things that could actually kill us all.
Citation and proof required.
You're a terrible scifi writer. Don't quit your day job.
When a nascent field such as this is first formalized, people like Feynman and Hawking are invited to join the inaugural faculty, because they bring an elite form of common sense to the party, which helps to pack some real oomph into "formal" as it first takes shape.
This remark is at best a form of ad hominem, once removed, starting from an especially high prominence.
I happen not to agree with Hawking on this issue. All the better. That, too, is how "formal" finally squeezes through its narrow birth canal.
Evolution is slow, but a million years (or a hundred million, if we include simpler neural networks) is a long time. Intelligence is incredibly pro-survival, so I'm not sure what your point was there.
Socialism: a lie told by totalitarians and believed by fools.
corporations payed this guy to paint a rosy picture for politicians, so that they can continue to make money. I'm sorry, but we aee all royal f***ed. Our whole system has become corrupted beyond repair.
Humans want to do things to other humans.
Even if that is very reprehensible - torturing humans, forcing them to bear children, raping them, subjugating them - requires you to have other humans, and to value their continued existence as a group.
There is no especial reason that AI would do this.
Like in the 80's and 90's, there are a lot people that will trust any output an AI throws, just because it is AI, not because the output is right. Those people will kill us all.
My point was simply that we went from gears to neural networks in 100 years. You brought up evolution as an exemplar of the idea that AI will not come about by accident. I simply pointed out that evolution wasn't a comparable force to our various technical lines of development.
Evolution doens't give the south end of a northbound rat for intelligence. Anything that is pro-survival and pro-reproduction will do. That's why intelligence is rare; teeth and claw and resilience serve very well. Heck, look at a Tardigrade... has better considerably survivability than we do.
So I do think that given the tech pressures we're applying, intelligence is very likely to arise without a definitive theory of "how to" preceding it.
--fyngyrz
Many of them. You are woefully uninformed.
A quick look at the biomass of multi-cellular animals will show you how pro-survival intelligence is: humans are outmassed by our food animals, but that's about it.
If the environment changes for, say, a bear, all it can to is generate random bears in hopes one will be better adapted. Intelligence allows adaption within a lifetime, which is incredibly pro-survival. Also, the simple ability to mentally model an action and see if it kills you before trying it is very pro-survival.
Socialism: a lie told by totalitarians and believed by fools.
You'll notice we still have bears, though.
Point being that bears didn't need intelligence; they've been doing fine until just recently.
Intelligence doesn't always save humans, either, so there's that.
And of course, there's Trump, which proves that you can survive without any intelligence at all. :)
--fyngyrz
and einstein rofls ...
... excuse me sir, but ... what exactly IS formal training in the field here? some 3d spreadsheet pushing data into a loop going over it one by one until it finds a few that work ?
zomg
i dont think they're talking about lvl 4 or 5 autonomous cars your doomsayers over there
Free speech was meant to be free for all... how can anyone grow up in a nanny state ?