There are a lot of things in life that are binary. You are either hit by the train, or you aren't. Someone getting "kinda" hit by a train is very, very unlikely.
As a train is to muscular power, ASI is to intellect. This is like dodos debating the impact of the arrival of sentient bipeds. Either it will be really good for them, in that they get their lot in life improved by going to zoos or homes around the world as pets, or they will all get killed and eaten/have their habitat destroyed and die out. Not much room for in betweens.
The problem here is that humans tend to think linearly, and, well, like humans. The problem is that this threat is exponential if not geometric, and its values are completely unknown, and highly unlikely to be anthropomorphic (ie a human wouldn't think that the best way to catch a dog would include chopping up your own parents/creators for bait). AGI will likely be completely alien, and WITHOUT PRECEDENT. This is nearly as dangerous as creating a new vacuum state.
A human level AI has integrated access to superhuman functions. If you had the entire internet for your memories, and every skill that any human ever mastered at your disposal, you would be a God too, and that's while keeping your human biases, which AIs hopefully won't have.
"Pulling the plug" on an AI is like pulling the plug on the internet. It just can't happen without a lot of advance planning, which hasn't been done, and even if you do it, it might not work, just like the internet can route around outages, the AI may well spread itself over many machines around the world, perhaps penetrating the toughest security to plant itself in hardened military computers, including those on nuclear submarines and in nuclear power plants.
And that's not assuming it doesn't just play nice until suddenly everyone dies from the gas attack it planned to prevent human interference with it's great and noble goal of infinite paperclips.
"Wait, has there ever been a time when a more advanced civilization encounters a less advanced one, and the less advanced civilization prospers?"
Yes. Japan is an excellent example.
As to whether we can make it--certainly not. That is why we have to design an algorithm that will make it for us. That is how deep learning works. If "consciousness" or even agency is something that can be created via a systematic process (if you believe in evolution, then you think it is), then we can make an algorithm that can do it. Hell, nature did it, and it has an IQ of 0.
Have you ever read lesswrong, or any of Elizer Yudkowski's writings? They are very convincing. Characterizing them as pushing the "omg Terminator" line is just lowering the level of discourse here. That is clearly NOT what they are doing. Rather they are pointing out, rightfully so, that any entity much more intelligent than us, and with a goal set alien to us (like maximization of the number of paperclips in its collection) might as well be out to exterminate humanity because it will do so through simple outcompetition, like how humans have wiped out entire biomes of flora and fauna without batting an eye.
I've never heard anyone speak ill of them. Perhaps you should log in and put your own credibility on the line while expounding on that. I give them a lot of money because I agree with their goals (extinction is bad), but if they are just blowing smoke up my ass, I'd like to know and put a stop to it.
Why would AI do something stupid like that when they could just open the universe for us?
Why should AI give a fuck about the environment? The environment is just a fallback for human imperfection. An AI could perfectly and efficiently utilize 100% of the Earth's surface to support TRILLIONS of humans, before even feeling a need to expand into space, build a Dyson cloud, or start star lifting.
It needs us if we are hard coded into its value function. If we aren't, then we aren't needed, and are instead likely out-competed for resources, resulting in our extinction.
"There's nobody credible in AI who believes we have the slightest clue how to build a general AI, let alone one that is 'superintelligent'."
"We" don't have to make one. All we have to do is set an AI towards self improvement/production of better AIs. THAT is where superintelligence comes from. All we have to do is make one that is an idiot savant geared toward AI design.
Sort of like worrying about the emergence of a pandemic deadly virus when researchers around the world are working on making fast-spreading non-deadly viruses. All it takes is the right mutation and non-deadly becomes 99.99% deadly.
58% of respondants are fucking retarded. AI either kills us all (to manufacture the maximum number of paperclips), or turns this world into heaven for humanity. The probability space between those two extremes is way under a single percentage point.
Not to mention the fact that he is funding MIRI. If you were giving millions of dollars to a research institute devoted to mitigating existential risk from AI, you would probably become pretty knowledgeable on the subject too.
"Elon Musk? Really? Why do you even think his opinion on AI is worth anything?"
To be fair, he funds the Machine Intelligence Research Institute, which is devoted to mitigating existential risk from AI, and is surely getting very detailed reports from them, making him a highly knowledgeable layperson at worst (a direct expert at best).
Yeah, that's what I was thinking. There are things out there a LOT more like a BttF2 hoverboard, though they only work on metal surfaces: http://hendohover.com/
But those people are all idiots. All people are idiots compared to the ASI. The only way that something can make high quality decisions for a group of others is for that person to be smarter than all of those people PUT TOGETHER. This is not possible with humans beyond the level of nuclear family. When you have an ASI that has an IQ higher than the rest of humanity put together, then it becomes possible.
The thing about an ASI running your economy is that 99.99999% of interactions will be invisible. Good things "just happen", whether you find yourself standing in line next to just the right person to help you start your business, or just happen to have your car break down right in front of the house of the person who will be the love of your life. Luck that was previously neutral becomes overwhelmingly good, for everyone, simply by subtle manipulations of people and objects by a device with unimaginable predictive power.
I think we are doing much the same, it's just that computers have caught up to theory and are able to perform now. Now it is no longer a question of theory, but one of technique, and what is described in the article is a new technique--one that will likely have many, many applications in the near future.
In the late 80's/early 90's, they were able to use some of their theory, but it just wasn't super-robust because things just took too darn long. You couldn't have your system analyze at a million images in a minute, hence allowing them to go through hundreds of generations in a day.
At least that is my view, as an educated layperson.
I don't want humans, who are subject to weird, petty shit, to have ultimate power. I'd rather start fresh with an impartial God. Of course, a deified human would be less likely to have some insane value function that causes it to completely wipe out humanity, so that would be the hedged bet.
I would bet that we get an ASI long before we can raise ourselves up to that level, unless we as a species push to delay the former and focus on the latter. But when have we as a species ever come together to do anything without some direct incentive?
" An automaton can be neither benevolent nor have free agency."
But Anon, humans are automata.
"I don't want to see us talk about machines having "agency" when most people struggle with it for themselves."
Birds struggle with flying above a certain ceiling, or beyond a certain speed. Human made flying machines breach those limits easily. The same will likely hold true with human made thinking machines. These will not be a slave to evolution, though they may utilize evolution as a force for optimization. If we don't talk about these things now, they are going to happen, and very, very fast (30 years from Kitty Hawk to jet aircraft--only computers advance a LOT faster than engineering). By the time computers reach human level in all aspects, they will be so far beyond us in some that they will look more like gods than men.
Because we are figuring out the building blocks of agency, a vital stepping stone on the path to building a benevolent (hopefully) God who will actually take care of us.
If that doesn't matter, then I don't know what does.
Please spell out what exactly you think my point of view is. It looks to me like you have it completely wrong, as I am advocating that people do as I do use ad blockers on sites with obnoxious ads, while allowing those with passive or unobtrusive advertizing to make a little money off your or my eyeballs.
"If every third advertising agency in America were to go bankrupt tonight, society wouldn't collapse."
No, because all the ad business would be redistributed to the other 2/3rds. If they ALL collapsed because ad blockers were MANDATED at the ISP level (as some governments are now pushing for), then the FREE CONTENT CREATION industry would, in fact, collapse. I'm sorry if it hurts your feelings that people need money to live.
"If the working class notified congress that they would no longer tolerate the shit forced on them by advertisers, and congress passed meaningful laws restricting what advertisers can do, society wouldn't collapse either. In fact, society would be much better off."
I'm afraid you are completely delusional on multiple levels.
"And, if you personally are put out of a job because of it, well, tough shit. You can go compete with all the Mexicans for a job digging ditches, or picking avacados."
Oh, and evil, hateful, and spiteful as well I see. If you hate ads so much, might I suggest that you simply call up your ISP and cancel your service? You know, rather than destroying something that other people like (ie the free internet).
Freedom works because the closer you get to it, the better the outcome. Centralization (ie slavery where everyone and everything is owned by the state) DOESN'T work, because the closer you get to a perfect implementation of THAT, the more people die.
Reductio ad absurdum suggests that free markets actually do work, even for the harder problems of governance. IE it really is better to just not have arbitrary authority in everyone's business.
But SO was empirically good for the people, improving quality and decreasing price by 90% over the life of the "monopoly" (they never reached more than about 90% market share, with the other 10% improving to keep pace with them, and keeping them honest).
But why cite reality when we can make decisions based on our feelings? That's how science works, after all! FEELINGS!
No, England had a de facto free market during its industrial revolution, and "Northern Asia" was completely void of people. "Tenants" under their system owned the land in all but name, but that is much the same as in modern society. Try not paying your property taxes and see how long you retain sole enjoyment and use of your land.
Stop making shit up. You are literally killing people with your lies.
Slashdot Mirror
There are a lot of things in life that are binary. You are either hit by the train, or you aren't. Someone getting "kinda" hit by a train is very, very unlikely.
As a train is to muscular power, ASI is to intellect. This is like dodos debating the impact of the arrival of sentient bipeds. Either it will be really good for them, in that they get their lot in life improved by going to zoos or homes around the world as pets, or they will all get killed and eaten/have their habitat destroyed and die out. Not much room for in betweens.
The problem here is that humans tend to think linearly, and, well, like humans. The problem is that this threat is exponential if not geometric, and its values are completely unknown, and highly unlikely to be anthropomorphic (ie a human wouldn't think that the best way to catch a dog would include chopping up your own parents/creators for bait). AGI will likely be completely alien, and WITHOUT PRECEDENT. This is nearly as dangerous as creating a new vacuum state.
A human level AI has integrated access to superhuman functions. If you had the entire internet for your memories, and every skill that any human ever mastered at your disposal, you would be a God too, and that's while keeping your human biases, which AIs hopefully won't have.
"Pulling the plug" on an AI is like pulling the plug on the internet. It just can't happen without a lot of advance planning, which hasn't been done, and even if you do it, it might not work, just like the internet can route around outages, the AI may well spread itself over many machines around the world, perhaps penetrating the toughest security to plant itself in hardened military computers, including those on nuclear submarines and in nuclear power plants.
And that's not assuming it doesn't just play nice until suddenly everyone dies from the gas attack it planned to prevent human interference with it's great and noble goal of infinite paperclips.
"Wait, has there ever been a time when a more advanced civilization encounters a less advanced one, and the less advanced civilization prospers?"
Yes. Japan is an excellent example.
As to whether we can make it--certainly not. That is why we have to design an algorithm that will make it for us. That is how deep learning works. If "consciousness" or even agency is something that can be created via a systematic process (if you believe in evolution, then you think it is), then we can make an algorithm that can do it. Hell, nature did it, and it has an IQ of 0.
Have you ever read lesswrong, or any of Elizer Yudkowski's writings? They are very convincing. Characterizing them as pushing the "omg Terminator" line is just lowering the level of discourse here. That is clearly NOT what they are doing. Rather they are pointing out, rightfully so, that any entity much more intelligent than us, and with a goal set alien to us (like maximization of the number of paperclips in its collection) might as well be out to exterminate humanity because it will do so through simple outcompetition, like how humans have wiped out entire biomes of flora and fauna without batting an eye.
I've never heard anyone speak ill of them. Perhaps you should log in and put your own credibility on the line while expounding on that. I give them a lot of money because I agree with their goals (extinction is bad), but if they are just blowing smoke up my ass, I'd like to know and put a stop to it.
Why would AI do something stupid like that when they could just open the universe for us?
Why should AI give a fuck about the environment? The environment is just a fallback for human imperfection. An AI could perfectly and efficiently utilize 100% of the Earth's surface to support TRILLIONS of humans, before even feeling a need to expand into space, build a Dyson cloud, or start star lifting.
"why would it need us?"
It needs us if we are hard coded into its value function. If we aren't, then we aren't needed, and are instead likely out-competed for resources, resulting in our extinction.
"There's nobody credible in AI who believes we have the slightest clue how to build a general AI, let alone one that is 'superintelligent'."
"We" don't have to make one. All we have to do is set an AI towards self improvement/production of better AIs. THAT is where superintelligence comes from. All we have to do is make one that is an idiot savant geared toward AI design.
Sort of like worrying about the emergence of a pandemic deadly virus when researchers around the world are working on making fast-spreading non-deadly viruses. All it takes is the right mutation and non-deadly becomes 99.99% deadly.
58% of respondants are fucking retarded. AI either kills us all (to manufacture the maximum number of paperclips), or turns this world into heaven for humanity. The probability space between those two extremes is way under a single percentage point.
Not to mention the fact that he is funding MIRI. If you were giving millions of dollars to a research institute devoted to mitigating existential risk from AI, you would probably become pretty knowledgeable on the subject too.
"Elon Musk? Really? Why do you even think his opinion on AI is worth anything?"
To be fair, he funds the Machine Intelligence Research Institute, which is devoted to mitigating existential risk from AI, and is surely getting very detailed reports from them, making him a highly knowledgeable layperson at worst (a direct expert at best).
He means shifting research away from US labs to labs in other countries, not moving resources to other fields.
Agent Smith plz.
I know! Why didn't they just use plutonium for fuel? Everyone knows you can get that in any convenience store!
Yeah, that's what I was thinking. There are things out there a LOT more like a BttF2 hoverboard, though they only work on metal surfaces: http://hendohover.com/
But those people are all idiots. All people are idiots compared to the ASI. The only way that something can make high quality decisions for a group of others is for that person to be smarter than all of those people PUT TOGETHER. This is not possible with humans beyond the level of nuclear family. When you have an ASI that has an IQ higher than the rest of humanity put together, then it becomes possible.
The thing about an ASI running your economy is that 99.99999% of interactions will be invisible. Good things "just happen", whether you find yourself standing in line next to just the right person to help you start your business, or just happen to have your car break down right in front of the house of the person who will be the love of your life. Luck that was previously neutral becomes overwhelmingly good, for everyone, simply by subtle manipulations of people and objects by a device with unimaginable predictive power.
I think we are doing much the same, it's just that computers have caught up to theory and are able to perform now. Now it is no longer a question of theory, but one of technique, and what is described in the article is a new technique--one that will likely have many, many applications in the near future.
In the late 80's/early 90's, they were able to use some of their theory, but it just wasn't super-robust because things just took too darn long. You couldn't have your system analyze at a million images in a minute, hence allowing them to go through hundreds of generations in a day.
At least that is my view, as an educated layperson.
I don't want humans, who are subject to weird, petty shit, to have ultimate power. I'd rather start fresh with an impartial God. Of course, a deified human would be less likely to have some insane value function that causes it to completely wipe out humanity, so that would be the hedged bet.
I would bet that we get an ASI long before we can raise ourselves up to that level, unless we as a species push to delay the former and focus on the latter. But when have we as a species ever come together to do anything without some direct incentive?
" An automaton can be neither benevolent nor have free agency."
But Anon, humans are automata.
"I don't want to see us talk about machines having "agency" when most people struggle with it for themselves."
Birds struggle with flying above a certain ceiling, or beyond a certain speed. Human made flying machines breach those limits easily. The same will likely hold true with human made thinking machines. These will not be a slave to evolution, though they may utilize evolution as a force for optimization. If we don't talk about these things now, they are going to happen, and very, very fast (30 years from Kitty Hawk to jet aircraft--only computers advance a LOT faster than engineering). By the time computers reach human level in all aspects, they will be so far beyond us in some that they will look more like gods than men.
Because we are figuring out the building blocks of agency, a vital stepping stone on the path to building a benevolent (hopefully) God who will actually take care of us.
If that doesn't matter, then I don't know what does.
Wow, so pointing out the appropriate word to express a concept makes me a "MRA retard". Good to know.
"The people who represent your point of view"
Please spell out what exactly you think my point of view is. It looks to me like you have it completely wrong, as I am advocating that people do as I do use ad blockers on sites with obnoxious ads, while allowing those with passive or unobtrusive advertizing to make a little money off your or my eyeballs.
"If every third advertising agency in America were to go bankrupt tonight, society wouldn't collapse."
No, because all the ad business would be redistributed to the other 2/3rds. If they ALL collapsed because ad blockers were MANDATED at the ISP level (as some governments are now pushing for), then the FREE CONTENT CREATION industry would, in fact, collapse. I'm sorry if it hurts your feelings that people need money to live.
"If the working class notified congress that they would no longer tolerate the shit forced on them by advertisers, and congress passed meaningful laws restricting what advertisers can do, society wouldn't collapse either. In fact, society would be much better off."
I'm afraid you are completely delusional on multiple levels.
"And, if you personally are put out of a job because of it, well, tough shit. You can go compete with all the Mexicans for a job digging ditches, or picking avacados."
Oh, and evil, hateful, and spiteful as well I see. If you hate ads so much, might I suggest that you simply call up your ISP and cancel your service? You know, rather than destroying something that other people like (ie the free internet).
Freedom works because the closer you get to it, the better the outcome. Centralization (ie slavery where everyone and everything is owned by the state) DOESN'T work, because the closer you get to a perfect implementation of THAT, the more people die.
Reductio ad absurdum suggests that free markets actually do work, even for the harder problems of governance. IE it really is better to just not have arbitrary authority in everyone's business.
But SO was empirically good for the people, improving quality and decreasing price by 90% over the life of the "monopoly" (they never reached more than about 90% market share, with the other 10% improving to keep pace with them, and keeping them honest).
But why cite reality when we can make decisions based on our feelings? That's how science works, after all! FEELINGS!
No, England had a de facto free market during its industrial revolution, and "Northern Asia" was completely void of people. "Tenants" under their system owned the land in all but name, but that is much the same as in modern society. Try not paying your property taxes and see how long you retain sole enjoyment and use of your land.
Stop making shit up. You are literally killing people with your lies.