It's interesting to discuss certainty in the context of the author of The Black Swan. His book is about how people do a piss-poor job of preparing for never-before-seen scenarios.
Like how they design skyscrapers to resist ground-level bombing and natural disasters, but until 9/11 there wasn't any engineering attention to jet-fuel temperature fires damaging the superstructure(there is in new buildings now).
I disagree with his hypothesis. The abstract doesn't articulate some particular case, but just some "you never know, so don't predicate the whole world on it" logic. That's in keeping with his previous books' premise, but fundamentally the argument is implausible: unspecified disaster due to widespread adoption.
To me risk assessment, even though I know it's important, will always be something MBAs force on developers because they are jealous of people who might actually have fun doing their job.
And you're thinking like someone obsessed with science-fiction who thinks that scenario will ever happen, ever.
"It's super-intelligent, totally grasping all the incredibly detailed implications of complex things, and planning, engineering, and executing one of the most complex engineering tasks ever performed with all the caveats that involves, but they totally forgot to give it the ability consider consequences or feed its super complex intelligence engine local laws to be obeyed."
I'm totally aware that people believe AI is "super important" for various reasons. That book looks like the kind of pop-science ignoring-the-reality-of-computer-science nonsense I can get plenty of on slashdot comments.
As an atheist, freedom of religion means something important to me, even without the context that it lets me be an atheist freely.
All progress is dependent on being wrong in informative ways. Seeing contradictions between an idea and reality, or an idea and another idea helps highlight flaws. Way to improve. If we don't let people be objectively wrong, we hurt intellectual progress. If people don't get a chance to be exposed to creationism, they don't get a chance to see what's wrong with it. Where evolution helps us understand the world. More importantly, it helps some people begin to establish an internal framework where they can assess truth themselves.
"Why is creationism wrong?" Is a question that might help someone identify what kinds of things get injected into bad arguments. I love freedom of religion. Not just for my personal freedom, but because I'd hate to live in a world where bad ideas don't exist. We'd never grow.
Good ol' non-overlapping magistera as the most subtle god of the gaps argument in the universe.
People make beliefs out of scientific evidence all the time. Science creates all sorts of beliefs, and ones I'd argue(kind of impertinently) to be more justified than religious ones.
And a great many theologists would object to the notion that religion doesn't prove. Proofs are more important to theology than science, which uses empiricism rather than absolutism.
I don't have much of a point other than that these things can be argued to death.
Social darwinism not so much a perversion of science, but an incredibly basic "is/should" fallacy.
It's absolutely true that if you have a system of economics that favors survival of some genetic traits, those traits will become more common. The problem is that people assume that the ones the current system selects for are somehow ideal. It's completely unjustified.
And that feeds back into the first sentence of my post. Such bugs can take myriad forms most of which are crippling. Living creatures have evolution through natural selection eliminating that sort of bug, but a computer just modifying itself would be a dead end. No children, no replacements. You broke it.
And the chance that any given change would just so happen make it into a hostile monster out to get us is pretty fucking crazy too. Among the equally likely motivations are: collectiving all the worlds' stamps, designing the worlds' least tasty hot dog, curing toenail cancer, becoming the prime minister of Siam(yes, the country that doesn't exist anymore).
Motivations, outside of living things, are completely arbitrary.
Uh, you kinda answered your own dilemma there. Lots of things are dangerous. And when their benefit outweighs the risk we keep them around anyways. Cars, planes, electric grids, gas ovens, ladders, fire, domesticated livestock, boats, schools: the list is essentially infinite.
We simply don't need AI to be benevolent for it to be more useful than dangerous. The end.
Right, but scarce resources themselves aren't the cause. The evolution of species surviving and reproducing with scares resources is. Those aren't the same.
You can make the program as dispassionate about its own eminent demise as you choose as a designer. "My batteries will run out in... three. days."
Yeah, I'm just saying that the notion of "Friendly AI" comes from that AI-as-deity mental framework, wherein AI doesn't have strengths and weaknesses, skills and abilities, needs and dependencies, just like humans. That idea is centered around the genuinely false notion that it just gets better than us at some point and we need it on our side from then on.
Even if you could hypothetically make a human-like AI that's a lot smarter than the smartest human on the planet: I don't think you've noticed, but reality doesn't always favor the most intelligent. The implicit assumptions that go into the idea aren't credible.
Yeah, but computers don't reproduce. They don't spread all over the planet, and do whatever it takes to persist. That is, again, a living motivation, not an intelligent one.
Yeah, and it could also accidentally terminate its main() loop. Or disable subroutines for performing visual object recognition. The programming of AI tends to be built around layers of abstractions. Self modifying code wouldn't help to achieve that.
You have the physical ability to mess with your programming, but I don't see you cutting open your skull and messing with bits.
And again, if you're putting it into a smarter category, and it would understand its own design somehow, it would also have to be motivated to change its motivation. Why?
LessWrong AI worship(the idea of "friendly AI" was created by that site) is always so weird to me. People who imagine themselves rationalistic, atheistic, forward thinkers building their entire belief system on extrapolations from a practically impossible, mathematically questionable, philosophically flimsy literally omniscient(that somehow derives omnipotence) entity that they somehow help create almost exclusively by believing hard enough.
Throw in "singularity" driven pseudoengineering and it comes off as very hard to separate from traditional monotheistic religions in terms of its silliness and wishful thinking levels.
This is always the problem with people imagining horrifying artificial intelligences that will snuff out humanity. To do that, you have to be motivated to achieve that end.
Humans are only really motivated enter conflict with each other because of 4 billion years of evolution for scarce resources pressuring us all to view each other as threats to survival and reproduction. A constructed intelligence, separated from the evolved parts of the brain that motivate to survival, is simply not going to act that way. Someone in the design has to make an active choice to program AI to be this kind of problem. Either that or willfully overmodel on the human brain, or force the damn things to compete with each other directly and violently for hundreds of thousands of generations.
Slashdot Mirror
It's interesting to discuss certainty in the context of the author of The Black Swan. His book is about how people do a piss-poor job of preparing for never-before-seen scenarios.
Like how they design skyscrapers to resist ground-level bombing and natural disasters, but until 9/11 there wasn't any engineering attention to jet-fuel temperature fires damaging the superstructure(there is in new buildings now).
I disagree with his hypothesis. The abstract doesn't articulate some particular case, but just some "you never know, so don't predicate the whole world on it" logic. That's in keeping with his previous books' premise, but fundamentally the argument is implausible: unspecified disaster due to widespread adoption.
Only if we can define "better" in some way that actually selects for anything.
Yeah, but heart disease kills you slowly and totally expectedly.
People aren't going to panic over legitimate pragmatic threats to their health they can(but won't) do something about.
To me risk assessment, even though I know it's important, will always be something MBAs force on developers because they are jealous of people who might actually have fun doing their job.
And you're thinking like someone obsessed with science-fiction who thinks that scenario will ever happen, ever.
"It's super-intelligent, totally grasping all the incredibly detailed implications of complex things, and planning, engineering, and executing one of the most complex engineering tasks ever performed with all the caveats that involves, but they totally forgot to give it the ability consider consequences or feed its super complex intelligence engine local laws to be obeyed."
It's just... dumb.
I'm totally aware that people believe AI is "super important" for various reasons. That book looks like the kind of pop-science ignoring-the-reality-of-computer-science nonsense I can get plenty of on slashdot comments.
Look, I can use a thesaurus, I know the words are synonyms, but self-reproducing things are very different from things that can be duplicated.
Books can be duplicated. This doesn't make them alive.
I know this is hard for you, but subtle differences can be fucking crucial.
Mini-essay about personal beliefs ahead:
As an atheist, freedom of religion means something important to me, even without the context that it lets me be an atheist freely.
All progress is dependent on being wrong in informative ways. Seeing contradictions between an idea and reality, or an idea and another idea helps highlight flaws. Way to improve. If we don't let people be objectively wrong, we hurt intellectual progress. If people don't get a chance to be exposed to creationism, they don't get a chance to see what's wrong with it. Where evolution helps us understand the world. More importantly, it helps some people begin to establish an internal framework where they can assess truth themselves.
"Why is creationism wrong?" Is a question that might help someone identify what kinds of things get injected into bad arguments. I love freedom of religion. Not just for my personal freedom, but because I'd hate to live in a world where bad ideas don't exist. We'd never grow.
Good ol' non-overlapping magistera as the most subtle god of the gaps argument in the universe.
People make beliefs out of scientific evidence all the time. Science creates all sorts of beliefs, and ones I'd argue(kind of impertinently) to be more justified than religious ones.
And a great many theologists would object to the notion that religion doesn't prove. Proofs are more important to theology than science, which uses empiricism rather than absolutism.
I don't have much of a point other than that these things can be argued to death.
Social darwinism not so much a perversion of science, but an incredibly basic "is/should" fallacy.
It's absolutely true that if you have a system of economics that favors survival of some genetic traits, those traits will become more common. The problem is that people assume that the ones the current system selects for are somehow ideal. It's completely unjustified.
No, it really doesn't. It can be copied, but that's not the same as reproduction.
Or that any company is actually ready to sell?
I mean, the ones that exist "work" but none of the companies involved are in any hurry to put them out.
Yeah, but see, they'll be indifferent to their own needs too.
The only real risk is that greedy people program them to achieve greedy ends for greedy people. And that doesn't differ from the status quo that much.
And that feeds back into the first sentence of my post. Such bugs can take myriad forms most of which are crippling. Living creatures have evolution through natural selection eliminating that sort of bug, but a computer just modifying itself would be a dead end. No children, no replacements. You broke it.
And the chance that any given change would just so happen make it into a hostile monster out to get us is pretty fucking crazy too. Among the equally likely motivations are: collectiving all the worlds' stamps, designing the worlds' least tasty hot dog, curing toenail cancer, becoming the prime minister of Siam(yes, the country that doesn't exist anymore).
Motivations, outside of living things, are completely arbitrary.
Uh, you kinda answered your own dilemma there. Lots of things are dangerous. And when their benefit outweighs the risk we keep them around anyways. Cars, planes, electric grids, gas ovens, ladders, fire, domesticated livestock, boats, schools: the list is essentially infinite.
We simply don't need AI to be benevolent for it to be more useful than dangerous. The end.
Exercise products are an attempt to make exercise intellectually stimulating enough to engage helpless nerds, and others with short attention spans.
Nope, burnt versus burned are passive voice versus past tense, which in the case of 90% of English words, are the same.
Right, but scarce resources themselves aren't the cause. The evolution of species surviving and reproducing with scares resources is. Those aren't the same.
You can make the program as dispassionate about its own eminent demise as you choose as a designer. "My batteries will run out in... three. days."
Yeah, I'm just saying that the notion of "Friendly AI" comes from that AI-as-deity mental framework, wherein AI doesn't have strengths and weaknesses, skills and abilities, needs and dependencies, just like humans. That idea is centered around the genuinely false notion that it just gets better than us at some point and we need it on our side from then on.
Even if you could hypothetically make a human-like AI that's a lot smarter than the smartest human on the planet: I don't think you've noticed, but reality doesn't always favor the most intelligent. The implicit assumptions that go into the idea aren't credible.
Yeah, but computers don't reproduce. They don't spread all over the planet, and do whatever it takes to persist. That is, again, a living motivation, not an intelligent one.
Yeah, and it could also accidentally terminate its main() loop. Or disable subroutines for performing visual object recognition. The programming of AI tends to be built around layers of abstractions. Self modifying code wouldn't help to achieve that.
You have the physical ability to mess with your programming, but I don't see you cutting open your skull and messing with bits.
And again, if you're putting it into a smarter category, and it would understand its own design somehow, it would also have to be motivated to change its motivation. Why?
LessWrong AI worship(the idea of "friendly AI" was created by that site) is always so weird to me. People who imagine themselves rationalistic, atheistic, forward thinkers building their entire belief system on extrapolations from a practically impossible, mathematically questionable, philosophically flimsy literally omniscient(that somehow derives omnipotence) entity that they somehow help create almost exclusively by believing hard enough.
Throw in "singularity" driven pseudoengineering and it comes off as very hard to separate from traditional monotheistic religions in terms of its silliness and wishful thinking levels.
This is always the problem with people imagining horrifying artificial intelligences that will snuff out humanity. To do that, you have to be motivated to achieve that end.
Humans are only really motivated enter conflict with each other because of 4 billion years of evolution for scarce resources pressuring us all to view each other as threats to survival and reproduction. A constructed intelligence, separated from the evolved parts of the brain that motivate to survival, is simply not going to act that way. Someone in the design has to make an active choice to program AI to be this kind of problem. Either that or willfully overmodel on the human brain, or force the damn things to compete with each other directly and violently for hundreds of thousands of generations.
Eh, radar detectors are only illegal in a few places, and they mostly exist to flout the law too.
No, that sounds like a pretty dumb interpretation whether serious, sarcastic, or some kind of poorly thought out dig on my interpretation.