Slashdot Mirror

How do you shut off a sufficiently intelligent Artificial General Intelligence? This is a harder problem then you might think. See for example: https://intelligence.org/2017/... The technical term is Corrigibility, and there is no solution yet.

That thing does not exist. Algorithms are improving glacially slow or not at all at this time.

Your comment is at odds with all of the research I could find on the topic. For instance this paper estimates that between 33%-50% of recent improvements in AI come from algorithmic improvements. In a quick Google search I couldn't find any research papers claiming hardware improvements are the only source of new breakthroughs in AI or any other computation heavy domain.

The problem is that we don't know how to make friendly AI. As in at some point, Artificial Intelligences will be able to beat humans at any task, at which point, how do you make sure that they don't destroy humanity (possibly through indifference). Even if you don't care about humanity, how do you make sure they do something interesting with the universe?

Various articles:
Stuart Armstrong's book Smarter than us discusses what happens when machines are smarter than humans:
https://intelligence.org/smart...
http://jjc.freeshell.org/Smart...
Bill Joy's article Why the Future doesn't need us on the dangers of robotics:
https://www.wired.com/2000/04/...
Tim Urban's article on superintelligence:
http://waitbutwhy.com/2015/01/...
http://waitbutwhy.com/2015/01/...

My favorite new tech book is Rationality from AI to Zombies by Eliezer Yudkowsky:
https://intelligence.org/ratio...
(or as a usable but not always perfect TeX document: https://github.com/jrincayc/ra... )

We are at least 50 years off from strong AI as in human level common sense about the world.

How certain of that are you? Are you willing to bet all of humanity on that? Experts disagree a lot about when we will have strong AI https://intelligence.org/2013/05/15/when-will-ai-be-created/. If it turns out to be sooner then this isn't a good situation.

The emerging Effective Altruism movement is full of young people figuring out how to make to make charitable donations go further. The difference in impact between typical charities and carefully targeted intervention spans many orders of magnitude. Top ranked causes tend to be in the areas of global health and catastrophic risk, particularly AI risk. A few links:

GiveWell - detailed evaluation of top charities
Giving What We Can - people who have pledged to give 10% or more of their income to the most effective causes they can find
Back of the Envelope Guide to Philanthropy - my own website; some very rough math-geek evaluations of charitable endeavors
The Most Good You Can Do - a recent book on the Effective Altruism movement by Peter Singer
Machine Intelligence Research Institute - MIRI focuses on AI risk

For staying in touch there is EffectiveAltruism.org, supposedly a very active FaceBook group (disclaimer: I don't use FaceBook), and upcoming effective altruism conferences at the Googleplex in Mountain View, in Oxford, and in Melbourne.

I would recommend that anyone thinking about machine intelligence read Smarter Than Us by Stuart Armstrong. You can get pay what you want for it from https://intelligence.org/smart... or since it is CC BY-NC-SA 3.0, you can also just download it https://drive.google.com/file/...

The book contains the following summary:

1. There are no convincing reasons to assume computers will remain unable to accomplish anything that humans can.
2. Once computers achieve something at a human level, they typically achieve it at a much higher level soon thereafter.
3. An AI need only be superhuman in one of a few select domains for it to become incredibly powerful (or empower its controllers).
4. To be safe, an AI will likely need to be given an extremely precise and complete definition of proper behavior, but it is very hard to do so.
5. The relevant experts do not seem poised to solve this problem.
6. The AI field continues to be dominated by those invested in increasing the power of AI rather than making it safer.

The only one of those statements I have much doubt about is 4. Even if the AIs are safe, they still probably will not be under human control.

The Machine Intelligence Research Institute believes very much as you do. They are working toward the long-term goal of developing (first, they need to *define*) "Friendly AI", which is to say, AI with the best interests of humanity in mind. If you want to contribute, they can use all kinds of help, ranging from financial donations (they are a 501(c)(3) nonprofit) to more researchers to join the team...

Even if they do, that doesn't mean all is lost. I think it's pretty likely an AI might decide to take our well-being into its own "hands", if for no other reasons than those that motivate humans to feed pets and take them to the vet, and to preserve natural beauty.

The question then is: what does it actually do?

There are ways to combat humanity's "inhumanity" other than enslaving (or eliminating) us, and some of those approaches are pretty good. Universal cultural memes that combat some of our worst tendencies, for example, would be a pretty good start, and there's already lots of support that such things work; human slavery itself, for example, has gone from an accepted and even expected fact of life to being typically viewed as abhorrent in just a few hundred years. A smarter-than-human AI could aid us in other ways too, such as deriving solutions to some of our problems of self-governance; the old joke about "democracy is the worst form of government, except for every other kind we've tried" implicitly reminds us that there are other ways which may be better yet. Maybe a friendly AI is one that offers guidance but will not itself lead, and helps us find our own way into superior future societies. Maybe it *is* a dictator, instead, but provides and enforces only the highest-level laws and leaves the minutiae to us.

The solution isn't likely to be found in a Slashdot discussion thread, but there is in fact a group of extremely smart researchers who focus on this topic: MIRI. If you see a machine revolt as a threat, and/or want to see AI done right, consider donating (or applying, if you've got what it takes to contribute there) to MIRI. Disclaimer: not related to the institute in any way aside from knowing of several members through their work.

The Machine Intelligence Research Institute (formerly known as the Singularity Institute) has a bunch of seriously smart people - AI researchers, behavior experts, etc. - working on figuring out how to avoid the doomsday scenarios you (and Musk) describe. The goal is "friendly AI"; a benevolent, or at least helpful, strong AI. If you believe (as I do) that AI is inevitable given the current progress of technology, then the MIRI is probably our best bet of surviving and benefiting from the technological singularity.

They need funding, though. Hey Musk, you want to put tiny part of those billions you've earned (I in no way deny that he's earned them) to work against this existential threat? Donate to MIRI and similar research groups, so those researchers can devote their working days to this stuff and more people can be brought on board!

It actually doesn't surprise me that he's concerned about this; SpaceX is nominally focused on mitigating the existential risk of a cataclysm on Earth (by getting a sustainable human population off of it). Of the two things, I think it's both more likely that a malevolent or unconcerned AI would wipe out humanity than that we'd manage to do ourselves in that badly, and that we can offset this sooner and more effectively than we can export enough of humanity to produce a self-sufficient extraterrestrial colony.

As a Singularitan, I recognize that super-AI may turn out to be impossible. After all, we don't have a theory that proves it possible yet. However, in the case that it is possible, the creation of the super-AI will be the single most important event in our observable universe. Thus it is worthwhile to spend some effort in getting it right.