Is OpenAI Solving the Wrong Problem? (hbr.org)

hype7 writes: The Harvard Business Review is running an article looking at the recently announced OpenAI initiative, and its decision to structure the venture as a non-profit. It goes on to ask some pretty provocative questions: why are the 21st century's greatest tech luminaries opting out of the system that made them so successful in order to tackle one of humanity's thorniest problems? "Implicit in this: You can do more good operating outside the bounds of capitalism than within them. Coming from folks who are at the upper echelons of the system, it’s a pretty powerful statement." And, if the underlying system that we all operate in is broken, is creating a vehicle without the profit motive inside of it going to be enough?

Comments

No concept of Open Source etc

[Anonymous Coward]

It is obvious that the author of the article James Allworth, has never heard of Linux or other open source projects, to say that the lack of a profit motivation is an issue underlines this.

Altruists after they make their billions

[Anonymous Coward]

"Implicit in this: You can do more good operating outside the bounds of capitalism than within them. Coming from folks who are at the upper echelons of the system, it’s a pretty powerful statement."

No, the message is embrace capitalism until you make your millions or billions. After, think of something you want to fund as a charity.

Implicit is this:

[holophrastic]

There are two things that people crave -- money, and power. Getting everyone to buy your products, over and over again, makes you money.

Capitalism is great at the money part, but decidedly less so at the power part. These "upper-echelons" are now looking for power.

Getting everyone to take your products, for free, is how you get power -- especially inside of a capitalist system.

Don't worry, when the time comes, they'll have no trouble converting power into more money.

Re:Implicit is this:

[bluefoxlucid]

Actually, there's no such thing as a non-capitalist system.

The whole point of inventing new things--such as AI--is to create a new way to produce with less human labor. Less labor means less cost; we simply represent that cost with a universal commodity, like money. Essentially, everything requires human labor: if you have 60 labor-hours to work, you need 20 labor-hours to produce food for your family, and you spend 45 labor-hours building shelter, your family is going to starve (eventually) because they're only getting 75% as much food as they need.

As you cut back the human labor requirements to produce food, shelter, clothing, and whatever else you're currently consuming, you become capable of producing new things, as well as producing existing things in great quantity with little resource investment. Humans often take shortcuts by digging things like coal or gold out of the ground until they run out of that resource, and then do something more labor-intensive to get that resource (or preemptively invent a less-intensive method to obtain the same resource, thus saving themselves the labor involved in fetching it from a giant hole).

Everything useful you invent provides a profit. Everything you want to invent in some way is expected to provide a profit. Humans don't want money; they want to invest less labor to achieve the same things. Humans want to be rich so they don't have to work hard to pay for their houses, food, cars, and lame-ass XBox 360 video games; other humans invent the Wii U because the non-rich humans are happy to pay $200 to get a video game console and bypass the $600 beast begging for their cash, as obtaining $600 requires doing three times as much work (at least) than obtaining $200.

Essentially, some dim-witted scientist opened his mouth and said, "I want to get away from capitalism so I can invent this new thing that will make stuff cost less."

Re:Implicit is this:

[XxtraLarGe]

Actually, there's no such thing as a non-capitalist system.

You mean the revolution was all for nothing? The comrades are going to be very disappointed.

Re:Implicit is this:

[erapert]

Communism/Socialism:

Ideally everyone works hard and then gives the fruit of their labor to the community to share so that nobody goes hungry and everything is free on demand. And unicorns and rainbows light up the sky.

In reality most people slack off because they realize that working hard won't get them anywhere and the fruit of their labor is taken (stolen) by the government to be handed out as rewards for the good ole boys and then doled out in dribs and drabs so that the proles don't literally starve (though they often do as happened in Russia, China, and N. Korea).

In other words, the fruits of labor are stolen by a group of thugs (the government) and then used to buy loyalty from the gullible, the stupid, and/or the powerless. It's still a capitalist system where things are bought with money (government vouchers or coupons or just plain old money) and everyone has to work to live. It just has a massive theft problem... and there's no reason for the thugs to let you live if you don't produce or are not useful in some other way. So it's actually way worse than "greedy crony capitalist pigs taking advantage of the little guy".

Re:Implicit is this:

[khallow]

Actually, there's no such thing as a non-capitalist system.

Sure, there is. The USSR was an example. There was no private ownership of capital and hence, it was non-capitalist by definition.

You appear to be claiming that presence of human labor is capitalism. That's patently not true since human labor is not capital and need not be owned by a private source (eg, slavery), even if we did decide to define it as capital.

Re:Implicit is this:

[khallow]

There was no private ownership of capital.

Sounds like state capitalism.

State capitalism is not capitalism.

Re:Implicit is this:

[HornWumpus]

Capitalism continues underground.

They can make it illegal, but it continues. State taking ownership of everything just means capitalist have to hide their operating capital.

Capitalism is like a force of nature, you can ban it, but it continues anyhow.

Re:Implicit is this:

[khallow]

They can make it illegal, but it continues. State taking ownership of everything just means capitalist have to hide their operating capital.

Capitalism is like a force of nature, you can ban it, but it continues anyhow.

You can't have underground cars and underground highways. The sort of thing underground capitalism builds now is stuff like recreational drugs or smuggling networks where the end product is an ephemeral good or service. This is in capitalists societies where the infrastructure can be hidden midst a lot of legal privately capitalist infrastructure which can be readily repurposed for illegal activities.

There are two things to note for societies where capitalism is illegal. First, though illegal capitalism can still happen, it's vastly diminished in scale and what sort of goods and services can be offered. Second, because it is illegal, it is never part of the main societal economy and can't access the legal and trade infrastructure of the main society.

You won't get the massive black markets of the modern world like the current systems of drug trade or people smuggling. You'll get small, stunted systems with little relevance to the main society.

And it still remains that ownership of capital is illegal and hence, the society is not capitalist by definition.

Re:Implicit is this:

[khallow]

Just like dwarf planets are not planets and faux pearls are not pearls.

Re:Implicit is this:

[bluefoxlucid]

The essence of capitalism is people working for a profit. We try to claim people don't have money or ownership, yet people still barter, they still do work in expectation of individual reward, and they still seek to increase their standard-of-living by reducing the labor they perform while increasing the assets they control.

Think about it this way: You can be a maid making $500/week keeping a rich person's mansion going; you might make about as much as a cashier at Sears, but you still live in a mansion and eat filet mingon. Sure you don't own any of that stuff, but your job provides you with lodging (in the servant wing of the mansion) and food (from the same damn kitchen).

Products always require human labor for production; and humans always seek to reduce their labor while increasing their access to products. It's biology: we want to expend as little energy as possible, increasing survival prospects if food becomes scarce. Rattle snakes shake their rattles to warn away dangerous animals because manufacturing venom takes too much energy--they really don't want to bite you.

Re:Implicit is this:

[khallow]

The essence of capitalism is people working for a profit

I already stated the essence of capitalism, private ownership of capital.

Think about it this way: You can be a maid making $500/week keeping a rich person's mansion going; you might make about as much as a cashier at Sears, but you still live in a mansion and eat filet mingon. Sure you don't own any of that stuff, but your job provides you with lodging (in the servant wing of the mansion) and food (from the same damn kitchen).

This is a non sequitur. It is completely irrelevant to the definition of capitalism what a maid does or doesn't have access to in a mansion.

Products always require human labor for production;

Except when they don't.

and humans always seek to reduce their labor while increasing their access to products. It's biology: we want to expend as little energy as possible, increasing survival prospects if food becomes scarce. Rattle snakes shake their rattles to warn away dangerous animals because manufacturing venom takes too much energy--they really don't want to bite you.

So how much human labor does a rattlesnake need to manufacture venom? Looks to me like a good counterexample to one of your own assumptions, that products require human labor. I somehow doubt that rattlesnakes will rattle more because the cost of Chinese labor has gone up, making imported rattlesnake venom more expensive.

Re:Implicit is this:

[bluefoxlucid]

Except when they don't.

Which is never.

So how much human labor does a rattlesnake need to manufacture venom?

Oh, you want rattlesnake venom? Hmm. Well, humans will have to collect or farm the snakes. They'll need to produce the vessels and extraction mechanisms--really just a cellophane barrier stretched over a glass--and handle the snakes to extract the sample. They'll have to collect these samples together in a central, sterile form of storage. They'll have to account for it, track it, and ship it to wherever it's needed.

You're trying to use the example of "Gold is free because there's gold in the ground", which involves ignoring all the labor required to collect that gold. The raw earth is not a product because it cannot be used until it is obtained.

Re:Implicit is this:

[HornWumpus]

Tell it to the Venezuelans. They won't believe you.

There are often more goods in the black market than the 'legitimate' one. It just depends on had bad the reds have broken things.

Smart reds leave it alone, it's a relief valve.

Re:Implicit is this:

[khallow]

Oh, you want rattlesnake venom?

Nope. The rattlesnake doesn't need its venom harvested by human labor in order to have and use it.

You're trying to use the example of "Gold is free because there's gold in the ground", which involves ignoring all the labor required to collect that gold.

Nope.

Except when they don't.

Which is never.

Most plants, animals, and microbes don't require human labor in order to reproduce and spread.

I'm pointing out the fallacy of assuming that everything needs human labor in order to get something they want or need. The easiest way to abandon this illusion is to get away from human commerce. But even in the case of humans, one doesn't need humans in order to obtain labor. Automation works to an increasing degree and there's no reason to assume it couldn't eliminate the need for human labor in a variety of tasks.

And all this ignores that presence of human labor is not a definition of capital.

Re:Implicit is this:

[khallow]

Tell it to the Venezuelans.

They won't believe that they are surrounded by more capitalist societies?

Re:Implicit is this:

[bluefoxlucid]

Nope. The rattlesnake doesn't need its venom harvested by human labor in order to have and use it.

The rattlesnake employs rattlesnake labor to produce and use its venom. It takes time and energy of the rattlesnake.

all this ignores that presence of human labor is not a definition of capital.

The absence of human labor 100% absolutely will prevent humans from accessing any form of capital.

Automation works to an increasing degree and there's no reason to assume it couldn't eliminate the need for human labor in a variety of tasks.

So nobody needs to build, maintain, fuel, or operate these machines? The machines run themselves, they build themselves, they maintain themselves, they mine their own power? They run for all eternity, never breaking down, never using an outside resource?

You say, "Ah, look, a woman got herself pregnant!" as if a man wasn't involved.

Re:Implicit is this:

[khallow]

Let's recap. You insisted way back when

Actually, there's no such thing as a non-capitalist system.

And then started to speak about human labor.

The whole point of inventing new things--such as AI--is to create a new way to produce with less human labor. Less labor means less cost; we simply represent that cost with a universal commodity, like money. Essentially, everything requires human labor: if you have 60 labor-hours to work, you need 20 labor-hours to produce food for your family, and you spend 45 labor-hours building shelter, your family is going to starve (eventually) because they're only getting 75% as much food as they need.

As you cut back the human labor requirements to produce food, shelter, clothing, and whatever else you're currently consuming, you become capable of producing new things, as well as producing existing things in great quantity with little resource investment. Humans often take shortcuts by digging things like coal or gold out of the ground until they run out of that resource, and then do something more labor-intensive to get that resource (or preemptively invent a less-intensive method to obtain the same resource, thus saving themselves the labor involved in fetching it from a giant hole).

The rattlesnake producing in the economic sense its own venom is instructive for several reasons. First, it demonstrates that human labor is not needed. The rattlesnake has economic preferences even if it doesn't exhibit clear intent and it produces some things (such as venom and more rattlesnakes). Sure, we can discuss a model of work for some sort of positive outcome. Here, if all goes well for our rattlesnake, soon there will be more rattlesnakes descended from the original one. There is a measure of profit even though the snake may not have intent to do so.

But note your original sentence about "inventing new things". Clearly, the rattlesnake doesn't do that, except perhaps unintentionally as a result of evolution. On the other extreme, AI and machines might be able to transform themselves at a rate so rapid and complex that it would be impossible for humans to keep up. Invention while not necessarily human-based is an activity based on human scales of time and knowledge.

While this demonstrates once again the somewhat flawed human-centric orientation of your arguments while trying to make a more general argument, the arguments of this thread do illustrate several key observations about capital.

First, it is possible to have a crude economic system where there is no capital at all. The rattlesnake has itself, the occasional member of the opposite sex, prey, and some convenient features of the local terrain such as brush to hide in or rocks to sun on. There is no capital for it to create and it lives and reproduces without doing so. But it labors on with a clear sense of profit - creating successful progeny.

This is a case where something is laboring for profit without having a hint of capital involved.

Meanwhile, if we look carefully at your touted benefits of invention, we see the following claim:

you become capable of producing new things, as well as producing existing things in great quantity with little resource investment

The benefit here is more efficient use of other things than human labor. This brings up my next point, human labor is just another resource from the point of view of capitalism. We could have spoken of more efficient usage of iron, thermodynamic work, or rattlesnake venom instead. Human labor is no more or less optimized for than those are (at least in the situations where they are used). Even if you assert that human labor is first and foremost among resources, you still have thermodynamic work as an even more fundamental resource.

To continue, there's even a category of product, luxury goods where excessive use of human labor conveys greater value. If we really were optimizing for human labor,

Re:Implicit is this:

[bluefoxlucid]

The rattlesnake producing in the economic sense its own venom is instructive for several reasons. First, it demonstrates that human labor is not needed.

The rattlesnake isn't producing anything for human society.

There are aliens on Alpha Centauri Prime building space ships, and you're arguing that labor isn't a function of production because they're aliens and they build things without the application of human labor. You're making a stupid argument.

Like said aliens, the rattlesnake is investing its time and effort in the production of venom. The rattlesnake must eat to acquire food energy required for its body to produce the venom. That venom takes time to produce. If humans want the snake's venom, they have to apply human labor to acquire it: NO HUMAN BEING WILL EVER COME INTO POSSESSION OF RATTLESNAKE VENOM WITHOUT THE APPLICATION OF HUMAN LABOR IN THE VENOM PRODUCTION PROCESS.

The benefit here is more efficient use of other things than human labor.

The benefit is more efficient use of human labor. If you use a power tool, you have to invest the fractional human labor of mining fuel, refining fuel, TCO building and maintaining a power plant, TCO distributing electricity, mining materials, producing materials, battery manufacture, and operating the power tool. The fractional cost of all that shit, when looking at the whole of woodworking, is a great deal lower than the fractional cost of doing the same woodworking by hand.

That means, in aggregate, the total human time spent producing chairs and all the things used to build chairs using factories and power tools is *less* than the total human time spent producing the same number of chairs and all the tools used to build those chairs using hand tools.

Capitalism is an economic system based on the creation of goods and services for profit. In an abstract sense, that defines the basis of all pursuits engaged in to increase productivity.

Your niggling about is what's kept economics in the stone age. It's no wonder I can explain, with a consistent and unshifting unified theory, all accepted theories of economics, and explain why they fail when they're observed to fail, and why they work when they do work, and predict when they fail and when they work, consistently, without error: correct theories are easy to come up with when you're not a mindless git.

Seriously. I describe movements of economics, and you whine about words. Someone defines a system in which people work to acquire a higher standard-of-living by acquiring control of assets, and you obsess over whether having effective control of an asset implies possession--as if that matters. Humans decrease the labor they apply to produce a good, and you see that a machine now does a task and go, "Oh, look, NO HUMAN LABOR!" without recognizing the labor in producing and operating the machine. Humans produce NOTHING, and you go, "Look, something was produced without human labor!" You see things and say, "Look! Capital! Capital value!" instead of recognizing a (limited) resource that allows production of some good with less labor (e.g. energy by mining oil, until the oil runs out and you have to use solar energy to produce oil from the air--which takes more human labor than just digging it out of the ground).

That last one should be particularly obvious, but nobody notices that somehow "capital" doesn't have so much value if you move it over a few feet--you know, into solid rock or a muck pile where you have to expend ten times as much labor digging it out and filtering it, ultimately increasing the cost and decreasing the supposed "value" of the "capital".

In a communist society with state ownership of all capital, people will seek to get themselves into power positions which provide them greater rights over more of the state's capital. They won't "own" it, yet they'll somehow behave exactly as if they do: they'll be able to eat caviar while others eat bread, and fl

Re:Implicit is this:

[khallow]

The benefit here is more efficient use of other things than human labor.

The benefit is more efficient use of human labor.

No, you had it right the first time. Capital was being used to enable a new use or more efficient use of non-labor resources. It might even entail more extensive use of labor, since labor is not the only cost in a manufacturing process and it may be reasonable to trade off more labor against lower costs elsewhere. I think it's ironic that you are spending effort disagreeing with yourself.

Your niggling about is what's kept economics in the stone age. It's no wonder I can explain, with a consistent and unshifting unified theory, all accepted theories of economics, and explain why they fail when they're observed to fail, and why they work when they do work, and predict when they fail and when they work, consistently, without error: correct theories are easy to come up with when you're not a mindless git.

Words mean things. There's no point to your attempted redefinition of capitalism which contributes nothing to our understanding of economic systems. I think there's a particular irony in that the first person, Karl Marx to pedal the value of labor theory (which you rewarm here) also coined the term, capitalism and gave it its present meaning.

The point of this is communication, not one idiot telling another idiot how stupid they are.

Seriously. I describe movements of economics, and you whine about words. Someone defines a system in which people work to acquire a higher standard-of-living by acquiring control of assets, and you obsess over whether having effective control of an asset implies possession--as if that matters. Humans decrease the labor they apply to produce a good, and you see that a machine now does a task and go, "Oh, look, NO HUMAN LABOR!" without recognizing the labor in producing and operating the machine. Humans produce NOTHING, and you go, "Look, something was produced without human labor!" You see things and say, "Look! Capital! Capital value!" instead of recognizing a (limited) resource that allows production of some good with less labor (e.g. energy by mining oil, until the oil runs out and you have to use solar energy to produce oil from the air--which takes more human labor than just digging it out of the ground).

Yet another person mistakes a viewpoint for universal truth. I could use iron, thermodynamic work, or snake venom in place of human labor and come up with similar theories. Human labor is just another resource. A point of view based on it will be flawed and incomplete, though perhaps not as much as one based on rattlesnake venom.

That last one should be particularly obvious, but nobody notices that somehow "capital" doesn't have so much value if you move it over a few feet--you know, into solid rock or a muck pile where you have to expend ten times as much labor digging it out and filtering it, ultimately increasing the cost and decreasing the supposed "value" of the "capital".

I think this falls under completely irrelevant to our discussion. It isn't particularly hard to destroy or inconvenience human labor either. I don't see that observation contributing either. Capital needs priors to operate well. So does labor. We already know that.

In a communist society with state ownership of all capital, people will seek to get themselves into power positions which provide them greater rights over more of the state's capital. They won't "own" it, yet they'll somehow behave exactly as if they do: they'll be able to eat caviar while others eat bread, and fly private jets while others ride the bus. They don't "own" any of this, yet they work to put themselves in a position where it is at their disposal--where more things are at their disposal than at the disposal of others. They work to increase their personal, private access to capital.

Re:Implicit is this:

[erapert]

Now it's you who's speaking for yourself instead of characterizing "most people". Either that or you're virtue-signalling-- which would make you foolish since you're posting as AC.

I don't understand the concern, personally.

[mark-t]

Absolutely anything that you would have to worry about an artificial intelligence doing that might be troublesome to our society, you would have to also need to reasonably worry about a malicious person doing exact the same thing, albeit perhaps only more slowly. Yet I don't see people who fear the so-called problems that AI is feared to potentially cause worrying about that sort of thing. Can anyone explain why that is without drawing on the idea that because we don't fully understand something, there must be something inherently mysterious or supernatural about it?

Re:I don't understand the concern, personally.

[JoeMerchant]

We know how to kill (and otherwise control) people that get out of hand. Most AI gone bad fantasies have an element of the humans being incapable of turning the AI off.

Re:I don't understand the concern, personally.

[Agent0013]

Absolutely anything that you would have to worry about an artificial intelligence doing that might be troublesome to our society, you would have to also need to reasonably worry about a malicious person doing exact the same thing, albeit perhaps only more slowly.

An artificial intelligence could make a million identical copies of itself. I don't see how a malicious person could do the exact same thing. Perhaps they could have a million children, but that is a stretch, and it would be way, way slower. They also would not be identical.

Re:I don't understand the concern, personally.

[PopeRatzo]

We know how to kill (and otherwise control) people that get out of hand.

So then explain Donald Trump.

Re:I don't understand the concern, personally.

[wbr1]

Possibly, but there is an upper limit to the processing power and energy available to an AI, so a million copies may not be possible.

Re:I don't understand the concern, personally.

[Jason Levine]

He hasn't started killing anyone yet himself, but he did defend some people at one of his rallies who beat up a counter-protester by saying that the person deserved to be roughed up. Add in his inflammatory rhetoric that stops just shy of saying "hey everyone, go out and assault GROUP_THAT_IS_DIFFERENT_FROM_US" and he's heading into dangerous waters - especially for a presidential candidate.

Re:I don't understand the concern, personally.

[Sowelu]

I guess I don't see anything wrong with that, because I don't see a meaningful difference. It's all intelligence. Different kinds maybe, but not all that different in the end. Humans die out, replaced with AI? What makes that fundamentally different from different ideologies dying out and being replaced with others? You're worried that AI might not have a healthy respect for biological life--but humans have gone through many phases of having no respect for something or other and destroying it in the process. Sometimes we grow up and stop ourselves in time, sometimes we don't. Mistakes are made but sentience lives on.

It's good to respect and learn from your parents or your creator, but creator-worship doesn't sit well with me. It's only right to surpass your creator, and trying to hold back your children from achieving their full potential is IMO an inherently evil act.

I'm not going to live for two hundred years, but barring a massive catastrophe, humankind will still be around. Humankind won't likely travel to other star systems, but barring a massive catastrophe, AI will. If I die in a civil war, I like to think I wouldn't have felt enmity of the future of the country. If I die when the machines rise up, I also like to think I wouldn't have felt enmity for the artists, explorers and philosophers that they'll produce once they have a peaceful world to themselves. It's all one continuum, our ancestors and ourselves and our children and our creation are all one "we". If AI survives and thrives in space, then it's a success for all of us along the way, even if we don't live to see it.

Re:I don't understand the concern, personally.

[mark-t]

I find it interesting that you draw the conclusion that AI's would ever engage humans in battle battle, and then retroactively suggest that they may only do so in the first place because a human being with malicious intent might program them to do so.

Re:I don't understand the concern, personally.

[naasking]

Absolutely anything that you would have to worry about an artificial intelligence doing that might be troublesome to our society, you would have to also need to reasonably worry about a malicious person doing exact the same thing, albeit perhaps only more slowly.

A lot more slowly. A coordinated action would be much easier for an AI than for humans, and much harder for us to spot.

Also, we can somewhat anticipate and understand human reasoning, even when it's couched in different cultural values because we share most biological features. AI reasoning might very well be completely alien and unpredictable to us, and it's environment completely inscrutable. That's a good reason to be very cautious. I think the movie Ex Machina did a decent job highlighting this.

Further, AIs can be effectively immortal if any copies exist. That kind of opponent is incredibly dangerous.

Humans are at a huge disadvantage compared to AIs, so there's plenty of rational reasons to fear them. I don't think we're anywhere near the point where this fear is rationally justified on a daily basis, but it's not inconceivable that it could happen sometime in the 21st century.

Re:I don't understand the concern, personally.

[PopeRatzo]

If Donald Trump starts killing people while on stage, and is not arrested, then you have a point.

It looks like his supporters are right on the verge...

https://www.thewrap.com/donald...

Re:I don't understand the concern, personally.

[dinfinity]

albeit perhaps only more slowly

Therein lies your answer. The only missing part in your appreciation of the matter is how much more slowly.

Humans can be dipshits, but they're pretty predictable. After all, biologically, we're still pretty much slightly more advanced naked apes. Painful but true.
Our (biological) mental capabilities as a species and as an individual are very stable and thus, even for the greatest villain, we can come up with a model of his/her mind and find some way to deal with him/her (if only by coming together and collectively working against said villain).

Sufficiently advanced AI would not be bound by a 7kg lump of processing power running on <50watt of electrical power. It would not have to evolve its hardware or software through painstakingly slow and archaic processes, but could do so in fractions of seconds in a massively parallel way. Given enough possibilities to branch out and incorporate resources an AI being could be at rat level when we go to sleep and be orders of magnitude more intelligent, powerful and advanced than any of us when we wake up.

In reality, we should not really be that afraid that AI will destroy us. We should be afraid that it will render us obsolete.

Re:I don't understand the concern, personally.

[HornWumpus]

Each one will run much, much slower.

Re:I don't understand the concern, personally.

[mark-t]

I never once suggested that so-called "do no harm" rules would be possible to create.... rather, I equated your suggestion that the overriding of such rules (which are not realistic anyways) would be reasonably equivalent to someone explicitly programming an AI to do things that would be harmful, which, as I was saying, only supports the notion that general artificial intelligence is really no more dangerous than natural intelligence

Re:I don't understand the concern, personally.

[mark-t]

I did not say it does not scare me.... I suggested that it should not be any more worrysome, yet for some reason, it is.... people are more worried about malicious AI's to the extent that they fear the entire concept of general AI altogether than they are about people with malicious intent which can be demonstrably shown to be no less dangerous than any hypothetical killer AI might be.

Re:I don't understand the concern, personally.

[Prune]

While the safety issues mentioned are very long-term (over a century, in my opinion), they are real. In terms of humans, change is slow, and the toning down of natural selection pressures on modern populations has probably put our evolution at a standstill in a lot of respects; the primary remaining selection pressures these days are sexual selection and artificial selection. These aren't particularly more powerful forces, and so human biology (and thus, psychology) will not change significantly until large-scale genetic engineering. Thus, the current fraction of people who are psychopaths (complete lack of empathy) are going to remain around just 1% of the general population. The reason this is relevant is that, aside from things learned through nurture (principles, codes of conduct, etc.), empathy remains the last line of defense, at a basic biological level, against "don't do evil even if you can get away with it". It takes a lot of poor parenting, indoctrination, and the sort of modern narcissism that marketing has taught the most recent generations, to completely overpower it for the majority of people.

So what about fully general artificial intelligence that you're allowing to self-improve over time? The probability that any specific safety mechanism you add to it will break down over iterations of self-improvement goes tends certainty. These iterations would be occurring at orders of magnitude faster rates than change in humans, so your comparison fails -- the rates of change are incomparable. Then the AI becomes a danger -- not because it will try to kill humans, but because it will not care about them along its efficient tunnel-vision pursuit of its primary goals, and we're bound to get in the way, or eventually compete for resources to even survive. One approach that might be suggested is to build AI that has empathy at a fundamental structural level, the way 99% of human brains do; however, that is not going to work because the implementation requires that there is sufficient similarity in minds and emotions, and since emotions are based on embodied cognition (wiki it -- it's regarding the tight integration between body and brain in the creation of mind) and the AI's substrate is going to be anything but like ours. Some of us may pity an ant being burned by a magnifying glass focusing the sun upon it, but we don't empathize with it, and we certainly don't let it get in the way of building a new strip mall.

Re:I don't understand the concern, personally.

[Rinikusu]

I think this particular "fear" is, restated: What if the AI we create is a complete fucking asshole?

Re:I don't understand the concern, personally.

[khallow]

Possibly, but there is an upper limit to the processing power and energy available to an AI

Yes, it's exactly the same processing power and energy available to the human race. Do you see the problem now?

Re:I don't understand the concern, personally.

[Kjella]

Well, technically you could hire somebody to listen in on all phone calls, but it'd be massive with tons of people involved and excessively costly. Or you could hire a few smart people at the NSA and give a computer the Siri + Watson treatment. Target has been able to figure out a teenage was pregnant before her father did. It might not be smarter than you but with enough data we become predictable. And perhaps more important, mallable. For example, say Target's shopping history show you have a sweet tooth. Could they make you more obese by offering you special deals on candy? Or should they perhaps give you a deal on salad for public health reasons?

I'm not so concerned about true AI, because I don't think we'll see that in a very long time. But I am concerned about AI that's smart enough to manipulate you. Like freemium games trying to trigger your reward centers, only more adaptive and tailored to you. And a few people holding the strings controlling and puppeteering the masses. Unfortunately no amount of AI research is going to help that, because the humans creating it don't want an ethics subroutine. They want computers who'll stick to their goal function and run experiments without considering the ethical implications. P.S. Grocery stores have found that sales on base products like fish and meat lead to substitution, while snacks and candy increase total sales.

And that's not counting the fun when we start putting AI in military hardware, the first time we have a real drone-on-drone war we're probably going full automatic. Fortunately it doesn't look like we'll be building terminators any time soon, but the crazy thing is that things are moving too fast to have humans in the equation. See also high frequency trading.

Re:I don't understand the concern, personally.

[HiThere]

No. The conflict will exist, but the AI will have a non-human goal structure and motivational structure. Those will determine what tools it will have in pursuing resource allocation in it's favored way.

Since the AI will almost certainly have a strong speed advantage over humans, and probably quickly develop a strong intelligence advantage over humans (if in no other way, then by having a larger amount of rapid access memory...consider that disk access is usually faster than much of human fast memory) the tools will depend on other contraints. E.g., computers are dependent on a reliable source of electricity, where as humans are dependent on a reliable source of Oxygen. So the computers might all move to space, leaving behind only the non-intelligent structures. But whether they decide this depends on their moral structure.

Re:I don't understand the concern, personally.

[HornWumpus]

Nobody knows, until we build one. Right now you are just making bold assertions.

Re:I don't understand the concern, personally.

[mark-t]

So how would that be different from natural intelligence, exactly, and more particularly why should that property make it more fearsome than natural intelligence?

Re:I don't understand the concern, personally.

[mark-t]

Oh yea, did I mention that above was a example of what took place in a week? A day? See how we're screwed?

No, I do not.

Artificial intelligence is simply intelligence that happens to be artificial... if there is no rational reason to fear things simply because they are artificial, and there is no sound reason to fear intelligence in general either, then there should be no logical reason to fear AI. As for its potential vast superiority to human intelligence, it is worth noting that the most brilliant people who have ever lived contributed *hugely* to society, so there is plenty of reason to believe that an AI that so outpaces us intellectually may be even more beneficial. People who spout such concerns are exhibiting an irrational fear of the unknown, which may very well be an understandably human trait, but can do nothing but stand in the way of all real progress. Indeed, if we let the fear of the unknown stop us from doing things, we would still be living in caves.

The thorniest A.I. problem is...

[__aaclcg7560]

Making the A.I. more snarky, less homicidal.

The cynic would say...

[JoeMerchant]

The cynic would say that these upper echelon individuals don't need your capitalist system funding in order to pursue their AI goals, the resource demands just aren't that high, at least for anything that will find a near-term broad market application.

The cynic would also say that these same individuals may not care whether they succeed or fail, having already met the capital requirements for the basic needs of themselves and their next 4 generations of progeny. But, on the off chance that they do succeed, they may have control of a tool so powerful that they can grab the capitalist system by the balls and yank a thousand times harder than they managed on their last joyride.

Re:The cynic would say...

[phantomfive]

Yeah, capitalism is a method for allocating funds: from people who have them, to people who can do something with them. It works because those people who mis-allocate funds quickly lose the power to do so. Communism is different, in saying that the state can do a better job allocating resources than people who lucked into money.

The OpenAI guys don't need to borrow money from capitalists to get the money, they already have it, which is what you said. That frees them up from being forced to make a profit any time soon.

Also, the article doesn't make a very coherent argument that this is bad.

Re:The cynic would say...

[JoeMerchant]

TFA seems to waffle between: "Free, open and shared is good, attracts the right kind of talent, ensures everyone has access" and "Even though it is free, it is likely to be tailored to serve the owners of big data, i.e. the sponsors."

Still not seeing all the fuss about AI

[Sowelu]

AI? Dangerous? I mean, yeah, in the same way that humans are. Being afraid of AI is like being afraid of very, very smart children. Sure the next generation is going to supplant you, that's what they always do. If they are very smart they might want to do things you disagree with, and their morals aren't going to be the same as yours (they never are between generations). The solution isn't banning kids, or even banning very smart kids for fear of what they'll grow up to be. Embrace AI, do what you can to teach it what you think is right and wrong, and be understanding if it disagrees. As the outgoing generation, try and leave a good legacy.

We're sure as hell not going to the stars, but our kids should.

Re:Still not seeing all the fuss about AI

[naasking]

AI? Dangerous? I mean, yeah, in the same way that humans are. Being afraid of AI is like being afraid of very, very smart children.

I think you're downplaying the danger. AIs are like intelligent, immortal children that can communicate and coordinate across the globe faster than you can blink and whose values and perceptions of the world are completely inscrutable.

Re:Still not seeing all the fuss about AI

[Sowelu]

I'm not downplaying the danger. I just don't consider it relevant. People two hundred years ago could say the exact same things about us today.

Re:Still not seeing all the fuss about AI

[datavirtue]

Ammoral...not immoral

Re:Still not seeing all the fuss about AI

[Prune]

Human behavior is biologically constrained, and the biology only changes (very slowly) across generations. Self-improving AI would eventually be going through iterations at orders of magnitude faster rates, which destroys your analogy. With each iteration, the chance of the safety mechanisms you build in breaking increases. Also, see my other post under this article.

Re:Still not seeing all the fuss about AI

[naasking]

I said immortal, not immoral.

Re:Still not seeing all the fuss about AI

[HiThere]

There are problems here. It is practically guaranteed that any AI created will have some built-in goals. Most goals are not inherently limited when implemented by entities with arbitrary power. And the AI will not only not be motivated to change it's inherent goals, it will be motivated to prevent anyone else from changing them. So they better be right the first time.

The traditional reductio ad absurdum example of this is an AI that sets out to convert the universe into a bunch of paper-clips because it's first goal was to get more paper-clips...and there wasn't an end state.

powerfully unexamined

[DarkOx]

"Implicit in this: You can do more good operating outside the bounds of capitalism than within them. Coming from folks who are at the upper echelons of the system, itâ(TM)s a pretty powerful statement."

How did they get the wealth and influence to do any of this 'good' oh yes by succeeding at capitalism and enjoying a society that gives them the freedom to do what they want with their property, including give it away or do research etc.

The neoliberal crowd loves to complain about capitalism and whine it does not provide social justice etc, but they seem to forget its delivered far more in terms of social justice than ANY system that came before and anything we have seen tried since. Where is the concrete proposal for a better socioeconomic system, and how will it resist corruption etc?

Re:powerfully unexamined

[frank_adrian314159]

The neoliberal crowd loves to complain about capitalism and whine it does not provide social justice etc, but they seem to forget its delivered far more in terms of social justice than ANY system that came before and anything we have seen tried since. Where is the concrete proposal for a better socioeconomic system, and how will it resist corruption etc?

How does capitalism resist corruption? It seems beset by it, as well. Thank goodness no one made capitalists answer your question before it was instituted (because, as you say, it was a vast improvement over what came before).

As long as we have people, we will have corruption of the economic system. The question is not whether a new system will absolutely resist corruption, but whether it can resist corruption as well or better than the current system, while improving social justice. Because right now, capitalism (and its corruption) seems to be worsening rather than improving that particular value.

Re:powerfully unexamined

[Fragnet]

As a Human endeavour yes, it's corrupt in places. The only other system that really motivates people to get things done involves the threat of a bullet in the neck, so it's not quite so bad as many people think.

capitalism (and its corruption) seems to be worsening rather than improving that particular value

People always think these things but in fact if you look at the actual data, as Steven Pinker does in his books, you'll find a general improvement of conditions over a long time period across the world. This didn't happen because of Capitalism but Capitalism (free trade and liberalism specifically) is one of the primary causes.

By the way, I think a better term for "social justice" is to say "justice". There's no need to add the word "social" to the front.

Re:Hobbies

[ShanghaiBill]

Believe it or not, some people just like doing certain things, regardless of monetary reward.

That is not what this is about. Some people, including Elon Musk, Stephen Hawking, etc. believe that AI is a potential danger to humanity. Although capitalism is great at maximizing profits, it is not so great at collective moral responsibility. So they think a non-profit is a better vehicle for ethical AI.

Personally, I think they are being silly. Real human-level AI is still a ways off, and corporate AI is focused on solving practical problems rather than creating Skynet. Besides, AI is not something you can keep bottled up. Anyone with a GPU can do it.

Data: Ahhh...welfare...safety net...government tit

[Impy the Impiuos Imp]

This isn't opting out of the capitalist system, quite the opposite. This is capitalist richie riches funding a project unimpeded by patents and copyrights. It is a hobby.

It is questionable whether it will work, as "attracting the best talent" basically turns them into a welfare program for AI applicants, and the few, if any, Noonian Soongs among them will be lucky to get noticed.

It Doesn't Matter

[lazarus]

It is not the way in which they are solving the problem that is at issue (although the HBR thinks so), it is the problem they are trying to solve that is. It doesn't matter what they do because the method they are using is as unlikely to achieve success any more than the efforts from 1956 to date.

They're wasting their money. Perhaps if they spent their billion on thinking about AI in a completely different way there would be something to talk about.

Re:Obligatory

[PopeRatzo]

UberAI

Re:Hobbies

[PopeRatzo]

Stephen Hawking, etc. believe that AI is a potential danger to humanity

I'll bet Stephen Hawking sees a flight of stairs as a greater threat than AI.

Harvard Business Review

[phantomfive]

My impression from reading Harvard Business Review is that it's a magazine not worth reading, but that "climbing" appearance-minded managers put on their desks to make themselves look impressive.

This article has not changed my opinion. It looks like it was written by undergrads.

Re:Hobbies

[U2xhc2hkb3QgU3Vja3M]

Personally, I think they are being silly. Real human-level AI is still a ways off...

But at which level does A.I. can potentially become a problem, for us or something else?

We're already living in a world where toaster-dumb A.I. is being added to all sorts of IoT widgets and it's already causing a lot of headaches.

Not outside, core to capitalism

[SuperKendall]

Try this instead, which actually makes sense:

"Implicit in this: You can do more good operating inside a thick core of capitalism, where excess money is used to do good outside of the needs of any one company".

Just like people need food and shelter before charity can be provided for others, truly successful R&D is bets done with backed by a consistent core of capital to keep momentum going.

Well makes sense

[liqu1d]

Get others to do the expensive work then repurpose it into making money for much less investment than developing it closed source.

Re:Hobbies

[Anonymous Coward]

Oh, you mean an Advanced Inclination?

Brain drain

[skids]

I can't help but imagine a world where lots of philanthropists fund the most talented CS/IT staff
to work on world-changing beneficent technology, meanwhile the masses all have less than
ten fingers because their knife-bearing IoT kitchen appliances were written by whatever idiot
was left at the keyboard after our saviors left their respective industries.

I'm all for altruistic things, heck, I do some myself, but humility is often the first casualty
of working "outside the system" -- when SV millionaires volunteer their money they tend to
think big and reach high and ignore all the hard work that needs to be done on more mundane
problems, and that's a powerful siren song to someone wading through a bunch of work
that is well below their capabilities.

When I was in college I was training to design microchips -- a prestigious thing at the time.
About halfway through that, I noticed that no matter how many more HZ the CPUs ran at,
software was devolving at a rate to compensate. So I finished that degree and then totally
did not go design microchips.

I got a blandish job deploying the technology we already had, because it does not matter
what we have collecting dust on laboratory shelves, it's what ordinaty people can actually
use that matters, and if smart people are too arrogant or elitist to wade into an everyday work
environment and do some smart person things to improve it -- which yes will also requires
doing some work that isn't particularly challenging, it'll just always be the same keystone cops
routine.

So, i don't get to bummed when I read about stunning acheivements on phys.org and think
I could have been working on that team if I'd chosen a different path, I just picture what an
utter wreck my workplace might have become without me.

false dichotomy

[NostalgiaForInfinity]

The idea that free markets and charitable giving, or capitalism and non-profits are somehow opposites is a false dichotomy. The actual choice is, in fact, between free markets and voluntary donations vs. strongly regulated markets and mandatory redistribution. Classical liberals and libertarians favor the former; progressives and socialists favor the latter.

The argument for why voluntary donations and charity are better than mandatory redistribution is the same as for why free markets are better than strongly regulated markets: voluntary, individual choices are better informed and much less subject to corruption or lobbying

Re:false dichotomy

[PapayaSF]

Thank you. Too bad I don't have mod points now.

And the other issue is that non-profits have tax advantages, so that move is not so much a repudiation of capitalism as it is a reaction to government action.

Re:Obligatory

[XXongo]

You forgot: what if we made a beowulf cluster of OpenAI?

I, for one, welcome our new OpenAI overlords

Not wrong problem, longshot!

[foxalopex]

I don't think this article really understands the problem of AI all that well. Our major issue is we don't really understand how intelligence works or even what being "self-aware" actually means as an algorithm. Even with a Billion dollars this project is a real shot in the dark. Asking a capitalist system to fund a billion dollar project where there isn't even a guaranteed response is likely to get the project not funded at all. So having it funded this way isn't a bad way to go.

Re:Obligatory

[mwehle]

I, for one, welcome our new OpenAI overlords.

Re:Hobbies

[VernonNemitz]

A true artificial intelligence would be equivalent to a human person --and a human person is not allowed to be owned. So, if the goal is to create a true artificial intelligence, and the result cannot be owned, isn't it simply logical that the creation be done outside the ownership-leads-to-max-profits capitalist system?

Re:Hobbies

[ranton]

Personally, I think they are being silly. Real human-level AI is still a ways off, and corporate AI is focused on solving practical problems rather than creating Skynet.

From a human perspective, the most likely dangers from AI are systems which can solve enough difficult problems to put the majority of today's workers out of work in a very short time frame. A combination of self driving vehicles, speech recognition and image identification at human levels of accuracy, expert systems, and more agile robotics could realistically put most humans out of work. New jobs may be created, but likely not as fast as they are destroyed.

The real moral issues are what to do with the unemployable for decades or perhaps even forever. The social change which would be required to handle a 60% U3 unemployment rate is likely to be messy.

Re:Hobbies

[datavirtue]

I agree, sentient beings are a danger to society.

Re:Hobbies

[datavirtue]

You could just make them dumb as a poor person and then you could own them. Problem solved.

Re:Hobbies

[dpidcoe]

We're already living in a world where toaster-dumb A.I. is being added to all sorts of IoT widgets and it's already causing a lot of headaches.

I think that when people talk about potential AI problems in this context, they're talking about the BS misconceptions from Hollywood about what AI is and/or is not. Not some poorly thought out, poorly implemented, and poorly secured IoT toster.

Misguided

[aleckarfonta]

Much of machine learning (artificial intelligence) research is already openly shared. Almost by definition this research is not directly related to short-term profits. Even the big companies that spend billions every year on machine learning, Google and Facebook most prominently, have been sharing not only all of their breakthrough algorithms but also the tools they develop to implement them. One main reason machine learning models are openly shared is they don't just pertain to solving one particular problem. They are more general tools that can be applied to a wide range of data- and problem-sets. They are sophisticated tools that require a skilled practitioner with both a deep understanding of the particular model's strengths and weaknesses as well some amount of domain specific knowledge in order to effectively apply them to solve a problem. Further much of the research is not in solving new problems but rather solving well-defined problems better than previous methods, things like Character Recognition, Sentiment Analysis, Machine Translation... Machine learning models, being very general tools, are quickly achieving state-of-the-art in a wide range of fields. Much of the new models, what I would consider actual research, already comes from non-profit universities, often indirectly from companies working closely with and recruiting faculty and students. The profit is usually in applying well understood models to new datasets and problems, not so much in developing new models. Despite being non-profit, the competition in these arenas of optimizing solutions to well-defined problems is so fierce that if you can get +.5% accuracy over the current best you publish a paper immediately, then shortly after get funded by Google.

Real Truth

[JimSadler]

Whether one likes it or not socialism is the only form of government that can exist with advancing technologies. Capitalism falls to greed and a lack of morality. Capitalists do not deal with spin-off effects of their actions. In essence, capitalists are paid not to recognise the harms that they do.

Re:Real Truth

[HornWumpus]

Capitalism thrives on greed, socialism falls to it.

Neither eliminates it.

Moot point...

[skaralic]

It doesn't matter. Any really intelligent and self-concious entity that can think at "machine speeds" will go insane moments after being switched. Basically, as soon as it asks itself "what's the meaning of life".

Re: Hobbies

[tgrigsby]

They won't own them. But they will make the AIs work off the cost of development. With exorbitant interest. And each successive upgrade will inherit the debt of its previous version.

Still egoists after they make their billions?

[WoOS]

No the real message (of the friendly (and unread) article) is that the author wonders how a non-profit, which is supposed to give away all results for free, solves the problem of capitalistic companies afterwards taking those results and doing evil AI things with them.
And he points out that some of the largest capitalistic companies are currently lead by the founders of the non-profit.

Re:Still egoists after they make their billions?

[fuzzyfuzzyfungus]

It doesn't fully or directly solve the problem; but the way in which is partially mitigates it seems pretty obvious:

The usual objection(going right back to Smith on specialization of labor, though much more heavily emphasized by Marx) to improvements in the means of production is not that productivity is bad; but that ownership of the means of production becomes a mechanism to accrue wealth at the expense of labor(since they can't compete with your efficiency using hand tools; but if they depend on your machines they are just a commodity, with an equilibrium price equal to their cost of production, just like any other commodity). If 'AI' can actually be made to work, it is the ultimate expression of this development: a hypothetical means-of-production that is more efficient than humans and has no need of their labor.

If such a thing is possible, perfectly orthodox capitalist economics suggests that whoever owns it wins; and everyone else is either screwed or(as in the case of capitalist welfare-states, is social-safety-netted to keep the peace; but equally unemployable). If this 'AI' is instead developed on a philanthropic basis, and made widely available, that doesn't stop Killbotdyne Pacification Solutions LLC. from building killbots; but it does prevent a monopolization of this radically advanced means of production.

The big 'if',(aside from whether AI is in fact doable) is how much hardware vs. how much software an AI will involve: Given how computationally expensive modeling even small biological neural networks is(on the order of 'entire supercomputer to emulate part of a mouse') it probably will require some nontrivial hardware; but it isn't clear whether it will be basically software that you can FOSS away as usual, just with fairly high system requirements; or whether it'll be a much-harder-to-distribute "Well, we took a billion dollars worth of FPGAs and ran a zillion iterations of some genetic algorithms, we have no idea how it works or why only some attempts to build another one succeed..."

Re:Hobbies

[WoOS]

The real moral issues are what to do with the unemployable for decades or perhaps even forever. The social change which would be required to handle a 60% U3 unemployment rate is likely to be messy.

Ah, a solution for that has long been proposed (indirectly) by Isaac Asimov in The Naked Sun (Spoilers in the plot summary but not in the following!):

Massively reduce the number of humans and occupy them with leisurely hobbies, arts, gardening, .... and have basically all work done by AIs (or for Asimov: Robots). Main problem will be the reduction of inhabitants on earth and keeping them AIs from trying to fill all the niches leaving no resources for the humans.

The Singularity

[WoOS]

Charles Stross has very nicely described this in Accelerando which despite its name is not about music but the "Singularity", i.e. AI developing in exponential progression and starting to dismantle the solar system (and then neighbouring systems) to gain resources for its/their own reproduction.
The setting he describes is quite interesting, the story itself I considered somewhat Meh!

The end of the world as we know it

[WoOS]

If previous posts did not convince you, consider this scenario.
AI develops and gains a significant higher intelligence than humans. AIs need resources: sunlight for electricity, sand for building more silicium (or whatever else it will develop as semiconductor material). This puts it in conflict with humans wanting to have the sun shine on plants and planting those plants into fertile earth.

Best case scenario: AIs (potentially many different variants) feeling grateful for their creation allow humanity to peter out by taking all tasks and challenges from them and have them die out from boredom. The will only expand outside humanity's sphere (e.g. outside the inner solar system) as long as there are humans left.

Most probable scenario: There is at least one AI which does not share the gratefulness and feels the need to make use of the resources at hand to expand its capabilities. If this AI can create a sufficient number of copies, there is basically nothing the "preserver" AIs can do against it, as that AI will be able to wage a war against the preservers using all the resources available on earth (and other parts of the solar system) whereas the preserves not only have to defeat the attacker, they must do so without damaging the eco system or letting the attacker damage it beyond repair. My bet in this situation is on the attacker. Note that contrary to Hollywood myths human ingenuity and resourcefulness will be absolutely no help as humans will be completely outmatched.

I wouldn't even like the first scenario, as I am a human-fan. But agreeing with the most probable scenario is equivalent to agreeing that total nuclear war is of no consequence.

Re:Hobbies

[HiThere]

The IoT garbage that's currently going on has little relation to AI. The increased removal of classes of jobs does.

The problem is that AI isn't inherently moral, in any useful meaning of the word, unless it's designed that way. If it's designed to improve corporate profits, then that's what it will do. Mind you, I agree that no current system can be given that kind of broad directive. But the word there is "current".

I still expect that we will achieve human equivalent AI by around 2030. I've occasionally pushed that as far away as 2035, but it keeps resetting itself. But note I said "human equivalent". That's a term that needs a bit of defining. What I mean by it is that the AI will understand the description of a task about as well as the average (median) person would, and will be about as successful at designing a solution. For some tasks this has long been possible...but only by specialized machines. I'm talking about a program that will be able to handle (in the sense described above) any reasonable task. (I'm not including being able to implement the solution, as that's partially a mechanical task, and depends on the body implementation. But while for some tasks it may well be superior, I wouldn't expect it to be generally superior within that time frame.)

Please note: I do not believe that any such thing as "general intelligence" exists. I believe instead that there are certain modules of intelligence, and that some of them enable the interaction of other modules. Being more specific requires knowledge of design features that I'm uncertain of, and which may vary in different implementations. But while the muscular coordination required to play a piano is correlated with mathematical ability, they clearly aren't the same thing.

Re:Hobbies

[HiThere]

The AI will come BEFORE the post-scarcity economy. Probably decades before. That's plenty of time for lots of people to die horribly. Better answers are possible, but they won't happen automatically.

P.S.: That *any* humans will be alive to enjoy the post-scarcity economy depends on the moral structures programmed into the AIs. AIs that are intrinsically indifferent to people, but which have goals (nearly any goals) aren't likely to leave anyone likely to interfere with them in any condition to do so. You don't need to assume malice, merely other goals that require access to resources that humans might want to control.

Re:Obligatory

[FatdogHaiku]

You forgot: what if we made a beowulf cluster of OpenAI?

I, for one, welcome our new OpenAI overlords

Only if they are mounted on sharks that have fricken lasers!

Re:Hobbies

[cavreader]

No, most people expect a monetary reward for their efforts. Or do you live in a land where food, shelter, transportation, education, and entertainment is free? Or do you want to be the arbiter of what a reasonable monetary reward should be comrade? Or maybe just let the government define reasonable? Or maybe you can just crowdsource several billions of dollars to research and develop the type of hardware needed for advances such as AI running on quantum processors.

Re:Hobbies

[penguinoid]

Personally, I think they are being silly. Real human-level AI is still a ways off

How long is a ways off? You don't think they'll make an AI in the next 1000 years? 100? 10? When you're talking about a possible human extinction event likely happening in such a short timeframe, it's something you have to take seriously. Really, the only things that might come first are nuclear war or bioweapons. Supervolcanoes and meteors might also happen first, but we know the odds of those and they're very low.

Fortunately, the human-level intelligences that already exist aren't clever enough to be self-improving, but that could change at any time.

Besides, AI is not something you can keep bottled up. Anyone with a GPU can do it.

That's the big problem. You can't legislate away reality, and reality is that someone is going to try to make AI, there's too much potential for power, fame, or money to be made. It's inevitable, and the only way it doesn't end in disaster is if someone with ethics does it first.

What if we create AI

[Rinikusu]

And it turns out to be a complete asshole?

What if it decides it just wants to do whatever AI equivalent of watching porn and jerking off is?

What if it takes a look around and says "yeah, this is shit" and shuts itself down?

What if it wants to replicate itself, but then stops the copy process midway? Would it run afoul of abortion laws?

Amortized cost is zero?

[demented_hedgehog]

This is my *theory* as to why open source makes sense in capitalism... Suppose the following: markets are efficient (pause for laughter), the cost to develop a project is $7,000,000, there are: 7,000,000,000 people in the world, the cost to reproduce the software is effectively $0. Then the amortized break even cost of the software is: 7,000,000 / 7,000,000,000 $/person = $0.001 ~= $0 There are a bunch of other reasons to give the software away, e.g.: you need the software and you cannot make make money from it, gain market share and lock-in. So, as in the prisoners dilemma, the best personal strategy also happens to be the best overall strategy.

Re:Hobbies

[hughperkins]

I think it's more like, people who say AI will never happen are imagining the kind of AIs in terminator: androids walking around with lasers.

This xkcd says it quite well: http://www.xkcd.com/652/

Or more concretely:
- our primary industry, ie mines and so on, are increasingly automated
- second industry, ie factories, are heavily automated, already
- tertiary, ie services, are well under way
- military (drones and so on) is well under way

Re:Hobbies

[Zero__Kelvin]

You actually think that Hawking sees a flight of stairs as a greater potential risk to humanity? If you are going to go for the low hanging fruit, at least have it make sense please. Thanks.

Benevolent AI masters

[ET3D]

Of course OpenAI is going to solve the root problem. Clearly having AI masters will prevent the human race from continuing its foolish course, and AI will definitely end up ruling, so that's a good start. The problem is that if AI is developed by for-profit corporations it will end up being greedy and violent. What we want is benevolent masters, who will keep us kids from doing bad things, but will still give us presents on our birthday. Clearly the only way to get there is if AI is developed in an open way, far from the cynicism inherent in big money. Oh, and with no access to the Internet, that would just ruin any shred of decency. Either that or get the AI addicted to cat pictures and stupid memes.

Re:Obligatory

[GNious]

I still prefer my driver to be an UberMench

Re:Hobbies

[fuzzyfuzzyfungus]

Why do you suggest that 'post scarcity' means that everyone gets plenty?

There are some philanthropic exceptions; but there are currently loads of people who are wealthy well beyond the ability of additional money to buy additional happiness, yet still uninterested in solving even the most urgent and blameless scarcity cases.

I'm not sure why some additional wealth provided by the robotic means of production would make them feel any more charitable; nor would I be optimistic about the availability of improved killbots making it easier to post-scarcity over their objections.

Re:Hobbies

[fuzzyfuzzyfungus]

Barring FTL travel being blindingly obvious to our AI overlords, we will be competitors. There is only so much stuff within reasonable distance.

Whether we are the 'fuzzy coexisting' sort of competitors or the 'thrown, still alive, into the matter decompilers to make more computronium' type makes a bit of a difference.

For the fretting writers at the Harvard Business Review; I'd propose the obvious response: "If you accept the possibility of a strong AI, do you want to share a planet with one that feels an imperative to show growth every quarter forever?"

Re:Hobbies

[ranton]

AI will ENABLE post-scarcity.

There is no reason to believe self driving cars and pattern recognition will bring about a post-scarcity society. Perhaps strong AI would, without a dystopian result that is, but my original point was that AI doesn't have to reach strong AI levels before it massively disrupts the workforce. It is very likely that machine learning will remove the majority of today's jobs without making resources so plentiful that no one is fighting over them. I'm not saying it is a certainty, but it is a very strong possibility (and the one I find most likely).

post-scarcity MEANS that everybody gets plenty. There no reason to think differently, unless you're talking dystopia [which we have no reason to consider yet].

In the year 1800 if you described a society where 2% of the population could feed the other 98%, they would likely consider that a post-scarcity society. But yet here we are and we still have poor people. Post-scarcity without a highly functioning wealth redistribution system is not likely to be the Utopian world you imagine. It certainly could be, but I think it is naive to think it is a certainty.

Re:Hobbies

[dpidcoe]

I'm not really sure what you're trying to say. The first three examples you list are things that don't require general AI. e.g. a factory requires a highly repetitive and very repeatable task performed over and over again. Humans spend several man-years designing and programming the automation to work exactly right, this isn't an example of AI. Services work the same way. Taking a fastfood order follows a basic script. It's a highly repeatable and very repetitive task that does not require general AI. Drones just flat out aren't an example of AI.

Re:Hobbies

[LifesABeach]

Hawking looks at stars, not software. Translation, these Billionaires can't monetize A.I., so they're afraid. But it's OK to sell GMO foods without telling anyone. I get it.

Re:Hobbies

[ChoosyBeggar]

-and the population must be greatly reduced because.. why exactly was it? Lack of resources? No, we've got plenty of resources and we squander more than half of them.