Meet Norman, the Psychopathic AI (bbc.com)

A team of researchers at the Massachusetts Institute of Technology created a psychopathic algorithm named Norman, as part of an experiment to see what training artificial intelligence on data from "the dark corners of the net" would do to its world view. Unlike most "normal" algorithms by AI, Norman does not have an optimistic view of the world. BBC reports: The software was shown images of people dying in gruesome circumstances, culled from a group on the website Reddit. Then the AI, which can interpret pictures and describe what it sees in text form, was shown inkblot drawings and asked what it saw in them. These abstract images are traditionally used by psychologists to help assess the state of a patient's mind, in particular whether they perceive the world in a negative or positive light. Norman's view was unremittingly bleak -- it saw dead bodies, blood and destruction in every image. Alongside Norman, another AI was trained on more normal images of cats, birds and people. It saw far more cheerful images in the same abstract blots.

The fact that Norman's responses were so much darker illustrates a harsh reality in the new world of machine learning, said Prof Iyad Rahwan, part of the three-person team from MIT's Media Lab which developed Norman. "Data matters more than the algorithm. "It highlights the idea that the data we use to train AI is reflected in the way the AI perceives the world and how it behaves."

Comments

I need to friend him on Facebook

[jfdavis668]

Sounds like a unique individual. I'll have to "friend" him on Facebook.

Not just machine learning

[Steve1952]

Anyone see any correlation between these machine learning results and results with real live humans? Now think about the effect of all the adaptive algorithms on social media driving individuals to ever stranger and more isolated information bubbles.

Re:Not just machine learning

[ShanghaiBill]

Yes, that's how learning works.

No it isn't. Humans can generalize and understand plenty of things they have not seen directly.

I can show a child three pictures of rowboats, and then show her a sailboat, and she will know that this is also a "boat" because it floats on water and is used for transportation.

ML doesn't work that way (yet). Even to recognize a rowboat, it would need THOUSANDS of examples, and it would not generalize by understanding the purpose and function.

Re:Not just machine learning

[Wycliffe]

No.

if the _ONLY_ thing you feed a child is porridge and beans, and then later in life introduce the person to chocolate ice cream? the person will preference to sweet foods, despite never having them previously.

That's not as true as you think. It's also interesting that you mention beans. In Taiwan, their ice cream is bean based and not near as sweet as in America. Most of their sweets are also not near as sweet and many Asians do not like the sweet candies in America. People raised in one country tend to prefer the foods and tastes they were raised on and delicacies in one country are sometimes disliked in another country. Humans don't have a universal set of tastes that they prefer over the others. Even inside a single culture, if you cut out sweets for a while then something extra sweet will taste disgusting to you after a while. You can train yourself to like stuff less sweet or more sweet. There are many things in many countries that are "acquired tastes" and don't come naturally.

Re:Not just machine learning

[TapeCutter]

What AI lacks is imagination, not intelligence.

What did they expect?

[peppepz]

Of course an image classifier will classify an unknown image depending on what images it has been trained on.

I hope nobody thinks this was surprising

[ScentCone]

If you limit anybody or any system to only a small, tunnel-vision view of only part of reality, that small pool of information IS reality. What would be news would an AI that forms a rose-colored-glasses sense of reality when only shown what's described. Or an AI that only perceives death and violence when shown unicorns and rainbows and (not mutilated) puppies. But when you limit a system's visual vocabulary to a small subset of consistently violent things, what else would one expect? The AI's got nothing else to draw on. GIGO.

When all you have is a hammer

[Yumi Saotome]

everything looks like a nail.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Nothing changes

[morcego]

"Garbage in, garbage out" still applies.

This sounds like a really stupid experiment

[gweihir]

If they had only trained it on fruit, it would have seen fruit in all the inkblots. Also, there is noting "dark" in the output of a classifier. It does not have any concept of such things (or of anything, really).

Re:This sounds like a really stupid experiment

[fazig]

The entire thing with calling it "psychopathic" is very questionable anyway.
The term psychopath is (actually obsolete) reserved for those people with the most severe antisocial personality disorders. Such a personality disorder requires a them to actively be antagonistic towards other people, by being manipulative, deceitful, callous, and hostile. Emphasis is on callousness, lacking empathy for those whose rights have been infringed. Feeling no remorse, guilt, or responsibility towards others. Add a good portion of sadism to that callousness, where the person actively enjoys the suffering of other people, and you've got yourself a psychopath.
Think of the character Patric Bateman in American Psycho, that was a rather good portrayal of an intelligent psychopath in the media.

Maybe they should have hired some actual psychologists. They would know that you can't rely on an inkblot test alone to get a good picture of a person's mind. But I suppose that would not make good pop science.

Ridiculous study

[carlhaagen]

If the only reference images the AI has been shown are such of gore and death, what possibly else would it refer to when shown inkblots? It's unbelievable that nobody raised an eyebrow over how unilateral, narrow and moronic this "study" was.

What a subtle way to describe the mainstream media

[lite99]

Data matters more than the algorithm Put this thought into your head the next time you are watching the news or reading about political or economical issues. We are being well fed, with garbage from the sagelike elite...? Quite surprising from MIT.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Reddit Can Do That To You...

[kenwd0elq]

Reading Reddit for too long can cause otherwise normal people to become insane. That isn't anything new; we've known that for years.

Ah, artificial intelligence ... wait, what?

[Artem S. Tashkinov]

"It highlights the idea that the data we use to train image recognition algorithms is reflected in the way the said algorithms calculate the world and how it behaves."

FTFY.

Please stop using the world "intelligence" gratuitously.

Thank you,
people of the world.

That's the point!

[lorinc]

That's the point!

Now think of all similar algorithms that will make decisions and the data they have been trained on. For example, the algorithm that process admission to a university, or the algorithm that computes the cost of your health insurance. You want these to have been trained on data that are favourable to you or at least neutral, but you'll never now unless the training data are public. Actually, and contrarily to other algorithms, with machine learning, you don't really care about the algorithm (with its millions parameters, it's completely opaque), you care about the training data.

Comment removed

[account_deleted]

Comment removed based on user account deletion