Software Recognizes Sarcastic Tweets

An anonymous reader writes "Even humans sometimes fail to recognize sarcasm and irony; can machines do better? An algorithm that identifies sarcastic tweets (PDF) on Twitter and sarcastic sentences in product reviews on Amazon will be presented next week in the International Conference for Weblogs and Social Media in Washington, DC, and in the Computational Natural Language Learning in Sweden in July. A team from the Hebrew University, Israel, has developed an algorithm that identifies sarcastic sentences by using a machine learning technique in which a small number of sarcastic sentences act as seeds for the software to learn and generalize upon. The algorithm can then identify sarcastic sentences that are nothing like the examples. The variety of recognized sarcastic sentences is impressive, though the results are not perfect. But again, we don't do it so well ourselves, do we?"

Comments

Software Recognizes Sarcastic Tweets?

[nacturation]

Yeah, sure it does.

Re:Software Recognizes Sarcastic Tweets?

[binarylarry]

I'm not so sure I'd jump to the conclusion that this is useful.

Determining the amount of sarcasm in bird calls doesn't seem to be an effective way to use research money IMO.

Re:Software Recognizes Sarcastic Tweets?

[clickety6]

The professor was lecturing the class.

"So while two negatives make a positive, two positives can never make a negative"

An answering voice came from the back of the class:

"Yeah, right"

 

You Know DRM is a Pervasive Problem When ...

[eldavojohn]

From the research paper:

Weight of various patterns and features. We present here a deeper look on some examples. A classic example of a sarcastic comment is: "Silly me, the Kindle and the Sony eBook can’t read these protected formats. Great!". Some of the patterns it contains are ...

You know DRM is pervasive as a very serious consumer problem when statistical research papers recognize user dissatisfaction with it as a classic example of sarcasm that floods reviews.

Re:You Know DRM is a Pervasive Problem When ...

[FatdogHaiku]

Oh Great, my Sarcasm Meter and my Bullshit Detector both exploded and now my UPS is on fi

This is great!

[Anonymous Coward]

This may help people with autism and Asperger's Syndrome recognize satire.

Re:This is great!

[Anonymous Coward]

You don't have to be sarcastic, they might really find this useful.

Re:This is great!

[Itninja]

As someone with Aspergers I have found that watching sitcoms is very helpful. Since nearly every character is being sarcastic most of the time, I learn through observing caricatures of reality.

Re:This is great!

[xouumalperxe]

I'd have though that even autistics would be able to recognize horns, goat legs, and a pan pipe? Oh, satire.

Re:This is great!

[fuzzyfuzzyfungus]

As someone with Asperger's Syndrome(yes, actual extensive-testing-and-medical-consensus-of-qualified-shrinks, not "well, I like computers and girls make me nervous"), I suspect that it won't be of much use for that purpose.

Many, though not all, Asperger's types actually have average to excellent parsing of written communications, or the strictly verbal component of other people's utterances(ie. the part that would get written down, if a transcriptionist were in the room). Odds are, most such people could easily outperform this algorithm(since, obviously, the purpose of the algorithm is to provide large volumes of adequate analysis for cheap, not to be human level).

The part of communication that is really difficult, though, is the nonverbal component, the stuff that doesn't show up in text. Tone of voice, expression, tiny muscular movements and reconfigurations around the eyes, that sort of thing. Since typical social standards of politeness and interaction actually discourage direct statement of things(ie. "Your story bores me." "Yes, I am interested." "No, go away") in favor of relying on subtle nonverbal communication of those message, this can be a real handicap. You care about what others around you are thinking, since you naturally want to be on good terms with them(or, even if you don't, you want to be on bad terms deliberately, not accidentally); but you just can't tell, unless somebody explicitly says something, which is rare, unless you've already really fucked up.

In fact, in my experience,(and yes, "my experience" = "N of 1" = "anecdote") I tend to find text-based communication comfortable for exactly these reasons. For normal people, strict text-based communication is harder, because they are denied the nonverbal cues that they normally take for granted. For me, I don't see the nonverbal cues that never mean much anyway, and we are both forced to rely on strict verbal expression, which is my best-practiced level.

Re:This is great!

[clone53421]

Just about everyone is less-than-average in some aspect. Asperger syndrome is an autism spectrum disorder, which means it’s a broad range going from just about normal all the way to really mentally impaired.

It’s not like blindness; it’s more like near-sightedness. Some people get it worse than others, and some people are just about impaired enough to be considered legally blind. However, everyone fits in somewhere on the autism spectrum... including people who are considered normal.

You can’t have just a touch of herpes, but you can of Asperger’s. Whether or not it makes you “disabled” is debatable.

Re:This is great!

[Anonymous Coward]

How are mods supposed to moderate anything in this thread?

Like the fair, independent, open-minded, thoughtful people that you know they are.

Re:This is great!

[Anonymous Coward]

Dude. He was being sarcastic.

Oblig. Simpsons

[billius]

A sarcasm detector, that's a real useful invention!

Tweet from the developer

[spleen_blender]

"Yeah, we totally developed a program to detect sarcastic tweets... #fuckinggenius"

I don't see this as a problem

[Merc248]

Just end your sarcasm tags before being sarcastic. This won't conform to W3C standards, however.

Re:I don't see this as a problem

[aicrules]

I opened my sarcasm tag about 28 years ago and don't plan on closing it any time soon. That has forced me to come up with a new language nuance that I like to call "more sarcastic than usual". But really that just means I add an extra, overemphasized "really" ahead of the point of super sarcasm.

Will have very good recognition rate...

[gweihir]

Given that sometimes not even humans understand when I am being sarcastic, I expect this software will have an exceptionally high recognition rate with very low false positives. A truly remarkable achievement and the one algorithm the human race has been waiting for!

Finally a way to verify Smith's Law

[Eric Smith]

"Any sufficiently optimistic statement is indistinguishable from sarcasm."

Oh, sure!

[medcalf]

Yeah, like that would work!

Testing

[Bobfrankly1]

The algorithm can then identify sarcastic sentences that are nothing like the examples.

Place it in my office. If it still responds at the end of the week, it's not working correctly. If it's overloaded and partially melted, we've got a winner.

For a real test

[medcalf]

I wonder what would happen if you applied their algorithm to any given slashdot post, particularly one on the Apple board.

Re:Ohh.. a sarcasm detector!

[sexconker]

When quoting the Simpsons, do it correctly.

Comic Book Guy: Oh, a sarcasm detector, that's a real useful invention.

Sarcasm, older than we thought

[HockeyPuck]

It dates back into some of the great classic works of our time... upon reading Romeo and Juliet one critic was overheard saying:

"Nice play Shakespeare..."

or upon solving a great mystery, Watson was once overheard saying, "No shit Sherlock."

Oblig Python

[$RANDOMLUSER]

Vercotti: Well, I had been running a successful escort agency - high class, no really, high class girls - we didn't have any of that. That was right out. And I decided. (phone rings on desk) Excuse me (he answers it) Hello......no, not now......shtoom...shtoom....right......yes, we'll have the watch ready for you at midnight.......the watch.....the Chinese watch....yes, right-oh, bye-bye mother (he replaces reciever) Anyway I decided then to open a high-class night club for the gentry at Biggleswade with International cuisine, cooking, top-line acts, and not a cheap clip joint for picking up tarts, that was right out, I deny that completely, and one night Dinsdale walked in with a couple of big lads, one of whom was carrying a tactical nuclear missile. They said I'd bought one of their fruit machines and would I pay for it.

Interviewer: How much did they want?

Vercotti: Three quarters of a million pounds. Then they went out.

Interviewer: Why didn't you call the police?

Vercotti: Well I had noticed that the lad with the thermo-nuclear device was the Chief Constable for the area. Anyway a week later they came back, said that the cheque had bounced and that I had to see Doug.

Interviewer: Doug?

Vercotti: Doug (takes a drink) I was terrified of him. Everyone was terrified of Doug. I've seen grown men pull their own heads off rather than see Doug. Even Dinsdale was frightened of Doug.

Interviewer: What did he do?

Vercotti: He used sarcasm. He knew all the tricks, dramatic irony, metaphor, bathos, puns, parody, litotes and satire.
....

Tech behind this

[adeft]

I bet it just looks for itallic text.

77% accuracy?

[pseudorand]

Their algorithm gets 77% accuracy. I think I can do better:

# Estimated accuracy: 92.1%
isSarcastic(tweet) { return true; }

Or does that only work for slashdot comments?

There's a problem with that.

[Estanislao Mart�nez]

The problem with that is that in American sitcoms, verbal irony is accompanied by non-verbal cues like facial expression, tone of voice, or, ugh, laugh tracks. Take away the cues, and deliver the sarcasm in a deadpan manner, and tons of people in the USA are completely unable to catch it, neurotypical or not.