Researchers Developing An Algorithm That Can Detect Internet Trolls

An anonymous reader writes Researchers at Cornell University claim to be able to identify a forum or comment-thread troll within the first ten posts after the user joins with more than 80% accuracy, leading the way to the possibility of methods to automatically ban persistently anti-social posters. The study observed 10,000 new users at cnn.com, breitbart.com and ign.com, and characterizes an FBU (Future Banned User) as entering a new community with below-average literacy or communications skill, and that the low standard is likely to drop shortly before a permanent ban. It also observes that higher rates of community intolerance are likely to foster the anti-social behavior and speed the ban.

Comments

In other words

[fustakrakich]

Automated censorship. Eh, saves us the trouble, I guess

Re:In other words

[dcollins117]

It would be much better to have a system that HIDES users content by default, than to delete it. Then, people scrolling all posts (including hidden) would be able to report mistakes in the system.

From my experience if you delete content or ban a troll, it just encourages them to troll more using a different account, usually from a different IP address.

The most effective way I found to deal with problem users is to make their bad comments only visible to them. That way it appears to them that they've had their say and no one responded to it. Without feedback to encourage them, trolls either quickly get bored and go elsewhere or sometimes they'll surprise you and produce better quality comments.

This is fucking stupid.

[Anonymous Coward]

Trolls are usually above average literacy and trying to skilfully cause a fight. It's easy to identify "illiterate" people and humans are way too quick to judge someone who cannot spell as having nothing to contribute or (worse) malicious, but these are not trolls. This is just another classist meme where the person is judged positively by the overcomplexity of their language and convolution of their sentences, as this must mean they have been educamated right.

BTW I went to a £30k/year British boarding school, so I have no axe to grind, nor insecurity about describing things as they are.

Re:This is fucking stupid.

[plover]

While I believe that people who are less sensitive tend to thrive more than others, I don't agree that "thicker skin" is a workable solution. Too many people have fragile emotional states and simply don't have the neural hardware psychological capacity required to dismiss the hate and insults that often happen on line. There have been some high-profile suicides among teens who were attacked online, and who knows how many people remove themselves from public comment because of the hate they've received? For safety reasons I don't think society should completely abrogate the forums to the trolls.

Does that not mean some people are overly sensitive? Sure. But just as we shouldn't velour-line the internet to cater to absolutely every person with a psychological disorder; we also don't have to tolerate the diarrhea that spews forth from the trolls. We don't have to draw a hard-and-fast line on the ground, either, and define "these words are always 100% bad in 100% of situations". Instead, we should be welcoming humans in the loop, asking them to pass judgment when needed. That gets us to a more fluid state than full automation. It also lets the user choose. Don't like the judgment process on Slashdot? Don't hang out on Slashdot.

I know full automated filtering is the holy grail of internet forum moderation, but as soon as you deploy a filter it becomes a pass/fail test for the trolls, who quickly learn to adapt and evade it. Human judges can adapt, too, and are about the only thing that can; there are simply too few for the volume of trolls out there. A tool like this might help them scale this effort to YouTube volumes.

Unicorns, skittles, rainbows, etc.

[pla]

within the first ten posts after the user joins

So, this algorithm only needs nine more posts than a troll will actually make per throwaway account, then?

That's some mighty fine police work there, Lou!

Think of the children

[Randon]

I can't wait til the anti-bullying crowd lobby for something like this wherever their children might be on the internet. Sounds like it is early days though, 20% false positives is pretty darn high.

Re:This, if true, will utterly destroy

[Anonymous Coward]

For the love of all (if anything) that is still good and holy:

TURN

YOUR PHONE

SIDEWAYS

I've refrained from formatting this post (any more) obnoxiously vertical to emphasize.

I run a feminist forum

[spacefem]

and volunteer to help test. We have a steady stream of trolls available for review, a truly endless supply.

Re:What is a 'troll'?

[Drethon]

Just because racial disparaging was acceptable back then doesn't make it right. Though I would question an arrest for speech.

How about state-sponsored trolling?

[mi]

I can see, how this may defeat (ab)users trolling for fun and not suspecting automated detection before it hits them (though, with only 80% accuracy, I dread the thought of the methods expanding out of the virtual realm).

But what about people "trolling" professionally — paid and/or otherwise compelled into it by a state or corporate actor pretending there to exist some kind of "grass-roots" movement? How would it deal with thousands of fake accounts mounting a coordinated assault, posting (while "liking" and "following" each other)?

Some times you may be able to catch accounts posting identical things at the same exact time (and ban them all in bulk), but Russians seem to have fixed that bug in their bots now...

This is turning into another battle like that, in which spammers have fought the best Information Technology minds into a standstill. I doubt, progress against forum-spammers will be much better than that — not when mere technology, however clever, is up against interests of a reasonably powerful state.

Re:This, if true, will utterly destroy

[Stan92057]

Your talking about racism, this article is about internet trolls They are not the same. A person with a different view is not a troll. A person with a different opinion then you is not a troll. Ive been tagged a troll because of my views a few times.Many here will post as anomoue because they know there opinions will be viewed and tagged as a troll.

https://en.wikipedia.org/wiki/Internet_troll

Re:This, if true, will utterly destroy

[Obfuscant]

yes, and now we get into the same sort of pointless useless territory as arguing about what "hacking" means

Because both "troll" and "hacking" have been made into pointless useless words through the magic of "common use" by common people who have no clue what they were supposed to mean.

"An algorithm that can detect trolls" is a meaningless statement. If it is an algorithm, it needs a definition to work from. That definition is not going to be based on historical or accurate usage of the term. In fact, the summary gives you a good idea what it will be based on:

It also observes that higher rates of community intolerance are likely to foster the anti-social behavior and speed the ban.

So, the "definition" of "troll" is going to be "people who display unpopular or angry behavior when confronted by an intolerant social media environment." Gee, anyone slashdotted recently? "Community intolerance" is not the problem, I guess, it's the reaction of people in a supposedly open forum to that intolerance.

There will be no direct definition as such. It will be an empirical model based on correlation between use of angry or unpopular phrases and the subsequent ban of the poster. That's the new "troll". Say enough stuff that people don't like, you're a troll.