Slashdot Mirror


Researchers Developing An Algorithm That Can Detect Internet Trolls

An anonymous reader writes Researchers at Cornell University claim to be able to identify a forum or comment-thread troll within the first ten posts after the user joins with more than 80% accuracy, leading the way to the possibility of methods to automatically ban persistently anti-social posters. The study observed 10,000 new users at cnn.com, breitbart.com and ign.com, and characterizes an FBU (Future Banned User) as entering a new community with below-average literacy or communications skill, and that the low standard is likely to drop shortly before a permanent ban. It also observes that higher rates of community intolerance are likely to foster the anti-social behavior and speed the ban.

41 of 279 comments (clear)

  1. In other words by fustakrakich · · Score: 5, Insightful

    Automated censorship. Eh, saves us the trouble, I guess

    --
    “He’s not deformed, he’s just drunk!”
    1. Re:In other words by Kyogreex · · Score: 5, Informative

      The original paper doesn't seem to be about automatic banning at all; that seems to have been added to the headline and the article linked to here (and therefore the summary). The paper says this: "automatic, early identification of users who are likely to be banned in the future."

      While that identification could be used for automatic banning, I think it would be more likely to be used to flag potential problem users, which could be very useful in determining which reported posts to investigate first rather than dealing with all of the "I don't like this post so I'm reporting it" instances.

    2. Re:In other words by Shadow+IT+Ninja · · Score: 2

      TFA basically says that you can detect trolls early on but, the faster you censor them, the more antisocial they become.

    3. Re:In other words by dcollins117 · · Score: 2, Insightful

      It would be much better to have a system that HIDES users content by default, than to delete it. Then, people scrolling all posts (including hidden) would be able to report mistakes in the system.

      From my experience if you delete content or ban a troll, it just encourages them to troll more using a different account, usually from a different IP address.

      The most effective way I found to deal with problem users is to make their bad comments only visible to them. That way it appears to them that they've had their say and no one responded to it. Without feedback to encourage them, trolls either quickly get bored and go elsewhere or sometimes they'll surprise you and produce better quality comments.

  2. This is fucking stupid. by Anonymous Coward · · Score: 5, Insightful

    Trolls are usually above average literacy and trying to skilfully cause a fight. It's easy to identify "illiterate" people and humans are way too quick to judge someone who cannot spell as having nothing to contribute or (worse) malicious, but these are not trolls. This is just another classist meme where the person is judged positively by the overcomplexity of their language and convolution of their sentences, as this must mean they have been educamated right.

    BTW I went to a £30k/year British boarding school, so I have no axe to grind, nor insecurity about describing things as they are.

    1. Re:This is fucking stupid. by SethJohnson · · Score: 2

      Of course, I haven't read the article, but I think the summary has applied the word "troll" in a different way than this. I think the researchers are seeking to reduce the racist, homophobic, etc. trash comments frequently posted to YouTube video comments.

      As you note here, a sophisticated troll is not easily detectable via AI.

    2. Re:This is fucking stupid. by Mr+D+from+63 · · Score: 4, Informative

      It is stupid to me because it does not solve a problem. Detecting trolls is certainly not a problem, dealing with them is. They need to work on algorithm for that.

    3. Re:This is fucking stupid. by marcello_dl · · Score: 4, Funny

      > Trolls are usually above average literacy.
      Your right.

      --
      ---- MISSING MISCELLANEOUS DATA SEGMENT --- [sigdash] trolololol
    4. Re:This is fucking stupid. by Mr+D+from+63 · · Score: 4, Funny

      No need. Remove anonymity from said sites and the problem is solved. Better yet, don't bother having user comments everywhere.

      posted by an AC. LOL.

    5. Re:This is fucking stupid. by Anonymous Coward · · Score: 4, Funny

      Douché.

    6. Re:This is fucking stupid. by Anonymous Coward · · Score: 2

      humans are way too quick to judge someone who cannot spell as having nothing to contribute

      I'd sooner judge grammar than spelling. Poor grammar indicates a fundamental misunderstanding of the language. Poor spelling only indicates carelessness or memorization issues.

      For example, I've noticed a recent trend (in American English) where native speakers are confusing adverbs with adjectives -- namely, by dropping the -ly suffix that defines most adverbs in English. For example, "I responded appropriate." This is a mistake I would fully expect from a non-native speaker, but a native speaker? I'm almost embarrassed myself. This is very basic stuff, and it's not just the spoken word -- they're actually writing like this. Another trend is where people confuse past tense (-ed) with the gerund. (A gerund is a noun made from a verb by adding -ing.) For example, "the car needs washed." Whatever this is, it's not English. The two correct options, of course, are "the car needs washing" (gerund) or "the car needs to be washed". Again, I'm almost embarrased myself.

    7. Re:This is fucking stupid. by plover · · Score: 4, Insightful

      While I believe that people who are less sensitive tend to thrive more than others, I don't agree that "thicker skin" is a workable solution. Too many people have fragile emotional states and simply don't have the neural hardware psychological capacity required to dismiss the hate and insults that often happen on line. There have been some high-profile suicides among teens who were attacked online, and who knows how many people remove themselves from public comment because of the hate they've received? For safety reasons I don't think society should completely abrogate the forums to the trolls.

      Does that not mean some people are overly sensitive? Sure. But just as we shouldn't velour-line the internet to cater to absolutely every person with a psychological disorder; we also don't have to tolerate the diarrhea that spews forth from the trolls. We don't have to draw a hard-and-fast line on the ground, either, and define "these words are always 100% bad in 100% of situations". Instead, we should be welcoming humans in the loop, asking them to pass judgment when needed. That gets us to a more fluid state than full automation. It also lets the user choose. Don't like the judgment process on Slashdot? Don't hang out on Slashdot.

      I know full automated filtering is the holy grail of internet forum moderation, but as soon as you deploy a filter it becomes a pass/fail test for the trolls, who quickly learn to adapt and evade it. Human judges can adapt, too, and are about the only thing that can; there are simply too few for the volume of trolls out there. A tool like this might help them scale this effort to YouTube volumes.

      --
      John
    8. Re:This is fucking stupid. by mobby_6kl · · Score: 2

      Thickness of skin has nothing to do with it, I'm pretty much impossible to offend or seriously piss off. The real problem with trolls is that they're a huge waste of everyone's time, even if you can ban/ignore them, you still have to read their posts at least once first.

  3. Re:This, if true, will utterly destroy by Captain+Splendid · · Score: 2

    What, no love for YouTube?

    --
    Linux, you magnificent bastard, I read the fucking manual!
  4. Dear algorithm by ArcadeMan · · Score: 4, Funny

    I don't want to talk to you no more, you empty-headed animal food trough wiper! I fart in your general direction! Your mother was a hamster and your father smelt of elderberries!

    1. Re:Dear algorithm by Kyont · · Score: 3

      [beep] [boop] [churn] [beep] User 2766669 identified as Python quoter. All further posts automatically accepted. Add automatic +1 Funny for ID ending in 69. [beep] [whistle]

      --
      You shall see a cow on the roof of a cotton house.
  5. Unicorns, skittles, rainbows, etc. by pla · · Score: 4, Insightful

    within the first ten posts after the user joins

    So, this algorithm only needs nine more posts than a troll will actually make per throwaway account, then?

    That's some mighty fine police work there, Lou!

  6. I chuckled... by benjfowler · · Score: 3, Funny

    ... at the term "Future Banned User" (FBU).

    Isn't that just a nice way of saying "somebody our algorithm flags as a complete asshat"

  7. "Old" vs "new" trolling by RogueyWon · · Score: 5, Interesting

    Your mistake is in using the "classic" definition of "troll" - somebody who sets out to deliberately cause fights on a forum. Trawl through the archives of Slashdot and you will find many instances of this kind of trolling - and yes, the people doing it are often highly literate (and, when they do it right, sometimes very funny with hindsight).

    But the term "trolling" has gone political these days and is routinely used to describe any form of online behaviour that the speaker doesn't approve of. So everything from outright criminal behaviour (eg. threats of immediate violence) at one end of the scale through to disagreeing with a forum's established groupthink (however respectfully) at the other.

    And yes, it has become a favourite term of the intellectually insecure, whenever they want to shout down an opposing point of view without engaging with it. In fact, conflating those two extremes I mention above under the same term is outright beneficial for the easily offended, as it allows them to group polite dissenters together with the mouth-foaming loons.

    1. Re:"Old" vs "new" trolling by meta-monkey · · Score: 4, Interesting

      Back in my day, trolling meant something!

      Ten plus years ago I used to troll /. as "Fux the Penguin" (some of my favs) and it was great fun. The system was:

      1) Get in early on a new story. You don't want to get buried under 100 comments.
      2) Lists and quotes are good. Everybody stops to read something with HTML formatting.
      3) Start reasonable. The first paragraph should sound rational.
      4) The next paragraph should include minor errors of fact or logic, but still be mostly reasonable. Just...wrong.
      5) The minor errors of fact and logic in the middle paragraphs should lead to a completely ridiculous conclusion that /.ers would hate, like running Windows, or requiring government approval for encryption technologies.
      6) Watch the post go to +5 insightful because mods don't actually read comments.
      7) lulz at people who write 8 paragraphs dissecting all my mistakes.
      8) -1 Troll.
      9) +5 Funny.

      Today the media conflates "trolling" with "abusive asshole." I think they misunderstand the word "troll." "Trolling" meant "fishing." To dangle bait for newbs to take and work themselves into a lather, and then laugh at those who don't get the joke. It was performance art. Today they think "troll" is referring to monsters who live under bridges. But no, people who stalk others on the Internet and hurl insults at them (or worse) are not "trolls." They are abusive assholes. It's sad.

      And it requires no skill. Trolling is a art.

      --
      We don't have a state-run media we have a media-run state.
    2. Re:"Old" vs "new" trolling by Megane · · Score: 2, Interesting

      Back in my day, trolling meant nothing!

      Twenty plus years ago, I used to hang out on alt.religion.kibology, where trolling was invented. Someone would post bait (hence the word troll derived from "trolling for newbies") to a newsgroup, adding an "audience" group such as alt.religion.kibology to the newsgroups header line. Stuff like mentioning "Majel Barrett Shatner" on a star-trek group, or intentional misspellings of whatever the group was obsessed with. Then you just sit back and enjoy your popcorn while you watch all the threads from one place. It wasn't even about annoying people as much as it was about what you could get with really pathetic "bait".

      Later, cross-group trolling was added, where a message would be posted to two or more groups plus the audience group. If you picked your groups right, they would flame each other quite nicely, and it would be time to get another bag of marshmallows.

      But yes, today's meaning of "troll" has shifted to what used to be just plain flames or flame-baiting.

      --
      #naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
    3. Re:"Old" vs "new" trolling by itzly · · Score: 3
    4. Re:"Old" vs "new" trolling by itzly · · Score: 2

      The "new" meaning (an individual that's deliberately abusive or deliberately fans the flames)

      Actually, the new meaning is: someone who holds an unpopular opinion.

  8. Poof! by RogueWarrior65 · · Score: 4, Funny

    There goes Gawker.

  9. Re:This, if true, will utterly destroy by Anonymous Coward · · Score: 3, Funny

    Yes, well, they should definitely ban people who can't point a fucking camera, and probably have them arrested

  10. Re:What is a 'troll'? by Chrisq · · Score: 4, Interesting

    Anybody who tells the truth that the scum in power don't want you to hear, apparently...

    In days when someone can be attested for quoting from a published book by Winston Churchill I have to agree.

  11. Research by Sir_Eptishous · · Score: 4, Funny

    Researchers have detected researchers detecting an algorithm detecting researchers researching.

    --
    We play the game with the bravery of being out of range
  12. Oblig XKCD by Anonymous Coward · · Score: 3, Funny

    https://xkcd.com/810/

    Seems relevant.

  13. Uh Oh... by edibobb · · Score: 3, Funny

    Looks like my days on the internet are numbered.

  14. Re:This, if true, will utterly destroy by Anonymous Coward · · Score: 2, Insightful

    For the love of all (if anything) that is still good and holy:

    TURN

    YOUR PHONE

    SIDEWAYS

    I've refrained from formatting this post (any more) obnoxiously vertical to emphasize.

  15. marking category cannot be used properly by Eravnrekaree · · Score: 2

    The word troll is a pointless word which is misused by people who mainly want to villify those who disagree with them, and excuse for people who do not want anyone else to be able to express opinions except for the ones they approve, to censor anyone elses opinions they do not like. Thus, the marking in almost all cases is abused and has no real purpose except for censorship. Obviously, since a message board should be a place for discussion and expressing of differing views and opinions, such is contrary to the purpose of message boards to begin with, to express ones views and to debate subjects.

    The fact is, expressing a view someone else disagree with is not something we should censor, and the tr*** accusation is just an excuse for censorship. As long as the poster honestly believes in what they are posting, its not a tro**, their are posting their view to express their position for the sake of the issue itself, rather than to annoy anyone. Maybe, a tr*** might be someone who posts things they do not agree with for the sole purpose to annoy. However, since it is impossible for anyone to know whether or not someone posting a message honestly believes in what they say, it is impossible to determine if a message is a tr***, or not. it is also impossible to know if someone is posting a view just because they are interested in a subject and have a view on it, rather than trying to annoy anyone.

    The fact is, if someone is annoyed by something, the person responsible for being annoyed is the person who is annoyed, its all in the eye of the beholder, some people will agree with something and others will disagree, you have to allow for a difference of opinions and views. It is always the case that someone will disagree with someone else says, it does not mean that the message was posted with the sole intent to annoy, but the reader of a message may still misconstrue or assume that even though it is impossible for them to truly know that. It is okay and important for people to be able to post messages they know will annoy others, because, anything can annoy anyone, its impossible to post a view or position on anything if one has to fear annoying someone.

    The tr*** thing could only apply to messages written with the sole intent to annoy, But as I said, its impossible for anyone to know if that was the sole intent, to be the sole intent, the person would have to not honestly believe in what they say, otherwise they are posting because they believe in what they say and think that its important.

    That is why the marking on a message cannot be used legitimately and fairly, there is impossible for anyone to know if a message is a tr8**. Thats why, we should remove the marking from messaging and bullitin board systems. As I said before, in 100% of cases the marking is abused, it cannot be used in any proper, fair way, because it is a fundamentally flawed feature.

    It would be best policy on these matters is that bullitin boards should have a rule against computer generated and mass posted advertising, but thats about it.

  16. Re:What is a 'troll'? by circletimessquare · · Score: 2

    i would describe a troll as the opposite:

    1. bullying of common folk

    2. and carefully anonymous, as a rule

    there's nothing wrong with whistleblowers, criticism, against leaders of industry, government bureaucrats and police, politicians, etc. but this isn't trolling. this is hiding to avoid *unfair* repercussions. there's nothing wrong with going after power as you allude to

    but hiding and attacking a regular guy is trying to avoid *fair* repercussions

    they spew venom only when they know there are no repercussions to their real life. so their behavior is less about truth and more about bizarre asocial need due to personality disorder

    if someone attacks someone with power, they can be anonymous, and that's not trolling

    if someone attacks a regular person, and uses their real name, that's not trolling

    but if someone attacks just a regular guy, from careful hiding, that's trolling

    and that's not free speech: that's a personality disorder degrading the commons for a sick asocial need. free speech means: i am free to speak, as myself, with my real name. i cloak myself only if i am going to be unfairly targeted. that's honorable. but if i hide to bully regular people, that's trying avoid fair repercussions. that's not honorable, that's trolling, and it must be stamped out to preserve forums. because where such behavior proliferates, everyone socially normal leaves: they don't want anything to do with such shit, and the forum dies. it's not censorship to squash anonymous hate directed at regular people, that's simply preserving common standards of decency which is the only way truly free speech can occur

     

    --
    intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
  17. I run a feminist forum by spacefem · · Score: 3, Insightful

    and volunteer to help test. We have a steady stream of trolls available for review, a truly endless supply.

  18. How about state-sponsored trolling? by mi · · Score: 3, Insightful

    I can see, how this may defeat (ab)users trolling for fun and not suspecting automated detection before it hits them (though, with only 80% accuracy, I dread the thought of the methods expanding out of the virtual realm).

    But what about people "trolling" professionally — paid and/or otherwise compelled into it by a state or corporate actor pretending there to exist some kind of "grass-roots" movement? How would it deal with thousands of fake accounts mounting a coordinated assault, posting (while "liking" and "following" each other)?

    Some times you may be able to catch accounts posting identical things at the same exact time (and ban them all in bulk), but Russians seem to have fixed that bug in their bots now...

    This is turning into another battle like that, in which spammers have fought the best Information Technology minds into a standstill. I doubt, progress against forum-spammers will be much better than that — not when mere technology, however clever, is up against interests of a reasonably powerful state.

    --
    In Soviet Washington the swamp drains you.
  19. What's the false negative rate by Sir_Substance · · Score: 2

    I'm secretly hoping the 80% success rate indicates that everyone else has a 1 in 5 chance of getting randomly banned, troll or not.

    I feel this would noticeably improve many online communities.

  20. Re:This, if true, will utterly destroy by Stan92057 · · Score: 4, Insightful

    Your talking about racism, this article is about internet trolls They are not the same. A person with a different view is not a troll. A person with a different opinion then you is not a troll. Ive been tagged a troll because of my views a few times.Many here will post as anomoue because they know there opinions will be viewed and tagged as a troll.

    https://en.wikipedia.org/wiki/Internet_troll

    --
    Jack of all trades,master of none
  21. Re:This, if true, will utterly destroy by Immerman · · Score: 4, Funny

    Nonsense. Next you'll be suggesting that the technology might be intentionally abused to silently bias "unmoderated" conversations about [REDACTED], which would be a frightful step toward &^%- - -
    [REMAINDER OF COMMENT DISCARDED AS TROLLING]

    --
    --- Most topics have many sides worth arguing, allow me to take one opposite you.
  22. Re:This, if true, will utterly destroy by circletimessquare · · Score: 2

    yes, and now we get into the same sort of pointless useless territory as arguing about what "hacking" means

    no one owns the definition of a word, and meaning changes over time. the common perception of the term "hacking" and the technically and historically more accurate usage of the term "hacking" are separate and equally valid. not because i say so, but because of the authority of common use

    likewise, the strict historical definition of "troll" and the more common meaning of any asocial hate filled speech being "trolling", which racism clearly qualifies as, also coexist

    now let's argue about the meaning of "terrorist"

    zzz

    --
    intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
  23. Trolling vs. Different Viewpoint by msobkow · · Score: 3, Interesting

    Unfortunately, many people think that if you express a different viewpoint or opinion than the masses that you're trying to start an argument or a fight. Why is society so hell-bent on crushing dissenting opinions? And not merely silencing them, but villifying them?

    I've often been tagged as "trolling" because I don't agree with the crowd. If you knew me personally, you'd know very well that I'm not trying to start a fight, just expressing my opinion. Just because it is not the popular viewpoint doesn't mean my views aren't valid.

    Here on Slashdot, I often see people flagged as being trolls just because they don't follow the masses. You'd think a site full of outcasts and oddballs like programmers and technologists would be more accepting of alternative views, but the exact opposite seems to be the case.

    --
    I do not fail; I succeed at finding out what does not work.
  24. Re:This, if true, will utterly destroy by Immerman · · Score: 2

    Right, CBS news proper is not. So where's the problem? You go into an unmoderated forum on their site, see a bunch of user-generated vitriol posted alongside an ad - are you really going to hold it against the advertised brand/product?

    Now it would be different if we were talking about an ad on HateFilledRacism.com, but even there the advertiser would get a substantial benefit from the echo-chamber effect - how often is anyone who would hold their choice of advertising venue against them going to actually visit the site to notice? They do run the risk of getting called out by a "virtuous" troll, and having public attention brought to their poor choice, but they have to weigh that risk (and the odds of being able to wave it off as a "communication mistake" with their ad-serving partners) against the business that it generates.

    --
    --- Most topics have many sides worth arguing, allow me to take one opposite you.
  25. Re:This, if true, will utterly destroy by Obfuscant · · Score: 3, Insightful

    yes, and now we get into the same sort of pointless useless territory as arguing about what "hacking" means

    Because both "troll" and "hacking" have been made into pointless useless words through the magic of "common use" by common people who have no clue what they were supposed to mean.

    "An algorithm that can detect trolls" is a meaningless statement. If it is an algorithm, it needs a definition to work from. That definition is not going to be based on historical or accurate usage of the term. In fact, the summary gives you a good idea what it will be based on:

    It also observes that higher rates of community intolerance are likely to foster the anti-social behavior and speed the ban.

    So, the "definition" of "troll" is going to be "people who display unpopular or angry behavior when confronted by an intolerant social media environment." Gee, anyone slashdotted recently? "Community intolerance" is not the problem, I guess, it's the reaction of people in a supposedly open forum to that intolerance.

    There will be no direct definition as such. It will be an empirical model based on correlation between use of angry or unpopular phrases and the subsequent ban of the poster. That's the new "troll". Say enough stuff that people don't like, you're a troll.