Google Releases an AI Tool For Publishers To Spot and Weed Out Toxic Comments (bbc.com)

Google today launched a new technology to help news organizations and online platforms identify and swiftly remove abusive comments on their websites. The technology, called Perspective, will review comments and score them based on how similar they are to comments people said were "toxic" or likely to make them leave a conversation. From a report on BBC: The search giant has developed something called Perspective, which it describes as a technology that uses machine learning to identify problematic comments. The software has been developed by Jigsaw, a division of Google with a mission to tackle online security dangers such as extremism and cyberbullying. The system learns by seeing how thousands of online conversations have been moderated and then scores new comments by assessing how "toxic" they are and whether similar language had led other people to leave conversations. What it's doing is trying to improve the quality of debate and make sure people aren't put off from joining in.

Comments

Good news!

[Anonymous Coward]

So all those posts from SJWs can easily be removed. I'm a fan of this.

"Toxic" comments huh?

[Mashiki]

Well, let's all bow down the moral arbiters of justice then. I'm sure that they'll be right on top of removing speech they disagree with. Then moving onto the useful idiots that cheered this on in the first place.

If you're willing to remove some speech because it makes you upset, there's nothing stopping others from doing the same to you later.

Re:"Toxic" comments huh?

[Mashiki]

Yeah here's the problem. Let's look at a site like neogaf, ever wonder why at one time it was the place to go and developers would post there and people would leak information. And now developers don't? Active participation is down? It was the moderation as you pointed out, but let's look at their definition of a troll, which basically boils down to "anyone who doesn't subscribe to the narrative." Let's look now, at what will get you banned. Have a contrary view of feminism? Banned. Support some ideas of a MRA? Banned. Have differing view points on global warming? Banned. Prefer the xbox vs playstation? Likely banned. Don't like your vidya characters to look like they were hit with a bat? Banned.

The problem is in many cases, moderators especially in this day and age use that as a form of power projection. A good example of this is /r/politics or /r/canadapolitics where you have moderators who ban people for pointing out factual information because it goes against the prevailing group think. Have a nice article about how forums become infiltrated by people pushing authoritarian viewpoints.

I ran a BBS in my teens, and it got large enough that I was considering applying for a regional fido:net hub. At nearly 700 people and 4 nodes, I picked moderators for my forums who weren't assholes, who usually worked, and if they stepped over the line they were given one warning then booted. As sysop, I expected my mods to be impartial. And if a friend was involved to pass the issue to another mod to deal with. I ran on Renegade.

Re:"Toxic" comments huh?

[johanw]

Posts are not removed, only given score. I can even conveniently select I want to see all posts, even those who are modded down. Having this treshold set to -1 shows my "trust" in the moderation process.

Re:"Toxic" comments huh?

[Kierthos]

First, freedom of speech does not mean freedom from consequences.

Second, all sides (liberals, conservatives, moderates) have trolls and jerks among them.

Third, if it's a forum run by a non-government source, the First Amendment doesn't apply. They don't have to let someone (or anyone for that matter) use their forum. Especially if they break the rules of the forum.

Fourth, there is a difference between open debate and toxic posting. I fully believe in open debate. You might not change my mind on a topic, but I welcome an honest and open debate. But some people don't want to debate. They want to shout down, or harass, or whatever. And unless you have a crack moderator staff, you can't always keep on top of this sort of thing, especially if it's a large/popular forum.

I speak from some experience on both sides of this. Some years ago, I was a moderator on an MMO forum for Dark Age of Camelot. It wasn't a forum run by the company that made DAOC, it was a fan forum. I ended up as a moderator (much to my surprise) for regularly giving helpful information and trying to be a good guy. While I was a moderator, we had to put people in various lengths of "time out" for being jerks, or abusing the forum rules, or spamming, or whatever.

Shockingly, the moderators weren't always the most popular people with the friends of the trolls.

On the other side of the fence, there was (possibly still is) a forum for a group called Christian Exodus. Among their stated goals is (as a last resort), the idea of moving enough followers to a specific U.S. state (South Carolina), to try and influence enough elections and legislators to make it a more Christian state. Actually, their absolute last resort was to have SC secede from the U.S.

I was banned from their forums because I would do things like remind them that the last time that SC seceded, it didn't go all that well, or remind them that there's more than one version of the Bible (they were very focused on returning to doing _exactly_ what the Bible said), and so on. I was enjoying myself, but I can't really say it came as a shock when I was banned from posting on their forum.

Awesome

[admin7087]

Great for our echo chambers, so we can hear more from people whose opinions we already have. What could go wrong with that? /s

The sign of our times: Censorship-Supression City

[adosch]

I can't speak for everyone else, but all this AI, machine learning, heavy algorithm, neural network, data mining that's been going on for well over a decade now and has become almost normal in terms of tech news conversation is really scary as hell.

For starters, the claim to the quote/unquote "internet" and plaguing social media is it's given absolutely everyone a platform to opinion-ate, alienate, berate, tolerate and flat out hate anyone, any topic, any agenda, any other opinion, idea, thought, preference, look, feel, ect. Let's face it: all that in itself alone as opened pandora's box to a metric shit-ton of people who flat out should not be sharing anything that bubbles in their skull. So now we all sit here with big thumb-tapping or keyboard-clacking loud mouths who can't act appropriately in a digital world.

But I have to say, when the hell did everyone become a bunch of sensitive sally's in terms of taking everything at face value, and buying into some internet handles drivel (or lack there of), hate speech. Look at slashdot and the anonymous coward approach? Hell at least we provide anonymity and low rank to toxic troll garbage here.

All that aside, we don't 'remove' it, cover it up and scrub it away because everyone likes to wave the I-am-offended-all-the-time flag. It becomes part of the culture, ambiance (if you laugh it it, I guess) and overall conversation. We don't un-ring bells, do we? I don't see how that's any different digitally.

By whom? For what exactly?

[s.petry]

Who will be filing the most complaints? The people with money to pay astroturfers and sockpuppets? That is what we have everywhere else, so why would Google's app be any different?

What will the complaints be about? Same thing we see everywhere else, which is anything not pro communist/extreme leftist?

This is a promotion of fascism, not freedom.

How this will kill Truth.

[geekmux]

If you have a troll problem, then moderate properly by banning. Censorship is not the answer because truth will ultimately suffer.

The definition of toxic will never be a constant, and I can already seen forums looking for revenue streams to favor those paying for certain "filters".

Re:Of course

[meta-monkey]

It's not some government control, and it's not something you can't already do on your own.

The issue of "government control" is moot when the companies that control the government also own the social media platforms. See my .sig.

Solves the WRONG problem

[DickBreath]

This solves entirely the WRONG problem.

What is needed is an AI that deletes all first posts. Think about how wonderful that would be. It would work, methodically, persistently and tirelessly no matter how many times a first post is submitted. No matter who submits it. No matter how many people try to get the first post.