Facebook Spares Humans By Fighting Offensive Photos With AI (techcrunch.com)

An anonymous reader writes from a report via TechCrunch: Facebook tells TechCrunch that its artificial intelligence systems now report more offensive photos than humans do. Typically when users upload content that is deemed offensive, it has to be seen and flagged by at least one human worker or user. Such posts that violate terms of service can include content that is hate speech, threatening or pornographic; incites violence; or contains nudity or graphic or gratuitous violence. The content that workers have to dig through is obviously not great, and may lead to various psychological illnesses such as post-traumatic stress disorder. AI is helping to eliminate such a terrible job as it can scan images that are uploaded before anyone ever sees them. Facebook's AI already "helps rank News Feed stories, read aloud the content of photos to the vision impaired and automatically write closed captions for video ads that increase view time by 12 percent," writes TechCrunch. Facebook's Director of Engineering for Applied Machine Learning Joaquin Candela tells TechCrunch, "One thing that is interesting is that today we have more offensive photos being reported by AI algorithms than by people. The higher we push that to 100 percent, the fewer offensive photos have actually been seen by a human." One risk of such an automated system is that it could censor art and free expression that may be productive or beautiful, yet controversial. The other more obvious risk is that such a system could take jobs away from those in need.

Comments

What could possibly go wrong?

[Bruce66423]

Skynet get to control what I'm posting. That will end well.

Re:What could possibly go wrong?

[Anonymous Coward]

I admin a facebook group and the "automoderation" is dumb, FB reports the tamest pictures and I can only imagine it's because the women have big boobs or something. Not even nudity, we're talking about.

Every "approve" I had to do was utterly a waste of time. The post doesn't even appear until I do. I imagine a lot of admins who don't want to run afoul of this start becoming much more conservative of their approvals than the group would normally be just so they don't run afoul of FB big brother.

Re:What could possibly go wrong?

[rainmouse]

I admin a facebook group and the "automoderation" is dumb, FB reports the tamest pictures and I can only imagine it's because the women have big boobs or something. Not even nudity, we're talking about.

Warning - following comment mentions disturbing violent content you may not wish to read about.

I used to work the night shift for a huge MMO played mostly by kids and young teenagers. One of my work queues was investigating suspicious weblinks the players posted, typically to fake clan websites. I would spend a few hours of a night hitting tiny urls for poop porn sites, scam sites, lolshock sites and sometimes paedophile grooming hangouts, suicide blogs etc that required notification to the authorities. It was pretty gross and mostly just the same lemonparties and guys eating poop over and over and over it would be really easys to automate that.... I always turned the darker stuff off immediately, but then one night near the end of a very long shift while really exhausted I saw video footage of 4 young teenagers beating a child to death with hammers.

Can't say why I watched it. I really, really wish I hadn't; If anything the sound was actually worse than the extremely graphic footage.
My point is that I only ever saw that one once, that's probably the kind of rare stuff that would slip past the AI's and hit the real people anyway. Although I can understand why they would create an AI to pre-detect this. I guess some employees are still going to be hit by things that will forever change them.

All in, I would rather human moderators perhaps with an AI to warn them about extreme content rather that than these incidents being used an an excuse for automated, draconian and potentially politically motivated automatic censorship.
I guess it's a long way off yet until we get our first AI whistle-blower.

Re:What could possibly go wrong?

[Anonymous Coward]

I honestly pray you reported what you saw.

I used to be an investigator for the largest ISP in the US. I saw so much pure evil, I left the security side of IT after 4 years. I've got a strong stomach, I saw blood in the military, I'm a "real man" as it were. But some content, especially involving minors is just beyond the pale.

Anyone who would harm a child in a malicious way needs to be executed. Full stop. I have children, and let me tell you that when you are aware of things like the above, you love your kids a little more when you see them at the end of days like that. Sadly, the left just coddles people that should be removed from society. They blame a bad childhood, this, that. It doesn't matter why someone does something evil. It's matter that they did it. Full stop. I don't give a monkey's toss that you're mental, angry, whatever. People that intentionally harm children should be culled from the population. Immediately.

Re:What could possibly go wrong?

[Opportunist]

Warning - following comment mentions disturbing violent content you may not wish to read about.

When did /. become a politically correct playground for the sensitive children that never grew up?

Besides, trigger warnings trigger me, you insensitive clod!

People In Need

[nick_davison]

"The other more obvious risk is that such a system could take jobs away from those in need."

Social Media Nipple Checkers Local 857, like my father and his father before him.

It's hard work on the Internet nippleface but we're a proud people.

Some people might say it's false drama, lamenting the decline of an industry that only goes back a dozen years but we original "ought fourer families" as we like to call ourselves have never known any other way.

I have friends in who were Internet Radio DJs for the four hours that was a thing until smart playlists replaced them. Many of them have never found employment since.

Re:People In Need

[advocate_one]

how good will it be at differentiating from artistic nipples, male nipples, offensive female nipples, accidental nipples? Until it can 100% spot and block only the offensive female nipples there's always a job for you...

Re:People In Need

[jouassou]

Or we could eliminate the problem by just not censoring nipples. If you get PTSD from seeing nipples on Facebook, see a psychologist...

Re:People In Need

[bondsbw]

You can milk anything with nipples.

Missing in the summary: The human misery

[Britz]

What is missing in the Slashdot summary is the misery of the human "digital sanitation workers", who usually have to sort that crap out. There has been some recent reporting on these unfortunate people. I believe this reporting is the reason why Facebook has come forward to show their effort, in order to counter the possible negative impact, if this hits US media outlets.

The German political foundation "Heinrich BÃll Stiftung" did a workshop on this phenomenon. Unfortunately there is little English language reporting I found, as for now. Here is a link to the original source (the workshop):

https://calendar.boell.de/de/e...

But one of the presentations is in English and available on Youtube:

https://www.youtube.com/watch?...

A couple facts:

- the service is called Commercial Content Moderation

- 150.000 people work in this industry in the Philippines alone
- the Philippines is the major site for this job, because while being cheap, they being Christian means they are supposed to have a good sense of what is considered appropriate content in the USA and Europa

- a lot of the workers report "issues" because of the extreme content they have to endure, including relationship problems and substance abuse
- they are not allowed to work longer than 24 month in this job, supposedly because of the issue mentioned above

Re:See you at -1!

[jandersen]

I've seen absolutely horrible things on the internet and I don't have PTSD.

People are different, and one of the great achievements of modern society is that we have developed a culture that shows some level of consideration to even the weakest members of society. And to be fair - there is always a possibility that it isn't the more sensitive that are too sensitive, but the less sensitive that are simply too callous. And in practical terms, if you really enjoy watching graphical portrayals of cruelty, then you will be able to find it, even if it isn't readily available, whereas if you don't, and it is everywhere, then your only option would be to stay away from most of the web; the burden, if you want to avoid something that is everywhere is vastly bigger than the burden of having to find something that isn't readily available.

Re:See you at -1!

[JaredOfEuropa]

Seeing the odd horrible picture probably won't hurt anyone. But having to make a decision to censor or not, thinking about the intent and the context, that is a heavier burden. Now multiply that by a thousand and make it someone's job... It's a well-known problem in police departments that have to go through and catalogue child pornography collections; people on that job don't last very long as a rule.

Re:When offensive becomes politically inconvenient

[AmiMoJo]

Remember when newly inaugurated Pres. Obama thought it'd be a good idea for FB login to be your official ID?

Nope, and the link you provided doesn't say he did. In fact what he suggested is something that many nerds have been asking for.

Imagine you could create pseudo-anonymous identities. You could have them signed by trusted government agencies to say that they have confirmed your real identity, without the need to necessarily share it with other organizations. If it gets compromised I can mark it as dead and set up a new one. Make it distributed, maybe block chain based.

He didn't suggest Facebook at all. You made that up entirely.

Re:See you at -1!

[Gilgaron]

I hadn't thought about it like that, but you're right: it is one thing to close a window quickly with something awful, but to have to consider the image you're going to have to wrap your mind around something awful for far longer.

Re:"Post traumatic stress disorder"

[JustAnotherOldGuy]

I've been more traumatized by not seeing boobs when I hoped I would. :(

I was just about to see them, but then something happened and didn't. I was depressed for weeks.

Seeing boobs would have uplifted me and buoyed my spirits.

here's your problem!

[tekrat]

"its artificial intelligence systems now report more offensive photos than humans do."

Then either the AI is more easily offended than humans, or there's simply less humans working. Maybe if they hadn't fired the whole department last month (except for one guy).

That's what we do here: We talk about a how a particular unit has been less productive so we can cut more heads, of course, knowing that the unit is less productive because we've already reduced them to a skeleton crew. And that's how MBA's get their bonuses while other people get pink slips.