Slashdot Mirror

← Back to Stories (view on slashdot.org)

Facebook Spares Humans By Fighting Offensive Photos With AI (techcrunch.com)

Posted by BeauHD on from the Draconian-laws dept.

An anonymous reader writes from a report via TechCrunch: Facebook tells TechCrunch that its artificial intelligence systems now report more offensive photos than humans do. Typically when users upload content that is deemed offensive, it has to be seen and flagged by at least one human worker or user. Such posts that violate terms of service can include content that is hate speech, threatening or pornographic; incites violence; or contains nudity or graphic or gratuitous violence. The content that workers have to dig through is obviously not great, and may lead to various psychological illnesses such as post-traumatic stress disorder. AI is helping to eliminate such a terrible job as it can scan images that are uploaded before anyone ever sees them. Facebook's AI already "helps rank News Feed stories, read aloud the content of photos to the vision impaired and automatically write closed captions for video ads that increase view time by 12 percent," writes TechCrunch. Facebook's Director of Engineering for Applied Machine Learning Joaquin Candela tells TechCrunch, "One thing that is interesting is that today we have more offensive photos being reported by AI algorithms than by people. The higher we push that to 100 percent, the fewer offensive photos have actually been seen by a human." One risk of such an automated system is that it could censor art and free expression that may be productive or beautiful, yet controversial. The other more obvious risk is that such a system could take jobs away from those in need.

11 of 127 comments (clear)

  1. What could possibly go wrong? by Bruce66423 · · Score: 4, Funny

    Skynet get to control what I'm posting. That will end well.

    1. Re:What could possibly go wrong? by Anonymous Coward · · Score: 4, Interesting

      I admin a facebook group and the "automoderation" is dumb, FB reports the tamest pictures and I can only imagine it's because the women have big boobs or something. Not even nudity, we're talking about.

      Every "approve" I had to do was utterly a waste of time. The post doesn't even appear until I do. I imagine a lot of admins who don't want to run afoul of this start becoming much more conservative of their approvals than the group would normally be just so they don't run afoul of FB big brother.

    2. Re:What could possibly go wrong? by rainmouse · · Score: 5, Interesting

      I admin a facebook group and the "automoderation" is dumb, FB reports the tamest pictures and I can only imagine it's because the women have big boobs or something. Not even nudity, we're talking about.

      Warning - following comment mentions disturbing violent content you may not wish to read about.

      I used to work the night shift for a huge MMO played mostly by kids and young teenagers. One of my work queues was investigating suspicious weblinks the players posted, typically to fake clan websites. I would spend a few hours of a night hitting tiny urls for poop porn sites, scam sites, lolshock sites and sometimes paedophile grooming hangouts, suicide blogs etc that required notification to the authorities. It was pretty gross and mostly just the same lemonparties and guys eating poop over and over and over it would be really easys to automate that.... I always turned the darker stuff off immediately, but then one night near the end of a very long shift while really exhausted I saw video footage of 4 young teenagers beating a child to death with hammers.

      Can't say why I watched it. I really, really wish I hadn't; If anything the sound was actually worse than the extremely graphic footage.
      My point is that I only ever saw that one once, that's probably the kind of rare stuff that would slip past the AI's and hit the real people anyway. Although I can understand why they would create an AI to pre-detect this. I guess some employees are still going to be hit by things that will forever change them.

      All in, I would rather human moderators perhaps with an AI to warn them about extreme content rather that than these incidents being used an an excuse for automated, draconian and potentially politically motivated automatic censorship.
      I guess it's a long way off yet until we get our first AI whistle-blower.

    3. Re:What could possibly go wrong? by Opportunist · · Score: 3, Insightful

      Warning - following comment mentions disturbing violent content you may not wish to read about.

      When did /. become a politically correct playground for the sensitive children that never grew up?

      Besides, trigger warnings trigger me, you insensitive clod!

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
  2. People In Need by nick_davison · · Score: 5, Interesting

    "The other more obvious risk is that such a system could take jobs away from those in need."

    Social Media Nipple Checkers Local 857, like my father and his father before him.

    It's hard work on the Internet nippleface but we're a proud people.

    Some people might say it's false drama, lamenting the decline of an industry that only goes back a dozen years but we original "ought fourer families" as we like to call ourselves have never known any other way.

    I have friends in who were Internet Radio DJs for the four hours that was a thing until smart playlists replaced them. Many of them have never found employment since.

    1. Re:People In Need by advocate_one · · Score: 3, Funny

      how good will it be at differentiating from artistic nipples, male nipples, offensive female nipples, accidental nipples? Until it can 100% spot and block only the offensive female nipples there's always a job for you...

      --
      Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
    2. Re:People In Need by jouassou · · Score: 3, Insightful

      Or we could eliminate the problem by just not censoring nipples. If you get PTSD from seeing nipples on Facebook, see a psychologist...

  3. Missing in the summary: The human misery by Britz · · Score: 4, Interesting

    What is missing in the Slashdot summary is the misery of the human "digital sanitation workers", who usually have to sort that crap out. There has been some recent reporting on these unfortunate people. I believe this reporting is the reason why Facebook has come forward to show their effort, in order to counter the possible negative impact, if this hits US media outlets.

    The German political foundation "Heinrich BÃll Stiftung" did a workshop on this phenomenon. Unfortunately there is little English language reporting I found, as for now. Here is a link to the original source (the workshop):

    https://calendar.boell.de/de/e...

    But one of the presentations is in English and available on Youtube:

    https://www.youtube.com/watch?...

    A couple facts:

    - the service is called Commercial Content Moderation

    - 150.000 people work in this industry in the Philippines alone
    - the Philippines is the major site for this job, because while being cheap, they being Christian means they are supposed to have a good sense of what is considered appropriate content in the USA and Europa

    - a lot of the workers report "issues" because of the extreme content they have to endure, including relationship problems and substance abuse
    - they are not allowed to work longer than 24 month in this job, supposedly because of the issue mentioned above

  4. Re:See you at -1! by JaredOfEuropa · · Score: 3, Interesting

    Seeing the odd horrible picture probably won't hurt anyone. But having to make a decision to censor or not, thinking about the intent and the context, that is a heavier burden. Now multiply that by a thousand and make it someone's job... It's a well-known problem in police departments that have to go through and catalogue child pornography collections; people on that job don't last very long as a rule.

    --
    If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
  5. Re:When offensive becomes politically inconvenient by AmiMoJo · · Score: 3, Informative

    Remember when newly inaugurated Pres. Obama thought it'd be a good idea for FB login to be your official ID?

    Nope, and the link you provided doesn't say he did. In fact what he suggested is something that many nerds have been asking for.

    Imagine you could create pseudo-anonymous identities. You could have them signed by trusted government agencies to say that they have confirmed your real identity, without the need to necessarily share it with other organizations. If it gets compromised I can mark it as dead and set up a new one. Make it distributed, maybe block chain based.

    He didn't suggest Facebook at all. You made that up entirely.

    --
    const int one = 65536; (Silvermoon, Texture.cs)
    SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
  6. Re:"Post traumatic stress disorder" by JustAnotherOldGuy · · Score: 3, Funny

    I've been more traumatized by not seeing boobs when I hoped I would. :(

    I was just about to see them, but then something happened and didn't. I was depressed for weeks.

    Seeing boobs would have uplifted me and buoyed my spirits.

    --
    Just cruising through this digital world at 33 1/3 rpm...