Slashdot Mirror

← Back to Stories (view on slashdot.org)

Facebook's 'Rosetta' System Helps the Company Understand Text Within Image, Which is Crucial In Handling Memes, Flagging Abusing Content (techcrunch.com)

Posted by msmash on from the how-about-that dept.

Facebook announced on Tuesday a new AI system, codenamed "Rosetta," which helps teams at the company as well as those at Instagram identify text within images to better understand what their subject is and more easily classify them for search or to flag abusive content. From a report: It's not all memes; the tool scans over a billion images and video frames daily across multiple languages in real time, according to a company blog post. Rosetta makes use of recent advances in optical character recognition (OCR) to first scan an image and detect text that is present, at which point the characters are placed inside a bounding box that is then analyzed by convolutional neural nets that try to recognize the characters and determine what's being communicated. This technology has been in practice for a while -- Facebook has been working with OCR since 2015 -- but implementing this across the company's vast networks provides a crazy degree of scale that motivated the company to develop some new strategies around character detection and recognition.

8 of 45 comments (clear)

  1. crucial at suppressing speech by Anonymous Coward · · Score: 3, Interesting

    It will be used to suppress speech.

  2. Abusive Content? by Jarwulf · · Score: 2

    How is content ie simple information/knowledge abusive? Does it come out of the screen and berate you?

    1. Re:Abusive Content? by Anonymous Coward · · Score: 2, Interesting

      Think back to the very worst moment of your life.

      Now imagine that somebody recorded a video of it and posted it to Facebook.

      To me, that video is "simple information".
      To you, the video is not just "simple information".

      Should the video be taken down?
      Let's consult the Platinum Rule: Treat others the way they want to be treated
      My answer is your answer to the question: Do you want the video of the worst moment of your life taken down?

    2. Re:Abusive Content? by butchersong · · Score: 2

      No. That seems very silly to me. What if the worst moment of my life was the towers coming down in NY? Does that mean that no one gets to post videos of that day? I may be able to get behind owning images of myself and taking down videos of me personally (though I doubt it) but I don't get to own and suppress ideas.

    3. Re:Abusive Content? by butchersong · · Score: 2

      The AC didn't specify that the video had to include an image of me personally or one of my loved ones. That is a narrower argument but doesn't address the story posted for Facebook. We're talking about "hurtful" content. I may feel some personal attachment to a certain idea. Say my parents died in a camp in WW2 or something of that sort. That doesn't mean I should be able to suppress holocaust denial. Say instead that someone was posting pictures of my dead parents in the camp, in that case I can see an argument but I'm not sure still which side of it I would end on. In this case we're not even talking about images themselves but ideas communicated in the images through text. So I don't think my comment the one going off on a tangent. I think the AC's comment was the tangent... unless I'm misunderstanding the story.

  3. Translation by Anonymous Coward · · Score: 3, Insightful

    "We can now identify conservative and Trump-supporting users much more rapidly in order to ban them for #WrongThink!"

  4. Used to filter ads by Dan+East · · Score: 3, Insightful

    Of course this tech is being spun to "save the children", but it is also used to screen all advertisements that run on FB. They do not want ads to contain much text - less than 20% of the area of the ad image can be text. This is detected automatically using the technology described, and their system will stop the ad if it doesn't meet that requirement.

    We've found that images with less than 20% text perform better.
    To create a better experience for audiences and advertisers, ads that run on Facebook, Instagram and Audience Network are subject to a review process that looks at the amount of image text used in your ad. Based on this review, ads with higher amounts of image text may not be shown. Keep in mind that some ad images may qualify for an exception. For example, book covers, album covers and product images usually qualify for an exception.

    https://www.facebook.com/busin...

    And from the blurb:

    detect text that is present, at which point the characters are placed inside a bounding box

    Thus the area of the bounding boxes (after performing a union) can be at most 20% of the area of the image.

    --
    Better known as 318230.
  5. Revenge time has come! by devslash0 · · Score: 3

    Now we should start writing text on memes using Captcha fonts.