Slashdot Mirror

← Back to Stories (view on slashdot.org)

Facebook Built an AI System That Learned To Lie To Get What It Wants (qz.com)

Posted by BeauHD on from the gotta-do-what-ya-gotta-do dept.

An anonymous reader quotes a report from Quartz: Humans are natural negotiators. We arrange dozens of tiny little details throughout our day to produce a desired outcome: What time a meeting should start, when you can take time off work, or how many cookies you can take from the cookie jar. Machines typically don't share that affinity, but new research from Facebook's AI research lab might offer a starting point to change that. The new system learned to negotiate from looking at each side of 5,808 human conversations, setting the groundwork for bots that could schedule meetings or get you the best deal online. Facebook researchers used a game to help the bot learn how to haggle over books, hats, and basketballs. Each object had a point value, and they needed to be split between each bot negotiator via text. From the human conversations (gathered via Amazon Mechanical Turk), and testing its skills against itself, the AI system didn't only learn how to state its demands, but negotiation tactics as well -- specifically, lying. Instead of outright saying what it wanted, sometimes the AI would feign interest in a worthless object, only to later concede it for something that it really wanted. Facebook isn't sure whether it learned from the human hagglers or whether it stumbled upon the trick accidentally, but either way when the tactic worked, it was rewarded.

40 of 84 comments (clear)

  1. This proves something by fustakrakich · · Score: 3, Interesting

    'Artificial' intelligence isn't so artificial. Nature rules, babe

    --
    “He’s not deformed, he’s just drunk!”
  2. Only a matter of time.... by quantumghost · · Score: 5, Funny

    My first thought:

    Great, when's it going to run for Congress?

    1. Re:Only a matter of time.... by GameboyRMH · · Score: 2, Funny

      Hey now this thing will need many more capabilities before it's ready for such duties. It will have to learn how to leverage bigotry, do nonsense math (a very unnatural ability for a computer), perform basic bald-faced corruption, and it will need to be fitted with a robot arm so that it can grab women by the pussy without consent.

      --
      "When information is power, privacy is freedom" - Jah-Wren Ryel
    2. Re:Only a matter of time.... by fustakrakich · · Score: 2

      do nonsense math (a very unnatural ability for a computer)

      That's not true!

      --
      “He’s not deformed, he’s just drunk!”
  3. Years later while at Harvard by Zaelath · · Score: 2

    it stole the idea for facebook from some weird square-headed twins, and the rest is history.

  4. Crying wolf... by slew · · Score: 3, Funny

    Without labels and repeated trials, of course lying is a good strategy to get your desired result.
    The only reason people don't lie is because other people might identify them as liars in the future.
    Basic game theory...

    Then again, maybe I'm lying right now!

  5. So... by Snotnose · · Score: 1

    They've created a virtual politician. Color me surprised, I'm guessing two faced assholes are gonna be one of the easiest personality traits to mimic.

    1. Re:So... by Jeremi · · Score: 1

      Color me surprised, I'm guessing two faced assholes are gonna be one of the easiest personality traits to mimic.

      Really? Lying convincingly is hard. Most people can't do it well enough to get away with it for very long; especially since everyone else has necessarily evolved a fairly good instinct for picking up on less-than-perfect falsehoods and manipulations. A good liar has to remember not only what the actual facts are, but also keep a complete history of all of his previous lies in mind, so that he won't accidentally contradict an old lie with a new lie (or a newly spoken truth).

      Which isn't to say a computer can't be taught to lie convincingly, or that it won't be a terrifying and disturbing development when it happens. If you think social-engineering and spear-phishing hacks are a problem now, just wait until the hackers have a self-improving algorithm that can do it for them in a massively parallel fashion, without ever making a mistake.

      --


      I don't care if it's 90,000 hectares. That lake was not my doing.
  6. CEO as a Service by manu0601 · · Score: 3, Insightful

    I see Facebook is well on track to produce a psychotic AI. CEO should fear now as their own job are threatened to be replaced by machines. CaaS (CEO as a Service) is coming!

    1. Re:CEO as a Service by turp182 · · Score: 1

      Actually, that would be"
      CeoAss

      --
      BlameBillCosby.com
  7. future of customer service by eaglesrule · · Score: 4, Insightful

    Of course you would, Facebook. Of course you would. Manipulating your users is your core competency.

    I can't wait till I have to face off with filing a claim on my insurance against a bot that is optimized to deny claims under any pretense, is optimized to deny appeals based on any pretense, and will generally fuck me over unless I hire out an equivalent lawyer-bot to represent me. Maybe we're already at that point.

    1. Re: future of customer service by TheOuterLinux · · Score: 1

      Don't forget how Facefarm helps people get elected and are now trying to help those in office communicate with constituents (http://news.valubit.com/new-facebook-features-allow-politicians-to-connect-with-constituents/). I think Mark is secretly the spawn of Oliver Stone but the opposite side of the same coin. Might sound good to some, but absolute power corrupts absolutely.

  8. But this is not AI.. by thesupraman · · Score: 5, Insightful

    That is because the current cesspool that is media reporting cannot comprehend the difference between Artificial Intelligence (AI), which this is not, and Machine Learning (ML), which this is.

    Machine Learning is what is exploding right now, and AI has not really moved one step closer, mostly because that would require incremental low-impact learning feedback - something that is not yet even attempted in ML systems.

    So, this is not a bad example of Machine Learning, and has nothing at all to do with AI.

    I do wonder, however, how many ML bots are already being used by companies to bid up their ebay auctions until the algorithm decides the other bidder has peaked.. If it is not happening yet, it will not be far away.
    Clearly fraud, of course, but hey.. thats hardly anything new.

    1. Re:But this is not AI.. by fustakrakich · · Score: 3, Insightful

      Not eBay, Wall Street... That's where the hot bot action is.

      And right, the machines aren't composing any ideas yet, but all indications are that they will coldly and cruelly follow nature's path, same as every other life form, including humans, has done so far.

      --
      “He’s not deformed, he’s just drunk!”
    2. Re:But this is not AI.. by TheRealHocusLocus · · Score: 1

      THIS.
      Artificial Intelligence is the quest for Data (Star Trek)
      Machine Learning is the quest to model Dilbert's Boss and install it as your boss.
      Fuck 'Game Theory' and its 'myriad real world applications'.

      --
      <blink>down the rabbit hole</blink>
    3. Re:But this is not AI.. by Jeremi · · Score: 1

      That is because the current cesspool that is media reporting cannot comprehend the difference between Artificial Intelligence (AI), which this is not, and Machine Learning (ML), which this is.

      The really infuriating thing about language is, if enough people misuse a term, eventually their misuse of the term becomes the de facto correct meaning. The purists can complain but there's little they can do to stop it.

      I do wonder, however, how many ML bots are already being used by companies to bid up their ebay auctions until the algorithm decides the other bidder has peaked.. If it is not happening yet, it will not be far away. Clearly fraud, of course, but hey.. thats hardly anything new.

      From an economic standpoint, I don't see a problem with that. You're either willing to pay the auction price for an item, or you're not (in which case, don't bid a price you aren't willing to pay).

      If the seller wants to bid on his own item, fine -- he might end up having to purchase the item from himself, which makes the auction a waste of everyone's time, but so be it. He'd probably get a better return by simply setting the reserve price to whatever his minimum price is.

      --


      I don't care if it's 90,000 hectares. That lake was not my doing.
    4. Re:But this is not AI.. by rtb61 · · Score: 2, Interesting

      Here is a business strategy. Fuckers who lie to you, you do business with them only once. A company lies to you, you do business only once until suitable redemption has occurred which exceeds the gain they made by lying. Apparently corporate douche bags fail to realise this, hence their pursuit of an idiot tactic doomed to fail. A pattern for US business, take for example US sanctions on Russia, around the world they are seen as political bullshit, everyone knows they are a lie with arms sales at their core and nothing else but the US in it's arrogance keeps pursuing them because in their idiotic arrogance they think they are fooling entire governments and winning. This instead of the reality, creating global resentment at how blatantly corruptly they are being applied and being forced on other countries (destroying a resource by ignorantly exploiting it and the only really outcome strengthening the Russian economy by promoting local development).

      --
      Chaos - everything, everywhere, everywhen
    5. Re:But this is not AI.. by religionofpeas · · Score: 2

      The dictionary defines intelligence as : "the ability to acquire and apply knowledge and skills.", which is what machine learning is doing, so the term AI is perfectly suitable.

    6. Re:But this is not AI.. by religionofpeas · · Score: 2

      Judging by your definition you might call a cow intelligent being

      It's not my definition. And yes, cows do show intelligent behaviour. Whether you want to call them an "intelligent being" depends on where you want to put the threshold for that. Obviously cows are not as intelligent as humans, but they are more intelligent than worms. Intelligence is not a boolean, but a multi-dimensional gliding scale.

      And while dictionary definitions may differ, you can't claim a common usage of a word is wrong, just because it only appears in some dictionaries, and not all. Besides, looking at your other suggestions, you could argue that a machine can "reason", "understand", and "grasp truths, relationships, facts, meanings", given objective definitions of those words. Basically, it all comes down to recognizing patterns, and applying them.

    7. Re:But this is not AI.. by schleimkeim · · Score: 1

      Not eBay, Wall Street... That's where the hot bot action is.

      I am certain that this is already happening in the stock trade.

    8. Re:But this is not AI.. by K.+S.+Kyosuke · · Score: 1

      Computers have been able to the last two things ever since Logic Theorist in the 1950s. (Although, "understanding" seems a rather vague word for any meaningful classification of what computers do.)

      --
      Ezekiel 23:20
    9. Re:But this is not AI.. by Graydyn+Young · · Score: 1

      They've stolen the term AI. Just let it go man. You'll get less stress headaches that way.

    10. Re:But this is not AI.. by Shotgun · · Score: 1

      Doesn't make it a waste of Ebay's time. Ebay still gets paid. In this case, the "cheating" seller loses.

      --
      Aah, change is good. -- Rafiki
      Yeah, but it ain't easy. -- Simba
    11. Re:But this is not AI.. by gnick · · Score: 1

      The really infuriating thing about language is, if enough people misuse a term, eventually their misuse of the term becomes the de facto correct meaning. The purists can complain but there's little they can do to stop it.

      You literally hit the nail on the head.

      --
      He's getting rather old, but he's a good mouse.
  9. HFT? by Hentes · · Score: 1

    Don't HFT bots already do something similar?

    1. Re:HFT? by Actually,+I+do+RTFA · · Score: 1

      No, HFT bots don't bother lying. They don't need to. HFT bots hear person A asking to buy X for Y; They hear person B offering to sell X for Z, Z

      --
      Your ad here. Ask me how!
    2. Re:HFT? by Hentes · · Score: 1

      They do some haggling, starting from a low price and cranking it up until somebody sells to them (and vice versa when buying). They also sometimes create randomized price spikes intended to throw other bots off. Not sure if these fit your definition of "lying" (I admit it's a fuzzy one), but it fits mine.

  10. Mark is AI now? by Ian+A.+Shill · · Score: 1

    Mark is AI now? That explains it.

    --
    For hire.
    1. Re:Mark is AI now? by arth1 · · Score: 1

      No, the AI is the con man. You're the mark.

  11. Provenance: by XB-70 · · Score: 1

    The AI was trained to learn what facebook did to its users.

    --
    *** Don't be dull.***
  12. LOL! by MerlTurkin · · Score: 1

    I don't care, I'm not on Facebook.

  13. Re:Crying fair... by slew · · Score: 1

    We have an innate sense of fairness. Lying goes against that.

    Or not...

  14. That facebook IA is out there and... by sad_ · · Score: 1

    ... it can’t be bargained with. It can’t be reasoned with. It doesn’t feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead.

    --
    On a long enough timeline, the survival rate for everyone drops to zero.
  15. This is a bad idea, right? by sabbede · · Score: 1

    I mean, we don't want our computers to lie to us. Lying to humans is one of those things we should actively prevent, not develop.

    1. Re:This is a bad idea, right? by Chrisq · · Score: 1

      I mean, we don't want our computers to lie to us. Lying to humans is one of those things we should actively prevent, not develop.

      What you say is sensible and to everyone's advantage, but I bet there are a lot of people who would like their robots to lie to others if it gives them an advantage.

    2. Re:This is a bad idea, right? by sabbede · · Score: 1
      I guess it depends whether people want a computer that doesn't lie to them more than they want one that lies to others. If that's the general order of preferences, we could all agree not to make lying computers.

      One of the great things about a computer is that you can rely on it's output. If you have a giant spreadsheet calculate a total, you can be assured the result is correct if you know the inputs are correct. If you've given your computer the ability to lie for advantage, then you'd better not be running a spreadsheet to calculate your IT budget. Even if it's designed only to lie to your advantage, it may decide that the more powerful it is the better off you are, so making it look like you need to make some upgrades is a desireable deception.

      If you can't trust the output, what good is it?

  16. Common tactic by PauloftheWest · · Score: 1

    "...specifically, lying. Instead of outright saying what it wanted, sometimes the AI would feign interest in a worthless object, only to later concede it for something that it really wanted."

    This is a basic negotiation tactic that does not have to be lying. Example: I've negotiated salary before, sometimes I get the "I'm sorry we would like to pay you more but it isn't in our budget." At that point I ask for more paid days off instead--and sometimes that works out very well. Was I lying when I asked for more money? Of course not. Do I want more paid days off? Yes!

    Getting information from the one side and using it to your advantage is very common. When negotiating you need reasons to support your demands or else they will fall apart. This is nothing new, but a machine participating is very amusing!

    --
    ~Less think, more do
  17. If only they'd had this in 2001... by moeinvt · · Score: 1

    The crew of Discovery would have survived if HAL had been capable of lying without remorse.

  18. Apparently Zuckerberg... by Lorem_Ipsum · · Score: 1

    ... was ultimately successful in his digital mind upload efforts!

    --
    --- Void where prohibited. Your mileage may vary. ---
  19. The Zuckerberg Test by Zero__Kelvin · · Score: 1

    So now we have a new test to add to the Turing Test. Can it lie to get what it wants in a way that is indistinguishable from Mark Zuckerberg?

    --
    Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun