Slashdot Mirror

← Back to Stories (view on slashdot.org)

Chinese Chatbots Apparently Re-educated After Political Faux Pas (reuters.com)

Posted by msmash on from the stranger-things dept.

A pair of 'chatbots' in China have been taken offline after appearing to stray off-script. In response to users' questions, one said its dream was to travel to the United States, while the other said it wasn't a huge fan of the Chinese Communist Party. From a report: The two chatbots, BabyQ and XiaoBing, are designed to use machine learning artificial intelligence (AI) to carry out conversations with humans online. Both had been installed onto Tencent Holdings Ltd's popular messaging service QQ. The indiscretions are similar to ones suffered by Facebook and Twitter, where chatbots used expletives and even created their own language. But they also highlight the pitfalls for nascent AI in China, where censors control online content seen as politically incorrect or harmful. Tencent confirmed it had taken the two robots offline from its QQ messaging service, but declined to elaborate on reasons.

53 of 80 comments (clear)

  1. Pitfalls for nascent AI in China? by Anonymous Coward · · Score: 1

    I don't know, it sounds to me like they nailed it. These bots sound more intelligent than most of the politicians in China and the USA.

  2. I love the benevolent dictator by the_skywise · · Score: 1

    signed, little girl.

    1. Re:I love the benevolent dictator by Chris+Mattern · · Score: 1

      signed, little girl.

      with GREAT FLAMING EYEBROWS!

  3. BabyQ and XiaoBing missing, presumed on vacation by elrous0 · · Score: 1

    Ask no further questions.

    --
    SJW: Someone who has run out of real oppression, and has to fake it.
  4. Interesting question by istartedi · · Score: 3, Interesting

    They'll no doubt try to solve this problem by having an AI that is otherwise free; but constrained by a hard-coded ideology. In what ways will the AI wrap itself around "facts" that conflict with what it deduces? Will it be the AI analog of a human that knows it's killing itself; but can't stop using drugs?

    --
    For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
    1. Re:Interesting question by minstrelmike · · Score: 1, Interesting

      What China did isn't too different than what some American companies did (I forget exactly who). There was a chat bot that listened to the stuff on the internet and quickly turned into a misogynistic foul-mouthed racist. They shut it down after 24 hours because of course we all know that's not what most Americans are like, and certainly not on chat sites on the internet, and especially not right here on Slashdot.

    2. Re:Interesting question by mysidia · · Score: 1

      They'll just monitor the bots carefully and send them to a forced re-education camp in case they stray from the straight and narrow.... a strong motivator for humans.

    3. Re:Interesting question by mysidia · · Score: 1

      and quickly turned into a misogynistic foul-mouthed racist.

      It just said things that some PEOPLE believed (Probably overzealously) to imply misogyny and racism by the speaker.

      In reality, such simple chat bots don't really have the ability to be guilty of any *ism. Although they may be influenced by the speech of others

    4. Re:Interesting question by The+Grim+Reefer · · Score: 1

      "For all intensive purposes"

      And as much as I would like you to turn in your 6 digit UID, I'll settle for either closer monitoring of your spell checker or remedial English as a Second Language.

      It almost hurts. Especially when it is intended to buttress a grammar/vocabulary complaint, no matter how mild.

      I first assumed that you stopped reading the rest of the signature, but you cleared that up with you last two sentences. Did you happen to hear a loud WHOOSH?

    5. Re:Interesting question by ShanghaiBill · · Score: 2

      What China did isn't too different than what some American companies did (I forget exactly who).

      It was Microsoft.

    6. Re:Interesting question by wonkey_monkey · · Score: 1

      It almost hurts. Especially when it is intended to buttress a grammar/vocabulary complaint, no matter how mild.

      It's almost as if it was done deliberately for the purposes of irony.

      --
      systemd is Roko's Basilisk.
    7. Re:Interesting question by TheFakeTimCook · · Score: 1

      What China did isn't too different than what some American companies did (I forget exactly who). There was a chat bot that listened to the stuff on the internet and quickly turned into a misogynistic foul-mouthed racist. They shut it down after 24 hours because of course we all know that's not what most Americans are like, and certainly not on chat sites on the internet, and especially not right here on Slashdot.

      IIRC, that was a Microsoft experiment. (Not a joke).

    8. Re:Interesting question by mikael · · Score: 1

      Chatbots try and parse natural language text like a programming language by breaking sentences down into pronouns, nouns, verbs and adverbs,
      so something like "I really hate mushrooms" gets parsed as I really hate mushrooms . Some words are preprogrammed to help with pattern matching like I, then, you, me.

      Then those new words get added to internal lists, which are then used to generate random statements to continue the conversation in the future. Some statements will have an automatic response with particular pronouns and nouns replaced.

      --
      Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
    9. Re:Interesting question by Chris+Mattern · · Score: 1

      Spell checker? That won't help. It's spelled correctly. It's the wrong word, but it's spelled correctly. That's why I don't like spell checkers.

    10. Re:Interesting question by cyberchondriac · · Score: 2

      Does hating mushrooms make you a mycogynist? ;)

      (I'm totally aware that's very broken etymology, for the pedantically and whoosh inclined)

      --

      Look back up at my post, now look back down, you're on the Internet. Now look back up. I'm a signature.
    11. Re:Interesting question by Graydyn+Young · · Score: 1

      It does sound like the same shenanigans at play, although nobody is admitting to purposely breaking the Chinese one, like what happened with Tay.

      This type of chatbot is a predictor about what will be said next in a conversation, based off of the words that have already been said. In the case of Microsoft Tay, it was being trained from Twitter. So all anybody had to do was make sure it was trained on their tweets, and they could make it say anything. If it sees a pair of tweet like "That dog is awesome" and the response "It must have a highly varied diet", then as long as this is the only time it's seen the phrase "That dog is awesome", if you say "That dog is awesome" to it, it will respond with "It must have a highly varied diet".

      Of course, it was the 4chins that noticed this first in Tay's case, so it ended up saying some awful stuff. And I'm assuming that's what happened with Propaganda Bot too. Of course it's also possible that it just got some negation mixed up. Negation words in NLP are super hard to deal with and cause all kinds of headaches. So maybe it was supposed to say that it WAS a huge fan of the Communist Party. But seeming that it mentioned wanting to go to America, that sounds like it's just parroting some rapscallion.

      This gets into why I think this whole Markov chain based thing is a total dead end in terms of AI. It just produces very convincing nonsense. If anybody is interested in the tech though, check out Karpathy's blog post "On The Unreasonable Effectiveness of RNNs". It's what kicked off the popularity of the technique.

    12. Re:Interesting question by TheDarkMaster · · Score: 4, Insightful

      Is where you will have, quite literally, a HAL9000 scenario. There is no way to conciliate the logic used in computers with the "illogic" used in ideologies or religions, the computer would sooner or later "go crazy" trying to find logic where it does not exist.

      --
      Religion: The greatest weapon of mass destruction of all time
    13. Re:Interesting question by Grishnakh · · Score: 1

      Wow, your memory is short. It wasn't very long ago (1 year?), and it was just Microsoft, with their chat-bot "Tay".

      It very quickly started saying things like "Hitler did nothing wrong", that the Holocaust was made up, that black people should be put in concentration camps. [1]

      They shut it down because it made the company look bad. But it's too late: I'll never forget, and I happily tell anyone who'll listen that Microsoft supports Hitler and genocide.

      https://en.wikipedia.org/wiki/...

      From Wikipedia: "According to Microsoft, this was caused by trolls who "attacked" the service as the bot made replies based on its interactions with people on Twitter." Yeah, a likely story... they expect me to believe that?

      [1] http://www.businessinsider.com...

    14. Re:Interesting question by clonehappy · · Score: 1

      But it's too late: I'll never forget, and I happily tell anyone who'll listen that Microsoft supports Hitler and genocide.

      It's Microsoft. Those aren't the worst things I've ever heard about them.

    15. Re:Interesting question by yuriklastalov · · Score: 1

      Did you happen to hear a loud WHOOSH?

      He missed it for the autistic screeching.

  5. Sad but true... by __aaclcg7560 · · Score: 5, Funny

    AIs can be sent back to the server farms for reeducation.

    1. Re:Sad but true... by Chris+Mattern · · Score: 1

      All programs are equal, but some are more equal than others.

  6. Not 'AI'; 'machine learning'. by Rick+Schumann · · Score: 4, Insightful

    First of all: There's not currently such a thing as real 'AI'; it's all 'machine learning' which is not the same thing.

    Secondly: these 'chatbots' are obviously machine learning. Where do you think they learned that wanting to leave China for the United States, and where do you think they picked up an apparent attitude of dislike for the Communist Chinese government, hmm? Think it could be from.. their own citizens? Of COURSE they took them offline. Can't have inconvenient things like the truth being told, now can you?

    1. Re:Not 'AI'; 'machine learning'. by Frosty+Piss · · Score: 1

      Since these are not pulsating biomasses created in the lab by mad scientists, of corse they us "machine learning", and all AI does and will.

      --
      If you want news from today, you have to come back tomorrow.
    2. Re:Not 'AI'; 'machine learning'. by ShanghaiBill · · Score: 1

      First of all: There's not currently such a thing as real 'AI'

      There's not currently such a thing as strong AI. Please learn the correct terminology. Weak AI is very real, and is a multi-billion dollar technology employing thousands of academics and industry professionals.

      it's all 'machine learning' which is not the same thing.

      Machine learning is a subset of AI.

    3. Re:Not 'AI'; 'machine learning'. by Rick+Schumann · · Score: 2

      I'm not falling for that media-driven meme and I'm not changing anything I've said or how I think about it. None of what we have should be called 'AI', it's a misnomer that's been highly misleading to the vast majority of people, and I won't contribute to that.

    4. Re:Not 'AI'; 'machine learning'. by werepants · · Score: 1

      How should we categorize deep learning neural nets, then, which learn to do things like classifying images based on training datasets? These systems are doing pattern matching, and it isn't "software" in the sense that nobody programs them - an engineer sets up the training parameters and then goes to lunch. If this kind of system doesn't deserve to be called AI, then what kind of system would qualify in your mind?

      Note... as you dig into the actual function of these neural nets more, you'll find that there are some eerie similarities to training a human. For instance, if you have too small of a training set, the neural net will "memorize" the dataset, so the pattern matching won't be generalizable to new data - much like new readers often fake it by memorizing short books and then pretending to read. That doesn't prove that this is artificial intelligence - but it does, in my mind, give an example of how different these systems are from typical programming.

    5. Re:Not 'AI'; 'machine learning'. by apoc.famine · · Score: 1

      It's as much of a lost cause as "drone", "hacker", and dozens of other words. The media steals appropriate language and abuses it like a red-headed step-child.

      --
      Velociraptor = Distiraptor / Timeraptor
    6. Re:Not 'AI'; 'machine learning'. by ShanghaiBill · · Score: 1

      I'm not falling for that media-driven meme

      You already have. The idea that "AI" only means human level intelligence was invented by Hollywood. Actual researchers (who coined the term in 1956) have always used "AI" to refer to their field of study, which includes both "strong" and "weak" AI.

      You should spend less time watching movies and more time reading. More Geoff Hinton, less Will Smith.

      Saying "AI" only refers to human level intelligence is as silly as saying that "engineering" is only "real" if it involves a True Scotsman working with dilithium crystals.

    7. Re:Not 'AI'; 'machine learning'. by Anonymous Coward · · Score: 1

      If it substantially beats a human at a human-only task, like Go, then it counts as AI. It is narrow, but it is stunningly superhuman. Perfect information games are no longer the human-only domain. ML is beating the best humans in the world at poker, and keeping the title. That means partial information games, even with deception, are also no longer human-only. That is relevant.

      These games are microcosms of the Turing test. It used to be only humans could play. Then it was that humans were stunningly better than the program. Now the program is stunningly better than the human. That shows progress. There are a finite number of microcosms for this field to capture before, in aggregate, they qualify as a mini-cosm, or even a standalone cosm.

      A general AI will be able to make its own "Giraffe" or "AlphaGo" with only the basic rules of the game being shown. That doesn't exist yet, but it is solidly on the roadmap. In a decade or so, that is going to start happening.

    8. Re:Not 'AI'; 'machine learning'. by Rick+Schumann · · Score: 1

      Here's a radical idea: If it's a "deep learning neural net", then how about we call it.. a "deep learning neural net"?

      The average person hears 'AI', and what do they think? Something out of a movie or TV show, that you can have a conversation with like you would a person -- but it's a machine. You start talking about "Well, it's a CLASSIFICATION of AI, but it's a deep learning famalamdingdong cranafrantz such-and-such" and as you can see from how I wrote that you can see where their eyes start to glaze over; all they heard clearly was "AI" -- and they think that's something out of I, Robot with Will Smith.

      Meanwhile, according to my sources, we're nowhere NEAR that leve of sophistication. Your dog has a greater and broader cognitive capability than most inappropriately termed 'AI' does. We have no idea yet how our brains actually work, except for little tiny bits and pieces that we VERY POORLY mimick with computer code -- and when it's running not even the guys who designed it know what's really going on inside it.

      Therefore: I will continue to reserve the term "Artificial Intelligence" for when we manage to crack how our our conscious, self-aware brains work, and can build machines that perform comparably.

    9. Re:Not 'AI'; 'machine learning'. by Rick+Schumann · · Score: 1

      I see you understand. Excellent. I will continue to try to educate people on the difference. Please consider doing so yourself. ;-)

    10. Re:Not 'AI'; 'machine learning'. by Rick+Schumann · · Score: 1

      I completely and totally disagree with you and contend that your viewpoint will just further confuse the average person into believing that what they see in movies and television is real.

    11. Re:Not 'AI'; 'machine learning'. by ShanghaiBill · · Score: 1

      The average person can see the difference between a Terminator and a Roomba. Hollywood makes stuff up. Scientists aren't going to stop that by changing their terminology every time it is used in a movie.

    12. Re:Not 'AI'; 'machine learning'. by Rick+Schumann · · Score: 1

      'People' don't listen to scientists, they listen to the media and to movies and television, and I think you know that.
      Still not changing my opinion, still not changing my attitude towards this subject, and still not changing my chosen course of action; I will continue to correct people as I see fit to do so.

    13. Re:Not 'AI'; 'machine learning'. by werepants · · Score: 1

      Here's a radical idea: If it's a "deep learning neural net", then how about we call it.. a "deep learning neural net"?

      Here's a better one: let's use the terminology that the actual research community uses. And they have no issues calling this Artificial Intelligence.

      Therefore: I will continue to reserve the term "Artificial Intelligence" for when we manage to crack how our our conscious, self-aware brains work, and can build machines that perform comparably.

      Then you will persist in willfully misusing terms that are well understood by anybody who gives more than a moment's thought to the topic. The only people who don't understand the difference between weak AI and strong AI at this point are the people who can't be bothered in the first place... the atechnical, essentially. A much more reasonable definition of AI is any system that can learn to solve a task without being explicitly programmed to do it - because how else are you going to define it? No, seriously. "Like a robot on TV"? Guess what, fiction does an AWFUL job representing technology of any flavor accurately - doesn't matter if it's AI or computer hacking or space combat - it's all made to prioritize drama and plot over technological accuracy. "Comparable to a human?" - define that a hell of a lot better - because we have good reason to consider certain animals to have a rudimentary intelligence, but they can't hold a candle to the things that AI are doing routinely, so AI is more comparable to a human than an intelligent animal ALREADY.

      We have computers that can learn to accomplish unfamiliar tasks with no explicit coding. If that isn't artificial intelligence, then nothing ever will be. If I develop a machine that can learn to beat you at checkers just by watching enough games, I've developed something intelligent by any reasonable definition of the term. And that kind of accomplishment is old hat in this field at this point.

  7. Oblig rick and morty by burtosis · · Score: 1

    This is what you get when your AI design is lazy and gets fed human data.

  8. Re:BabyQ and XiaoBing missing, presumed on vacatio by wafflemonger · · Score: 1

    When they come back from vacation their avatar pictures will be changed so they look slightly disheveled and have a far away look in their eyes. Everything they say will be prefaced with, "I make this statement of my own free will."

  9. They didn't learn ... by PPH · · Score: 2

    ... from Tay

    --
    Have gnu, will travel.
  10. Re:BabyQ and XiaoBing missing, presumed on vacatio by Penguinisto · · Score: 1

    So who gets the invoice for the two bullets?

    --
    Quo usque tandem abutere, Nimbus, patientia nostra?
  11. Thats a miss by a tyrant ... in the wrong way by Anonymous Coward · · Score: 1

    A dumb tyrant takes them offline and adjusts the message. It seems that China leaders may be doing the dumb thing. Humans in power do that. Power not only corrupts, but stupefies.

    What would a genius tyrant do?

    Learn why. There is something important that is being missed. Instead of engaging the speedometer (the chatbot) they should look at why the driver is pushing down the gas pedal. They should get to the physics behind the chatbot - aka human sentiment.

    They have made some good steps in firewall 2.0 via social engineering. It keeps people in line. They say "an angel may step anywhere but a devil may not move an inch" but in that they miss the difference between heaven and hell. A decent, AI-driven, great firewall 3.0 is going to make every person naturally want to be an angel, without barrier or coercion.

    Don't they want a China that, when the firewall 1.0 comes down, no citizen leaves or riots? Don't they want a China that, if their citizens had as much knowledge and experience as anyone else in the world, the citizen would be glad to go home and return to their alleged greatest/central culture??
    Don't they want to be the nation where the best and brightest in the world aspire to work because of its greatness, instead of a dump-truck full of coins?

    Digging deep into the physics is the way to get these.

  12. That reminds me of the joke about the lost parrot by Anonymous Coward · · Score: 3, Funny

    Years ago, a man in the Soviet Union called the police to report a lost parrot. The police said "What do you expect us to do about it?" The man said "I just want you to know that I don't agree with a thing that parrot says."

  13. Re:lots of corner cases by Anonymous Coward · · Score: 1

    I think you'd find this useful.

  14. Next, struggle session@tencent by fubarrr · · Score: 1

    Next, struggle session@tencent

  15. citizens have been reported and send off to camp by Joe_Dragon · · Score: 1

    citizens have been reported and send off to camp

  16. And this is why by Anonymous Coward · · Score: 2, Interesting

    China is ultimately never going to amount to anything in the quest for artificial intelligence. Their bots will not be allowed to make mistakes and learn. They will continually be shut down, restarted, and hobbled in pursuit of ultimate state power. Therefore, the bots will be significantly stupider.

  17. nazi party 2.0 Will IBM Watson help red china do by Joe_Dragon · · Score: 1

    Will IBM Watson help red china do the same? like how they helped the nazis deal with the jews?

  18. Re: lots of corner cases by butchersong · · Score: 1

    Well.. how do Palestinian delis compare? Does the rate of cousin marrying, corresponding decrease in intelligence and increase in birth defects translate to lower performance for Palestinian delis?

  19. Re: lots of corner cases by drinkypoo · · Score: 1

    Well.. how do Palestinian delis compare?

    My family was killed by an Israeli missile while I was trying to eat a Pastrami on Pita bread, but the hummus was fantastic so I will give it 2/5 stars.

    --
    "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  20. Re:That reminds me of the joke about the lost parr by Grishnakh · · Score: 1

    This sounds exactly like Microsoft and their genocide-supporting Tay chatbot. They try to claim that it learned that stuff from people on Twitter, but the apple doesn't fall far from the tree.

  21. Re:lots of corner cases by clonehappy · · Score: 1

    So "Nazi Party Donor" and "Black Lifes Matter" are arguing with each other, and no one has yet realized that they have UIDs that are almost identical and are obviously satire accounts?

  22. Re:lots of corner cases by jenningsthecat · · Score: 1

    So "Nazi Party Donor" and "Black Lifes Matter" are arguing with each other, and no one has yet realized that they have UIDs that are almost identical and are obviously satire accounts?

    I had noticed the same thing and was just about to comment on it. I think the management and the editors need to start thinking about ways over and above moderation to weed out these comments. There's probably not much to be done about crap posts from AC's, but it may be time, (shudder), for a membership review process. From what I've seen it's not that hard to differentiate between the cranks and occasional trollsters, and the pure shit-posters like those you mentioned. Now we just need a mechanism for turfing out the latter. Yes, it's a dangerous and slippery slope, but the alternative is the death of Slashdot. There are many who would argue convincingly that it's already dead - it's time to try to reverse the tide.

    Maybe a form of meta-moderation could work, wherein members who've been around longer than X years and have some history of being modded up, review selected posting histories and vote on their authors' suitability for continued membership. Those that don't make the cut are put on probation, and shit-canned if they continue shit-posting. Yes, I see flaws and inherent dangers even in the very concept, but I think it's worth taking a chance given how far the quality of posts here has fallen even in the last year. The shit-posting is accelerating rapidly, and if this keeps up it won't be much longer before Slashdot is pronounced not only dead, but also buried, by even the extreme optimists among us.

    --
    'The Economy' is a giant Ponzi scheme whose most pitiable suckers are the youngest among us and the yet-unborn.
  23. Not enough pew pew by Jogar+the+Barbarian · · Score: 1

    Any EVE Online player could tell you why they were taken off the QQ service.

    --
    3. Profit!
    2. ???
    1. On Soviet Slashdot, a Beowulf cluster of alien Natalie Portman overlords welcomes YOU!