Slashdot Mirror


University of Columbia Researchers Translate Brain Signals Directly Into Speech (columbia.edu)

dryriver writes: There is good news for people who have limited or no ability to speak, due to having suffered a stroke for example. Researchers at Columbia University have managed to turn brain signals in the auditory cortex of test subjects into somewhat intelligible speech using a vocoder-like system with audio output cleaned up by neural networks. The findings have been published in the journal Nature. Here's an excerpt from the Zuckerman Institute's press release, which contains example audio of a number sequence being turned into robotic speech: "In a scientific first, Columbia neuroengineers have created a system that translates thought into intelligible, recognizable speech. By monitoring someone's brain activity, the technology can reconstruct the words a person hears with unprecedented clarity. This breakthrough, which harnesses the power of speech synthesizers and artificial intelligence, could lead to new ways for computers to communicate directly with the brain. It also lays the groundwork for helping people who cannot speak, such as those living with amyotrophic lateral sclerosis (ALS) or recovering from stroke, regain their ability to communicate with the outside world."

84 comments

  1. Transcript by Anonymous Coward · · Score: 2, Funny

    Dear aunt let's so double the killer delete select all

    1. Re: Transcript by Anonymous Coward · · Score: 0

      You're missing: Let's set

      Sent from my brainwaves.

    2. Re:Transcript by Anonymous Coward · · Score: 0

      I see they used this on Bobby Tables.

  2. Captain Pike will be thrilled! by spiritplumber · · Score: 3

    Seriously though, this is amazing.

    --
    Liberty - Security - Laziness - Pick any two.
    1. Re:Captain Pike will be thrilled! by PolygamousRanchKid+ · · Score: 3, Informative

      Seriously though, this is amazing.

      I agree. However, I'm afraid that the police, NSA, CIA, DHS and their pals will discover the usefulness of this neat new toy.

      Well, at least they might stop water boarding people.

      Before you enter the US, we'd like you to put on this funny hat, and ask you a few questions. Don't bother answering; the computer will speak your thoughts for you.

      --
      Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
    2. Re:Captain Pike will be thrilled! by Anonymous Coward · · Score: 0

      "Technical difficulties" might translate to: we are currently waiting for a long enough government shutdown to apply 3 years worth of security patches and fix other critical problems at the FBI office in Palo Alto.

      CROFLOL!

    3. Re:Captain Pike will be thrilled! by Anonymous Coward · · Score: 1

      Seriously though, this is amazing.

      I agree. However, I'm afraid that the police, NSA, CIA, DHS and their pals will discover the usefulness of this neat new toy.

      Well, at least they might stop water boarding people.

      Before you enter the US, we'd like you to put on this funny hat, and ask you a few questions. Don't bother answering; the computer will speak your thoughts for you.

      Seriously though, this is amazing.

      I agree. However, I'm afraid that the police, NSA, CIA, DHS and their pals will discover the usefulness of this neat new toy.

      Well, at least they might stop water boarding people.

      Before you enter the US, we'd like you to put on this funny hat, and ask you a few questions. Don't bother answering; the computer will speak your thoughts for you.

      Look on the bright side, perhaps they'll be less scared of the left wing if they can finally confirm beyond any doubt that all that most of us are thinking about 99% of the time is boobies and weed, ... except the women, they are mostly thinking about shoes .... and weed.

    4. Re: Captain Pike will be thrilled! by Anonymous Coward · · Score: 1

      Or the computer's speech synthesis code sees "non-white" or "from bad country" or "fuck with them" flagged in their profile data and starts spitting out things like "alalalalalalalahhhbhhbh" or "death to America" bullshit regardless as to what the person is actually thinking.

      After all the computer said it so it must be true......

    5. Re:Captain Pike will be thrilled! by Oswald+McWeany · · Score: 1

      Look on the bright side, perhaps they'll be less scared of the left wing if they can finally confirm beyond any doubt that all that most of us are thinking about 99% of the time is boobies and weed, ... except the women, they are mostly thinking about shoes .... and weed.

      I never think about weed...except when spraying roundup. It's all boobies for me.

      It would be embarrassing though...

      We would like to ask you a few questions, ignore this mind reading device...

      Now, Mr. McWeany,
      oh shit! they think my name is really McWeany, that's an alias, my real name is Dickbutt
      OK, Mr.Dickbutt...
      Why did my brain just say Dickbutt, that's not my name either... I'm so scatterbrained, I wonder if I left the oven on
      Where were you on the evening of the 17th?
      The female agent has nice lips, I wonder how they feel...
      The 17th
      This guy is an arsehole... how am I supposed to know what happened on the 17th... my dating anniversary with my ex was on the 17th... I wonder what she's doing now, she had nice lips too

      On second thoughts- mind reading devices would be a waste of time on me.

      --
      "That's the way to do it" - Punch
    6. Re:Captain Pike will be thrilled! by Anonymous Coward · · Score: 0

      I was thinking how awesome this would have been to have while Hawking was still alive.

    7. Re:Captain Pike will be thrilled! by The-Ixian · · Score: 1

      Except this would only be the things that you would be saying. There is a big gap between thinking something and forming the words and speaking them...

      Even if it was just your thoughts, who's to say that the first thing that comes to mind is truth?

      --
      My eyes reflect the stars and a smile lights up my face.
    8. Re:Captain Pike will be thrilled! by newbie_fantod · · Score: 1

      I'm afraid that the police, NSA, CIA, DHS and their pals will discover the usefulness of this neat new toy

      My worry too. I'm guessing that since the technology monitors the auditory cortex, it's not going to pickup on anything other than what you would consciously want to say - maybe a passing neuroscientist can weigh in.

    9. Re: Captain Pike will be thrilled! by Anonymous Coward · · Score: 0

      Why, he didn't produce anything of value in the past two decades at least.

  3. "I'm hungry. Boobs! What should I order? Boobs!" by SensitiveMale · · Score: 5, Funny

    "Yes, I'd like to order a Big Mac BOOBS! Good Lord look at those boobs! Ummm, some fries and a My God those boobies are huge! Umm and diet coke. Damn, those of tig ass bitties!"

    That's what every guy's brain is thinking.

  4. Re:"I'm hungry. Boobs! What should I order? Boobs! by Anonymous Coward · · Score: 0

    Whatever you say log cabin.

  5. Push to talk? by Anonymous Coward · · Score: 1

    I just hope the system has push to talk button...

  6. In other news ... by Anonymous Coward · · Score: 0

    Sales of waterboards dropped precipitously due to this new interrogation technology.

  7. Obligatory by Miles_O'Toole · · Score: 4, Funny

    Now they just have to figure out how Trump manages to generate speech while demonstrating no measurable brain activity.

    --
    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.
    1. Re:Obligatory by Anonymous Coward · · Score: 0

      Orange man bad

    2. Re:Obligatory by Anonymous Coward · · Score: 0

      Orange man bad

      And once again you nailed it Mr Coward, now was that so hard?

    3. Re:Obligatory by infolation · · Score: 1

      Oh, that's already been figured out. Seth Meyers has been using the University of Columbia's technology for a few months now and broadcasting the results in the form of...

      The Tiny Voice In The Back Of Donald Trump's Head

      For example, Trump's inner voice on Don't Lose The Queen

  8. Brain microphone, not brain speech synthetizer by abies · · Score: 5, Interesting

    Please note that while it seems like they cracked translating thought into speed, it is really about something very different. They decode what person is hearing into sound. This is very, very far from being able to interpret actual thoughts - they attach themselves to very early stage of hearing process, where things are still looking very much like sound waves. Additionally, it is tested on just 10 words (numbers from 0 to 9). On top of that, given neural nets are magic, I wouldn't be surprised if instead on brain activity, they focused on some measurement artifact, which is caused by skull vibrating to the sound, turning all that setup into overcomplicated and underperforming microphone. Sure, it is a worthy research, but it is light years away from thoughts to speech conversion.

    1. Re:Brain microphone, not brain speech synthetizer by teslar · · Score: 3, Interesting

      Yes, this.

      The only thing I would add is that we do know that higher-level cognitive processes re-activate sensorimotor parts of the brain; so thinking the number "nine" might cause some activation similar to hearing that number.

      So, assuming they didn't indeed just pick up on some non-neural artifact, this shows that there is potential - at least in the ideal case where we know the signal is as good as it's ever going to get. It's one step in the direction of being able to deal with weaker, noisier versions you might get from reactivation.

      But yeah, noone's going to read your thoughts anytime soon.

    2. Re:Brain microphone, not brain speech synthetizer by petes_PoV · · Score: 2

      This is very, very far from being able to interpret actual thoughts - they attach themselves to very early stage of hearing process, where things are still looking very much like sound waves.

      Yes. My first thought when I read the headline was: what language is the speech in?

      But it seems that this work, while interesting in an academic sense, is far from what the headline indicates.

      --
      politicians are like babies' nappies: they should both be changed regularly and for the same reasons
    3. Re:Brain microphone, not brain speech synthetizer by mentil · · Score: 1

      translating thought into speed

      Apparently, the parent needed more thought and less speed in order to avoid that typo.

      --
      Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
    4. Re: Brain microphone, not brain speech synthetizer by Anonymous Coward · · Score: 0

      It's implied to be the same language that the subject hears which is the one used to train the machine

  9. University of Columbia? by 93+Escort+Wagon · · Score: 1

    So these are scientists living in Bogota? Those Columbian researchers are pretty amazing. Must be all that coffee.

    --
    #DeleteChrome
    1. Re: University of Columbia? by Anonymous Coward · · Score: 0

      One researches cocaine. The other researches while consuming it.

    2. Re:University of Columbia? by mentil · · Score: 1

      Once they can read minds they can figure out who the narcs are.

      --
      Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
    3. Re:University of Columbia? by Anonymous Coward · · Score: 0

      Once they can read text, they can figure out the name of the university where the research was done. And maybe get the title of the submission right.

    4. Re:University of Columbia? by Anonymous Coward · · Score: 0

      You're thinking of Colombia.

  10. Explains Trump! by Anonymous Coward · · Score: 0

    I suppose Trump was one of the test subjects?

  11. Re:"I'm hungry. Boobs! What should I order? Boobs! by Anonymous Coward · · Score: 1

    It doesn't translate your subconscious thoughts, just the vocalized region of the brain.

    Unless you're schizophrenic, then I guess this makes sense.

  12. Hopefully now... by Anonymous Coward · · Score: 0

    ..... we can figure out what Mike Tyson is saying.

  13. Assist Law Enforcement by Anonymous Coward · · Score: 0

    “You have the right to remain silent.”

    Too bad if you keep your mouth shut and the brain blabs. Like you do not have to incrimate yourself but your finger to unlock your phone is free game. And courts will likely return similar verdicts.

    1. Re: Assist Law Enforcement by Anonymous Coward · · Score: 0

      Any court that admits "evidence" extracted using device like this would be in police state dreams only. It is so clearly unconstitutional to do this invasive "search" of a person's thoughts to force them to incriminate themselves. The founders were clearly willing to let a few guilty people walk over trampling the rights of everyone.

  14. Re:"I'm hungry. Boobs! What should I order? Boobs! by Anonymous Coward · · Score: 0

    And it appears he eats at McDonalds, the most deliberately self-debilitating condition of all.

  15. What if by Anonymous Coward · · Score: 0

    you have Tourettes syndrome?

    1. Re:What if by BlackOverflow · · Score: 1

      There will be much swearing.

    2. Re:What if by lgw · · Score: 2

      What if you have Tourettes syndrome

      Fun fact: profanity comes from a different part of the brain from normal speech. That's why Tourettes can even exist. That part of the brain is, as much as we can reason between species, the part of the brain that other apes and monkeys use for warning vocalizations. Literally "oh shit it's a snake!"

      --
      Socialism: a lie told by totalitarians and believed by fools.
  16. Read my mind by MicroSlut · · Score: 2

    Have they tried using this on a dog yet?

  17. Re:"I'm hungry. Boobs! What should I order? Boobs! by Anonymous Coward · · Score: 0

    And it appears he eats at McDonalds, the most deliberately self-debilitating condition of all.

    Yeah, but when you order two cheeseburgers, and put a bit of ketchup on top of each one, they look like boobies....

  18. Publication by Anonymous Coward · · Score: 0

    Its not published in Nature, it is published in Scientific Reports which is owned by the same publishing company that publishes Nature.

    Itâ(TM)s harder to get stuff in Nature and yes I have gotten stuff in there. If you donâ(TM)t have a major advance they bounce it back saying hey why dont you try our other journal?

  19. That sounds like autoregression with extra steps by MrTree · · Score: 1

    Here, clean speech was used as both the input and output of the system. The subjects listened to the speech, and the speech was the target, having being reconstructed with the brain activity. (This technique uses invasive electrocorticography; they happened to have some epileptics around to use as subjects)

    The target was on the speech perception. Speech production would need to focus on another region of the brain (Maybe Broca's Area or straight to the motor cortex). The training data for speech production would be significantly different - no autoregression possible in that approach. It's an order of magnitude difference.

    Still, it's the first step towards actual telepathy, so it's got that going for it, which is nice.

  20. ATHF did it! by SoundGuyNoise · · Score: 1

    Frylock made one for Travis.

    --
    You never expect irony, do you?
    Want to be a professional wrestler? Visit www.iyfwrestling.com
    @iyfwrestling
  21. shift by Anonymous Coward · · Score: 1

    Be good for people in comas and the brain dead.

  22. Re:"I'm hungry. Boobs! What should I order? Boobs! by Anonymous Coward · · Score: 0

    Your sex life has deteriorated beyond the value menu.

  23. No, for obvious reasons... :-) by Anonymous Coward · · Score: 0

    "You shall now call me Snowball ..."

    https://www.youtube.com/watch?v=waLxMl6K3zA

    1. Re:No, for obvious reasons... :-) by Ol+Olsoc · · Score: 1

      "You shall now call me Snowball ..."

      https://www.youtube.com/watch?v=waLxMl6K3zA

      Where are my testicles Summer?

      --
      The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
  24. Re: "I'm hungry. Boobs! What should I order? Boobs by Seewhatidonehere · · Score: 0

    Nou imagun dis wen id iz jus somwha inteligibibil. If you cant speak use the fucking app to order your food. Supposimg you still have fingers or a nose you can poke the icons with on the screen. Also the excerpt says the device " replicates what the person HEARS, with unprecedented clarity "..

  25. Re: "I'm hungry. Boobs! What should I order? Boobs by Seewhatidonehere · · Score: 0

    Excuse me who are you to say that subconscious thought can't go through the vocal region of the brain? If that was the case we would never speak stupid things subconsciously... Fact is, so each involves thought, and thoughts are hard to control sometimes even for the brain, let alone a device that is practically an amiga console in comparison to what the brain is capable of.

  26. Re: Thoughts required. by Anonymous Coward · · Score: 1

    The question everyone wants the answer to: if you use it on Trump, would it say "hambudgers" or "build my wall"?

  27. Re: Thoughts required. by Anonymous Coward · · Score: 0

    "The question everyone wants the answer to: if you use it on Trump, would it say "hambudgers" or "build my wall"?"

    RTFS, you need brainwaves for it to work.

  28. Know Your Universities by Anonymous Coward · · Score: 0

    Columbia University is most assuredly not the University of Columbia.

    1. Re:Know Your Universities by Swave+An+deBwoner · · Score: 1

      Thank you. I came here to post exactly that. "Columbia University" != "University of Columbia".

  29. Re:"I'm hungry. Boobs! What should I order? Boobs! by sad_ · · Score: 2

    from TFA;

    “In this scenario, if the wearer thinks ‘I need a glass of water,’ our system could take the brain signals generated by that thought, and turn them into synthesized, verbal speech,” said Dr. Mesgarani.

    it clearly says it will vocalize what the brain is thinking. unless there is an easy way to turn that off/on i don't think anybody actually wants something like that.

    --
    On a long enough timeline, the survival rate for everyone drops to zero.
  30. Re: "I'm hungry. Boobs! What should I order? Boobs by Vintermann · · Score: 3, Interesting

    It would be creepy as hell, but very cool, to get a recording of the voices schizophrenics hear.

    --
    xkcd is not in the sudoers file. This incident will be reported.
  31. There must be a mistake by elrous0 · · Score: 2

    We hooked it up to Ariana Grande and it just keeps printing out "Need attention" over and over.

    --
    SJW: Someone who has run out of real oppression, and has to fake it.
    1. Re:There must be a mistake by Anonymous Coward · · Score: 0

      FFS I'm in a meeting laughing. +1 you bastard

  32. The Singularity is Here! by GLMDesigns · · Score: 1

    Ok. That's simply click-bait.

    If you're able to turn thoughts into speech then you're able to record it. How soon until memories can be saved? Such as the Potterverse' Pensieve.

    These advances are close to fantastical. And the world goes ho-hum.

    --
    If you're scared of your govt then you need to further restrict its powers
    Vote 3rd Party in 2016 and beyond
    1. Re: The Singularity is Here! by Anonymous Coward · · Score: 0

      I think the recording would be: up, up, down, left, right
      This is already possible. But how much more? Doubt too much

  33. Re:"I'm hungry. Boobs! What should I order? Boobs! by Anonymous Coward · · Score: 1

    You laugh but this already is happening with eye trackers. Seriously, look on youtube for "fail" videos wherein people are judged for looking at a cute girl (or guy) in some other place besides their head. Pewdiepie for example.

    My mind is mine, no one elses.

  34. Re: "I'm hungry. Boobs! What should I order? Boobs by Anonymous Coward · · Score: 0

    Slashdot upcoming the bro culture? Lol

  35. Re: Thoughts required. by Anonymous Coward · · Score: 0

    Why can't it be both? "Build me a hamburger wall around my desk. I'll make McDonald's pay for it. On second thought, make it double thick. I'm hungry."

  36. Poor NPC's by Anonymous Coward · · Score: 0

    This only helps 60% of the population since the other 40% do not think in the manner that this would work for.

  37. Stop, Willy Wonka sighed, knowing nobody listens by Impy+the+Impiuos+Imp · · Score: 1

    Governments will argue this can be used to detect criminal evidence. We need laws to stop this cold because it will be abused in dictatorships.

    --
    (-1: Post disagrees with my already-settled worldview) is not a valid mod option.
  38. Re:"I'm hungry. Boobs! What should I order? Boobs! by TuringTest · · Score: 1

    That's what every guy's brain is thinking.

    I'm a DICK gay guy DICK, you insensitive DICK clod!

    --
    Singularity: a belief in the "God" idea with the "demiurge" relation inverted.
  39. Re: Thoughts required. by Anonymous Coward · · Score: 0

    "My Covfefe is the best, believe me."

  40. Washington University vs University of Washington by Anonymous Coward · · Score: 0

    Washington University is in St. Louis, and Columbia University is in New York. The University of Washington is in Washington. The University of Columbia is... well... order matters.

  41. RTFA by Anonymous Coward · · Score: 0

    FFS that's not at all what the article says.
    It was playing match squares with words, not even close to reading your minds.
    Fake news!

  42. we have ways to make you talk by Anonymous Coward · · Score: 0

    version 2.0

  43. New? by Anonymous Coward · · Score: 0

    Isn't this what booze is for?

  44. Re:"I'm hungry. Boobs! What should I order? Boobs! by Ol+Olsoc · · Score: 1

    it clearly says it will vocalize what the brain is thinking. unless there is an easy way to turn that off/on i don't think anybody actually wants something like that.

    Yeah - I think Ima let that thing miss me.

    If someone asks me a question that indicates they weren't paying attention, I do think "You stupid fuck!", while in practice going back to what I said and re-pointing it out.

    I pick up concepts quickly. When others don't, I might think "y'all are a little slow", while in practice I sit politely.

    If an attractive woman walks into the room, I think thoughts for a moment that every heterosexual male would. In practice, I say "Hello, how are you today?"

    Look on the bright side, #metoo will have a lot of new men to destroy.

    --
    The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
  45. Re:"I'm hungry. Boobs! What should I order? Boobs! by Zaiff+Urgulbunger · · Score: 1

    It would be like the Belcerabons who were sentenced to telepathy. Everyone would have to go around talking continually about inane trivia.

    > OHHH, YOU'RE VERY TALL AREN'T YOU?
    >> YES I AM VERY TALL. LOVELY WEATHER TODAY ISN'T IT?
    etc etc.

  46. U of C? by Anonymous Coward · · Score: 0

    Who the hell has ever called it the "University of Columbia?"

  47. Alas, just a bit too late for Stephen Hawking by The+Snazster · · Score: 1

    But scientists just announced a laser that can cause a person to hear words. Connect this gadget up to that laser (and maybe put it in a bionic eye) and boom, something not too different from telepathy.

  48. Error in title. by Anonymous Coward · · Score: 0

    University of Columbia is *NOT* the same as Columbia University.

  49. Re: Thoughts required. by Anonymous Coward · · Score: 0

    I've considered this question. Here is my answer:

    "Hamberders"
    "Covfefe"
    "Wall"
    "Stormy"
    "Bigly"
    "Ivanka"
    "Pussy"
    "Vladimir"
    "Hamberders" (again)
    "Powerful"
    "Muller"
    "Pussy" (again)

  50. The ultimate invasion of privacy by Rick+Schumann · · Score: 1

    I'm sure that people could be trained to defeat this technology for when it's used for interrogation purposes, but still: the Ultimate invasion of privacy, and the possibility of a new dark era of human history: thought crime.

  51. Re: "I'm hungry. Boobs! What should I order? Boobs by Ol+Olsoc · · Score: 1

    It would be creepy as hell, but very cool, to get a recording of the voices schizophrenics hear.

    Boy howdy would it. I have always been intrigued by Schizophrenic speech patterns. But to hear the voices in their heads alonng with them? Whoa!

    --
    The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
  52. Re:"I'm hungry. Boobs! What should I order? Boobs! by k6mfw · · Score: 1
    reminds me of this episode "Do Larger Breasts Equal Bigger Tips? | MythBusters" https://www.youtube.com/watch?...

    They would have collected a lot more data besides tip amounts.

    --
    mfwright@batnet.com
  53. Re: "I'm hungry. Boobs! What should I order? Boobs by Anonymous Coward · · Score: 0

    Go drown in santorum, then, pederast.