Slashdot Mirror


Amazon Workers Are Listening To What You Tell Alexa (bloomberg.com)

Amazon reportedly employs thousands of people around the world to help improve its Alexa digital assistant. "The team listens to voice recordings captured in Echo owners' homes and offices," reports Bloomberg. "The recordings are transcribed, annotated and then fed back into the software as part of an effort to eliminate gaps in Alexa's understanding of human speech and help it better respond to commands." From the report: The team comprises a mix of contractors and full-time Amazon employees who work in outposts from Boston to Costa Rica, India and Romania, according to the people, who signed nondisclosure agreements barring them from speaking publicly about the program. They work nine hours a day, with each reviewer parsing as many as 1,000 audio clips per shift, according to two workers based at Amazon's Bucharest office, which takes up the top three floors of the Globalworth building in the Romanian capital's up-and-coming Pipera district. The modern facility stands out amid the crumbling infrastructure and bears no exterior sign advertising Amazon's presence. The work is mostly mundane. One worker in Boston said he mined accumulated voice data for specific utterances such as "Taylor Swift" and annotated them to indicate the searcher meant the musical artist. Occasionally the listeners pick up things Echo owners likely would rather stay private: a woman singing badly off key in the shower, say, or a child screaming for help. The teams use internal chat rooms to share files when they need help parsing a muddled word -- or come across an amusing recording.

Sometimes they hear recordings they find upsetting, or possibly criminal. Two of the workers said they picked up what they believe was a sexual assault. When something like that happens, they may share the experience in the internal chat room as a way of relieving stress. Amazon says it has procedures in place for workers to follow when they hear something distressing, but two Romania-based employees said that, after requesting guidance for such cases, they were told it wasn't Amazon's job to interfere. [...] Amazon, in its marketing and privacy policy materials, doesn't explicitly say humans are listening to recordings of some conversations picked up by Alexa. "We use your requests to Alexa to train our speech recognition and natural language understanding systems," the company says in a list of frequently asked questions. In Alexa's privacy settings, the company gives users the option of disabling the use of their voice recordings for the development of new features. A screenshot reviewed by Bloomberg shows that the recordings sent to the Alexa auditors don't provide a user's full name and address but are associated with an account number, as well as the user's first name and the device's serial number.
An Amazon spokesperson said in a statement to Bloomberg: "We take the security and privacy of our customers' personal information seriously. We only annotate an extremely small sample of Alexa voice recordings in order [to] improve the customer experience. For example, this information helps us train our speech recognition and natural language understanding systems, so Alexa can better understand your requests, and ensure the service works well for everyone."

They added: "We have strict technical and operational safeguards, and have a zero tolerance policy for the abuse of our system. Employees do not have direct access to information that can identify the person or account as part of this workflow. All information is treated with high confidentiality and we use multi-factor authentication to restrict access, service encryption and audits of our control environment to protect it."

Further reading: How To Stop Amazon From Listening To Your Recordings

25 of 137 comments (clear)

  1. Wait, you DIDN'T think that was happening? by SuperKendall · · Score: 5, Insightful

    The article seems to present this as some new info, I assumed this was happening all the time, otherwise how else can Alexa improve?

    it's incidentally also why I don't have anything like Alexa or other voice assistants in my house, but if you are sending audio to Amazon hey guess what, something or someone is going to listen to that audio. DURRRR.

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
    1. Re:Wait, you DIDN'T think that was happening? by postbigbang · · Score: 5, Insightful

      No, no one should be suprised-- but they will be. Alexa isn't the only one, just the one currently exposed. How does one improve Alexa? Certainly training.

      Or, recycling it.

      Humanity has long desired servants. The servants are controlled by their masters, who are not you.

      --
      ---- Teach Peace. It's Cheaper Than War.
    2. Re:Wait, you DIDN'T think that was happening? by gweihir · · Score: 4, Insightful

      This is a standard tactic: First lie directly, then less directly, then admit a bit, then admit the whole. The average person is stupid and will only see the small steps, not the large overall one and will accept the whole thing. Works time and again.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    3. Re:Wait, you DIDN'T think that was happening? by Askmum · · Score: 4, Interesting

      I never thought about it because I conciensly will not buy such a product, but the way this article is written makes me think of the SciFi novels I read as a teenager. Those novels where there is a mostly hidden but all-encompassing upper layer of monitoring that we at that time thought would only happen in totalitarian states like the GDR or Russia.
      Guess what. It's here. And now. Maybe not yet in the way as in the former east-block, but give it some time.

      Sure, it can have good side-effects. Someone hearing a child cry for help, why wouldn't you alert 911 to get help. But boy is this a worrying development.

    4. Re:Wait, you DIDN'T think that was happening? by dwillden · · Score: 2

      Yep, Google admitted they did this outright years ago. That was a big part of the purpose for their Google Voice phone and voicemail systems. To get voice and speech recordings to train their voice recognition systems.

      But what people still don't seem to get is that these devices are not recording 24/7. They monitor for the trigger names to activate, "Hey Google", "OK Google", "Hey Alexa", "Hey Siri", "Hey Cortana" "Hey Galaxy" etc...

      So when the devices are activated then they record. Now they can be triggered by similar sounding phrases. When OK Google first became available on Radio Host on my local talk station had a phrase he used that occasionally triggered my phone. My Amazon Dot gets triggered occasionally when someone calls my son by his name Alex. But otherwise they only listen for the trigger phrases and only record once triggered.

      This is mostly not news. I do feel for the employees who hear distressing captures and can't do anything about them. I doubt they are even listening in real time and hopefully all the captures they process are anonymized making it even more difficult to be able to do anything about what they hear. I would hope that these companies all strip the time, location and network details of the captures before sending them to their employees.

      The purpose of the capture is to train the service to better understand the words and the variables of human speech idiosyncrasies, not actually monitor what is being said. Teach it to handle variations in accents and dialects. I grew up in a town called Layton. Most residents swallow the T in a glottal stop. People who didn't grow up there do pronounce the T. These systems need to be able to tell those differences as well as more extreme accent and dialectic differences in pronunciation.

      --
      I'm too lazy to compose a creative sig.
    5. Re:Wait, you DIDN'T think that was happening? by swillden · · Score: 2

      We can call it "listening" or "phrase recognition". They're the same. Now, maybe when phrase #1 is detected, action #1 is preformed. But then if phrase #2 is detected, then action #2 is preformed. However, that action #1 talks to you, and you're made aware of that, doesn't mean that action #2 needs to let you in on it all.

      Do you have any evidence -- at all -- that these devices scan for something other than the configured hotword?

      Note that since the companies that make them are publicly-traded, there's a legally-enforceable expectation that when those companies say that the devices don't search for any other phrases, they're telling the truth. If anyone could prove that they weren't it would get the companies in significant trouble with the FTC and SEC. This is especially true for Google, which is operating under the terms of an FTC consent decree put into effect after the Google Buzz incident.

      Oh, and if the devices were doing anything else, it wouldn't be at all difficult for skilled engineers to tell by disassembling the devices and observing what parts to what, when. Or by reversing the firmware. In addition, we're talking about Silicon Valley companies, where the engineering staff is notoriously willing to blow the whistle on anything they perceive as bad behavior.

      Given all of that, the company leaders would have to be complete idiots to allow anything like what you postulate to be done.

      --
      Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
  2. Look on the bright side by Tablizer · · Score: 2

    Well, at least such could be helpful to the next person who asks for "Natalie Portman to moan romantically about a Beowulf cluster of hot grits." It does get better each time ... my friend asks for it.

  3. why do people use these things? by schklerg · · Score: 4, Interesting

    Seriously, I have no objections to technology, but the people behind the technology are not even remotely trustworthy. Until we hit some semi-utopian Star Trek civilization, I cannot trust the machines due to the untrustworthy people behind the machines.

    --
    Be Excellent To Each Other
  4. Will learn the value of privacy when its too late by joe_frisch · · Score: 2

    There are a myriad of ways this can go badly. Everything from misinterpreting conversations leading to arrests to blackmail of politicians.

  5. Get rid of your tracker too. by jbn-o · · Score: 4, Insightful

    If you're like a lot of people, you already have a cell phone (more properly known as a tracker because that's what it does most of the time) so you already have the same spying capability in your house, on your person, and you likely choose to carry that around with you everywhere you go. Even technical users don't expect that the portable spy devices are listening whenever the proprietor wishes (and there's no indicator to tell the user when the mic is hot). You shouldn't own a tracker either.

  6. People don't believe it though. by Narcocide · · Score: 2

    I knew this was happening. Everyone I've warned about though, just plays it down. They like to believe that the chances of being listened to are very slim. They like to believe the privacy controls are sufficient and reliable.

  7. Re:Interesting by TheGratefulNet · · Score: 4, Interesting

    they (we) need to teach kids, early, ideally in school that:

    if a politician or businessman is saying something, its generally a good bet that he's lying.

    business has no ethics, not anymore. it will do anything to make a profit. lying is just a tool they allow themselves.

    kids need to learn this so that we can fix it next generation. they have the benefit of the internet so they have no excuse to grow up not knowing this.

    --

    --
    "It is now safe to switch off your computer."
  8. We take the security and privacy... by gweihir · · Score: 3, Interesting

    ...of our customers' personal information seriously.

    Translation: We do not give a fuck about you or your privacy. We will keep these recordings forever and eventually monetize them any way possible.

    From what I have seen, the more a company stresses how it values privacy, the less it actually does. The "Big Lie" approach at work.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    1. Re:We take the security and privacy... by tsa · · Score: 5, Funny

      They do take the security and privacy of their customers’ info seriously. They keep it locked away and sell it only to people they trust. It would be a bad idea to sell it to someone who might not pay for the info.

      --

      -- Cheers!

  9. Re:Interesting by gweihir · · Score: 4, Insightful

    While I agree on the problem, the solution is not going to work as most people (and most children) are stupid. Just look at the decisions they make. They know that politicians are lying, yet they still vote for the one that tells the better lies. They know corporations are just after money, yet they believe the ads. They vote against their own freedom, against their economic well-being and against their future. They are driven by fear, greed, hate and arrogance, and rationality makes only very rare appearances, if at all. And the "leaders" are cut from the same cloth.

    I am sorry to say this, but this installment of the human race is fucked, and it is doing all the fucking to itself. Sure, there is a minority (may 10-15%) that actually understands how things work, that can think independently, that can verify facts and that can recognize a thing for what it is. But these are far too few. It is almost as if this planet is a failed experiment as to whether this mix of independent thinkers and others works and I think we can safely say it works badly and no way to fix it that can actually be implemented is known.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  10. Re:If you care that much, poison the well by gweihir · · Score: 4, Insightful

    Be on a blacklist that makes this ineffective within 15 minutes. The ones running this operation may have absolutely no ethics, but they are not terminally stupid.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  11. No you don't by Anonymous Coward · · Score: 2, Informative

    You only have this surveillance if you gave Google access to the mic permission.

    I suggest you go into your phone and access permissions, microphone and turn off everything but the camera app and phone apps.

    The big problem with Android is you cannot deny apps NETWORK access, so I'd like to stop the phone and camera apps accessing networks.

    1. Re:No you don't by hawguy · · Score: 5, Insightful

      You only have this surveillance if you gave Google access to the mic permission.

      I suggest you go into your phone and access permissions, microphone and turn off everything but the camera app and phone apps.

      The big problem with Android is you cannot deny apps NETWORK access, so I'd like to stop the phone and camera apps accessing networks.

      You have a lot of faith that the creator of the operating system didn't give themselves a back-door way to enable the microphone anytime they like, regardless of what you've set the permissions to.

      While I feel pretty confident that when I deny a random app's access to the microphone, that app really can't access it, but I have less confidence Google themselves can't turn on the mic anytime they want.

  12. Re: Will learn the value of privacy when its too l by astrofurter · · Score: 3, Interesting

    Presumably both of those things have been happening for quite a while now. The system is operating as designed.

  13. Re:Interesting by SuricouRaven · · Score: 4, Informative

    The article does not claim otherwise. It only claims that humans may listen to what is said *after* the trigger phrase, or after something which the box misinterprets as the trigger phrase.

  14. The truth is... by LordHighExecutioner · · Score: 2, Funny

    ...that Alexa actually has no AI at all. It just records audio commands, sends them to a central server, where human monkeys listen to conversations, and make Alexa act accordingly. It is just like the Truman show, only bigger. Probably the same happens for Siri.

  15. Re:What should be done? by dwillden · · Score: 2

    Because ideally Amazon should be anonymizing the voice captures. All the agents should get is the voice clip, with at most a regional indicator. But otherwise they should get no details about who said it, where or when it was said. The point is to transcribe the words recorded and feed them back into the system with the capture so that it can learn to recognize those variants of the words.

    Maybe in the future if there is demand, they could consider adding a sub routine that could identify captures that indicate serious life threatening crimes and feed those directly to the local authorities.(who would then listen and if they decide it is an actual criminal act, rather than dialogue captured from a nearby TV,) then they could pull the location and respond. But not via transcribers in another country who may be working on captures that are hours or even days old.

    And what does making a WMD sound like? Is it a WMD or is it a chemical experiment for school? Or a home inventor at work building the next super duper vacuum to sell? Or a couple retired military buddies talking about some of the IED's they ran across in Iraq back in the Day?

    --
    I'm too lazy to compose a creative sig.
  16. Re:Interesting by skaralic · · Score: 3, Insightful

    While I agree on the problem, the solution is not going to work as most people (and most children) are stupid. Just look at the decisions they make. They know that politicians are lying, yet they still vote for the one that tells the better lies. They know corporations are just after money, yet they believe the ads. They vote against their own freedom, against their economic well-being and against their future. They are driven by fear, greed, hate and arrogance, and rationality makes only very rare appearances, if at all. And the "leaders" are cut from the same cloth.

    I am sorry to say this, but this installment of the human race is fucked, and it is doing all the fucking to itself. Sure, there is a minority (may 10-15%) that actually understands how things work, that can think independently, that can verify facts and that can recognize a thing for what it is. But these are far too few. It is almost as if this planet is a failed experiment as to whether this mix of independent thinkers and others works and I think we can safely say it works badly and no way to fix it that can actually be implemented is known.

    Relax. The world is complicated. Progress is not a straight line. People are free to make their own decisions, your understanding is not required. On the whole, we are doing amazingly well. The "challenges" of today are nothing compared to what we faced in the past. Nothing.

    Get some perspective.

  17. Re:Interesting by gweihir · · Score: 2

    Dead wrong. The challenges of the past were not global and quite a few civilizations managed to wipe themselves out by their stupidity throughout history. Today we do not have redundancy in this way anymore, everything is far too global.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  18. Re:Interesting by skaralic · · Score: 2

    Dead wrong. The challenges of the past were not global and quite a few civilizations managed to wipe themselves out by their stupidity throughout history. Today we do not have redundancy in this way anymore, everything is far too global.

    Yes, WW2 was not a global thing.

    The Black Plague was not a global thing.

    In total, the plague may have reduced the world population from an estimated 450 million to 350–375 million in the 14th century.[8] It took 200 years for the world population to recover to its previous level.

    https://en.wikipedia.org/wiki/Black_Death

    Remind us, again, what are the existential challenges that you're facing in your life?

    Learn some history. Get some perspective.