Slashdot Mirror

← Back to Stories (view on slashdot.org)

How Google's Pixel 2 'Now Playing' Song Identification Works (venturebeat.com)

Posted by BeauHD on from the behind-the-scenes dept.

An anonymous reader shares a report from VentureBeat, written by Emil Protalinski: The most interesting Google Pixel 2 and Pixel 2 XL feature, to me, is Now Playing. If you've ever used Shazam or SoundHound, you probably understand the basics: The app uses your device's microphone to capture an audio sample and creates an acoustic fingerprint to compare against a central song database. If a match is found, information such as the song title and artist are sent back to the user. Now Playing achieves this with two important differentiators. First, Now Playing detects songs automatically without you explicitly asking -- the feature works when your phone is locked and the information is displayed on the Pixel 2's lock screen (you'll eventually be able to ask Google Assistant what's currently playing, but not yet). Secondly, it's an on-device and local feature: Now Playing functions completely offline (we tested this, and indeed it works with mobile data and Wi-Fi turned off). No audio is ever sent to Google.

4 of 129 comments (clear)

  1. works offline? by Anonymous Coward · · Score: 3, Insightful

    How in the actual fuck is this possible? They have an audio an audio signature of every song built in?

    1. Re:works offline? by lucm · · Score: 4, Insightful

      Why shit on mp3 and try to re-invent the wheel with vectors?

      First, nobody is shittng on mp3. As for the reason to use tiny vectors instead of storing big mp3 files, I'm not sure why I have to explain it to you but it comes down to two things.

      1) Storage
      2) Availability of advanced, high quality vector processing libraries like BLAS or LAPACK

      this being said, it was just my guess, for all I know maybe they are storing data in sqlite3 or in the headers of a jpeg file that shows your mom pleasuring herself with a maglite.

      --
      lucm, indeed.
    2. Re: works offline? by Zero__Kelvin · · Score: 2, Insightful

      It uses a microphone and analog to digital converter, there is background noise, and they don't have a known start and stop point. The incoming bitstream is not by any stretch of the imagination an invariant, so however it works "that ain't it."

      --
      Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
    3. Re: works offline? by omnichad · · Score: 3, Insightful

      Checksum is shorthand terminology - music fingerprinting is probably more accurate, but everyone knew what they meant.