Slashdot Mirror

← Back to Stories (view on slashdot.org)

How Google's Pixel 2 'Now Playing' Song Identification Works (venturebeat.com)

Posted by BeauHD on from the behind-the-scenes dept.

An anonymous reader shares a report from VentureBeat, written by Emil Protalinski: The most interesting Google Pixel 2 and Pixel 2 XL feature, to me, is Now Playing. If you've ever used Shazam or SoundHound, you probably understand the basics: The app uses your device's microphone to capture an audio sample and creates an acoustic fingerprint to compare against a central song database. If a match is found, information such as the song title and artist are sent back to the user. Now Playing achieves this with two important differentiators. First, Now Playing detects songs automatically without you explicitly asking -- the feature works when your phone is locked and the information is displayed on the Pixel 2's lock screen (you'll eventually be able to ask Google Assistant what's currently playing, but not yet). Secondly, it's an on-device and local feature: Now Playing functions completely offline (we tested this, and indeed it works with mobile data and Wi-Fi turned off). No audio is ever sent to Google.

6 of 129 comments (clear)

  1. works offline? by Anonymous Coward · · Score: 3, Insightful

    How in the actual fuck is this possible? They have an audio an audio signature of every song built in?

    1. Re:works offline? by lucm · · Score: 4, Insightful

      Why shit on mp3 and try to re-invent the wheel with vectors?

      First, nobody is shittng on mp3. As for the reason to use tiny vectors instead of storing big mp3 files, I'm not sure why I have to explain it to you but it comes down to two things.

      1) Storage
      2) Availability of advanced, high quality vector processing libraries like BLAS or LAPACK

      this being said, it was just my guess, for all I know maybe they are storing data in sqlite3 or in the headers of a jpeg file that shows your mom pleasuring herself with a maglite.

      --
      lucm, indeed.
    2. Re: works offline? by Zero__Kelvin · · Score: 2, Insightful

      It uses a microphone and analog to digital converter, there is background noise, and they don't have a known start and stop point. The incoming bitstream is not by any stretch of the imagination an invariant, so however it works "that ain't it."

      --
      Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
    3. Re: works offline? by omnichad · · Score: 3, Insightful

      Checksum is shorthand terminology - music fingerprinting is probably more accurate, but everyone knew what they meant.

  2. Dear Google by Anonymous Coward · · Score: 0, Insightful

    My phone said it closed Firefox and Here maps to save power.... but I haven't run these for days.

    IMPLEMENT A FOOKING CLOSE APP FEATURE, FFS.

    Why are these apps even hanging around once I've finished with them. The choice to keep apps running and only close them when Android thinks they should be closed WAS LOST WITH v5. Prior to v5, apps would be gracefully packed away, after v5 you made the assumption that apps would/could save their state completely and restore it and forcibly wiped background apps from memory regardless of state when you chose.

    These choices were dumb. Apps cannot restore server connections or completely restore the state every time, which is why people them force kept the app open to prevent you closing it randomly.

    You then compounded this fooking dumb choice, with these 'power saving' things. Stupid features which even close MP3 players when the screen is off.

    How about you step back and do this properly.

    Apps are closed by the user when the USER is finished with them. You don't randomly close stuff and apps don't need special permissions or steps to stop you randomly closing them.

    Is it so hard? Firefox and Here should not be running when I've done with them, there should be a proper close. You don't need to fix this shit you created with more shit.

    At the moment you cannot make software reliably run on Android because of the OS. Only mini-apps can reliably use it. Everything else needs to register a special notification (to say its running, WTF...) or they will be closed at random in their run cycle.

    Backtrack on this dumb "no close needed" idea. It's had its day.

  3. Re:Weasel Words by Anonymous Coward · · Score: 1, Insightful

    This is google we are talking about. ALL your personal data is immediately supplied to the Democrat party and the NSA/CIA deep state crisis actors who are WAITING to use it to confiscate your guns and emprison you and your family.