Cell Phones That Learn the Sounds of Your Life

An anonymous reader writes "Researchers at Dartmouth College have developed new software that uses the microphone on the iPhone to track and interpret a user's everyday activities using sound. The software, called SoundSense, picks up sounds and tries to classify them into certain categories. SoundSense can recognize completely unfamiliar sounds and runs entirely on the phone. It automatically classifies sounds as 'voice,' 'music,' or 'ambient noise.' If a sound is repeated often enough or for long enough, SoundSense gives it a high 'sound rank' and asks the user to confirm that it is a significant sound and offers the option to label the sound. In testing, the SoundSense software was able to correctly determine when the user was in a particular coffee shop, walking outside, brushing her teeth, cycling, and driving in the car. It also picked up the noise of an ATM and a fan in a particular room. The results [PDF] of the experiments were recently presented at the MobiSys 2009 conference."

Comments

Privacy

[sopssa]

For those of us who dont want to pay for it, dont worry! Next year the goverment will install it to your phone for free!

Re:Privacy

[Anonymous Coward]

US consumers are safe for next 10 years though: even the government won't be able to force telcos to activate a new feature.

Re:Privacy

[hattig]

Why not?

Your bill can now have:

"Federal Government Location Audio Sensing Fee : $4.99"

on it.

Re:Privacy

[davester666]

That was the sound of you shooting someone. For a monthly charge of $500 on your cell-phone bill, the phone won't call the police. Accept the charge?

Yes No

Re:Privacy

[MartinSchou]

Not only did you get it wrong, but most of the people replying to your post got it wrong as well.

GPS comes in several varieties. The common one used in cheap handheld units are using the C/A signal. This gives you an accuracy down to about 5 meters (due to multipath and atmospheric interference). The accuracy is determined by the precision of the built in clock which in term determines the size of your unit. C/A gets you down to about 5 meters accuracy.

Add in DGPS and you more or less eliminate atmospheric interference which can get you down to about half a meter. Technically 20 inches can count as a few inches when compared to the 200 otherwise.

P-code (the military) gives you down to about 2 meters accuracy by comparison. Not sure how much better they get with DGPS, but I'd suspect it'd get them to 1/10th just like the C/A does.

For the best accuracy you won't be relying on L1 and L2 directly (decoding the signals), but will be looking at the carrier phase change which requires bigger and better antennas as well as a much more precise clock which is why most of these are big, bulky and used for surveying more than anything.

If you're moving around (airplanes pictures and very likely road surveying as well) you can get down to about half a centimeter, but expect from 5 mm to 10 cm). When not moving for a significant amount of time, you can get down to 2 mm to about 3 cm)

As for one of the replies claiming that GPS only gives you a 2D location, this is rubbish. You need a clear view of a minimum of four satellites to get a proper height as well as longitude and latitude. The accuracy of each of these coordinates varies as well . However the biggest inaccuracy you're likely to face when dealing with GPS is using the wrong datum for your map. It's fairly easy to end up with coordinates several hundred meters from the correct one, merely by forgetting to switch datum when moving into a new area.

If you're not using the US GPS but instead rely on Glonass (not done yet) you get some advantages. Since each satellite is running its own discrete frequency, you're essentially able to rule out atmospheric errors. This adds to the cost of the unit though, as it now needs a much more advanced radio receiver. I can't remember if Galileo uses one or multiple frequencies. I think it's supposed to use two, but it's not that important.

Granted, it's been a while since I worked heavily with GPS theory, so feel free to correct me (if you can cite properly). I'm using Wikipedia because I doubt most of you are able to read Danish which is the language my text books on the subject.

new and improved ways to tweet your life

[ilblissli]

so in other words this will effectively let me tweet when i'm pooping without having to type it up on my phone. it may also be handy finding out if people have washed their hands after such an event ;)

ob. Clippy

[RobertB-DC]

"It sounds like you're going pee."

Then, if male: "Please, remember that there are ladies in the house, and put the seat down when you're done."
Else, if female: "If you're out of toilet paper again, I can call your sister to bring it to you."

Social networking?

[Ambitwistor]

From TFA:

"The SoundSense software was able to correctly determine when the user was [...] brushing her teeth [...] Choudhury says that enabling the software to learn to recognize new sounds will be essential for practical applications. 'A system that can recognize sounds in a person's life can be used to search for others who have the same preferences'"

That sounds like great functionality. " If you like brushing your teeth, you may be interested to know that 21 out of 23 people on your Contacts list also enjoy brushing their teeth."

Re:Oh great....

[CastrTroy]

Actually not a bad idea. A lot of time I would like to wear my iPod while doing house work, but I don't because I can't hear the baby cry. If I had an iPod that could recognize when the baby was crying, and play it over the headphones in place of my music, so that I knew the baby was crying, then I would really appreciate this feature. Same goes for somebody calling out my name. Even if there was a 1 second delay, it would be awesome.

In response to "Why?"

[bobetov]

I see a lot of tags/comments asking what this is useful for. There are a few uber-nerd things like recording your life and whatnot that I'm not going to get into, but the big one is determining location.

There are a TON of sweet things you can do with accurate location information, but the one that I'm most yearning for is to control my bluetooth, wifi, ringer volume, etc based on where I am during the day.

I'm an Android user, and there's a very nice applet called Locale that attempts to do this, but it proves to be pretty useless. The reason is that you're either using GPS (drains battery, doesn't work indoors) or wifi (drains lots of battery, and is the primary thing you want to control) to figure out where you are. If using the microphone and cpu is cheaper in energy, then this will be a big win.

Beyond the energy use argument, one of the main things you want to control is bluetooth - again, it drains batteries when on, and is not generally useful. But it's EXTRAORDINARILY useful in the car if you have a hands-free setup. Again, figuring out when you're in a car is hard via GPS or wifi, but this technique would seem to knock that one out of the park.

So, in summary, having your phone know where you are in your daily routine allows it to be more intelligent about what services and functionality it enables, and thus makes your cell phone that much smarter and more valuable.

Re:Next level of twittering

[jammindice]

Actually this sounds like the echo location batman used in the Dark Night at the end except for being self contained on the phone versus using just the mic from all phones and processing centrally

Clippy!

[Anonymous Coward]

It sounds like you are having trouble reaching orgasm. Would you like me to
1) Moan real loud
2) Show erotic video
3) or, just vibrate