Slashdot Mirror
Cell Phones That Learn the Sounds of Your Life
An anonymous reader writes "Researchers at Dartmouth College have developed new software that uses the microphone on the iPhone to track and interpret a user's everyday activities using sound. The software, called SoundSense,
picks up sounds and tries to classify them into certain
categories. SoundSense can recognize completely unfamiliar
sounds and runs entirely on the phone. It automatically classifies
sounds as 'voice,' 'music,' or 'ambient noise.' If a sound is repeated
often enough or for long enough, SoundSense gives it a high 'sound
rank' and asks the user to confirm that it is a significant sound and
offers the option to label the sound. In testing, the SoundSense
software was able to correctly determine when the user was in a
particular coffee shop, walking outside, brushing her teeth, cycling,
and driving in the car. It also picked up the noise of an ATM and a fan in a particular room. The results [PDF]
of the experiments were recently presented at the MobiSys 2009 conference."
For those of us who dont want to pay for it, dont worry! Next year the goverment will install it to your phone for free!
@#$%#@!
3 categories. Less audio sensitivity than a nomad. Lame.
Have the phone tell everyone what you're doing at every moment in time. Someone get on that. Then when it happens, we'll have a YRO article about how this could be an invasion of privacy!
"The Y chromosome is genetic. The odds are very good that if you are male then your father was too." -Internet Commenter
1984 wold be boring. No chance at all.
so in other words this will effectively let me tweet when i'm pooping without having to type it up on my phone. it may also be handy finding out if people have washed their hands after such an event ;)
Lets be honest people. If a device is capable, someone will write the software to enable it. This shouldn't be surprising or shocking. When 'wearable computers' started getting buzz it was because people were walking around with web-cams attached to their heads seeing everything they could see and slashdot thought it while amazingly geeky, was cool. This isn't that different except there's no soldering required.
To be honest, we haven't even seen the worst of it yet. Considering the deluge of FPGA and EEPROM powered embedded devices out there you'd best be scared of the things that are _hard_ to reprogram, not the ones with complete IDEs and API documentation available.
I'm more concerned about someone snooping on me from my Jura Capresso than I am from my cell phone.
Like I really my phone to tell me my wife is bitching at me, or the kids are whining...
With this amazing new software, your smart phone can notify you when you are in a coffee shop, and when you are at home relaxing. Will wonders never cease?
I can see some use for the stuff mentioned at the end, where the software would classify periods of time to make them easier to search through, but I don't have any particular desire to have a continuous recording of my life.
Nerd rage is the funniest rage.
"It sounds like you're going pee."
Then, if male: "Please, remember that there are ladies in the house, and put the seat down when you're done."
Else, if female: "If you're out of toilet paper again, I can call your sister to bring it to you."
Stressed? Me? Of course not. Stress is what a rubber band feels before it breaks, silly.
From TFA:
"The SoundSense software was able to correctly determine when the user was [...] brushing her teeth [...] Choudhury says that enabling the software to learn to recognize new sounds will be essential for practical applications. 'A system that can recognize sounds in a person's life can be used to search for others who have the same preferences'"
That sounds like great functionality. " If you like brushing your teeth, you may be interested to know that 21 out of 23 people on your Contacts list also enjoy brushing their teeth."
could be to anonymously feed thousands of opt-in users current sound input into a public algorithmic service which provides data facilitating the creation of truely random numbers. Similar to the use of atmospheric noise by sites like Random.org; but this could be better because the sound input devices are decentralized and always moving.
There are the obvious problems to overcome with attempts to game this sort of system, but I think it an interesting idea.
Did you ever wake up in the morning, with a Zombie Woof behind your eyes? -- FZ
FTA: To address privacy concerns, they designed SoundSense so that information is not removed from the device for processing. Additionally, the program itself doesn't store raw audio clips. A user can also tell the software to ignore any sounds deemed off limits.
I can think of lots of fun and interesting uses for this kind of monitoring. And I can't think of many reasons to limit what sounds my phone can hear. Total capture and processing of the sounds of my life could have remarkable implications which I'm sure Charlie Stross or somebody is already writing about.
I'm concerned over the privacy issues. And these are total dodges on the privacy front. By the time the useful information has been processed, listed, and categorized, the raw data doesn't count for much. It's the end result of the algorithm, not the data that goes into it, that is of interest to advertisers, nanny states, and Generally Evil People. Unless maybe you're trying to use the phone as a bug. Fortunately, I rarely say (or arguably write) much of interest to anyone but the voices in my head.
The crux of the privacy issue is this, for me: I want *control* over *who* gets to looks at the results of the reality mining algorithm. If it's going to an app that determines how best to advertise back at me, well, fuck that. Or some bureaucrat who wants to make sure I'm not engaging in troublesome activities; fuck that too. If that's the price, I'll forgo the experiment and just keep track of things myself, thanks.
Now my phone will ask why it's 4:20 all the time ;)
"I bow to no man" - Riddick
OK, so jokes aside, could not this be used to build profiles of areas where you might be trying to hold a coversation and apply noise cancellation activily to the phone? blocking out all the background noise for the person you are talking too, perhaps even making it a bit eaiser for you to hear in the process?
I Need someone to rebuild a Digitech Digital Delay pedal for me....for me...for me...for me.
Finally, an app that can automatically record how many times a day we were rejected by women. It should be called iGiveUp and have an icon of a geek with a gun in his mouth.
The application in question categories sounds and thus tries
to label them. This is typically done using AI algorithms like
a combo of a neural net and fuzzy logic. That is pretty impressive to have it running on a mobile
phone of all things.
I think it's very cool
that we have come to the point that we basically have a device
in our pocket that resembles a combination of the stuff mentioned
in books like the Hitchhiker's Guide and the Ender saga.
How long before it learns all of the curse words in your life?
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
I wonder how many of these SoundSense users will end up having the "awooaaaa" sound of a booting Mac get classified as a "significant sound".
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
What would be the point of identifying the sounds? Would the phone do anything useful with the info? And, why would anyone put useless software on a phone? OK, I know most of the software people use is useless, but this sounds unbelievably useless.
The whole government? If so, I'm pretty sure they'd never be able to agree on how to get to my house.
What useless fearmongering. When cell phones first became pervasive, people had this same kind of hysteria: Oh no, we're all carrying microphones attached to a phone network! The government can work with cell phone companies to eavesdrop on our lives! 1984 was just a decade late!
After all the repetition of not outlawing technology that has legitimate alternative uses (i.e. p2p software), it's sad to see how many kneejerk fear responses there still are to each new advance that could be misused. The point of the article isn't that cell phones now have microphones that can be enabled, it's that the software is getting better at sound identification in context. And that has plenty of good uses, which the article lists, if you could disturb your paranoia to take look.
Yes, I readily agree that spying via mining of the resulting data could easily accomplished. But let's be serious: most of these phones already have a GPS receiver -- you're not panicking about "the government" tracking your every movement already, are you? And if you are -- just don't buy one. Nothing's making you use a phone with these capabilities.
Friday afternoon, when all the crazies come out.
"ATM machine" appears in two summaries in one day? That sure is redundantly redundant.
And it rings you to ask if it's a significant sound?
After 30 minutes...I'll get a buzz saying:
"Do you want me to remember the fart sound I keep picking up? And by the way, you better get some fresh air"
WTF? Over?
After looking at the charts I would have thought that the pink asterisk would have muted out the red dot... Maybe the red dot includes the TV?
... though I suppose most of the Slashdotters would say "Someone really is out to get me!" Maybe. Let's ignore that for a moment, and address a question that arose multiple times in the comments. What possible use was this developed for?
Research. That's right. Research. Actually finding out what people are doing/eating/etc. is actually really hard. People's recollection is full of holes. Think about it: What were you doing exactly 1 hour ago? Most people will be able to name the task, but not what specific activity in that task they were doing. Where you talking? Where you typing. Where you taking a two minute mental break and staring off into space? Yeah. It's hard to recall these inane details. But things like how many people you have contact with, how often you're in public-public (and not in an office, etc.) are hard to get at. How many times have I been to the café down the street in the last week? Hell if I know. But if I volunteer for a study - let's say, time spent in a restaurant by number of colds people get (totally made up on the spot for the purpose of illustration), this way the researcher doesn't have to trust a potentially erroneous recollection... they can get another, good estimate. It's made to be opt-in, and for research of this, that or the other thing.
Is there a potential for abuse? Sure. There's lots of things with abuse potential. But I hate to break it to you, but they tracked Pablo Escobar in 93 from his phone. And it's only got easier since. If you want perfect privacy, don't get a cellphone.
how much battery power will this suck down?
We had Furbies doing this several years ago and nobody freak'd out
People don't care about privacy, they just don't. Everyone here on /. upset about govenment civilian spy programs and such, and then everyone else gives up everything about themselves, willingly and deliberately, on sites such as Facebook.
Let me say that again, just so it's clear - most people just /don't care/ about privacy.
And /they/ are the ones that we need the laws to protect, those not smart or competent enough to take care of themselves.
Bringing the point back to the article - as soon as some flashy popular website springs up that takes this stuff into account, you'll have /millions/ of people uploading this information about themselves.
Is it still an invasion of privacy when people give up the information willingly?
I see a lot of tags/comments asking what this is useful for. There are a few uber-nerd things like recording your life and whatnot that I'm not going to get into, but the big one is determining location.
There are a TON of sweet things you can do with accurate location information, but the one that I'm most yearning for is to control my bluetooth, wifi, ringer volume, etc based on where I am during the day.
I'm an Android user, and there's a very nice applet called Locale that attempts to do this, but it proves to be pretty useless. The reason is that you're either using GPS (drains battery, doesn't work indoors) or wifi (drains lots of battery, and is the primary thing you want to control) to figure out where you are. If using the microphone and cpu is cheaper in energy, then this will be a big win.
Beyond the energy use argument, one of the main things you want to control is bluetooth - again, it drains batteries when on, and is not generally useful. But it's EXTRAORDINARILY useful in the car if you have a hands-free setup. Again, figuring out when you're in a car is hard via GPS or wifi, but this technique would seem to knock that one out of the park.
So, in summary, having your phone know where you are in your daily routine allows it to be more intelligent about what services and functionality it enables, and thus makes your cell phone that much smarter and more valuable.
Looking for a Rails developer in Chapel Hill?
Finally, a phone that can automatically disable the ringer when the phone's owner is having sex (or snoring).
I've abandoned my search for truth; now I'm just looking for some useful delusions.
...your phone listens to you!
"It sounds like you are sitting next to a fan.
Would you like help?"
There is no way in hell an app constantly running and listening to the mic is going to last 8 hours.
I'm on my second iPhone, both the original and the 3G, there is no way in hell they have the battery life to do this.
If by 'everyday life' the author means sitting in front of your PC with the USB cable attached, then okay, not my definition of everyday life, but okay. Otherwise, I call bullshit.
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
Now I can make use of my extended battery that I bought off of woot today. Seriously though, it would be a pointless app to drain and deteriorate the already poor life of the battery. But as others have posted, the application would be better suited for something other than the iPhone.
It sounds like you are having trouble reaching orgasm. Would you like me to
1) Moan real loud
2) Show erotic video
3) or, just vibrate
Once the phone learns what the sounds are, you could bridge the gap, and just have it auto-twitter what your doing... but make sure you have a blacklist of items NOT to post... such items; Pirating music/movies, having sex with that hooker while your gf/wife is at work, grunting so hard you blow an o-ring after eating chillicheese dogs for a week from A&W...
keep track of my flatulence and orgasm noises and share stats on those with my friends. Now thanks to technology I can!
Arbitron is already using something somewhat like this.
They have their "test families", or whatever they're called, carry small devices they call "portable people meters". Television programming includes sonic markers outside of the audible range, which these devices pick up via a small mic.
One would think that being able to identify television/radio programming without pre-inserting said inaudible watermarks could simplify their process.
So if you fart a lot you'd get beano ads on your iPhone?
They did exactly that.
Your personal black box
Well, I hope for me it can recognize the sound of hot lingerie models having screaming orgasms! Heh heh heh!
No, not really. :-( Pretty much just silence here.
iPhone: That clicky sequence you made just now, I hear that a couple dozen times a day, what is it?
user: That's my password.
iPhone: Thanks, let me just back that up for future reference.
user: <BLINK> "Where's the battery on this thing?"