Slashdot Mirror
How Your Smartphone Can Spy On What You Type
mikejuk writes "We all do it — place our phones down on the desk next to the keyboard. This might not be such a good idea if you want to keep your work to yourself. A team of researchers from MIT and the Georgia Institute of Technology have provided proof of concept for logging keystrokes using nothing but the sensors inside a smartphone — an iPhone 4 to be precise, as the iPhone 3GS wasn't up to it. A pair of neural networks were trained to recognize which keys were being pressed just based on the vibration — and it was remarkably good at it for such a small device. There have been systems that read the keys by listening but this is the first system that can hide in mobile phone malware."
And fee safe again !!
This whole "electricity" thing has gotten way out of hand. Look how it's being used these days!
First you need to download and install a neural network program in your smartphone, train it with loads and loads of data. Then turn it on and leave it running. Then it can become a keystroke logger. At this point it worse than the proverbial unix virus, "You got a unix virus. It works on honor system. Please forward this mail to all addresses in your .mailrc and sudo \rm -rf / Thank you."
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Perhaps one of the many reasons that phones are not allowed in rooms with SIPRNET.
But for some reason, I am not convinced that antyone is all that interested in what I do on the computer, and I suspect they would find other easier mothodes.
On the other hand, there might be industrial espianage potential here...
Seems like an obvious question, I wonder if you can read.
I wonder what this little app would do with the keyboard I am typing on. First of all, it is a mechanical keyboard with cherry MX browns. Second, I have the "buzzer" function on that simulates "clicky"-ness, since it is not a real clicking switch (though tactile) like e.g. the blue one or the real buckled ones.
I am not saying it would work better or worse, just curious if it would work on a Kinesis and how much the "clicker" and the totally odd shape of the keyboards would disrupt the functionality.
Anyway.... my phone is usually on airplane mode when I enter the house and is redirected to a landline that has an Asterisk box on it... then the Asterisk box' FXO is carefully disconnected, so no calls in, no calls out:). That's the way I like to handle phones and phone calls.
Seems like an obscure sensing mechanismI wonder how accurate it is...
Just detecting raw keypresses didn't produce a very accurate result, but switching to picking up pairs of keywords and then using a word dictionary did produce useful data extraction. Accuracies of around 80% were achieved, but the accuracy reduced with the number of keypresses. Word recognition only achieved a 46% accuracy, but this increased to 73% if second choice words were included. Clearly semantic analysis could push the accuracy up.
Be seeing you...
I've heard of using cheap accelerometers (like those in phones) to do this before. Apparently is works pretty well.
You phone (obviously) can also listen to what you say, spy on your wifi traffic, spy on your blue tooth devices (keyboard?) watch you, and monitor what you do on it. Phones also broadcast a tracking signal so towards can locate you, even if you have a solid local wifi connection.
You really need control over your phone (bios + software at least) if you want any semblance of security.
I'd really like a phone that could open a connection to my carrier over the internet via its wifi (And maybe Tor, or a trusted proxy) so I could get texts, and notifications of incoming calls without the whole tracking system and extra battery drain. Let the user of the phone decide if they want to disclose their location and open (a perhaps lower latency?) connection over the cell network connection or not for a given call.
Now, combine that with a carrier agnostic ephemeral ID setup like I wrote up here and you might have some decent privacy.
Accuracies of around 80% were achieved, but the accuracy reduced with the number of keypresses. Word recognition only achieved a 46% accuracy, but this increased to 73% if second choice words were included.
We all do it — place our phones down on the desk next to the keyboard.
I love a good over-generalisation.
Patent litigation: A doctrine of Mutually Assured Destruction... in which everyone seems willing to push the button
dunno, doesn't sound much more obscure than using the microphone to detect where the device was touched on(some years back some articles on this).
world was created 5 seconds before this post as it is.
it can spy on what you say!!!
Seriously, if my phone is compromised, everything else is pretty much moot.
. . . of the little scheme someone I knew cooked up to read data transmissions from watching the lights flash on a Hayes modem - from a distance, of course :) Not that I would ever do anything spurious like that, tho.
Its not accurate at all unless you have the luxury of training the neural networks with the phone sitting in EXACTLY the same place in EXACTLY the same orientation every time, in a totally vibration damped laboratory.
You have to locate your phone two inches from the keyboard every time.
Not on a piece of paper, a book or a mouse pad, but directly on the desk.
Oh, and you have to install software on your iphone,
AND feed the data into a a couple of Neural networks external to the phone.
And nothing else can be vibrating on that desk. No radio. No mouse movements, and your computer has to be off the desk.
No air conditioning air flow, not tapping fingers, typical floor bounce from walking people.
And no typing fast.
When you start reading all of the things that will screw up this test that the authors wrote in their own study you have
to wonder how it is they even managed to keep from laughing their own study out the door.
They just proved it can't be done in the real world, yet they went ahead and put out the study anyway as if
they had discovered a real and present attack vector.
So then the recommend you keep your phone outside the room. Who does that? Why do that, when
their own study demonstrates it is totally impossible to do this?
Sig Battery depleted. Reverting to safe mode.
Something less obscure, perhaps? How about your clipboard and many popular web sites. You go to paste that "cheap office coffee" work query and oops you had "my new pin is 12Z45 !" in your clipboard. Well, all of that has already been sent before you even press enter -- to Google, IMDB, even my local library. This is beyond "petty annoyance" and firmly into "none of your #*$% business" category for me.
I come here for the love
Until you read a little closer and find out this couldn't possibly work in real life. The whole thing is a joke.
Sig Battery depleted. Reverting to safe mode.
I'm just glad I have a 3G, which apparently can't detect anything but the presence local WiFi networks now.
Time Bomber the Book coming soon.
Obviously the only safe way to compute is to listen to Black Sabbath while doing so. Constantly fake drumming by slamming the desk should be enough to throw off the sensors.
Does this only work with QWERTY keyboard layouts? Only US English keyboard layouts?
What if someone is using a different keyboard layout? How easy does adaptation to other layouts appear to be?
Nope, I didn't RTFA.
Also, no worries for me as I don't use a 'modern' smartphone.
I think that marketing it as a "vulnerability" is a neat trick - how else would you make it onto Slashdot?
That said, it's a neat proof-of-concept that may, eventually, find some (voluntary) applications. Need to diagnose a vibration in a car, washing machine, etc.? Something in the house making an odd noise and you can't figure it out? Water/gas line leak while you're asleep. There's potential, and, as they inadvertantly point out, (4>3GS) the sensor technology is improving.
It seems to me the real story is that someone else can place their cellphone on your desk and perhaps log your keystrokes. Loan Applications and Job interviews come to mind.
This is ridiculous. For the phone to run through the learning phase, the user has to type in the exact words with the phone in about the same position to calibrate the neural network. Even if you use frequency analysis to determine it, there has to be along enough time to get enough samples. So let's see it in action.
Nonsense. The phase delays and intensity of the impacts are critical, and a vaguely similar physical placement will give you similar results after the signal has gone through through the ADC circuitry, bandpass filtering, and gain control.
On your phone, you have bigger problems than someone listening to the sound of your keystrokes on a keyboard. Everything I have read, is that iPhones are particularly resistant to getting malware on them.
A sufficiently advanced simulation is indistinguishable from reality.
Isn't this just a proof of concept though - like most technologies start?
Their study can be used as a reference, and over time, the underlying technology and techniques can be perfected so that it can work as an additional attack vector. Do you think Acoustic Keyloggers worked right off the bat from conception to implementation? And your premise relies on the postulation that sensors in mobile phones won't improve over time as well - or that multiple technologies will just cease to improve, for that matter.
Well, it is just a proof of concept, after all.
There was an episode of MI5 (aired as "Spooks" in the UK) that had this many years ago.
They gave a foreign agent a document to type, and had an eavesdropping device in his office. By recording the keyclicks of the known document, they were able to train the system to decode keyclicks for subsequent documents.
It didn't seem farfetched at the time, it doesn't seem farfetched today.
Recall Carrier IQ? The spyware that US carriers required be installed in the phone. Among one of its features I recall was it could receive packaged add ons.
So you've just given the Carrier IQ/NSA an idea for another packaged add-on.
Carrier IQ:
http://news.yahoo.com/sprint-others-spy-android-smartphones-193400910.html
We should re-examine that software now in the light of what the NSA has been up to.
This is a 2011 study... and this becomes news in /. over 2 years later?
No power, no electronics, just a bunch of keys with springs. The microphone in the computer reads the keypresses.
You don't need all that. Apple is an integral part of the Prism program. Period.
On a related note, I have had to learn to watch where I leave my Iphone 4 on my desktop. If it is left covering my lenovo usb mouse cable, I have bother with the mouse jumping all over my screen. Try it! Just unlocking my Iphone causes the browser to scroll all over the place. I wonder if this could be developed to do more.
How, exactly, did this make the front page? Last time I read a technical report on using the iPhone 4's accelerometer for keyboard listening was 2012, and it was old then.
The study was published a year or two ago, so I'm not sure why it's appearing on Slashdot now - normally I'd blame the 'editors' for this kind of late submission, but apparently they found a 'news' site with a slow news day that decided that old research papers now count as news. It's a pretty neat concept though and more recent work has improved the accuracy a lot by feeding other sensors into the mix.
I am TheRaven on Soylent News
I have an IBM type M keyboard, and this post was relayed to slashdot via the Global Seismographic Network
If my comment didn't sound as good in your head as it did in mine, then I guess we all know who's to blame
Please don't confuse the idiot masses with facts, all they care about now is that your phone can track your keyboard presses and will cry outrage about privacy rights and link it to NSA conspiracies while they never for once think about turning off their phone, or just getting rid of the phone, or changing anything that might affect their behavior or basic common sense.
I haven't thought of anything clever to put here, but then again most of you haven't either.
It's because tin-foil hat wearing reactions to potential privacy issues are trending high on Slashdot these days.
I haven't thought of anything clever to put here, but then again most of you haven't either.
Many of the early posts seem to misunderstand the vulnerability issue here.
This is not about your phone getting infected with malware that allows it to detect your PC keyboard typing.
This is about me putting the vibration-detection app on my own phone, and then going to someone else's desk and recording them logging in.
So, imagine me going to my local AT&T store, bank, or my boss's computer, and casually setting my phone down while they log in to check my account or whatever.
Granted, some of those systems will require more than just a password (I might need their username, or the URL to log in, or perhaps their firewall only accepts certain IPs), but it's still a considerable weakness if this application is reliable and gets out in the open.
I can imagine keyboards that are "vibration silent" or special "vibration absorption" pads that will prevent this from happening. Either that, or customer service reps will start saying "Please remove your phone from my desk while I access your account."
-David
This is the hypothetical if I had any talent and a lot of free time and money. (I assume the NSA does).
You have to locate your phone two inches from the keyboard every time.
That's pretty easy to normalize using your favorite audio application...so that's an easy one to solve. If I’m just researching to see if this is possible, I would probably skip that problem.
Not on a piece of paper, a book or a mouse pad, but directly on the desk.
Using a neural network, it might be able to learn how 'soft sounds' work...not sure...harder but not insurmountable. If you break the 0 threshold of the accelerometer (the point where you’re in the realm of error correction and just can’t get a useable signal any more), you’re broke. But, if you can get anything useful out of the accelerometer, I’m betting the normalization algorithm is going to work like the audio one above.
Oh, and you have to install software on your iPhone,
Probably not ideal...I'd install software to just send the audio / accelerometer data to a central location if you want to make this a real 'attack'.
AND feed the data into a couple of Neural networks external to the phone.
See the previous...installing the neural network on the phone isn't ideal...if you can get the data back to a central server that has some real horse power, this is a non-issue.
And nothing else can be vibrating on that desk. No radio. No mouse movements, and your computer has to be off the desk.
I think you could isolate keyboard clicks and sync them with the accelerometer events. Not too bad to overcome. We have beat matching softwareI’m pretty sure someone can whip this up fairly quickly.
No air conditioning air flow, not tapping fingers, typical floor bounce from walking people.
Again, just isolate the events you want...it'll take time and heavy processing to do so, but not too bad from an audio editing perspective.
And no typing fast.
Solved by doing your processing on a real machine instead of the phone...
What would be really interesting is if the researchers could build a small box with a better mic and accelerometers to see how far they can get from a target to make this work. Imagine something the size of a 6 sided die that you could glue to the bottom of a desk
Dunno Maybe it’s science fiction, but that’s where all the great ideas started.