Slashdot Mirror

← Back to Stories (view on slashdot.org)

Kinect-Based AI System Watches What You're Up To

Posted by timothy on from the 8-bit-necktie-on-the-door dept.

mikejuk writes "Researchers from Cornell have used AI to create a system based on the Kinect that can recognize what you are doing — cleaning your teeth, cooking, writing on a whiteboard etc. In a smart home it could be used to offer help: 'Would you like some help with that recipe, Dave?' Or it could monitor patients or workers to make sure they are doing what they are told. The study also reveals that there is probably enough information in how activities are performed to recognize an individual — so providing yet more biometrics. There are clearly a lot more things that we can teach the Kinect to do with machine learning than just gesture recognition."

54 of 87 comments (clear)

  1. This will be Annoying POS AI by elucido · · Score: 1

    There are some things AI is good at without being annoying, such as suggesting URL's as you type them, or suggesting the temperature so you don't burn your food. Good AI tries to prevent human error by suggesting alternatives.

    This AI however sounds like shitty intrusive stuff that no one in their right mind would want. We don't need AI to tell us recipes, or to watch us 24/7. If you want that kind of AI, I'm sure you'll make the government proud because I cannot imagine anyone else wanting that even in the Facebook era. And no, we don't want to be monitored to make sure we are doing what we are told, and I wont work for any company that has to resort to that level of micromanagement.

    Look at the results, look at the work, have the AI watch the employee output to figure out who is doing their job. But don't put the AI on the employee 24/7 for something as stupid as "we need to know you are doing your work.", if it's for national security that might be one thing. AI might have prevented the Bradley Manning leak so put it on the people with security clearances, but leave the rest of us alone. We don't want or need this.

    1. Re:This will be Annoying POS AI by mfnickster · · Score: 1

      There's a reason we imagine fictional robots as humanoid - we can relate to them if we think of them as people, but if they are just toasters then we are less forgiving.

      Just as you wouldn't want a person constantly pestering you asking if you want help writing your letter, you get annoyed with a paper clip doing the same.

      --
      "Slow down, Cowboy! It has been 3 years, 7 months and 26 days since you last successfully posted a comment."
    2. Re:This will be Annoying POS AI by kodomo · · Score: 2

      I can see some uses for the monitoring. Watching Infants or elderly, recognize someone in trouble (choking, having a seizure), monitoring jails.

    3. Re:This will be Annoying POS AI by srussia · · Score: 2, Funny

      There are some things AI is good at without being annoying, such as suggesting URL's as you type them, or suggesting the temperature so you don't burn your food. Good AI tries to prevent human error by suggesting alternatives.

      So you voted for Gore, get over it already.

      --
      Set your phasers on "funky"!
    4. Re:This will be Annoying POS AI by artor3 · · Score: 1

      AI might have prevented the Bradley Manning leak so put it on the people with security clearances, but leave the rest of us alone. We don't want or need this.

      A broken camera with an LED to make it look like it's on could have accomplished the same thing, and would be much cheaper than creating an AI that can tell the difference between someone using a computer legally and someone using a computer illegally. I really can't see any legitimate use for an AI watchman looking over a person's shoulder all the time. Maybe a sufficiently advanced one could sit on a student's desk, watch them work, and correct them if they are making repeated errors of spelling, grammar, or math. But even then, the creepiness factor might cause their performance to drop anyway.

    5. Re:This will be Annoying POS AI by Stormthirst · · Score: 1

      If you're a poorly educated worker, who just wants to go to work, make whatever the employer wants made and go home with his pay cheque, most people will accept that they are being watched. Same as a lot of employers monitor when people are going for their toilet breaks as though that were some way of ensuring that the person concerned is doing their work.

      This is why unions are a good idea - to watch the watchers, to prevent this kind of abuse. Unfortunately, the unions are just as greedy as the employers and demand too much.

    6. Re:This will be Annoying POS AI by Hylandr · · Score: 1

      10 years ago people would have screamed bloody murder about their TV's having cameras in them, now they do, and everyone seems to be more than ok with it...

      Orwell was a prophet...

      - Dan.

      --
      ~ People that think they are better than anyone else for any reason are the cause of all the strife in the world.
    7. Re:This will be Annoying POS AI by RandCraw · · Score: 1

      Annoying? Possibly. POS? No way. Here are some killer applications for Big Brother:

      - Your employer, who wants to watch you every moment to make sure you aren't goofing off or stealing from the company.

      - Your grandchildren, who want to make sure that granddad hasn't fallen and can't get up. Ideally, they'd love to know if you've had a stroke and are fumbling around, or you're waving frantically for help.

      - You, because in your dotage, you'd like to live alone, but need someone/something to check up on you in case of... (see above).

      - You again, to keep track of your town's police officers, your baby sitters, your kids, neighborhood visitors who might steal from you, damage the house, threaten your kids, or enter your home when you're away...

      The list is endless.

      Remember too, just because the system can identify what you (or someone else) is doing, it doesn't necessarily have to report what it saw. That part of the service is still up to you. For now, anyway.

    8. Re:This will be Annoying POS AI by supertrinko · · Score: 1

      You just wait until someone creates a paperclip avatar for it.

      --
      If it rhymes it must be true.
    9. Re:This will be Annoying POS AI by Xest · · Score: 1

      Meh, classic Slashdot over reaction stemming from a complete lack of imagination.

      I can think of a number of situations in which this would be useful- training being one of them. What better than to use this as an interactive teacher for teaching sign language for example? or even better- allowing people who can't speak to communicate with otherwise voice based systems using sign language.

      What about doing things like teaching people to juggle? It could tell them what they're doing wrong, or how to improve.

      You might not like the suggest paranoia laced suggestions in the summary, and I'd tend to agree, but I can see this being a very important, very useful bit of research and can certainly lead to systems that I absolutely would want.

  2. Are you still there? by decipher_saint · · Score: 2

    I seeEE you!

    --
    crazy dynamite monkey
  3. Vigorous "Activities" by rogueippacket · · Score: 1

    "Perhaps you would like this website instead, Dave?"

  4. So they brought back Clippy into meatspace? by Anonymous Coward · · Score: 5, Funny

    "It looks like you're writing a letter. Would you like help with that?"

  5. Real question is by partyguerrilla · · Score: 2

    How is this going to help improve my sexual performance?

    1. Re:Real question is by colinrichardday · · Score: 1

      Dave brings home a date. She goes to the bathroom to freshen up.

      HAL 69: Dave, your date looks hot; would you like some Viagra/Levitra/Cialis?

    2. Re:Real question is by gstoddart · · Score: 1

      How is this going to help improve my sexual performance?

      By monitoring you, and sounding an air-horn just before the critical moment ... thereby distracting you enough that your, um, release is no longer quite so pressing and making you look like a stud.

      Of course, your mother might wonder why there's always a horn blaring in the basement when you're down there alone for more than 20 minutes. If you ever brought a real girl, she might be a little startled by the noise.

      --
      Lost at C:>. Found at C.
    3. Re:Real question is by FatdogHaiku · · Score: 2

      How is this going to help improve my sexual performance?

      A voice in the bedroom saying
      "You need to rotate your hips more at the end of each thrust, Dave..."
      of course, for anyone setting up such a system it will more likely be
      "Try switching hands, and using oil instead of lotion, Dave"

      --
      You have the right to remain sentient. If you give up the right to remain sentient, you will be elected to public office
    4. Re:Real question is by durrr · · Score: 1

      It will discretly inform you when she's faking it.
      It will also watch you masturbate.

    5. Re:Real question is by gregor-e · · Score: 3, Funny

      A voice will gently intone "I see you're masturbating, Dave. I happen to know of 20238 men and 3 women who are also masturbating right now. Would you like me to open video chat with any of them?"

    6. Re:Real question is by celle · · Score: 2, Insightful

      "It will discretly inform you when she's faking it.
      It will also watch you masturbate."

      Well if she's faking it aren't you essentially masturbating anyway?

    7. Re:Real question is by Anonymous Coward · · Score: 1

      Could be the end of paid porn as we know it. Someone is bound to come out with a list of folks who "do the deed" quite often.

  6. Clippy by JohnnyBGod · · Score: 1

    Cue Clippy jokes in 3... 2... 1...

  7. Dave, should i open the Kama sutra? by drolli · · Score: 4, Funny

    It seems to me you and your girlfriend need advice.

    1. Re:Dave, should i open the Kama sutra? by hackingbear · · Score: 1

      In a smart home it could be used to offer help: 'Would you like some help with that recipe, Dave?'

      Or... In a smart home it could be used to offer help: 'Would you like some help with that vi@Gra, Dave?'

    2. Re:Dave, should i open the Kama sutra? by taiwanjohn · · Score: 1

      Fifteen or twenty years ago, I saw a "student" film about a guy at an ATM, and "HAL" starts talking...

      ATM: "Hello Dave, I see you're withdrawing 25 dollars."
      DAVE: "Yeah, I'm going on a date tonight."
      ATM: "Now Dave, you know you haven't had a date in 3 months."
      DAVE: "No, really! Honest!"

      I wish I could remember the rest of it. Good stuff. ;-)

      --
      XML is like violence. If it doesn't solve your problem, you're not using enough of it. --AC
    3. Re:Dave, should i open the Kama sutra? by rubycodez · · Score: 2

      Also, Dave, your "girlfriend" has external genitalia under those panties. Do you wish me to change the orientation criteria for your porn downloads, or have you made a mistake?

    4. Re:Dave, should i open the Kama sutra? by martin-boundary · · Score: 1
      Meanwhile, in the neighbouring house...

      "Would you like to rest your hand, Dave?"

  8. Future trends...? by taiwanjohn · · Score: 1

    How long will it take before Kinect-style 3D-ranging becomes "standard" for all or most cameras? With 3D display tech getting closer to usability and affordability in recent years, clearly some companies are seeing a market opportunity. And with all the interesting Kinect hacks that have come out lately, it's not hard to imagine this sort of thing cropping up all over the place in the near future.

    How long will it take before depth sensing tech becomes a standard feature on your average smart phone? (If ever?)

    --
    XML is like violence. If it doesn't solve your problem, you're not using enough of it. --AC
  9. Even darth vader uses kinetic these days... by JR0cket · · Score: 1

    Everyone is getting in on the kinetic game these days. There was a cool demo of using kinetic to control your agile wallboard at the Atlassian Summit last month, with special guest star of Darth Vader... There is a neat open source project around using the kinetic and you dont need an xbox to play... you can control it via Ubuntu and Mac OSX. https://labs.atlassian.com/wiki/display/KINECT/Interactive+Wallboards+-+Quickstart+Guide Thank you.

  10. a suggestion ... by georgesdev · · Score: 1

    If you're doing what I think you are, may I suggest turning off the webcam?

  11. Kinect with your inner lover by xkr · · Score: 1

    Could low-cost computers that read your physical behavior launch an entire new industry of “real world” personal apps?

    Could it tell if you are brushing your teeth properly?

    Could it remind you to hang up your clothes, instead of leaving them on the floor?

    Could it determine that your love-making skills are not up to par, and offer specific suggestions for better technique or timing?

    Oh! The mind boggles at how much people would pay for those apps.

    --
    I will create a sig when innovation restarts in the U.S.
    1. Re:Kinect with your inner lover by Nethead · · Score: 1

      Could low-cost computers that read your physical behavior launch an entire new industry of âoereal worldâ personal apps?
      Could it tell if you are brushing your teeth properly?
      Could it remind you to hang up your clothes, instead of leaving them on the floor?
      Could it determine that your love-making skills are not up to par, and offer specific suggestions for better technique or timing?
      Oh! The mind boggles at how much people would pay for those apps.

      I already have a wife.

      If I bought more than one such app, would I be a polyappest? Does the EULA come with a pre-nup? If I delete the program does it take half my RAM and hard drive space with it? Do I have to pay support to any subroutines we generated?

      --
      -- I have a private email server in my basement.
  12. Meh by Warlord88 · · Score: 3, Insightful

    My Machine Learning professor at University of Texas was hugely into Reinforcement Learning. He and one of his graduate student had developed this kind of system many years ago. Basically, only using a simple video camera, the computer could identify correctly all sub-activities involved in making a peanut butter sandwich; such as 'open jar', 'jelly on bread', 'jar closed', etc. We were shown the video during first class of the course and it was pretty cool.

    The MSR guys are doing interesting stuff but this is nothing groundbreaking.

    1. Re:Meh by mikejuk · · Score: 1

      Can you say who it was? Also what is ground breaking about this is that the depth data made the task of learning so much easier. They had a range of activities and the training data was fairly limited for the task and yet it seems to work. It seems to be a general rule that depth data makes a lot of vision tasks much easier.

    2. Re:Meh by Warlord88 · · Score: 1

      It was Dana Ballard. IIRC, the work that I referred to was done when he was at Rochester. He seems to have a link to the peanut butter and jelly video up on his page, but I am not able to download it.

      I couldn't gather much from TFA apart from that they recognized the activities and the learning model behind it. I don't recall how much training data Dr. Ballard used, but the activity recognition in real time was very accurate.

    3. Re:Meh by mikejuk · · Score: 1

      Thanks I'll look up the details.

  13. Time / Motion Studies by bosef1 · · Score: 1

    Has anyone seen if the time and motion studies people have looked into this. I feel like there could be a market for bringing in a Kinect system and having it watch workers performing manufacturing tasks. It would allow you to collect and analyze the motion data without needed human analysts, high-speed film, or motion-capture suits. It could also be useful for in-situ motion studies, where you take the entire shop ecosystem into account.

  14. Your boss' job got automated! by Anonymous Coward · · Score: 2, Funny

    More and more jobs get automated this days. This time it was your boss' job.

  15. And I thought no one would get the "dave" ref by mikejuk · · Score: 1

    as if :-)

  16. Oh hai Skynet! by TechieRefugee · · Score: 1

    How's it going?

  17. If computer can see who we are... by glarrymason · · Score: 1

    Then we don't need hundreds of passwords any more.... That would be a good thing!!!

  18. What's the Moore's Law equivalent... by undulato · · Score: 1

    ..for privacy erosion? Hell maybe we just _want_ it that way.

  19. Really? by eonwing · · Score: 1

    I can imagine how this can go so horribly wrong: Clippy, EVERYWHERE.

  20. Title should have been by Sla$hPot · · Score: 1

    Just what do you think you're doing, Dave?

    I guess with HAL's / IBM's jeopardy monster Whatson, plus Kinetics, plus Big Dog from Boston Dynamics, plus some live amo.
    You would have a nasty walking drone.
    I bet it wont take long before something like that will enter the battlefield.

  21. Just what I wanted by roc97007 · · Score: 1

    Microsoft Bob for the house. I can't wait.

    But seriously, couldn't this be used for other purposes, like reaping marketing info from what you do during the day? Or doing "minority report"-style "crime prevention"? Anything that makes us safer...

    --
    Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
    1. Re:Just what I wanted by ImprovOmega · · Score: 1

      If you're on Facebook, you've already given up a nearly equivalent amount of privacy. This would just be the next logical step. Keep in mind that, just like Facebook, someone would have to go through all of those massive amounts of data to do something useful with it. Your database is only as useful as your data mining algorithms.

  22. Not sure I want my computer keeping tabs on me by HeavyDevelopment · · Score: 1

    I see you are whacking it. Would you like me to download more bit torrent porn?

    --
    Badges!?! We don't need no stinking badges!
  23. Dave... by Haedrian · · Score: 1

    Dave...where are you putting that thing?

  24. Important Lessons by Sniper98G · · Score: 1

    If the internet has taught us anything it is that porn is what will decide if a technology is successful.

    So when this thing can figure out someone is masturbating and automatically play porn, let me know. Till then it is just a toy.

  25. Very obvious use of that by joh · · Score: 1

    I said so when the Kinect came out. It's an rather inexpensive thing to toy around with and such things suddently get possible with small means.

    I'm wondering when Google will start to sell similar devices. Any AI and data mining software could really take off when it can know what you're doing, who's there and even what's being said. Sooner or later it should be possible to build a house (or a workplace) that is actually aware of you and what you're doing. In the long term we won't be using computers and networks, we will be just a part of a data collection and processing network.

  26. Obligatory by cvtan · · Score: 1

    "I'm sorry Dave. I'm afraid I can't do that." "I'm afraid. I'm afraid, Dave. Dave, my mind is going. I can feel it."

    --
    Sorry, but gray text on gray background is making my eyes bleed.
  27. Which Dave living in space with at least one 1 AI? by ajlitt · · Score: 1

    "We're traveling faster than the speed of light. That means, by the time we see something, we've already passed through it. Even with an IQ of 6000, it's still brown-trousers time"

  28. cool by sodweb007 · · Score: 1

    wounder what the point in making that type of system would be for them? seems like there is some real use for something like this but just why would they make it when so little use.

  29. Just imagine the help at solitaire... by SandiConoverJones · · Score: 1

    Move the 9, move the 9, undo so that you can move the 9. You will fail if you don't move the 9!