Demo of a New "Sixth Sense" Technology

TEDChris writes "Here's an intriguing attempt at a versatile new tech device that tries to augment the wearer's five senses. It comes out of Patty Maes's group at the MIT Media Lab. By combining a computerized personal projector with a camera and linking both to the Net, a host of surprising new applications becomes possible. This 8-minute demo created a lot of buzz at TED last month and was posted online today. Would love to know what the Slashdot community makes of it."

Comments

Sixth Sense

[Anonymous Coward]

Turns out that the character played by Bruce Willis was shot dead at the beginning of the movie.

Warning: the preceding was a spoiler.

Re:Sixth Sense

[PCM2]

Dr. Manhattan killed Rosebud? That was Charles Foster Kane's favorite sled!!

Re:Sixth Sense

[PPH]

I thought Willis' career was mortally wounded by Hudson Hawk.

Re:Sixth Sense

[fractoid]

Wait, that guy in that movie I had to watch for English class... he made beer?

Re:Sixth Sense

[SenseiLeNoir]

The only issue i have is that there is actually already more than five senses in the human body.

In addition to:
- Sight
- Hearing
- Taste
- Smell
- Touch

There is:

- Balance and acceleration
- Temperature
- Kinesthetic sense (the part of the brain that tells the position of various parts of our limbs. Previously this was thought to be related to touch, but its been found in weightlessness, our brains can still "sense" the position of our limbs.)
- Pain (different to touch, as pain can exist without touch.

There are also other senses used for respiratory, etc.

One possible application

[Locke2005]

Now you really CAN see dead people!

Re:One possible application

[tlhIngan]

And you won't have to imagine someone without their clothes. You can just turn on the mode which shows everyone like they would sans theirs clothes. And if you get caught doing that, you also risk getting killed, in which case you will not only be able to see dead people, you'll become one with them. Yippie.

Wasn't there a Sony camcorder over a decade ago that could do this?

ISTR that it was recalled due to its "night vision" mode turning into more like "x-ray vision", except stopping at just under the clothes... I wonder how many cameras were actually returned, and how many were re-sold to others who wanted that particular feature.

Re:One possible application

[CatsupBoy]

Reuters

Linked to from the references section of this wikipedia article:

Infrared photography

spidy sense!

[jollyreaper]

Now all we need are web casters, ultra-sticky material for the hands and feet, and someone to beat Tobby MacGuire with a bar of soap in a sock if he comes anywhere near it.

I already have more than five senses

[holophrastic]

I, as a typical human, have plenty more than five senses. I would have hoped that people's understanding of their own body would have continued past grade-school.

But in any event, I welcome yet another sense beyond my current twenty-something.

Re:I already have more than five senses

[DrLang21]

This isn't a "sense" at all. A "sense" implies something that provides continual information about your immediate environment. This is just another human interface to a computer. Nothing to see here.

Re:I already have more than five senses

[JustinOpinion]

I, as a typical human, have plenty more than five senses. I would have hoped that people's understanding of their own body would have continued past grade-school.

Yup. This is a pet-peeve of mine, too. Humans have between 9-16 senses (or more), depending on how you want to count/divide them. The "5 senses" idea dates back to Aristotle... and we've learned quite a bit about the world and the human body since then. Frankly it's ridiculous that even in grade school children are told that humans have 5 senses: it's patently false. And it's quite easy to demonstrate otherwise (e.g. ask a person if they can sense which way is down).

It bugs me to no end that these kinds of basic science mistakes are repeated ad nauseum.

Re:I already have more than five senses

[von_rick]

The purpose of putting the term "sixth sense" in quotes tells you that it clearly isn't a sixth sense in literal terms. They are just calling it so because they couldn't find any other term which would make people sit through the whole demo, which btw was ultra-sleek.

Re:I already have more than five senses

[Curtman]

This is just another human interface to a computer. Nothing to see here.

Did you watch the video? There is definitely something to see here.

I expected to have a foodarackacycle and a rosie robot by now, at least a flying car. But this will do for now.

Re:I already have more than five senses

[petes_PoV]

The thing about school is that it biggest tacit lesson is to give the expected answer - not the correct one. As the OP, I use this exact question (along with how many colours in a rainbow) as an example.

It goes down great with the kids, though the teachers I know, hate it.

Re:I already have more than five senses

[DrLang21]

I did. Looks like a specialized PDA with internet access. We were supposed to have hand held devices that would do similar things with RFID tags by now. If that can't make it to the market, this stands no chance. I predict that the technology is far too expensive for a consumer device.

Re:I already have more than five senses

[Marxist Hacker 42]

Exactly. Now the cyborg implant of a magnet in the middle finger of the left hand- that at least does SOMETHING NEW.

Re:I already have more than five senses

[Goldsmith]

Depends on who you ask...

I would say we have only 4 senses: chemical, light, pressure and temperature... but part of my job is to work on electrical analogues of sense and often this boils down to the most basic properties. For example, telling which way is down is just an application of a pressure sensor, even though it's nothing like a sense of "touch".

Of course, I understand completely that a neurologist is going to have a different opinion, which is correct in its own way, and probably more similar to how a computer scientist would think of things.

But, yeah, the 5 senses thing is pretty dumb.

Re:I already have more than five senses

[tsm_sf]

Should I get off your lawn?

Re:I already have more than five senses

[edelholz]

She said that the components cost about $350 as is.

Re:I already have more than five senses

[MozeeToby]

1 & 2. Brightness and Color
3. Hearing
4. Pressure
5. Pain
6. Temperature
7. Smell
8-13. The different Taste Receptors
14. Balance/acceleration
15. Proprioception - Knowing where your body parts are. If you don't believe this is a sense check out the Pinocchio Illusion
16. Vasodilation in the skin (blushing)
17. Sensing a full bladder/bowels
18. Intestinal discomfort (not actually pressure nerves, that's just how your body perceives them)

Re:I already have more than five senses

[Curtman]

I did. Looks like a specialized PDA with internet access.

Yeah, just like that except you aren't holding a PDA in your hands, it's projecting on to surfaces that you are looking at.

I predict that the technology is far too expensive for a consumer device.

Again, I predict that you didn't watch the video. They built it with "off the shelf" parts for $350.

Re:I already have more than five senses

[Anonymous Coward]

1. Sight
2. smell
3. touch
4. taste
5. hearing

Care to name your other twenty something senses?

PS: there should be no hyphen between "grade" and "school". Dodn't you learn that in grade school?

Nope I dodn't learn that in grade school

Re:I already have more than five senses

[holophrastic]

Pain, proprioception, thermoception, orientation, direction, acceleration, balance, . . .
Ever seen a cop perform a field-sobriety test? You know, before breathalizers? It specifically tests balance and proprioception -- stand on one leg, eyes closed, and touch your nose. The only way it tests any of your five is because you need to hear the cop speak the directions.

Build a robot, or design any physical machine, and see how many "sensors" you need before it can do anything. Your laser printer has a dozen sensors just to align the paper!

In my world, possibly the same as yours, "grade-school" is indeed hyphenated; in part because it is a unified term. In "grade school", "grade" is an adjective where "school" is a noun. In such syntax, "grade" modifies "school". I was not referring to a school on a hill, or to school which teaches about grading. Now, I was talking about a school which is gradual, and hence "grade school" would have worked, I was not referring to the entire academic system in which education is taught gradually. I was referring to the subset of years consisting of grades 1 through 6(ish), commonly coined "grade-school".

Similarly, I could have used "elementary-school", however "elementary school" would have been a school that teaches the periodic table, or the basic elements of some other industry.

See, "adjective noun" is a general form of English, where each word is considered according to its individual definition. "adjective-noun" is a specific form of English, where the compound-word (or "compounded word", because "compound" is a noun, and "compounded" is the adjective here) is considered according to a non-Englist lexicon, often industry-specific jargon.

They taught me your way in grade-school, when they told me that I was in grade school. That's my point. My education continued beyond grade-school where I learned that I had not only attended grade school but I had also attended grade-school; and I learned the important distinction.

But I'll ask you the same question I ask of people who argue "whom" versus "who". "How many times have you said the word 'whom' in the last year?" Many of them realize that they've never used it, and that's when they realize that they must be making some mistake. So in your case, when was the last time you used a hyphen? If you answer is unreasonable, then clearly you aren't utilizing the entire English language properly.

A lot of people have been dropping hyphens over the last sixty years -- twice my life-time. But hey, people say "there's five of them over there" because they don't seem to realize how ignorant and hick-town it sounds when expanded to "there is five . . .".

Re:I already have more than five senses

[lawaetf1]

Sure.. just like the "components" of my server costs only $5,000. Then there's the $30k database license sitting on it. And the 2A of power it draws, and the $1k/month internet connection.. and my salary.. etc. Her hardware cost reference is to promote the "why don't we have this now?" reaction.

Anyone else notice the instances of him using the device (bookstore, grovery store) were conspicuously dimly lit? Not knocking what is certainly a clever packaging of components in an experimental doodad.. but would you buy a device you couldn't use outside?

Re:I already have more than five senses

[node 3]

I would say we have only 4 senses

And I would say you're too smart for your own good. A sense is an aspect of the physical world you can detect, it's *not* the type of physical phenomena being utilized in the sense. Otherwise, all senses are just chemical (or electro-chemical, if you want).

The sense of up and down is distinct from the sense of rough or smooth, even though both use pressure, just like a radio antenna and a roll of photographic film are distinct even though they both measure electromagnetism.

Re:I already have more than five senses

[Tryle]

I've seen his mom, I would not be watching the demo. Your logic is flawed.

Re:I already have more than five senses

[Curtman]

All this thing does is look up information on something presented to it. The iPhone can be used to look up the same information, with the only disadvantage being that you have to manually search for it.

Yeah the iPhone recognizes whatever object/person you put in front of it and displays information about it directly on it. I must have missed that one in the app store.

Re:I already have more than five senses

[arkhan_jg]

8-13. The different Taste Receptors

Now you're just being picky. Why not separate the cone cells into S,M and L, or split the sense of smell down into the thousands of distinct odours we can detect?

Besides, you're missing the point. The 5 senses description is part of the lies-to-children approach to teaching. You teach the basics by giving them a simplification or outright falsehood that conveys the meaning well enough - and that lie is good enough for that purpose, even for an adult layman.

Take physics, with newton's laws of motion compared to special relativity. Newton is good enough for general purposes, even if it is wrong with a more detailed understanding. Or the Bohr model of the atom; I remember when I did my A-level in chemistry, and the first thing my teacher told me was everything I'd been told about the atom up to that point was wrong. The same thing happened at university.

It's all part of specialization; the mechanical engineer looks at the human senses, and compares them to their human-built equivalent sensors, which is an entirely different approach to that of a biochemist.

I guess

[oodaloop]

this sixth sense doesn't help to identify dupes, since this, or something very much like this, was just on /. a few weeks ago.

Overlay on my glass instead...

[sam0737]

I hope I can get the information overlay'ed on my glass instead of projecting out. First it should get better contrast, second I don't need to display what I am looking to the public.

Put the calibration aside, I would need to start wearing glass...Or should we get the video overlay signal injected into the brain?

Re:Overlay on my glass instead...

[nine-times]

Yeah, that was my reaction. It's one thing if all the information being displayed would be a standard default, but as soon as you customize what data to show you, you're already displaying private information about yourself whenever using this. It'd be much better if it was displayed in a format that was private, that only you could see.

Beyond that, if it's something mounted in your glasses, it seems like it opens the potential (perhaps) to track eye movement and therefore guess at what you're looking at. That might open the door to have it make more intelligent guesses as to what kind of information you're looking for, instead of just displaying information about whatever happens to be in front of you.

But I gather from the video that this was all just supposed to be a starting point or proof of concept rather than an actual product. Maybe given an investment, building it into glasses would be more feasible.

Wearable Display?

[GerardAtJob]

The use of a retinal display could complement this thing nicely... but since microvision have all and every patent on this... and only create stuff for military purpose, we won't see anything like this soon...

You would think that

[One Brave Prune]

with all that technology the girl at the end would have a calculator watch.

Augmented senses: SID

[macraig]

I already know how to augment a person's senses: it's called SID (Sensory Integration Disorder). Anyone with SID is automatically the proverbial canary in the coal mine. Doesn't require any awkward paraphernalia, either, just a few rearranged genes! You probably already know one of these SID people, like the guy who screams at the neighborhood kids to stop that infernal racket!

Re:I have a sixth sense.

[retech]

Pity KDawson doesn't have that ability.

Re:Some of this is reminiscent

[Locke2005]

You can already do this in a limited fashion with Google Maps on the Google phone. Using the built-in compass and tilt sensors, it can display a Street View image that changes as you pan the phone to correspond with the direction the display is pointed. Which opens up the possibility for someone to write an app for the Google phone that takes the camera image in real time and displays it on the LCD with superimposed 3D virtual modifications that everybody running the app could then see at the same location. Just imagine the multiplayer games you could play with that! (Ignoring, of course, the safety implications of a bunch of people navigating busy streets by looking through their cell phones.) As far as I know, this should be doable on the iPhone as well (and possibly other devices).

Re:How the Hell is this a sense, or even useful?

[carlmenezes]

A lot of technology is the combination of existing pieces. What makes it useful is how the pieces are combined. Are search engines useful for that matter - they're essentially queries running on databases right?
Besides, it's a demo of a work in progress. You could have come up with this - but did you? Did you even think about it and envision it as completely as it was presented in the video, let alone implement it?
This is a work of genius. Please do not belittle it.

What could possibly go wrong?

[Locke2005]

Imagine all the great opportunities for gaslighting people you don't like you could create by hacking into this device while somebody else is wearing it!

Solution in search of a problem

[Locke2005]

When is the MIT Media Lab going to start working on something that is actually USEFUL to the common person? Say something in the field of teledildonics, for example.

Surprising?

[Alrescha]

"a host of surprising new applications becomes possible"

Surprising? New?

No. Please read some Vernor Vinge. To stay on topic, I recommend "Fast Times at Fairmont High", which covers the concept of augmented reality quite well. Someone wake me when technology catches up to that.

That said, I think it's wonderful that someone is working on it.

A.

Someone Please Tell Me

[DynaSoar]

Just what really revolutionary devices have been developed and put into common use by MIT Media Lab? I see a lot of hype from them, and it's getting less and less realistic and more obviously pie-in-the-sky. Science in the popular media only requires this condition and that's where Media Lab seems to live now. Real applications require more. What concretely have they done, previously and lately?

If they're stuck in theory mode, so be it. But then they should present their theories as such, not as super duper gaming gizmos on the verge of revolutionizing everyday life.

I'm still waiting for my jeans with the embedded keyboard they "demo'd" a decade or more ago.

Where are the intention gestures?

[izeskepticles]

The tough part in a "real world environment" is understanding the users intentions. They left out all of the hand gestures you would need to explain your intentions to the computer. Which makes this far more complex to interact with than shown here.

We only have five senses?

[MartinSchou]

Hmm ...

Taste, smell, vision, hearing, touch, balance, temperature, spatial.

I suspect I'm leaving several more out, but which ones am I supposed to ditch?

Comment removed

[account_deleted]

Comment removed based on user account deletion

"Super hearing" happens

[dbIII]

With just a crack in the skull in the wrong place you too could have super hearing. It's apparently also very difficult to fix because there are a lot of important bits of brain right next to it.

Don't forget the noise generator

[inKubus]

Also there's the noise generator, also known as the creative mind. If you count senses as basically a signal generator (IE: eyes generate a signal based on the light levels, ears generate a signal, etc.) then the creative mind is definitely one as well. Too many people think of the senses as the sense itself plus the filtering system of the brain (and the recording and cataloging sections as well). Looking at them as a package prevents one from seeing that the filtering, recording and cataloging systems work independently of the actual senses. In reality, your entire nervous system is one sense, with various adapters (eyes, ears, etc) connected to it. But because of the massive capabilities of the adaptors, other, unseen capabilities are often ignored. Like all systems that carry signals, the nervous system is subject to noise, interference, and other unintended signals. The filtering and cataloging systems in the brain can be put to work on this noise, and literally create something (a new thought) out of nothing (noise). For lack of a better way to label it, this is the creative mind, where inspiration comes from. Likewise, the processes can also be used to sense characteristics of the adapters that may not be their primary purpose. The ear, for instance, is designed to sense air pressure, but it can also help with balance because the air pressure adapter is also affected by gravity. I think a lot of the brain/sense structure is misunderstood simply because we go our entire lives only believing there are 5 senses and not seeing the true nature of ourselves. I think that is where you were going with your comment, just wanted to finish the thought.