Slashdot Mirror
Program Allows Ordinary Digital Camera To See Around Corners (theguardian.com)
An anonymous reader quotes a report from The Guardian: Science may never tell us what lies round the next corner, but researchers have come up with the nearest thing: a computer program that turns a normal digital camera into a periscope. In a demonstration of "computational periscopy" a U.S. team at Boston University showed they could see details of objects hidden from view by analyzing shadows they cast on a nearby wall. Vivek Goyal, an electrical engineer at the university, said that while the work had clear implications for surveillance he hoped it would lead to robots that could navigate better and boost the safety of driverless cars.
In the latest feat, Goyal and his team used a standard digital camera and a mid-range laptop. The researchers, writing in the journal Nature, describe how they pieced together hidden scenes by pointing the digital camera at the vague shadows they cast on a nearby wall. If the wall had been a mirror the task would have been easy, but a matt wall scatters light in all directions, so the reflected image is nothing but a blur. They found that when an object blocked part of the hidden scene, their algorithms could use the combination of light and shade at different points on the wall to reconstruct what lay round the corner. In tests, the program pieced together hidden images of video game characters -- including details such as their eyes and mouths -- along with colored strips and the letters "BU." The program takes about 48 seconds to work out a hidden scene from a digital image, but the researchers believe it could be sped up with a faster computer. Eventually, it may be fast enough to run on video footage.
Goyal also said "it is even conceivable for humans to be able to learn to see around corners with their own eyes; it does not require anything superhuman."
In the latest feat, Goyal and his team used a standard digital camera and a mid-range laptop. The researchers, writing in the journal Nature, describe how they pieced together hidden scenes by pointing the digital camera at the vague shadows they cast on a nearby wall. If the wall had been a mirror the task would have been easy, but a matt wall scatters light in all directions, so the reflected image is nothing but a blur. They found that when an object blocked part of the hidden scene, their algorithms could use the combination of light and shade at different points on the wall to reconstruct what lay round the corner. In tests, the program pieced together hidden images of video game characters -- including details such as their eyes and mouths -- along with colored strips and the letters "BU." The program takes about 48 seconds to work out a hidden scene from a digital image, but the researchers believe it could be sped up with a faster computer. Eventually, it may be fast enough to run on video footage.
Goyal also said "it is even conceivable for humans to be able to learn to see around corners with their own eyes; it does not require anything superhuman."
We can already do this most times. I do it all the time to determine if someone or something is coming around a blind corner. It's just called paying the fuck attention, a skill people seem to not care too much about these days.
Good technology for unexpected turns. No worse than a zoom lens
Is that a man carrying a cucumber?
They could just download an FPS hack and use that.
Goyal also said "it is even conceivable for humans to be able to learn to see around corners with their own eyes; it does not require anything superhuman."
I'd think this was more of ASSUMING or taking a guesstimate of what was around the corner. Actually seeing would be superhuman. Still a cool idea if we'd have something that could do this in real time at a fast pace. I'd assume it'll be a bit more accurate than what we'd guess, especially if we were excited/worried/scared/etc.
researchers believe it could be sped up with a faster computer.
Ima gonna git me one of dem "researcher" jobs..
How does it account for differing light sources? Depending on where the light is coming from an object can cast all kinds of shadows.
Wanna buy a shirt?
https://www.redbubble.com/people/stealthfinger/shop?asc=u
If we could learn to make sense of fuzzy images, there would be no need for glasses.
Science may never tell us what lies round the next corner...
But it already has, here, on slashdot.
What, did they recycle DECs Alpha processors for this application to warrant a "digital" logo on this story? Sheesh, kids these days...
You just face the camera at 90 degrees on the selfie stick. So simple! amazing no one thought of this before.
Some drink at the fountain of knowledge. Others just gargle.
This could never be done with a mirror.
Where in a scene with a camera and a projector, the scene can be from the point of view of either the camera or projector.
Pretend each pixel is a projector that is projecting an image of what's in front of it (they're casting shadows, so it's a negative image). The color of the pixel is the color around the shape it's projecting, but not the color of the shape. (Think of putting you finger in front of a flashlight.) For each pixel you'll get a slightly different image (shadow) of the same object since they're projecting from different angles (put your finger in front of a bundle of 3 flashlights and you'll see threeish shadows of your finger). Do a little math to figure out what the shape is (your finger). Now that you know the shape, find the centers of all the projected shapes. Now do some geometry to go from the centers back to the original object and then back to the original points of light. Now you know the location of that shape's light source and its color. Repeat that for each copy of the shape you can find. Using each recalculated pixel you can reconstruct the original lighted image. The more shapes you find, the higher the resolution you'll get.
It's actually really obvious when you think of it. I don't know if this is something new or sometime that was in research papers 60 years ago and is simply being rediscovered since we have cheaper hardware to do it.
They claim humans can learn to do this. We already do it, just not with enough resolution to be able to draw a detailed picture of what is there. A computer can do this far better than a human.
I like how we make the impossible thing a possible thing by simple redefining what the impossible thing is.
In this case, we define "seeing around a corner" as making judgements, with high margins of error, about shadows on a wall; in the case where the conditions are right for the shadows to exist. There could be a huge hole or a tall brick wall in the road, around the corner, and this method would not show that. You are not "seeing" anything.
Enhance, stop. Move in, stop. Pull out, track right, stop. Center in, pull back. Stop. Track 45 right. Stop. Center and stop. Enhance 34 to 36. Pan right and pull back. Stop. Enhance 34 to 46. Pull back. Wait a minute, go right, stop. Enhance 57 to 19. Track 45 left. Stop. Enhance 15 to 23. Give me a hard copy right there.
As you may have learned from the recent lunar eclipse, a shadow is composed of a penumbra and umbra. The light is completely occluded in the umbra, partially occluded in the penumbra. So if you consider any single point source of light from the hidden scene, certain parts of the wall receive light from it, other parts do not. The algorithm then works from that to back out the original scene (light sources). "Accounting for differing light sources" is exactly what it's doing to figure out the original scene.
Where your comment is relevant is that the object creating a shadow can be of arbitrary shape. In this case it was a fixed and known shape, which simplified computing the image. I dunno if this technique would work with an arbitrary scene and an arbitrary-shaped object blocking the light. If the object were close enough to cast an umbra, you could deduce its shape and refine the image over multiple iterations (basically guess small changes in the shadow object's shape to see if it results in the algorithm yielding a sharper picture). But if you're only getting a penumbral shadow, I'm not sure if there's an algorithmic solution that'll work for all scenes and all blocking shapes.
The program takes about 48 seconds to work out a hidden scene from a digital image, but the researchers believe it could be sped up with a faster computer.
Yes a faster computer "could" speed up the process. Great reporting there!
Thats like saying a faster car could get me to work sooner. It might, but I suspect the cars in front of me and all the lights would prevent me from fully utilizing the speed of any car. However, re-evaluating the route I take, or using a bicycle might utilize the current car I have much better than getting a faster one.
Whenever writing something critical, first make it work. Then make it so that anyone can understand it in detail. Then profile and find the bottlenecks. Finally remove or mitigate those bottlenecks as much as possible so that it still works and can be understood. A faster CPU should be the last resort.
Enhance. Enhance. Enhance.
MATTE YOU MORONS
Just like on CSI, when they find a glint of light reflecting off a lamp, magnify, and enhance the image so they can figure out who the killer was. Easy peasy!
As the scientist said, it is conceivable to see around corners, in a sense.
Soon we may see as part of a soldier's kit: a headcam connected to his Android Digital Buddy app.