Slashdot Mirror
The Nonphotorealistic Camera
An anonymous reader writes "This article on Photo.Net describes a new type of imaging technique that finds depth discontinuities in real-world scenes with multiple flashes added to ordinary digital cameras. As depth discontinuities correspond to real 3D object boundaries, the resulting images look like line drawings. The same technique was used at this year's SIGGRAPH to create a live A-ha 'Take On Me' demo."
It's interesting to see that people finally wanted to try to obtain from their hardware what they'd usually expect Photoshop filters to do.
I am for example very happy with my Motorola v550 cell phone camera which takes the trashiest but also most colorful nunrealistic photos.
Trolling using another account since 2005.
A live "Take On Me" video?
:P
People always ask how we'll know when technology will go too far, and I think we've just found out.
Is here
has slowed to a crawl, Here's the cache.
You are mistaken. The final image is the 'equivalent' photoshop filter, and it's showing what a poor job it does in contrast to the third image, a good image, from the multiflash.
Try reading the article, the last 3 images use different techniques and are for comparison. Jeez
I wonder if this technology could be extended to allow one to quickly take a picture of a real world object and turn it into 3D models (for use in 3D Studio etc.). Obviously one would have to take multiple pictures (six?) to get a proper all round representation of the object.
Just a thought.
How about having a camcorder with several differently coloured light sources? By analyzing the correspondingly differently coloured shadows one could create depth information in real time.
Add this to moving around a room while filming it. It should be possible to create an accurate 3D-representation even with today's technology.
If the colours of the light sources we're properly matched any discoloration could probably be eliminated as well.
Food for thought.
.: Max Romantschuk
This could be very useful when you need to postprocess an image - like apply a segmentation algorithm.
Several segmentation algorithms exists. Ususally, they look at the color/brightness of an area and uses that to do the segmentation. Adding knowledge of spatial position to an image will help segmentation immensly. I'm not sure that 3 small flashes is enough. The examples provided are not exceptional - the same results could be obtained without that special camera. Nevertheless, the idea is good.
Underholdning.info
I know it ain't really a 3d mapper but is it a quick way to grab info that could be later given a more in-depth scan?
Could this technology be modified to produce a good 3d mapper?
What's it's claim 2 fame? Shadow-comparison , right? Length-of-shadow=height-of-object, yes?
You see, edge detection is funny.
Real world discontinuities, what they mean is, you run an edge detection algorithm on the distance signal.
This will not find edges in newspaper print.
No edge detection system is perfect - even this which uses spatial edges.
There is no real new technology, the multiple flash cameras are amazing and beat any faked edge detection hands down.
I do think they have awesome capabilities to allow computers to do what our eyes do, which is segment and label areas of our vision, and may make the blighters useful!
Computer aided sensory vision [audio descriptive alerts] could do amazing things for blind people, if we get there before they make artificial eyes better than 16x16 resolution.
#hostfile 0.0.0.0 primidi.com 0.0.0.0 www.primidi.com 0.0.0.0 radio.weblogs.com
This technique sounds like it could be useful for 3d reconstruction problems. The main issue in, for example shape from stereo algorithms is accurately finding depth discontinuities, and it can be nigh on impossible with a textureless, evenly lit surface.
Having said that, I'm not sure whether it would be better than existing solutions for that sort of thing, for example structured light.
Disclaimer: I am not a expert on graphics technology.
But look at the second image in the final set, it's clearly able to detect the edges of things. I'm not even sure what the filter in the last image is for.
And I'm not sure what you mean by "reproducing what can already bt produced". There are other multiple-image processing engines that can do line drawings and even 3d from multiple sources, but the thing is, they all require multiple cameras and calculating the slight offset in objects from different sources.
What's interesting about this new technique is that it uses the shadows from the flashes to determine edges and depth. Doing it entirely with lighting without multiple cameras is a really neat hack, imho.
First off, as the 2 posts above pointed out, the composite image is one in the top right,not bottom right. I would say the top one is useful but your free to feel however you want.
However,if you read carefully you see that the nifty aspect is that it gives depth information to images, even monochrome images. Just to start with this has applications for internal medicine (i.e. laparoscopy). This is cool.
What do you say to the man that has nothing? Cast it away!!
Oh, no. Will traced overlay pictures return for manuals if this is faster then optimizing the original CAD design for a consumer? I hope not, manuals can be pretty clear these days, I have no desire to return to those traced illustrations of the past even if it can be done automatically instead of by hand. Taking the time to optimize the original CAD technical drawings of a product in Freehand gives much better results.
The technique can be great, reading stone tablets with a flash dome is so much easier then trying to look at the real surface, but for most (consumer) uses two lenses will give more opportunities (to remove the background, to see "3D".
The depth edge maps bear a superficial resemblance to phase congruency maps. It's the best edge detection method I've come across, and works on ordinary 2D images. Check out some examples on Peter Kovesi's pages, there's also some code for download.
Escher was the first MC and Giger invented the HR department.
Could this tech be used to help robots, or any computer really better understand it's evironment visually? As I understand it one of the problems facing robot optics is the lack of depth perception and identifying object bounderies, if they used optics in the nonvisable spectrum and basically walked around with they're flashes strobing happily along would that help these problems? The only problem I see with that is multiple robots flashes interfering with each other, so maybe it's only be used sparingly when absolutely needed? Or is this technology completely inappropriate for this application?
I can't believe no one's mentioned that this would probably be pretty good for robot image processing.
With four flashes, the first thing you better do before any fancy schmancy edge detection algorithm is run the red-eye removal filter!
Don't encourage him, it's a bad habit.
Jolyon
Please read my Canon EOS tech blog at http://www.everyothershot.com
This would be great for technical manual writing. Help you take pictures of the mechanicla interfaces.
Evolution or ID?
Thanks for getting that song stuck in my head. Thank you very much.
Smoke me a kipper, I'll be back for breakfast.
3D analysis requires stereo-pair of images, like this. Alternative would be to use some kind of radar or sonar, measuring time-differences of bounced signals, etc. Those and other methods for 3d digitizing.
Preserve old classics: copy your collection onto all hard drives.
I wonder if such a technogology could be used for biometric facial recognition. Since the lightsources are internal, it would be relatively simple get consistent refrence points from it.
Also, it would not be *AS* processor intensive, so you could take more photos from more angles.
Using autofocus, and a short depth of focus, you isolate figures even in crouds. Isolate the target from multiple photos, so you have more than one agle for a biometric.
If we can track the target in motion, we can assume that FRONT is aproxomately the direction they are traveling. Use and IR flash so that people don't get all paranoid (not saying they don't have a reason).
Even with glasses and a beard change it would be tough to fool the system.
I would rather be ashes than dust!
A nonphotorealistic camera for Lexmark's entire line of nonphotorealistic printers!!
"Capital punishment makes the state into a murderer. Imprisonment makes the state into a gay dungeon-master"
In one of my to-do notebooks I've got 3 camera setup sketched out and some maths. This will get you radial depth on a moving object as the shots are taken all at once. Also handy if you're moving! It's not a new idea. Stereo cameras and viewers were used for remote sensing from the early days of the camera. I used to look at aerial 2-D photos of national forests in my dads office back in the early 60's.
You can do the same thing with a 'normal' digital camera if you are taking digital photos while moving and keep track of the relative position of the camera and subject
Now I'm the grandest Tiger in the Jungle!
I think this is a quantum level above the Photoshop filters
So, you mean, this is the tiniest possible improvement over Photoshop filters?
Could b a useful technique for robot vision perhaps? It may improve edge detection for that too and reduce the CPU cycle count necessary to calculate 3D spatial representation.
At last, there's finally some technical justification for the sequential flashing lights on the hood of Knight Rider's car. It must have been part of Kit's vision system.
For those that are uneducated in graphics, the engine photos show two comparative methods:
The TOP row shows how the camera output is good enough to be used as a technical drawing- it requires very little modification or touch-up.
The BOTTOM row shows how a Photoshop filters butcher the image and the result is completely useless. No amount of touch up could help that image.
Furthermore... NO THIS CAMERA CANNOT BE USED ON MONOCHROME IMAGES. It can't be used on any kind of images, and it isn't a post-filter. There isn't any edge detection involved.
The 4 flashes cause shadows to be cast in 4 different directions and creates a composite from the difference. If the subject DOESN'T cast a shadow, then the camera won't work.
I assume this camera cannot be used to photograph the outdoor scenes, simply because the flashes will not render shadows at that great distance.
This is an brilliant method though, and the results are excellent (look at how the details in the spine pop out).
"Capitol punishment"? You know, some of them do deserve it...
I don't have a sig.
Speaking of non-photorealistic and real-time, this reminds of me NPR Quake.
Dan East
Better known as 318230.
As the article contains a picture of a girl, there will be untold numbers of comments about that girl, as this is /. So, as a convenience, please put them up as replies to this post, to save everyone else the redundant reading. Thanks.
Care about electronic freedom? Consider donating to the EFF!
All these years, people were trying to make cameras more photorealistic, when, what they really should have been doing is making cameras non-photorealistic. I guess my 2-megapixel camera is lightyears beyond all those new fangled cameras ou see in the store.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
Or for Dijjer users, click here.
Down she goes...
Imaging techniques that control the illumination are not new. This is a nice and simple application of them, though.
"I'm just here to regulate funkiness."
This technology is a long way from 3-D. First, this camera can only estimate relative depth not absolute depth. Thus, it might determine that the foreground object is half the distance to the camera as the background object, but have no estimate of the numerical distance of either object - the foreground could be 3 feet from the camera and the background would be 6 feet or the foreground could be 5 feet from the camera and the background would be at 10 feet.
Furthermore, this technology only sees edge discontinuities where a foreground object sits in front of a background object. Thus it cannot tell the difference between a circular disk in the foreground or a sphere in the foreground. Actually it is worse than that because the rounded edge of the sphere will cause errors in the estimation of the relative depth of the sphere vs. the background.
Even with these limitations, the technology could be quite useful in robotics. Combining multiple edge images using optical flow and knowledge of the robot's motion would yield a more accurate 3-D depth map at least for the purposes of navigation.
As for extending the technology, a second camera would do wonders for pinning down the distances to each observed edge. The system would still need separate software magic for mapping the front surfaces of objects (e.g. discerning the difference between a 3-D sphere and a 2-D disk).
Two wrongs don't make a right, but three lefts do.
Go here for the mirrordot version of the article. Its nice to know someone solved the slashdot effect for slashdot. dot dot dot Impressive they cache the images too.
Had to mention this for those who didn't catch it in 2001. Some students in Wisconsin created a Quake II mod that converts the Open GL rendering engine output to non-photorealistic sketches. Looks like the A-ha video in realtime. I'd really like to see someone bring this to more modern 1st-person-shooters like Doom 3 or Quake 3.
NPR Quake.
$5 / month hosted VPS on linux = awesome!
Funny that you should mention A-ha, because "ah ha!" is pretty much what I said to myself when I read TFA. The offset flash on most cameras is usually viewed as a liability that screws up your photos, but these guys have turned around and taken advantage of the effect.
Simple idea, well executed. Ah ha!
It bothers me a lot that stereo photography has been around so long yet isn't ubiquitous yet. Modern digi-cams don't do this. You said it's been around for ages, I hope most people know you mean more than decades. A quick google search tells me 1839 at the latest. What is stopping it?
Putting 2 sensors on a digi cam (photo or video) is not a difficult trick. You store the images in a format that supports 2 channels (left/right) and you can view them on any monitor with a simple pair of USB controlled glasses that flicks back and forth blacking out each eye. Also there are already 3d monitors out there that work without glasses.
Print out one channel for a 2d image or use photoshop filters to create red/blue 3d prints. Or even send images to a printer and get back those wheels used in those orange stereoscope toys.
If I had this ALL my pictures would be 3d. For that matter all movies should be 3D. IMAX has a workable solution but I think every movie should be shown this way. People would even buy their own personal polarized glasses that are more comfortable than the pairs handed out at the show.
I've been eyeing a digital-SLR for quite some time, for the cost of one of those I'd gladly turn my attention to a 3D capable camera with lower quality. And if the grandparent post is right something similar should be possible for SLR cameras without using 2 huge lenses. Although I'd submit that you can't always control the lighting.
Every now and then a red/blue 3D image comes up on APOD or elsewhere and I kick myself for not having a cheap pair red/blue glasses.
Being a tech writer, I've spent hours and hours fooling with photos (35mm 20+ years ago, 35mm scanned 10+ years ago, 35mm printed on clear overlay and taped to screen for tracing, digital, Photoshop, Photoshop filters, etc.) In more than 20 cases that I can quickly recall off the top of my head, this process would have solved 90% of my problems in depicting and pointing out some issue that needed a clear unambiguous illustration or diagram. I think that my friends in the medical illustration field (who regularly are given photos of bloody surgeries to "clean up" and illustrate for medical reports and journals) would also find this incredibly helpful.
FYI, they're called "View-Master," and apparently they're no longer available in the vertical-wheel red/orange style I had as a kid.
Or you could hire an underemployed illustrator. :(
http://alternatives.rzero.com/
This effect reminds me of John Bolton's artwork in Niel Gaiman's Harlequin Valentine.
-Peter
One of them is Canesta that makes a photo sensors that can make pictures that include deep maps.
To my surprise I see that they are running a contest were your can win 10000 $.
But I don't have time to participate myself, because I am writing on my masters. So enjoy the contest.
I can see these non-realistic photos replacing our passport photos very soon.
It makes me wonder what the CAMERA looks like. I'm picturingh a sort of X shape with flashes sticking out about a foot from the camera. Kind of unwieldly. Obviously, the further out you move the flashes from the camera, the more pronounced the shadow, but I wonder what the effective distance of this is.
-- Having a Creationist Museum is like having an Atheist place of worship
If autofocus (or any other method) from differnet angles allows for this enhancement, this technique can be used to 'cut' the image into different focus layers.
Piece the layers together, and you get a photo that has depth of field and is much sharper at each level.
The layer information could be stored seperately for later processing or combined with only a little fudging to give a weighted blur to the non-primary layer(s). Keeping the layers seperate and doing a comparison would also allow editing tricks such as cutting out objects at a specific depth or performing color enhancements on each level.
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
> Just to start with this has applications for internal medicine
Opening the patient to insert the flash lights ?
The first step to achieving Artificial Intelligence is good robotic vision rendering real life into 3d.
God spoke to me.
What struck me is how this is reminscent of human vision. It helps explain the depth perception that I do have, despite my lack of stereoscopic vision. It makes me wonder if we already have this algorithm built into our visual processing. It seems so useful it would make me wonder if we don't have this algorithm in us, why not? There certainly is evidence for filter elements in our visual processing that provide the right kind of spatial processing for it. This could be a great leap for revaling the mysteries of human vision and the brain.
Target acquistion would be an immediate application. .
Lots of ranging euipment exists , but sorting targets is the hard part.
If the imaging can be done almost as well with 3 cameras on vertices of an equilateral triangle then a minimun spatial discrimination platform would be the 4 vertices of a 4 sided pyramid
A deployment of 4 of these platforms ( coordinated )in a pyramid formation would yield a minimal system : with a robust well imaged internal space and a "watching" your back external space.
I'm not in favor of it, but military funding has always driven technology as far back in history as you care to go.
A vertices centric 4 sided pyramid is also probably your best minimal military 3d formation for the projection of fire power in space and the mutual protection of combatants Has this ever been war gamed/
http://www.3dvsystems.com/index.html
This technique makes outlines for real-life objects, thus providing the an inportant step for turning real-life objects into cartoons.
Contribute to civilization: ari.aynrand.org/donate
Target acquistion would be an immediate application
I'm not so sure about this because of the very limited range of this scheme. It relies on illuminating the entire scene four times for each estimation cycle which implies that it will not work very well at ranges of more than 10-20 feet (without extremely bright flashes) or work well in broad daylight (without extremely bright flashes). Finally, the system only works where there is a background for the foreground object shadows. The power required for the flash increases with range squared (range to background, not range to foreground) and ambient illumination so long range is is not feasible
Perhaps for robotic navigation in enclosed spaces using an IR flash (so as not to to be blindly visible) might work. Of course, I won't venture to guess whether this system can work if the ambient illumination is time-varying (e.g., fluorescent), but that's a whole other problem
If the spatial edge detection were saved (highly compressed) as a companion image, and there were a way to describe this feature in EXIF, then manipulation of the captured image in digital cameras would be that much easier.
Imagine: You take a picture of someone/something, and want to cut out just the subject... like chromakey, but with an arbitrary background... the camera saves the spatial boundaries, and all you'd need to do is indicate which "blob" you're interested in preserving.
That would be sweet.
However, for deep-field images where the shadows may not show up (too far away), this is still a little lacking.
My guess is that using 3 CCDs, arranged in a triangle, and figuring out based on image correlation where the foreground/background borders are located, would be better. This would also remove the need for specially timed, artificial light sources.
My degree was in Technical Illustration. I knew then, (15 years ago), that it was inevitable that computers were going to make the field obsolete. (which this technology appears to *finally* achieve to a large degree). So I got into computers.
However, I *do* believe that this statement:
Additionally, an endoscopic camera enhanced with the multi-flash technology promises to enhance internal anatomical visualization for researchers and medical doctors.
Won't work out. This flash technology relies on the parellax effect from having the four flashes originate from spacially different locations. That's not really possible with an endoscope. Unless you want to stretch your patient's rectum until it resembles Mr. Goatse's.
These are my friends, See how they glisten. See this one shine, how he smiles in the light.
makes the parent a troll? Stating that you are unimpressed - albeit in a rather trite way - is not trolling. Bad moderators...
I've never used a newsgroup aren't they all porn?
Wait... 3D-PORN!?
There's a picture of the camera at the top of the article.
But from the looks of it, this might be useful for doing something of a cel shaded/anime tyrpe movie without the animation part, just do it live action, but apply the tech for the cartoony look.
Get me a meat pie floater!
and you should have what you need. Two cameras for stereo, the range finder for distance and scale and something to crunch all the numbers.
That laser method sounds really interesting, but I'm doubtfull. I'm too lazy to calculate how long it takes light to move, say one milimeter, but your talking picoseconds... Can you really synchronize at that level? If the object was static you could send out multiple pulses, so wouldn't need to take multiple pictures, but it would still need to be super-precisely synchronized.
Oh. Wow. Small picture. I completely missed it. I'd imagine you need the flashes further apart the farther you get away from the subject material.
-- Having a Creationist Museum is like having an Atheist place of worship
Wouldn't it be better to use multiple lenses rather than multiple flashes? It would be similarly simple, yet provide much better data on relative position.
That's some pretty impressive edge detection, thanks for pointing it out. A related problem is to identify areas in an image that are the same thing. The best algorithm I know of for segmenting images is Leo Grady's brand new iso parametric graph partitioning method. His work is at http://cns.bu.edu/~lgrady/. His PhD thesis is probably the best place to start.
Give that man $1,000,000.
This makes me wonder how expensive it would be to simply add a secondary lens/CCD to the side of the primary lens to take a second picture that will provide parallax information for depth. The secondary CCD could be smaller and cheaper because the resolution doesn't need to be as high. You may even be able to make it even cheaper by using a black-and-white CCD and comparing only the luminance values. I suppose this would only work well with a camera that has no zoom lens. Maybe you could just tape two digital cameras together and make sure you push both buttons really fast. ;-)
In surveillance applications, replace those four flashes with infrared illuminators (to avoid desturbing the subject) and you just might find some serious uses for this technology. - Video-based object counter (used for counting people in malls, cars in parking places and more) that will never detect two people/cars/products as one. - Motion detection that ignores computer monitors and TV's. - [Serious privacy issues ahead] Facial recognition using surveillance cameras might get a lot easier if you could get the outlines of the whole face to start with.
The reason stereo photography hasn't caught on is that it requires special viewing aparatus, whether it be a Viewmaster, special glasses (red/blue, polarized, or shutter) or a special monitor. As far as equipment, check out this site.
"I'm not impatient. I just hate waiting." - My Dad
It seems like this could be a really cheap way of giving robots good depth and spatial perception. It would be easy to simply measure the width of the lines vs. time. Certainly for less adventurous tasks like Roomba-ing or accident avoidance in cars this could have some interesting applications.
If it allows depth perception to some degree, it can be truly useful for the blind's artificial vision (as in the brainports article).
Even at a 16X16 image would avoid bumping into white chairs next to a white wall, something other vision systems don't do yet.
Now if we could avoid annoying the seeing with constant quadruple flashing (perhaps using infrared light), we'd have something!!
Microsoft is pure dog-ma. FreeBSD is pure cat-ma.
In terms of computing, I expect this to be pretty cheap (at least from their quick description). The hard part is that 4 flash are going to eat your batteries like no tomorrow, and if you're trying to do snapshots, you want to grab the 4 images as quickly as possible.
The edge value for any given pixel is essentially max(4_pixels)-min(4_pixels). If you add a second buffer to the camera with 5 layers (R,G,B,Min,Max) you can do the computation as you read from the buffer, If you add 2 bits to the R,G,B layers then you can compute an average without overflow. with a 600megabyte/second memory bandwidth, you should be able to take and calculate a 4megapixel image in under 1/30 of a second, and a 1 megapixel image in about 1/125.
The limiting factor, at that point, becomes the speed of your imaging chip.
Free Software: Like love, it grows best when given away.