Slashdot Mirror
Cheap 3D Computer Vision?
InspectorPraline writes "According to this article at the New York Times [free reg req'd], a tech firm known as Tyzx is developing optics technology that will have three-dimensional capability -- using two cameras attached by a high-bandwidth connection to a custom processing card inside a PC. The article makes one believe that the system would have a top speed of as much as 132 stereo frames per second, which could be very useful in security systems. Of course, the real question is who's behind the cameras, but we can all drool over the other possibilities, right?"
No more taping the red and blue filters from my Mag-Lite to my eyelids any more! :-)
See the company's website for better details on the used technology, here are some interesting publications, this one (PDF) is the core: Real-time Stereo Vision for Real-world Object Tracking.
can real 3d be obtained with just two cameras?
or is it merely 2.5d
Regardless of where the cameras are, is there not still a plane which the cameras/software cant determine the "height" of
Dont you need 3 cameras minimum for proper 3d?
It will be virtually impossible to palm chips or any other sleight (spelling?) of hand tricks that people do at card tables. I'm sure there's millions of other more interesting possibilities, but that, and stopping pickpockets, are the ones that arrived immediately in my head..
He tried to kill me with a forklift!
This is taken from the document Real-time Stereo Vision for Real-world object tracking:
.... the DeepSea chip may not be able to find a valid match for every pixel in the image. Large unformity lit areas of scene may have pixels of identical intensity; for pixels in such area, no single match can be found. Pixels that correspond to an object that is invisible to one imager but the other also do not have matching pixels.
... Once the matching process is complete, the range of each pixel can be calucated using the horizontal disparity of the matching pixels, the focal lenghts of the lenses and the distance between them. The DeepSea chip designates the range or anormalous pixels as invalid. :)) See also a HP document covering partly the same matter.
<clip>
The DeepSea chip is hardware implementation of the census correspondence algorithm invented by Tyzx staff... The algorithm's key concept is transforming a pixel's numeric absolute intensity value into a bit string that represents the pixel's brightness relative ot it's neighboring pixels. For each pixel, The DeepSea chip examines the pixels surrounding area called a neighborhood. A typical neighborhood is 7x7 pixels centered on the subject pixel. Comparing a subject pixel's intensity to its neighbours, the chip produces a relative intensity map (show in the document, page 8).
</clip>
(typos are mine)
Soon we'll finally be able to properly verify if Britney's boobs are indeed unnaturally big... without getting into a sexual harassment lawsuit.
If I have understood correctly, this is for tracking/sensing movements accurately in 3 dimensions and being able to record them in binary, not for reproduction of images in 3D onto a screen with viewing glasses and all that stuff. Indeed, pretty good 3D technology is available but the pr0n industry relies on the cheapest technology available to make the most money - at least as a general rule this is the case. The current massive pr0n market has been enabled by Internet and digital media, but they ain't going to poney up loads of cash for this kind of technology.
Pr0n is not necessarily for the discerning film critic after all, known rather for hand relief and titilation of couples who like a bit of that. Not for amazing technology and three dimensional shots. Would you pay more for 3D DVD quality pr0n??? DVD works for the pr0n industry due to form factor and ease of quality pausing of the frames, at least that's what I reckon ;-)
Conversion Rate Optimisation French / English consultant
The technology employed (both hardware and software) is limited. CMOS sensors of the type described suffer from poor signal to noise as well as interlacing artifacts. Pixel jitter is of major importance in machine vision and I doubt these sensors offer much clock control over and above the 1 pixel mark (if any).
The matching algorithm described is very primitive, assuming rotation in depth between views doesn't effect the scene projection into the image - ooh but it does. The concensus matching algorithm is very simple and whilst it does recognise the problems of illumination variation it fails to solve the problem in a manner you could describe as robust. Also contrary to popular belief you cannot robustly recover depth from every pixel n the image! There is no evidence that the human vision system does it (without knowledge of the object) so why are people trying it? Even if you ataempt it you are going to need some way of telling which data is more accurate than not in order to start using the results. Edges are your best bet and I didn't see any evidence of preprocessing described in their system (although to be fair I only read it breifly).
I appreciate that this is supposed to be a cheap system and thus its limitations are probably to be expected. Might be fun to play with for a hundred Euros or so.
For more state of the art look at what is possible you could do better than take a look at TINA an open source machine vision system with a very sophisticated stereo depth estimation algorithm (we even built a chip to accelerate it!)
-- "Can't sleep, clowns will eat me!"
I don't know what "inexpensive" means. It's all relative, and no real point of reference is given. If it truly is inexpensive, this could open up a market for lots of new products which track objects in 3D (real) envirnments where it just never made economic sense before.
Product ideas anyone?
-Pete
Soccer Goal Plans
But it is no longer in production and it is patended.
Works with any software as it is attached at the front of the screen. Surface mirrors and the idea of doing the view-master 'on screen'
I'll keep mine for a long time.
A description and pictures of it here
Patent here with description.
Nahh.. easy of quality pausing ? I don't think so. DVD works for the pr0n industry because of all the cheap kids who own a 60$ DVD player and no VHS. Not that there's anything wrong with that, but it does represent a very early technology jump.
Just think back of the early 80's when audio CDs started hitting the market; lame cd players would cost 400$ and up, and the discs themselves were hard to find, but eventually gained popularity and eclipsed 4-track tapes. It took years for the transition to progress, and now if you're seen buying a music tape the clerks will be wondering where you've been living for the last fifteen years. But fifteen years ago if you were purchasing a CD, those same clerks (ok, their parents) were probably wondering where you got all the cash for a cd deck/discman, and pretty much everyone in the street would chat you up about your shiny cd player. Same thing's happening with video, right now we're somewhere in the middle, as DVD is well on its way to widespread acceptance in the home market.
Of course the pr0n industry has little choice but to follow the technology trends. Nowadays everyone wants 4-hour multi-angle ass-to-ass compilations with running commentary by the not-so-great Rocco himself, and since DVD discs are so compact, they can stuff more of them in the bottom dresser drawer underneath their socks.
-Billco, Fnarg.com
The two cameras approach requires relatively high performance. Is there are reason why combination of digital camera and laser based distance meter (accuracy is measured in millimeters) would not be more accurate, reliable and require less computational performance.
:) To me atleast this aproach is also easier to comprehend than some magic algorithm.
Take image, feed the laser distance-o-meter, which scans the distances and embeds the results with the imagedata. We could even have a matrix of the lasers for example to measure the distance on a single shot, for example at 8x8 (64) beams would be already good for scanning an area of few square meters - if the objects that we are looking for area bigger than insects, ofcourse
Close one eye. Can you still estimate the distances of objects around you? Of course you can. This demonstrates that there's much more to depth perception than stereo vision.
Stereo vision is inherently limited. It requires that the objects have sufficient texture so that points on the two stereo images can be correlated. Our depth perception relies on much more than stereo e.g. common sense knowledge about the world, intution about shading and lighting, etc.
but we can all drool over the other possibilities, right?
You mean 3d pr0n?
Riiiight, and those X10 cameras are for surveillance too.
EDISON, is a free C++ toolkit that performs edge detection and image segmentation. The image segmentation portion is based on mean-shift analysis.
A colleague and I are currently in the process of porting portions of EDISON to Java.
There is a company called Point Grey Research
(http://www.ptgrey.com/) that has external binocular and trinocular stereo units for sale that use firewire. They don't do the processing on the unit, but have algorithms that run on standard PCs to process the data for you. Pretty interesting little guys, the computer vision lab where I got my degree (http://cvrr.ucsd.edu) had 3 of the triclops camera systems. They have a new one called the bumblebee that looks to be cheaper and maybe do processing onboard?
There are linux SDKs available also. Note my version of Mozilla (version 1.0) doesn't load their page correctly, maybe some IE messy code?
__ No registration required to read this message. They did it in the Matrix.
No, we are not related.
Kryzx
"I don't know half of you half as well as I should like, and I like less than half of you half as well as you deserve."
No more radar guns for police (now you'll need an invisible car)
Fighter planes that don't need radar (but will need scads of cameras all over it -- both visible, infrared, and tetrawave)
Computerized athletic officiating (which may finally kill the politics of skating and gymnastics)
Better identity recognition software (now you don't have to face the camera)
Custom-tailored clothing (no more scanning mechanisms)
Automated grocery checkout (the machine identifies the fruits & veggies so that the clerk doesn't have to type in a 4-digit produce code)
Another reason for George Lucas to go back and re-film all 6 episodes into digital 3-D.
It's not about displaying the stereo image for a human, it's about a machine being able to interpret imagery and 'understand' what it sees around it - machine vision. Algorithms for automatically determining depth from stereo are usually imperfect and may require several passes through both images.
we can all drool over the other possibilities
It's always the same on Slashdot - somebody will eventually end up talking about p0rn...
I'm disgusted!!!
I'm struggling to think of any other products the technology could be useful for.
:)
There's always the universal correct answer to that kind of question - porn
Triple-D cups in 3-D
-
- - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.
Basically, the computer needs to find the matching objects in each image and how much they're displaced, then compute the distance map based on the displacement map. The second part is easy, but the first part is challenging and computationally intensive. Simple algorithms I've worked with look for identifyable edges in one image and try to find that edge in the other image. Starting with greatly reduced images to develop a "guesstimate" map, the resolution is increased in each pass using the previous results to determine what area to search (so the entire image doesn't need to be searched). This is just one of many approaches to pulling depth from stereo. I'm not sure what kind of algorithm Tyzx is using or what kind of accuracy they're claiming (if they're advertising that info). In machine vision, time complexity is an issue. In other applications, such as mapping and generating 3D models, accuracy is a bigger issue.