3D Visualization Moves Forward

Chris writes "Showing for the first time at the Society for Information Display (SID) conference in Boston was a three-dimensional display with 100 million volume pixels or "voxels". The Perspecta is a hardware and software combination that projects 3D images inside a 500 mm transparent spherical dome. Images 250 mm in diameter can be seen from a full 360 degrees without goggles, allowing the viewer to walk around the image. It can be used to visualize protein structures and to plan surgical and radiation treatment by locating the exact position of a tumour on an x-ray or mammogram. It could also be used in air traffic control, prototype designing and security scanning of luggage. Perspecta uses Texas Instruments' digital light processor technology and a spinning projection screen, which sweeps the sphere." We've done some previous stories about this globe from Actuality Systems. The trend seems to be toward simulating 3D with high-resolution flat screens, though.

Comments

I see probelms with the approach

[inkfox]

I missed the Actuality story. This was my take on that (also included below). It doesn't look like the new version is much of a jump forward...

Now, niftyneat as it looks, I see a few problems...

First and foremost, you're going to be stuck representing solid 1-color materials, wireframes and ghosts with this. You're also not going to be able to make objects appear to be lit correctly. Why? Because the display has no idea what angle you're viewing it from. I'll explain.

Hold your thumb in front of the screen. It's blocking some part of the display, right? Move your head back and forth a little, and it will block different parts. Raise your head up a little or drop it down and it will block different parts again. The thing is, the display has no idea where you're looking from, so every part needs to be visible at all times. It can't clip out bits that are behind other things like a traditional 2D display. The result is that if you show a screen full of text, and draw a thumb in front, you still see the text through the thumb. Both will appear to act like ghosts.

Now, consider drawing a Coke can with a flashlight shining on the side. Again, it has no idea which side you're viewing from, so it's got to draw all sides of the can. The thing is, as you move about it, the logo on the front of the can shouldn't be visible when looking at the back of the can. Similarly, when you look at the side opposite the flashlight, it should be all dark. But since the display uses volumetric texels, it has no idea about the facing of each texel. Every texel's going to be drawn, so a you'd see the backward logo when looking from the back, and you'd see what boils down to a really confusing lighting situation when viewing from the non-flashlight side. It's like ghosts or colored X-Rays.

If you're still with me, that covers the reason for no shadows or non-uniform dull, not-too-shiny surfaces.

Next problem is - it's gonna be SLOW! Sad, but true! If it were a 3D bitmap representing equal units of a cube, that would be one thing. Unfortunately, it represents slices of a bitmap rotating through space.

Now, let me say this: Computers hate round things. Arcs, swooshes, ribbons, none of these are much fun for a computer to draw (comparatively speaking), much less, to render into.

Normally, polygon raster operations boil down to setting up a bunch of lines, one per scanline, and for each, figuring out how you progress across the line in measured, discrete steps. "I'm starting here in the texture, and I'll be there in the texture. I need to get there in 32 screen pixels, and I advance n units through regular steps of screen, texel and 1/z space." This tells the computer do the expensive calculation once, and just do 32 iterative steps to render the 32 pixels on that scanline. Any modern 3D engine is actually optimized to do the expensive stuff 1-2 times, creating the per-scanline numbers iteratively as well.

The only places where this approach doesn't work are where you're clipping against the edge of the display area. Clipped triangles are traditionally an order of magnitude more expensive to render than non-clipped ones. So much so, that terrible tricks are used to avoid them or reduce them to categories of special cases that can be tackled to attempt to avoid reverting to a true clip. For example, many display systems actually create waste RAM in a border around the screen. If a triangle doesn't penetrate the waste area, the rasterizer will go ahead and draw (or pretend to draw) the dummy pixels. It's only in the case where triangles are partly on screen, but go even beyond the dummy area that the hideously expensive render functions are called. Drawing millions of pixels per second that you know the user will never see? That sure points to a problem!

Enter the circular slice-based display space.

Here, for every single pixel, you've got to find which bits of a render go through. Essentially, you have to clip against the front and back of every single triangle you render as you calculate each slice. You're taking the worst hit on every single triangle!

What's even worse is that a single 'frame' (half rotation, assuming the rotating display plane is visible from front and back) consists of just shy of 200 renders. This means you're taking that 1% worst case scenario and repeating it 100% of the time, and repeating it about 200 times per frame. And because you're dealing with an arc for the rotational advancement (remember, computers like even, linear, discrete steps), you're dealing with curved surfaces instead of little cubes and the planes of a view frustum. Essentially, you're looking for the union of an arbitrary material and a stuffed piece of macaroni instead of merely finding the portion that fits within a little box. This makes the checks for pixel penetration several orders of magnitude more expensive and makes it even more expensive to attempt to reuse data from one slice to the next.

Hee. Plus the display is connected to your PC via SCSI2W, which is also a not-too-minor detail. You've got over 100 million pixels to send across per 'frame'. Even if they're just 1-byte pixels (256 color), and partial updates, that's asking a lot of a dual-channel 20MHz(?) bus.

Mind, we're still discovering things today which would have sped up rendering on our Commodore 64s. The computational cost will come down over time as more ways are developed for rendering in non-uniform/curved space, and as different spatial representation methods are explored. This is a nifty advance, certainly a step closers than the silly lenticular lens based 3D systems and the layered LCD-over-CRT approaches.

Still. Think of a ghosty AutoCAD on a 286. Look, but don't touch. We've still got a ways to go before 3D games and movies become a reality.

But don't get me wrong: It's a neat advancement, and it gives me hope. If I could borrow one, I think I'd make a noisy whirring ghost town snow globe. The shape just begs for it. And I'd love to get cracking on trying to find efficient algorithms for the unusual render space. *sigh.* $60k though. Maybe eBay can help me out on this one in another 20 years.

Re:Aaaaaand, why bother.

[vrt3]

However, if you were to cover one eye and look at a 3D object, you get a totally different sense of that object

To be honest, I don't. I get exactly the same sense whether I use one eye or both. Reason is that in most circumstances only one eye at a time is focused. My left eye is short sighted, while my right eye is far sighted. Very strange, and as an effect I'm bad at perceiving depth. Good enough for normal life; I have no problems driving a car, but it makes parking more difficult.

I can see stereographic images, but it's very hard to get both of my eyes focused simultaneously (up to 1 m I can see sharp with both eyes, further away only with my right eye; depends on lighting conditions though, on a sunny day it's clearly better). Looking trough a binocular or stereomicroscope gives me full 3D view, since they allow me to adjust the focus separately for each eye.

3D techniques that rely on both eyes getting a different image won't have much effect for me, unless the screen is in the range where both my eyes can adapt and focus enough. Or maybe I should get glasses.