Slashdot Mirror
Ray Tracing for Gaming Explored
Vigile brings us a follow-up to a discussion we had recently about efforts to make ray tracing a reality for video games. Daniel Pohl, a research scientist at Intel, takes us through the nuts and bolts of how ray tracing works, and he talks about how games such as Portal can benefit from this technology. Pohl also touches on the difficulty in mixing ray tracing with current methods of rendering. Quoting:
"How will ray tracing for games hit the market? Many people expect it to be a smooth transition - raster only to raster plus ray tracing combined, transitioning to completely ray traced eventually. They think that in the early stages, most of the image would be still rasterized and ray tracing would be used sparingly, only in some small areas such as on a reflecting sphere. It is a nice thought and reflects what has happened so far in the development of graphics cards. The only problem is: Technically it makes no sense."
Adaptive rendering would seem to be the way forward. Ray tracing has the advantage that you can bail out when it gets complicated, or render areas to the desired resolution. This means a developer can prioritise certain regions of the scene and ignore others: useful during scenes of fast motion, or to bring detail to stillness. The result is similar to a decoded video stream, with detail in the areas that are usefully perceived as detailed. Combining this with eye position sensing (for a single user) would improve the experience.
That completely depends on your point of view.
__ Someday, but not this morning, I'll finally learn to use the preview button.
I guess one has to state the obvious in that by moving to a process which is not implemented in silicon, as with current graphics cards, the work must necessarily be done in software. That means it runs on CPUs and that's something Intel is involved in where as when you look at the computational share of bringing a game to your senses right now, NVIDIA and ATI/AMD are far more likely to be providing the horsepower than Intel.
But really, even if this wasn't a vested interest case (and it may not be, no harm exploring it after all) - the fact remains that we don't actually need this for games. Graphics hardware has gone down an entirely different route whereby you write little shader programs which create surface visual effects on top of the bread and butter polygons and textures. This is a well established system by now and has a naturally compressive effect. It's like making all your visual effects procedural in nature rather than giving objects simple real-world textures and then doing a load of crazy maths to simulate reality. It works very well. Rememeber a lot of the time you want things to look fantastical and not ultra-realistic so lighting is some of the challenge.
Games aren't having a problem looking great. They're having a problem looking great and doing it fast enough and game developers are having a problem creating the content to fill these luscious realistic-looking worlds. That's actually what's more useful, really. Ways to aid game developers create content in parallel rather than throwing out the current rendering strategy adopted world wide by the games industry.
There is a project 'The OpenRT Real-Time Ray-Tracing Project' (not so much open despite name, but noncommercial code available) out there, and presumably Blender should be there soon.
CC.
TaijiQuan (Huang, 5 loosenings)
I remember some scenes that I would create with PoV to sometimes take several hours for a single frame to complete. Now we're looking at doing it in real time. Amazing.
... on the subject, from someone that doesn't have a vested interest in seeing real time ray tracing in games becoming a reality.
http://realtimecollisiondetection.net/blog/?p=38
I keep finding people who think that ray tracing is some kind of "perfect" rendering algorithm.
Actually I think of ray tracing as of the bubble-sort of computer graphics. It is absurdly naive, completely inefficient, and totally not necessary, especially for real time graphics. There are lots of different approaches to rendering that are much more flexible, efficient, and feasible.
I was a founder of one of the Midwest's first rendering farms back in 1993, a company that has now moved on to product design. Back then we had Pentium 60s (IIRC) with 64MB of RAM. A single frame of non-ray traced 3D Studio animation took an hour or more. We had probably 40 PCs that handled the rendering, and they'd chug along 20 hours a day spitting out literally seconds of video. I remember our first ray trace sample (can't recall the platform for the PC, though) and it took DAYS to render a single frame.
I do remember that someone found some shortcuts for raytracing, and I wonder if that shortcut is applicable to realtime rendering today. From what I recall, the shortcut was to do the raytracing backwards, from the surface to the light sources. The shortcut didn't take into account ALL reflections, but I remember that it worked wonders for transparent surfaces and simple light sources. I know we investigated this for our business, but at the time we also were considering leaving the industry since the competition was starting to ignite. We did leave a few months early, but it was a smart move on our part rather than continue to invest in ever-faster hardware.
Now, 15 years later, it's finally becoming a reality of sorts, or at least considered.
Many will say that raytracing is NOT important for real time gaming, but I disagree completely. I wrote up a theory on it back in the day on how real time raytracing WOULD add a new layer of intrigue, drama and playability to the gaming world.
First of all, real time raytracing means amazingly complex shadows and reflections. Imagine a gay where you could watch for enemies stealthily by monitoring shadows or reflections -- even shadows and reflections through glass, off of water, or other reflective/transparent materials. It definitely adds some playability and excitement, especially if you find locations that provide a target for those reflections and shadows.
In my opinion, raytracing is not just about visual quality but about adding something that is definitely missing. My biggest problem with gaming has been the lack of peripheral vision (even with wide aspect ratios and funky fisheye effects). If you hunt, you know how important peripheral vision is, combined with truly 3D sound and even atmospheric conditions. Raytracing can definitely aid in rendering atmospheric conditions better (imagine which player would be aided by the sun in the soft fog and who would be harmed by it). It can't overcome the peripheral loss, but by producing truer shadows and reflections, you can overcome some of the gaming negatives by watching for the details.
Of course, I also wrote that we'd likely never see true and complete raytracing in our lives. Maybe I'll be wrong, but "true and complete" raytracing is VERY VERY complicated. Even current non-real time raytracing engines don't account for every reflection, every shadow, every atmospheric condition and every change in movement. Sure, a truly infinite raytracer IS impossible, but I know that with more hardware assistance, it will get better.
My experience over the years was ALWAYS with static images that were raytraced. They looked great, but it wasn't until I experienced raytraced animations (high res, many reflective and transparent layers with multiple light sources and a sun-source) that I really saw the benefit and how it would aid in gaming.
The next step: a truly 3D immersive peripheral video system, maybe a curved paper-thin monitor?
I get tired of hearing this talk about real time ray tracing. They might be able to get basic ray tracing at 15 frames per second or more. But it won't matter, the quality won't be as good as some of the high quality images that you see that take hours to render. Sometimes days.
See, the two are incompatible because the purpose is different. With games, the idea is "How realistic can we make something look at a generated rate of 30 frames per second". But with photorealistic rendering the idea is "How realistic can we make something look, regardless of the time it takes to render one frame."
And as time goes on and processors become faster and faster, the status quo for what people want becomes higher. Things like radiosity, fluid simulations and more becomes more expected and less possible to do in real time. So don't ever count on games looking like those still images that take hours to make. Maybe they could make it look like the pictures from 15-20 years ago. But who cares about that? Real time game texturing already looks better than that.
Although I have a hard time arguing in the realm of 3D lighting, I will direct attention to the Beyond3D article, Real-Time Ray Tracing: Holy Grail or Fool's Errand?. Far be it of me to claim that this article applies to all situations of 3D lighting, it may be that Ray Tracing is the best choice for games, but I for one am glad to see an article that atleast looks into the possibility that Ray Tracing is not the best solution; I hate to just assume such things. Indeed, the article concludes that Ray Tracing has its own limitations and that a hybrid with rasterisation techniques would be superior to one or the other.
Demented But Determined.
As already mentioned, openrt is not open source. A good open source RTRT I've looked at is Manta. http://code.sci.utah.edu/Manta/index.php/Main_Page
... what are you comparing there? You're almost comparing a technology with an implementation.
And to the (+5 Insightful) naysayer who says that the future of games will be in shaders not in RT
You can implement a ray tracer on the gpu. I.e. through the use of shaders.
No kidding?? Well if you drive a car with a 16 cylinder, 1500 HP engine, it's a LOT faster than a 4 cylinder compact. More on this story as it develops.
There, fixed that for you.
Raytracing the shiny first-intersection makes a lot of sense, even if it doesn't sell more CPUs. Sure, some day we will all have stunning holistic scene graphs that fit entirely within the pipeline cache of the processor, but it's not yet time for that.
Every change in developing a game engine requires changes in the entire toolset to deal with how to produce assets, how to fit within render time limit budgets, and how to model the scene graph and the logic graphs so that both are easily traversed and managed.
In the meantime, we have a pretty nice raster system right now, with a development strategy that provides for all those needs. You might not think that fullscale raytracing would upset this curve, but I'm not convinced. What do you do when a frame suddenly is taking more than 1/30sec to render, because the player is near a crystalline object and the ray depth is too high? How do you degrade the scene gracefully if your whole engine is built on raytracing? We've all played games where things like this were not handled well.
I contend that game AI is sometimes more advanced than academic AI because game developers are results-oriented and cut corners ruthlessly to achieve something that works well enough for a niche application. The same goes for game graphics: 33 milliseconds isn't enough to render complex scene graphs in an academically perfect and general way, it will require the same results-oriented corner-cutting to nudge the graphics beyond what anyone thought possible in 33ms. If that means using raytracing for a few key elements and ray-casting/z-buffering/fragment-shading the rest of the frame, game developers will do it.
[
What's the Amiga have to do with raytracing? well, let me explain:
When the Amiga was released, it was a quantum leap in graphics, sound, user interface and operating system design. It could run full screen dual-playfield displays in 60 frames per second with a multitude of sprites, it had 4 hardware channels of sound (and some of the best filters ever put on a sound board), its user interface was intuitive and allowed even different video modes, and its operating system supported preemptive multithreading, registries per executable (.info files), making installation of programs a non-issue, and a scripting language that all programs could use to talk to each other.
20 years later, PCs have adopted quite a few trends from the Amiga (the average multimedia PC is now filled with custom chips), and added lots more in terms of hardware (hardware rendering, hardware transformation and lighting). It seems that the problems we had 20 years ago (how to render 2d and 3d graphics quickly) are solved.
But today's computing has some more challenges for us: concurrency (how to increase the performance of a program through parallelism) and, when it comes to 3d graphics, raytracing! Indicentally, raytracing is a computational problem that is naturally parallelizable.
So, what type of computer shall we have that solves the new challenges?
It's simple: a computer with many many cores!
That's right...the era of custom chips has to be ended here. Amiga started it for personal computers, and a new "Amiga" (be it a new console or new type of computer) should end it.
A machine with many cores (let's say, a few thousand cores), will open the door for many things not possible today, including raytracing, better A.I., natural language processing and many other difficult to solve things.
I just wish there are some new RJ Micals out there that are thinking of how to bring concurrency to the masses...
OK when models get further away in games, they get less detailed, both in terms of textures and polygons. Now if you have a light, with a simplified character, casting a shadow onto a wall quite a long way away, this simplification might become very much more obvious.
In a lot of cases in computing, doubling the number of pipelines (read: adding a second core, for example) does not, in fact, double performance unless the problem being worked on is highly parallelizable. For example, this is why one can not accurately describe a machine with two 2.5GHz processors as a "5GHz machine". Most computation that personal computers do on a day to day basis does not scale well, and the average user will reach a point of diminishing returns very quickly if they add many cores to increase performance for these tasks.
So all he's demonstrating here with his "16-core" experiment is that ray-tracing is a highly parallel process, and that throwing lots of cores at it will work effectively to increase performance without reaching that point of diminishing returns (at least, not reaching it very quickly.) Yes, we expect 16 cores to be faster than 4 cores or 1 core, but he's saying that when we're ray-tracing we can expect 16 cores to be almost four times faster than four cores and almost sixteen times faster than one.
That guy should ask the movie industry whether mixing rastering and raytracing makes sense or not. The defacto rendering engine, Pixar Photorealistic RenderMan, thinks that frankenrendering is a worthwhile thing.
Ray tracing looks hyper real on scenes full of plastic and mirrors but it's useless for rendering real-world scenes where radiosity effects dominate.
No sig today...
So, what type of computer shall we have that solves the new challenges?
A custom chip that has hundreds or thousands of dedicated raytracing processors that run in parallel. Raytracing is embarrassingly parallelizable, so it's far better suited to specialized processors than vectorizing.
Saarland University, the people who designed OpenRT in the first place, were getting 60+ frames a second on a hardware raytracing engine in 2005... and their raytracing engine only had 6 million gates and ran at 75 MHz. Today's GPUs have 100 times as many gates and run at 8 times the clock. A dedicated raytracer built with nVidia's or ATI's or Intel's resources instead of Saarland University's could give you that thousand core processor right now.
Somehow along the way I made a bad choice in life and now must live with 0 Karma.
Ray tracing solves two problems well - shadows and reflections.
A hybrid renderer might produce slightly better shadows than we have today but we still need orders of magnitude more power before it happens. Right now we're pushing the limit of graphics cards without ray tracing. Adding ray tracing at each pixel will make your pixel shaders hundreds of times slower.
No sig today...
the fact remains that we don't actually need this for games.
Your post is heavily dependent on availability of suitable hardware. Software can be ported and recompiled to new platforms, but hardware-dependent software has a short lifespan precisely because getting usable hardware doesn't last particularly long. There's a lot of otherwise good enjoyable games which are unplayable now because they depended on the early Voodoo cards or other unrepeated graphics hardware. Now with CPU power ramping back up (relatively speaking), we can pull away from lifespan-limiting dependence on dedicated graphics hardware, and move the full rendering process back to generic CPU hardware.
Torques me off when I want to play something but can't - not because I don't have the horsepower, but because it requires obsolete cards.
BTW: The article notes that RT vastly outperforms raster on very high poly count scenes - that alone is reason to switch.
Can we get a "-1 Wrong" moderation option?
Professer Philipp Slusallek of the University of Saarbruecken demonstrated a dedicated raytracer in 2005, using a 66 MHz Xilinx FPGA with about 6 million gates. The latest ATI and nVidia GPUs have 100 times as many transistors and run at 6-8 times the clock with hundreds of times the memory bandwidth. Raytracing is completely parallelizable, and scales up almost linearly with processors, so it's not at all unlikely that if those kinds of resources were applied to raytracing instead of vectorizing you'd be able to add a raytracer capable of rendering 60+ FPS at the level of detail of the very latest games into the transistor budget of the chips they're designing now without even noticing.
Here's a debate between Professer Slusallek and chief scientist David Kirk of nVidia: http://scarydevil.com/~peter/io/raytracing-vs-rasterization.html .
Here's the SIGGRAPH 2005 paper, on a prototype running at 66 MHz: http://www.cs.utah.edu/classes/cs7940-010-rajeev/sum06/papers/siggraph05.pdf
Here's their hardware page: http://graphics.cs.uni-sb.de/SaarCOR/
Not that much. Because GPU achieve ultra high parallelism using SIMD (single instruction multiple data) mechanisms.
And those aren't very efficient with very diverging code path.
i.e.:
- in traditional triangle rasterisation a lot of pixels are calculated at the same time for the same triangle. Thus a lot of thread will be running the exact same shader code on the GPU. SIMD is a nice increase of performance.
- in RayTracing, all pixels (all rays) start at the same point. But as rendering advance (as ray travel away from the camera) they will start to get different paths. In the end not all rays have taken the same path and executed the same shaders at the same time.
Thus you won't get as much improvement from a GPU as you would expect compared to other algorithms.
CELL's SPU or other chips that have several units that could take diverging paths (the latest generation of SPARC Niagara chips that don't share 1 single math core for all 32 threads) would probably perform well.
On the other hand, modern GPU are starting to have a large number of separate SIMD processors, so even if they couldn't take full advantage of their SIMD capability (as explained above), they could still achieve good performance by running 1 thread per multiprocessor.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Raytracing scales up far better than rasterization. Adding triangles to a raytraced scene has far less effect on it than to a rasterized scene, because you don't have to render anything that's not actually part of the scene... and you don't have to run what is effectively a separate rendering pass to eliminate parts of the scene to limit the hidden variables, and the processing for each collision is much simpler so you can fit thousands of dedicated raytracing processors in a hardware raytracer where the same transistor budget might support 8 or 12 rendering pipelines.
Look at what they were doing in 2005, with 1% of the gates, 1% of the memory bandwidth, and 15% of the core clock speed of today's GPUs: http://graphics.cs.uni-sb.de/SaarCOR/
If I understand the concept correctly, this combined with headtracking (the Wii guy?) will revolutionize the gaming world. Of course there's a fat chance I didn't get it at all, 'coz I don't have the time to read TFA at work...
Uugh. I'd like to say depth map shadows work perfectly well in games, and it'd be nice to stay with it.
Raytracing is a lot more intensive on the hardware which to me says that the video card companies will be pushing for game developers to implement it just so they can sell more powerful cards.
I guess the next logical step after raytracing would be global illumination, and real time displacement maps. None of which the gaming industry actually needs.
Once again what the gaming industry needs is creativity not video cards with 2gb of on board memory.
I have nothing compelling to say
Mod parent up. The debate and the SIGGRAPH paper are more on topic than the rest of the usual junk posted in the comments. Additionally, thanks for sharing the debate, I hadn't read it before.
That is an optimization that is used today. It is NOT a law. Think the real world, just because that huge billboard is miles away doesn't mean some guy runs up to it and tears down the paper and puts a low res version up on it. The entire world in RL and 3D has the same detail no matter where it is.
As you shoot the ray, it finds the entire world the same size and detail. This is actually one of the problems, for proper raytracing you can't use a lower res model for faraway objects because then the scene might indeed end up as you say, reflecting low-res stuff up close.
However, using lower res models is a crappy shortcut anyway, while it currently does lessen the rendering burden it in turn asks for considerable effort to supply the various version of models and textures.
There is another problem with it. Zooming/magnification. Say you are looking at a far away character, now you switch to your sniper scope and all of sudden you see a stick figure, doesn't work does it. Same with camera's that might show that model up close while you are far away. Notice how many outdoor shooters have you wade through grass obscuring your close range vision, while a sniper sees you standing on clear open ground.
I remember Intel making a lot of noise about this optimization by cutting detail on far away objects years ago, it worked but only for games with a fixed view of the world where you couldn't all of sudden get a closeup. In regular rendering you can pull the same stunt, If a building is ALWAYS going to be background, you only need to build the part you see, many a 3D scene is like an old hollywood set, nice facades and nothing in back.
As we get faster hardware, hopefully we can do away with the old hacks to come up with a scene that can be rendered. Quad core PC's don't even cost that much right now, willbe intresting to see what a few more years will bring. Mind you, if all goes to the CPU, I expect AI will take a hit.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
I'm not disagreeing with the author ( I did RTFA ), but I want to say there is some ray tracing ( in a sense ) already in some modern games. Specifically, some types of parallax mapping can be considered to be bastard red-headed stepchildren of raytracing.
What you have is ray tracing in texture space, but that texture is brought to the screen via conventional scanline rasterization methods. Sort of. My glsl parallax shader code sucks though ( looks all gelatinous close up ) so I'm no expert....
lorem ipsum, dolor sit amet
I left out the word million several times in my initial statement. DAMN YOU NO EDITING!
rhY
I hold very few opinions. I hold information based on observation and fact. If you wish to disagree, please use facts.
I can't wait to play a game where I'm a shiny silver ball floating above a checkered marble floor.
One of the problems with the current techniques is that programmers have to write shaders and artists have to make normal maps, high and low poly versions of models, and so on. It may be that raytracing reduces some of the need to do this work, and thus making games cheaper and faster to create.
I discovered raytracing by looking at that http://tirania.org/blog/archive/2007/Nov-16.html . It's a fun way to quickly see how raytracing works :-)
It's worth noting that in order to do decent looking shadows using a rasterizer, you need to use structures that simulate a shadow by way of a shadow volume. At some point, having scene graphs with a multitude of light sources will simply overwhelm a rasterizer by virtue of requiring too much space/time, due to how this kind of technique works. So the current standard is only useful within a certain envelope of scene complexity and computer power, rather than offering itself as an endlessly scalable solution.
Another thing to consider is that a ray tracing engine is practically a drop-in replacement for a standard triangle rasterizer; BSP maps, shaders and meshes work fine with ray tracing. If that's not enough, ray tracers come equipped with lots of extras: free shading, free mirrors, z-buffering isn't really needed, and you can use algorithms to define surfaces instead of relying high poly-count models. So as consumer-grade computers get more and more internally parallelized, this will become the clear way ahead for gaming graphics since it's more robust, and doesn't require a major shake-up of your art department's toolchain.
I just got off the phone with a Mr. Romano - apparently, Everybody Loves Raytracing!
David Kirk's explanation is totally correct. The need for "accuracy" in gaming is not high enough to justify the downside of ray tracing. Multipass rendering and shaders are obviously sufficient for 99% of the games given the state of GPUs and the quality of today's modern game engines. I consulted with nVidia back in the 90s on multipass techniques based on my PhD work at Upenn (available at my current webpage: http://www.pages.drexel.edu/~pjd37/diefenbach96thesis.pdf ), and these so-called "hacks" can approximate physics-based calculations very closely for non-critical applications.
It's amusing to read this. This guy apparently works for Intel's "find ways to use more CPU time" department. Back when I was working on physics engines, I encountered that group.
Actually, the Holy Grail isn't real time ray tracing. It's real time radiosity. Ray-tracing works backwards from the viewpoint; radiosity works outward from the light sources. All the high-end 3D packages have radiosity renderers now. Here's a typical radiosity image. of a kitchen. Radiosity images are great for interiors, and architects now routinely use them for rendering buildings. Lighting effects work like they do in the real world. In a radiosity renderer, you don't have to add phony light sources to make up for the lack of diffuse lighting.
There's a subtle effect that appears in radiosity images but not ray-traced images. Look at the kitchen image and look for an inside corner. Notice the dark band at the inside corner. Look at an inside corner in the real world and you'll see that, too. Neither ray-tracing nor traditional rendering produces that effect, and it's a cue the human vision system uses to resolve corners. The dark band appears as the light bounces back and forth between the two corners, with more light absorbed on each bounce. Radiosity rendering is iterative; you render the image with the starting light sources, then re-render with each illuminated surface as a light source. Each rendering cycle improves the image, until, somewhere around 5 to 50 cycles, the bounced light has mostly been absorbed.
There are ways to precompute light maps from radiosity, then render in real time with an ordinary renderer, and those yield better-looking images of diffuse surfaces than ray-tracing would. Some games already do this. There's a demo of true real-time radiosity, but it doesn't have the "dark band in corners" effect, so it's not doing very many light bounces. Geometrics has a commercial real-time game rendering system.
Ray-tracing can get you "ooh, shiny thing", but radiosity can get to "is that real?"
but not from the surface, that is in the middle. Light starts from a source and then bounces about the scene until it meets the camera. Most light rays don't intersect with the camera before they are fully absorbed. The trick is to start from the camera and go outwards hitting surfaces and bouncing about until you hit some light. Starting from both ends might lead to better results. I can't see how starting from a surface would help much, but then I am not a ray trace developer.
Sorry, but that design is worthless. Any academic, or indeed student, with a few $1000 can buy a Xilinx system and design a 66MHz chip. What they tend to do next is assume that all they have to do is buy a more expensive Xilinx system and the chip will suddenly run at 2GHz. Nothing could be further from the truth. The design changes completely.
The fact is that an EXISTING CPU running at 2GHz can outperform an EXISTING FPGA raytracer running at 66MHz.
My money's on stuff that actually works.
Hybrids do make a lot of sense. The author's argument is the need for a spatial partitioning structure if one mixes ray tracing with rasterization. This is a no-brainer; you'd have such a structure anyway.
In fact, his points actually show why a hybrid is perfect: most surfaces are not shiny, refractive, a portal, etc. Most are opaque - and a rasterizer is much better for this (since no ray intersection tests are necessary). He shows pathological scenes where most surfaces are reflective; however, most shots do show a lot of opaque surfaces (since Quake 4 does not feature levels where one explores a glass labyrinth or something).
Yes, if a reflective surface fills the entire screen, its all pure ray tracing - and guess what, that is exactly what happens in a hybrid. Hybrid does not exclude pure ray tracing for special cases.
Ideally, we'd have a rasterizer with a cast_ray() function in the shaders. The spatial partitioning structure could well reside within the graphics hardware's memory (as an added bonus, it could be used for predicate rendering). This way haze, fog, translucency, refractions, reflections, shadows could be done via ray tracing, and the basic opaque surface + its lighting via rasterization.
Now, I keep hearing the argument that ray tracing is better because it scales better with geometric complexity. This is true, but largely irrelevant for games. Games do NOT feature 350 million triangles per frame - it just isn't necessary. Unless its a huge scene, most of these triangles would be used for fine details, and we already have normal/parallax mapping for these. (Note though that relief mapping usually doesn't pay off; either the details are too tiny for relief mapping to make a difference, or they are large, and in this case, traditional geometry displacement mapping is usually better.) So, coarse features are preserved in the geometry, and fine ridges and bumps reside in the normal map. This way, triangle count rarely exceeds 2 million triangles per frame (special cases where this does not apply include complex grass rendering and very wide and fine terrains). The difference is not visible, and in addition the mipmap chain takes care of any flickering, which would appear if all these details were geometry (and AA is more expensive than mipmaps, especially with ray tracing).
This leaves us with no pros for pure raytracing. Take the best of both worlds, and go hybrid, just like the major CGI studios did.
This sig does not contain any SCO code.
Radiosity rendering and ray tracing are very complementary, in that one does a really good job with diffuse/global lighting effects, and the other does a much better job with local lighting, refraction, and reflection. Neither one alone gives what I'd consider really realistic images, at least without "cheating" in one way or another. Some combination of techniques is probably the best approach for realistic rendering.
Dedicated. *sigh* So I have this huge expensive chunk of silicon, and all it's good for, is playing games? That's swell when I happen to be playing a game. What about the other 95% of the day?
Cool, but does it run Linux? ;-)
(Note: I'm not asking for a driver; I'm asking for a port.)
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
The obvious problem with layering hacks on top of hacks to make your games look better is that it takes more time and money to pay programmers to develop them.
However, one of the clear trends in gaming is spending more on game art and artists than on programming and programmers, and maybe a less obvious problem is that working with hacks is a lot harder for the artists, as well. As an example: setting up shadows in rasterization is not too difficult, codewise (it's fairly expensive, timewise, but clearly workable if you don't have too many lights).
However, from an artist's point of view, rasterized shadows are wonky, counterintuitive, and fiddly to use. You have to specify which lights cast shadows of which objects on which other objects. The specific angles of lights, cameras, and the scene in question can cause ugly artifacts, which may or may not be possible to eradicate. This is particularly a problem in interactive play, where it may not be practical to preview all possible combinations of the above.
So, an overlooked advantage of ray tracing may be to provide a better artistic *medium* for game developers -- if you can afford it....
radiosity can get to "is that real?"
I suggest that any technique that actually computes global illumination can get that response. Radiosity is certainly one way of computing global illumination, but so is ray-tracing: starting from the surface, there's no reason why you can't bounce rays off multiple other surfaces to compute the effects of multi-bounce illumination. Ray-tracing isn't limited to visibility determination from the viewpoint.
Yes, you need a lot of rays to compute global illumination: the chance that you hit a light source if you're just blindly probing from the surface is low after multiple bounces. Yes, this is computationally very expensive. But it parallelizes well, which is Intel's point. Also, there has been much research to cut down on the cost and to intelligently shoot the rays so you guarantee you end up at a light source after multiple bounces.
I don't get it.
So I have this huge expensive chunk of silicon, and all it's good for, is playing games? That's swell when I happen to be playing a game. What about the other 95% of the day?
What's that got to do with raytracing?
If your video card cost more than about $20, why isn't that dilemma already burning in your heart?
Cool, but does it run Linux?
Does any video card run Linux?
If you argue with GlaDOS, you will not receive cake.
Why bother.
Any academic, or indeed student, with a few $1000 can buy a Xilinx system and design a 66MHz chip. What they tend to do next is assume that all they have to do is buy a more expensive Xilinx system and the chip will suddenly run at 2GHz.
1. When I referred to those company's resources, that includes their engineers.
2. Even assuming there's something inherently preventing them from running a raytracer faster than 66 MHz, it's an embarrassingly parallel problem. Even if all you can do is stick (say) thirty-two 66 MHz RPUs on a chip with a hundred times the gates, it's still going to get almost a 30:1 speedup rendering the whole scene. And that 2005 chip was getting performance comparable to its contemporary CPUs already.
3. If ATI/AMD or Intel started out right now building an RPU, and it took them 2 years to first silicon, if they ended up with a design that was no more than 30 times faster than the 2005 RPU (and I doubt they would be that bad), it might "only" be doing raytracing competitive than the 8-core CPUs they'll be selling... but it'd use a fraction of the power. There's no way you're going to have an 8-core CPU in a laptop in 2 years, but you could have a 32-way RPU in it.
...I'm surprised no one has mentioned one of the most serious issues with ray tracing: cache coherency. As processors have gotten faster, the bottlenecks are shifting drastically; being able to feed data to your processor quickly enough is as important as being able to get it through the processor's pipelines.
Traditional rasterization is about as cache-friendly as you can get; feed your GPU a list of triangles to rasterize, along with small chunks of additional data here and there, and watch it fly through them. Texturing can cause a little bit of thrash since arbitrary orientations mean that you're not going to be sampling your texture maps in consistent patterns (and things like mipmapping throw additional wrenches into the works), but the major GPU manufacturers have done a ton of work on those issues and they're pretty well worked out at this point.
Now, contrast this with raytracing. Even for primary rays, it can be hard to exploit ray-to-ray coherency; the snazzy spatial partitioning algorithms that give the bragged-about O(log n) asymptotic time can send you romping about your scene memory in fairly arbitrary fashion, and while trusting that you're going to intersect the same objects as the last (primary) ray cast is a good start, tests still need to be done to make sure that nothing has intruded on the scene any closer to the camera than the last ray's nearest hit. And secondary rays (lighting, reflections, etc) -- the ones needed for all the effects that have people interested in ray tracing in the first place -- are much, much worse; the direction of reflections off of a curved surface might as well be random, from a caching perspective; it's all but impossible to do any predictive preloading there except in certain degenerate cases. We're still nowhere near the object counts needed for ray tracing's ostensibly-better asymptotic performance to wipe out the hugely better constant factor that good cache behavior gives rasterization algorithms, and I wouldn't be shocked if we never get there (but that's a rant in its own right).
The need for "accuracy" in gaming is not high enough to justify the downside of ray tracing.
But that's not the reason you need ray tracing. You need ray tracing because it scales up so much better than ad-hoc approximations as the scene complexity increases. It gives you "good enough" with less power.
Look, Slusallek's RPU was getting far better raytracing results than a 2005 P4 CPU, on hardware that's about comparable to a mid-90s Pentium MMX. It's three years later. Intel's showing "good enough" performance now with maybe 8 times the CPU of that 3 year old Pentium-4 Slusallek was talking about.
Building an RPU array with 2008 technology instead of 1996 technology? I would be amazed if you couldn't get at least as good looking a game as you're getting from a 9-series GPU for a lot less hardware... which means a lot less money, and far lower power requirements...
It says that a quad core processor gets 16.9 frames at 256x256 resolution.
Wolfenstein 3D on a 386SX/16 at 320x256x8 :-)
Stick Men
Douche.
You're kind of missing the point; yes, the ray-tracing crowd already knows that ray-tracing is slow, though it's not as slow as public perception (influenced by non-realtime renders like Pov-RAY) would have you believe. What the ray-tracing crowd sees (and they have been seeing this from a long ways off) is where the computational complexity lines cross, where it suddenly becomes cheaper to use ray tracing. Sure, ray tracing on modern hardware is pretty slow; maybe my (several year old) dual-core AMD64 3800 could be made to render ray-traced graphics comparable to an N-64 rather than a Wii, but the great thing about ray tracers is that they don't slow down much when you throw more geometry at them. Rendering a scene with a million triangles only takes twice as long as rendering a scene with a thousand. Once hardware is fast enough to do low-complexity scenes at a good resolution and framerate, it takes only a very small hardware improvement to render extremely high-complexity scenes. The real limitations soon become "how many triangles can we squeeze into physical memory on one machine?" and "how fast is the memory bus?", not "gosh, I wish these ray-intersection tests were faster."
Keep in mind that a 96x performance improvement sounds like a lot, but that's only about ten years if Moore's law holds, and the article points out that Intel already has demos running at 90fps at HD resolutions. (I haven't seen them myself, so I don't know if they're rendering complex scenes with shadow rays and textures or what, but a current quad core processor ought to be able to do complex scenes at quite a bit better than 256x256@15fps, with a well-written renderer. (Not mine; I would be thrilled if it was that fast.)
There are, certainly, some performance issues that need to be worked around; rebuilding an acceleration structure when something moves is expensive (BIH trees are better at this than KD-trees, but they're still O(nlogn)), so more care must be taken to treat static and dynamic parts of the scene differently. Lots of lights can bog a scene down, and certain shapes of scenes are worse than others. Mixing large and small triangles is bad for BIH trees. These are all things that have to be understood in order to make a fast game, at least until the hardware gets fast enough that no one cares about framerates anymore.
I expect maybe we'll see some sort of raytraced game on par with tuxracer this year, and a real high-budget game maybe in one or two years, and then ray tracing will be mainstream in about five, when the average person will have a fast-enough machine to handle it.
This is a rather good point; at some point, adding more polygons doesn't do anything to make an unrealistic scene look more realistic. This is true for raytracing and polygon rendering alike. Ray tracing has some advantages here, for what it's worth; it scales better with huge scenes, and it can represent non-triangular primitives natively (though all the fast ray-tracers I've seen only deal with triangles). I wouldn't call reflection a non-issue; currently, no one cares because current implementations aren't very good, and there aren't any better alternatives to compare with. Same with refraction. Ray-tracing can do soft shadows, but they're computationally more expensive (at least, all the approaches I know are).
Ray tracing is just the next step towards realism. Once we start doing ray tracing, we can move on to global illumination. Photon mapping is a GI algorithm that works like ray tracing in reverse; it casts rays out from the light source, and then they can bounce of off or be absorbed by the objects they hit (depending on surface properties), and their final location is stored as a point in a data structure called a photon map. Then, when you ray trace, you use the local density of the photon map to approximate the amount of light. Photon mapping can be used to simulate ambient light, caustics (patches of light reflected off of or refracted by shiny things), and subsurface scattering in a way that is physically correct and unbiased. See "The Light of Mies van der Rohe" animation on this page for an example of ambient light, or here for some images of caustics.
Ray tracers can do all the things that polygon renderers can do, plus a bit more. Once the hardware gets fast enough (and it looks like that will happen within a few years), there's no real reason not to use ray tracing. Photon mapping is more expensive (there's an nlogn sort involved in constructing the photon map), so it will probably be quite a while before we start to see real-time global illumination updates, but that's the next big step, and we can't go from here to there with the polygon rendering algorithms we're using now.
You're right, that's a point that a lot of people don't realize; acceleration structure can take a long time to build. However, kd-trees aren't the fastest trees to build; BIH trees can be built much faster. They're both nlogn, but the BIH tree construction algorithm is an in-place sort and uses a predictable amount of memory (it's actually remarkably similar to quicksort). BIH is usually a bit slower to use when you're actually tracing the scene (maybe ~%70 performance of kd-tree depending on scene and implementation), but they can be constructed maybe a couple orders of magnitude faster. (A link to that paper and others can be found here.)
The big difference between the two approaches is that KD-trees are a spacial subdivision, and any objects on both sides of the split plane are referenced from both sides of the tree; BIH is an object subdivision scheme: objects are never duplicated, but the space occupied by both sides of a branch may overlap.
It's possible to get around some of the animation problems by having a top-level BIH tree, that contains a bunch of separate trees, one for each movable object and one for the static scene. (That won't work very well for scenes with a million snowflakes blowing in all directions, but it should be okay for most games.)
Amazing how Daniel Pohl can master the intricacies and subtleties of raytracing optimizations but can't understand the difference between "then" and "than".
The problem is that fast general-purpose CPUs use a lot of silicon, while specialty processors use very little silicon and are much much faster than GP CPUs. In other words, your PS3's Cell is going to perform algorithms like Folding@home 30 times faster than an average CPU, while your graphics card can even double that.
So the idea is that instead of building a chip with 100 general-purpose cores (of which you only have a use for about 20), you put only put on 20 of those, but you also have 20 cores that are good for things like compression algorithms, 20 that are good for encryption algorithms, 20 that are good for graphics algorithms, and maybe 20 that have reprogrammable microcode.
It might seem like a waste of silicon to include specialty cores you can't use all the time, it's a waste of power to run 20M transistors to do something that 1M can do twice as fast.
dom
finally developers will be able to trace those shiny little bugs perfectly. i'm sure that will make the overall quality of the games a lot better.
O_o
I wonder how this post could be modded 'interesting'??
When the raytracing guys talk about scaling, it was increasing the number of FPGA present on the chip (possible because the memory bandwith is low) not scaling it's frequency.
That said I don't believe much in raytracing for games in the short/medium term:
- software only ray tracers are still too slow
- dedicated HW solutions have the huge problem that they cannot render existing games
- apparently it's hard making GPU do ray tracing efficiently
- plus it's likely that the resolution of games is going to increase to HD resolution in a few years which will delay again raytracing (O(n) in resolution compared to O(log n) for rasterisation).
- game companies are very conservative.
This is a bit offtopic, but anyway:
I used Amiga for a long time (approximately from 1987 to 1997) as my only platform and liked many things on it, but what do you mean with the bit about the 'some of the best filters ever put on a soundboard'? AFAIK, the Amiga audio hardware had only one filter, and that was the low-pass filter that had no user-controllable properties other than being switchable on and off. In this particular regard, the Paula chip was a step down from the famous SID, which had quite flexible filtering capabilities.
In order to not to be completely offtopic, I agree that having a pile of cores is good for non-trivial tasks, although I wouldn't be throwing specialized hardware out yet; after all, the GPUs these days are fairly flexible and can be adapted for many tasks outside just rendering polygons for the latest 3d-shooter.
Everyone who makes generalizations should be shot.
Resolution isn't a big deal: I already run my computer at 1280x1024, which is slightly better than HD... and over the past 10 years good affordable monitors have really only gone from 1024x768 to 1280x1024... higher resolution monitors than that cost more than most computers.
:(
No, the big problem with raytracing is your last point: game companies are very conservative.
Even if nVidia could stick several parallel RPUs in the next generation of graphics chips pretty much for free (and they could, when an RPU would come out to about 1% of their silicon budget) it'd be five years before any game other than open source-ones like Doom and Second Life used them.
Of course Intel could stick 32 RPUs in the next Core Hyper Quadro II without noticing the transistor budget, but they won't. Maybe AMD/ATI will put it in the next Biathlon XXX...
This isn't entirely true; it is quite possible to build an acceleration structure that contains not triangles but other acceleration structures. For instance, you might have a top-level kd-tree that holds a hundred other kd-trees, where each of those other kd-trees is the pile of triangles representing some movable object in the scene. If one of those objects moves, you only have to recompute the top-level tree, and (if the object's geometry has deformed as well), the tree of that thing that moved. You definitely do not have to recompute everything just because one triangle moved.
Also, we now have BIH trees (link to the paper is here); these can be re-sorted much more quickly than kd-trees (the current favorite for static scenes). They use a fast in-place sort very much like quicksort. Dynamic scenes can be a bottleneck in certain cases (a million swirling snowflakes, for instance), but in the general case it's not a show-stopper.
Actually, primary rays are quite a bit cheaper for a raytracer to compute than secondary rays (reflections, shadows). There are ways to exploit the coherency and do far less work (packet tracing and MLRTA, for instance). In a hybrid scheme, the graphics hardware would only be unloading the easiest portion of the workload.
The point the article was trying to make is that if your ray tracer is fast enough to ray-trace the entire screen, there's no compelling reason to even use rasterization; the ray tracer is going to be fast enough to do that by itself, too. Hybrid schemes might make sense if ray-tracing is too slow to render the whole screen at a decent framerate (basically, where we're at today), but in a few years that will very likely no longer be the case.
The reason the movie studios don't use raytracing more often is that raytracing is not memory-parallel, and they need to be able to render scenes that can't fit in the memory of any one computer. Here is some info on how Pixar used ray-tracing in the movie Cars.
Here is an (in my opinion) intelligent and well-informed rebuttal to the article you linked to that I came across today.