Ask Slashdot: How Did Real-Time Ray Tracing Become Possible With Today's Technology?

dryriver writes: There are occasions where multiple big tech manufacturers all announce the exact same innovation at the same time -- e.g. 4K UHD TVs. Everybody in broadcasting and audiovisual content creation knew that 4K/8K UHD and high dynamic range (HDR) were coming years in advance, and that all the big TV and screen manufacturers were preparing 4K UHD HDR product lines because FHD was beginning to bore consumers. It came as no surprise when everybody had a 4K UHD product announcement and demo ready at the same time. Something very unusual happened this year at GDC 2018 however. Multiple graphics and GPU companies, like Microsoft, Nvidia, and AMD, as well as other game developers and game engine makers, all announced that real-time ray tracing is coming to their mass-market products, and by extension, to computer games, VR content and other realtime 3D applications.

Why is this odd? Because for many years any mention of 30+ FPS real-time ray tracing was thought to be utterly impossible with today's hardware technology. It was deemed far too computationally intensive for today's GPU technology and far too expensive for anything mass market. Gamers weren't screaming for the technology. Technologists didn't think it was doable at this point in time. Raster 3D graphics -- what we have in DirectX, OpenGL and game consoles today -- was very, very profitable and could easily have evolved further the way it has for another 7 to 8 years. And suddenly there it was: everybody announced at the same time that real-time ray tracing is not only technically possible, but also coming to your home gaming PC much sooner than anybody thought. Working tech demos were shown. What happened? How did real-time ray tracing, which only a few 3D graphics nerds and researchers in the field talked about until recently, suddenly become so technically possible, economically feasible, and so guaranteed-to-be-profitable that everybody announced this year that they are doing it?

Comments

It isn't out of the blue

[kiminator]

The short answer is that it isn't brand-new. As the article mentions, nVidia has been doing real-time ray-tracing demos for about a decade. Various tricks and approximations are used to make this a reality. Game developers have largely shied away from it because similar results can be achieved with typically better performance using other methods.

This announcement, particularly with the involvement of Microsoft, indicates that the companies finally feel that the technology is mature enough to actually be used in a game. My guess is that it will likely still be some time before it is put to use. The fact that it was announced at the same time likely indicates that the three companies have been working together for some time behind closed-doors to agree upon the DirectX Raytracing API.

Re:It isn't out of the blue

[Anonymous Coward]

Indeed, they could do this many, many years ago, the problem was always having enough rays to make it worth while. Ray tracing a couple rays in real time is relatively easy, but tracing enough of them to get results that are better than the current system was hard.

I remember messing around with ray tracing programs back in the '90s and they were great, but it took huge amounts of time to render a scene. Far too long for even simple board games, let alone a typical 3D game.

At some point, it was inevitable that ray tracing would be preferable to the standard raster graphics that we've been using. The file sizes necessary to get anywhere near what raytracing can do would have made it impossible to keep up. Having better algorithms and specialized computation units focused on it just moves that point closer to the present.

The thing I'm really looking forward to though is that they should be able to scale with new advances in monitor technology. Both in terms of pixel count and in terms of aspect ratio in a way that wasn't realistic previously.

Re:It isn't out of the blue

[kiminator]

Also, apparently their current demos are running on some severely beefy hardware (e.g. here), and they're only using raytracing for a portion of the scene. This may play a part in gaming eventually, and it's good that the APIs are getting out there, but it probably will be a while before it makes it to the next Call of Duty game.

Re:It isn't out of the blue

[Shinobi]

Even if ray-tracing isn't used for graphics, you can use it for the sound engine, with the benefit of making it hardware accelerated. Would make a whole lot of games more interesting, for example Stealthers like Thief, horror games etc etc.

Terminology

[B.Stolk]

Be careful with your terminology.

Real Time Ray Tracing with one primary ray and one shadow ray for each pixel, was viable last year as well (at 1080p.)
But this will not render indirect light, thus no Global Illumination.

You may be referring to Real Time Path Tracing, where you need to shoot a lot of rays for every pixel.
This is currently not possible, and also not possible in this year's GDC demos: I think most of the demos were hybrids (rasterizing+tracing), and definitely not full Global Illumination.

Re:Terminology

[ganv]

You are touching on an important issue here. 'Real-time' and 'Ray-Tracing' are both open to definition: At what screen resolution, frame rate, and number of rays? And is it a hybrid or full ray tracing solution? That asked, I am very interested to know the answer in the original post: Is this a tipping point where they finally decided the hardware is good enough to market the ray tracing they have been working on or is there some substantial improvement in algorithms or in custom hardware dedicated to ray tracking?

Ummm, this isn't too shocking

[Anonymous Coward]

Real time ray tracing was a topic since I was in college 20 years ago. A lot of PhD students have done their research. Algorithms optimized. Hardware advancing. Futurists at major corporations (fancy name for people responsible for monitoring tech) saw hardware evolving.

So Microsoft probably saw that it was going to be viable in 5 years (or whenever) in 2013. They probably starting developing an API in conjunction with major hardware manufacturers. So they all worked together to bring ray tracing to the masses eventually.

Now all this work is paying off and consumers get the end product.

This is no different than major corporations using augmented reality to validate construction designs. Ya, they are doing that atleast in labs. Take the 3D model as built and validate that it is as built. Or quickly look at something in the real world and figure out where the 3D model data is and figure out in a second what vendor supplied the broken part. Ya, this stuff is happening!

Re:Smaller transistors

[Anonymous Coward]

You're probably not paying enough attention to planes. They crash less frequently than they did and airports are switching to systems where the planes make a smooth approach without the stair-step approach pattern that was previously used.

Other than that, there's not going to be much difference because by the '80s planes were just that good. Any other changes are ones that are mostly only going to be noticed by people actually working with them.

The computing technology stuff tends to be more accessible and more present in our day to day life than it was, so we tend to notice it more. But, there again, we're getting close to the point where additional improvements aren't going to be as obvious as they were previously. I've had this computer now for a few years and the previous one was quite a few years old when I replaced it. That would have been inconceivable to me 20 years ago as the technology was improving that quickly.

Well, not all of a sudden...

[Junta]

Demos of real time raytracing have dated back to 2009 or earlier, albeit with various limitations. Raster based rendering is faster and going to raytracing means much better lighting, at the expense of some resolution/geometric detail/hardware requirements.

I think what is being seen is that we've been well beyond the point of diminishing returns as far as raster can reasonably get in terms of better quality. Sure, we can cram more and more polygons and sure we can raise the bar to 4k resolution, but the bang for the back is small. Given that situation, video card market has an issue, they need to do demand generation.

So pushing 4k for gaming helps, and 8k would be nice, but it's just really hard to tell the difference at this point.

If that's a hard sell, then VR certainly can knock things back if it gets traction. With wider FOV, stereoscopic rendering, and optimal experience being at least 90Hz, that would certainly deliver. However, as much as I am a fan of it, it's far from a given that VR is ever going to be large enough to drive adoption. at volumes that can sate the business needs of the GPU vendors.

So raytracing is a third option to make best of breed graphics card noticeably struggle with something that's very visually apparent. Suddenly the market content with status quo of ever refined raster graphics simply must make the leap to have some marketable advancement.

The people pessimistic about any such advancement just continuously have their expectations calibrated to how fast it can perform raster graphics. Raytracing does mean having to step back, but we have enough headroom to take the hit.

Re:Sinple

[KingMotley]

I would guess they figured with graphics cards having 3500+ cores and ample memory for massive lookup tables, suddenly it seems feasible. That or a patent just expired, or both.

It is HYBRID, not full Realtime ray tracing

[Anonymous Coward]

It is using rasterization for the scene, and then hybrid ray tracing for the reflective, shadow, and opacity components were it applies. This is not true ray-tracing, but rather another rendering "trick" to optimize for speed. Most of our rendering for games today is approximations, because it is good enough. Same thing here.

Not feasible for gaming

[PhrostyMcByte]

Ray tracing is an impressive technical feat, but the argument against it for gaming still stands:

However fast you get at ray tracing, you can instead use that power for rasterization and do far far more.

The day may come where that gap doesn't matter anymore, or where we find a way to overcome it... but for now, I think this technology will primarily be used to help accelerate very simple environments, and more complex for offline rendering.

Re:Not feasible for gaming

[shaitand]

"However fast you get at ray tracing, you can instead use that power for rasterization and do far far more."

Not when you are real time rendering the graphics to be integrated on the fly along side real time light vectors flying at the retina. Think pokemon go, without the phone, or the screen, and with the pokemon actually sitting on your counter with it's legs hanging down.

Re:Not feasible for gaming

[squiggleslash]

The major innovation here is that it'll soon become possible to render, in real time, silver balls, cones, and cubes floating above a checkerboard, something that's just not possible with regular 3D graphics.

Raster 3D Graphics seem to be tapped out

[rsilvergun]

Remember the jump from the SNES/Genesis to Playstation/Saturn? How about the first time you saw a Dreamcast game? Those were big leaps. But PS2 to PS3? If you'd been paying PC games you'd seen stuff on par with PS3. And PS3 to 4 was hardly a leap at all.

The trouble is modern graphics have gotten _hard_ to make. Pixel shaders are a bitch. They're too labor intensive. What's needed is something that lets you do great graphics with less man hours and fewer bugs. If ray tracing isn't gonna do that then it might as well be PC's answer to 3D TVs. Especially if it's only kicking out 30 FPS.

Re:Raster 3D Graphics seem to be tapped out

[Kjella]

The trouble is modern graphics have gotten _hard_ to make. Pixel shaders are a bitch. They're too labor intensive. What's needed is something that lets you do great graphics with less man hours and fewer bugs. If ray tracing isn't gonna do that then it might as well be PC's answer to 3D TVs.

I think that problem extends far beyond graphics, if you're going for realistic graphics it also has to behave and interact with everything like it's real. Like if the wind is blowing everything has to be fluttering in the wind. If you're making a dog it can't just look like a dog, it has to move like an actual dog with bones and muscles dragged down by gravity and leave paw prints in the mud. To say nothing of humans, uncanny valley here we come. Heck, I think it'd be difficult just to properly simulate taking a baseball bat to inanimate objects. All the dents and chips and scrapes and smears and splinters are very difficult to get right unless you bring in a ton of material science and deformation models. Usually we just fake it with an animation or two, replacing the "whole" model with a "broken" model and skipping all the hard parts. But it only looks half real...

...and clever Algorithms

[Roger W Moore]

According to this video it is not just more powerful hardware but also that they came up with the idea to use only a fraction of the rays normally required and to then used a power denoise algorithm to generate the final image.

Re: ...and clever Algorithms

[Anonymous Coward]

This pegs it. Ray tracing is being used selectively as part of the standard raster pipeline we have today. It's still not feasible for real time raytracing for every pixel and handling reflections perfectly may never be doable.

Re: ...and clever Algorithms

[Pseudonym]

Just to be clear, it's not "reflections" (as in mirrors) that's the problem necessarily, it's the fuzzy effects: diffuse, glossy, translucent, aerosols, etc.

It may never be doable because of Blinn's Law.

Re: Someone's cheating

[sound vision]

Nvidia makes custom hardware to play raster games too. They call them "video cards". I hope they can cheat the ray tracing right into my computer.

So first off, the Nvidia/MS thing is crap

[locater16]

The first thing to understand is that the Nvidia and MS press releases are complete crap. Oh it "works". The UE4 Star Wars thing is real enough. It's also running on $12,000 worth of GPUs at 30fps in 1080p, and is only partially raytraced.

Now does this mean realtime raytracing isn't here? Well the answer is no, it is here. The game Claybook is entirely raytraced, unlike the DirectX demos. It's in early access and runs on nothing more than an Xbox One or PS4, and does so at 60 frames per second. Here's the trailer: https://www.youtube.com/watch?... Now THIS is possible because graphics programmers have gotten quite clever over the years. The cleverest bit is called signed distance fields. This can be thought of a volume of points, or boxes, that all store the nearest distance to a solid "surface". Going through this structure allows you to raytrace very very quickly, as you know how much empty space you can skip each time without hitting anything. And since this data is relatively small for each point it doesn't use up a lot of memory either. It's so fast and low memory you can run a demo in your browser here: https://www.shadertoy.com/view...

Obviously there's a bunch of other clever programming going on in Claybook and other titles. But SDF's are the biggest thing to understand. That and that the MS Raytracing API is totally uninteresting from a performance perspective. In fact it's rather awful.

Re:So first off, the Nvidia/MS thing is crap

[UnknownSoldier]

> The UE4 Star Wars thing is real enough. It's also running on $12,000 worth of GPUs at 30fps in 1080p

Uhm, try $50K for the Nvidia DGX Station running on four Volta GPUs.

> Well the answer is no, it is here.

Outcast, back in 1999, did real-time ray tracing and voxel rendering

The only difference is 20 years later we can do it hardware.

Re:Smaller transistors

[OneHundredAndTen]

Other than that, there's not going to be much difference because by the '80s planes were just that good.

What has been touted as big passenger-visible advancements in commercial airlines in the last few years is slightly larger windows, slightly higher pressure in the cabin, and slightly higher moisture. That's it. As for safety, while it is true that airplanes are safer than they have ever been, the truth is that they were already extremely safe in the 70s. The sad, depressing truth is that it takes exactly the same time to fly from London to San Francisco as it did fifty years ago. That's half a century ago. It is indeed cheaper than it used to be, but the experience is, if anything, more miserable than it has ever been. And, bearing in mind that the airline industry has become a race to the bottom in the last 10 to 15 years, the experience is going to become, if anything, worse.

No, It didnt.

[Joviex]

You fundamentally misunderstood the technology; partial solutions for partial rendering with interpolation is not 100% raytracing.

Re:Smaller transistors

[Joce640k]

The sad, depressing truth is that it takes exactly the same time to fly from London to San Francisco as it did fifty years ago.

It's unfortunate that we can't change the laws of physics, yes.

It is indeed cheaper than it used to be

Ticket price is linked to fuel consumption. The price of air travel would rise sharply if we tried to fly any faster.

Re: Smaller transistors

[Anonymous Coward]

Cars dont have carburetors and run for 100,000 miles with essentially no maintenance beyond wear components. My guess is you never owned a car before the 80s to have any idea how much things have changed.

Re:Smaller transistors

[hipp5]

We don't even have the Concorde anymore. Airplanes still look, act, and sound the same as in 1969 when the 747 first flew. Cars, houses, clothes, buildings, roads, food, all more or less look the same.

They still look roughly the same because that's a pretty good geometric design. But that doesn't mean they are the same. There have been huge advances in aviation technology over the past 50 years. For example, the average fuel burn of new commercial jets fell 45% between 1968 and 2014. Modern jets have better engines and are made of modern materials (carbon fibre is becoming more and more common). This means cheaper prices and longer distances for direct flights.

Buildings certainly do not look the same. New homes in the '60s were usually 1100 SF or smaller (at least in my area). Compare that to new homes now that are 2000 SF, much more efficient, and have a lot more modern conveniences. Now they're not filled with asbestos and lead paint.

And I could go on and on with food, clothing, and the rest.

Complexity

[fyngyrz]

Handling reflections perfectly is trivially easy with raytracers, and there have been plenty of real-time raytracers even for low-end consumer hardware in the 90's.

Even with a simplistic ray model, the more objects there are in the scene, the more complex the trace becomes. Until you hit an object, you don't know at what angle the ray diverges from that object; it may, for instance, go through several partially transparent objects before it reaches a light source, or it may reflect off of several items that are colored, or metallic (which, if done properly, introduces further complexities.)

The classic ray tracer has to examine every object N for every pixel generated M:. So the more objects and pixels there are, the longer it takes to render the scene. Complexity is straight-up NxM. This is without even thinking about things like soft shadows.

That is the bottleneck; and you're not going to solve it for all scenes just by throwing hardware at it.

You can solve it by simplifying the scene until the hardware can keep up, and as near as I can tell, that's what's being described here.

Or IOW, there's more hype here than truth.

the uneven future, since forever

[epine]

Or even 30 years ago ... well ray tracing was anyway.

Back in 1980, the University of Waterloo mathematics and computer science building had a locked public display case featuring artifacts from the senior-level computer graphics course, most of which involved ray tracing (standard chessboard-reflected-in-shiny-sphere kinds of things, but with the scenes aggressively simplified—like only three chess pieces of a dozen polygons each and the board reduced to sixteen squares).

What separated the great from the good was treatment of subtleties such as getting the specular highlighting right. I was young and naive and didn't have Wikipedia at my fingertips, so I don't recall much more.

They also had a public information kiosk back in 1980 which used some kind of polygon-fill graphics language to render an interactive dial-up information browser in all the best oversaturated colours. Just like the Internet, if the Internet consisted of exactly one host, and it was dead slow. But shiny! Everyone tried it out—for exactly three minutes (that would be about your third screen rendered).

They also had rooms full of card readers, which they couldn't bother themselves to eliminate until some fancy anniversary shindig a few years later.

In one terminal room you'd have IBM 3270 block-oriented displays, in the next you'd have the god-awful WIDJET terminals, in the next you'd have IBM PCs running APL (for statistics students), in the next you had Commodore SuperPETs, custom tweaked by the Computer Systems Group or related ecosystem (these people were later responsible for the Watcom C++ compiler).

WIDJET (Waterloo interactive direct job-entry terminals) was basically a JCL front-end with the ability to store about six whole files on non-card storage, where mostly you sat twiddling your thumbs waiting for your runs to pass through the job submission queue.

One such room was more advanced and you could *edit* your next assignment with your previous run still in the queue, but you had to sign up way ahead of time to actually get a seat in this room.

In the other room, you might submit a compile, sit there for ten minutes watching it bounce up and down the queue (upper-year jobs had priority) and then decide "oh, I did something wrong in the previous assignment" and you'd click "edit" on some previous source file, only to discover the message "job submission cancelled" popping up on the status line of your display and since you'd already spent years with the immense power of the TRS-80 or Apple II at your fingertips, you'd bash your head into the desk and wonder how you'd become mired in this technological institution of hazing, abuse, and mediocrity.

The SuperPETs were okay, but the custom Pascal did some kind of partial compile to catch syntax errors before running the interpreter. The error reporting was beyond horrible. 50% of all syntax errors were the same message: "syntax error near or before end of file." It was much like Donald Trump tweaking "WRONG!" at you if you got a single brace out of alignment.

"Sorry! I'll hunt through my entire program looking for the one I messed up." This worked for localized changes. But generally what you actually did after a big edit was added or removed braces at random until you found the problem through a tedious process of bisection (which, however, was an order of magnitude less tedious than dealing with WIDJET, even in the good room).

My first year at Waterloo was the biggest computer science mindfuck of my entire life.

Eventually I discovered you could kind of get into a groove with the IBM 3270 terminals—as archaic as they seemed—if you blinked longer than normal after each press of the ENTER key. Plus, these had chat, so you could send "nice sunrise" messages to all your friends at 05:00.

Fro