Cinematic Game Graphics

CowboyRobot writes "LucasArts engineer Nick Porcino has an article detailing what to expect from graphics in the next generation of game systems including the "influence of cinematic realtime rendering, the promise of advanced lighting techniques and high-dynamic range images, the uses of the rendering pipeline, and the future of multiprocessor-based rendering and advanced geometry." These will allow run-time rendering of high quality backgrounds and characters, ultimately resulting in games that are closer to full-blown Pixar animations, allowing better narratives and more immersive user experiences."

Comments

Which games have recent impressed you?

[adamgreenfield]

This article makes for interesting reading for developers and gamers alike.

On a more basic level, I can think of several games I have played that really impressed me graphicly. One that springs to mind (that got a mention in the article) was Quake III from id Software. Another game I remember being impressed with was WarCraft 3 from Blizzard (The way it let you angle the camara and sort of fly from an overhead view into tight third-person was awsome).

With all the cinimatics we're seeing in games (both for the computer and the console) I can only begin to imagine what the future holds for this industry, but I think gamers can be sure of one thing. They definatly have something to look forward to :)

Where are the "Sound Acceleration" cards?

[Jezral]

Off-topic, I know, but I've been wondering...

ok, so we've been able render Toy Story in real-time for a while...

But, where are the cards that can generate the sound of one arbitrary object hitting another? I don't just mean positional sound of pre-recorded samples, but really create the sounds from scratch (or an "audio-enabled model").

Re:Where are the "Sound Acceleration" cards?

[AKAImBatman]

You do realize that you're asking for what amounts to RayTracing for Audio? The idea of realtime raytracing has just barely begun to penetrate the research community, and they have to cheat like hell to make it work. Trying to calculate material properties, force applied, vibration vs. tearing, listener posistioning, doppler, etc. in realtime would amount to far more CPU power than is currently in the hands of the public.

Re:Where are the "Sound Acceleration" cards?

[Jerf]

But, where are the cards that can generate the sound of one arbitrary object hitting another? I don't just mean positional sound of pre-recorded samples, but really create the sounds from scratch (or an "audio-enabled model").

Not to put too fine a point on it, but... we don't know how to do that .

Even what you may have heard of is wild, wild cheating compared to what you describe.

I've given some (intelligent, educated) thought to this issue, and here's the problem. Light is adequately simulated with a line in the macroscopic world. It is, technically, a wave, but it is so high frequency that we can ignore it unless we're trying to simulate at quantum scales. This keeps the complexity down to merely polynomial, and some smart people have figured out how to shrink that polynomial down to a surprising degree.

Sound is not so friendly. Imagine some reasonably complex object, like, oh, a chair. We'll even cheat and make it all out of one material and go ahead and make it as geometrically simple as you like, as long as it's a chair, which is to say, at least three legs and a sitting surface, preferably with some sort of back support. Now, mentally give the chair a tap.

Now, odds are you're not too practiced at this sort of mental visualization, but here's what happens. For the sake of argument, let's tap it on the direct middle top of the sitting surface. Let's cheat some more and assume that one impact is what makes the sound, rather then an oscillation at the tap point. (See how much we're cheating, and we'll still end up with an uncomputable scenario.) So the tap radiates outward from there and starts wiggling the legs. The sound partially bounces off the legs, and goes back, and some of the sound wiggles into the legs. The sound bounces all around in the leg, and every time it bounces off of the edge, it loses some of the sound to the atmosphere and some of it bounces back. By the time it hits the bottom, it's bounced several times. (See, sound can't turn except at one of those boundaries, where it is essentially absorbed and re-emitted.)

Meanwhile, the same thing is happening in the back of the chair. Plus, on the sitting surface, we have reflections of reflections of reflections to deal with, and thanks to the wonders of resonance, we absolutely have to track each and every one of them until they hit a really low level. In a fraction of a second, we have hundreds upon hundreds of seperate waves to track, and they aren't even rays, they are "wavefronts".... imagine a wavefront hitting the edge of the chair at an angle, like an ocean wave bouncing off the beach. It doesn't bounce like a particle, it is two entirely new waves, the one that reflects and the one that continues on.

Basically, we can't even simulate this horrid simplification, the real world is even worse. Sound is highly, highly parallel. Ultimately, sound simulation is firmly exponential and the constants are very, very high. Maybe if those magical quantum computers come online we'll get this, but we'll quite possibly never get it with conventional technology; we're always going to have to cheat.

(One can try to imagine a transformation of the chair where the sound travels in a straight line in some space, but I'll be damned if I know what that actually looks like in real code, nor am I sure that it would be any easier to compute then a straight-forward simulation anyhow. Bright ideas in this regard should probably not be posted on Slashdot and saved for your PhD thesis in Mathematics/Physics/Computer Science; they'll all be waiting with bated breath.)

Re:Where are the "Sound Acceleration" cards?

[Anonymous Coward]

I respectfully disagree with the notion that dynamically generated sound is impossible to do with today's hardware. Yes, perfectly raytracing sound would be computationally infeasible, but guess what - we don't need to raytrace it. That's totally overkill, and no one would even notice.

For example, take a bullet impact. We have three sounds to deal with - the sound of the impact, the sound of the reverberation, and (depending on angle and material) the ricochet.

It should be a matter of simply mixing a "palette" of small sounds (metal impact 1, metal impact 2, metal impact 3, metal reverb 1, metal reverb 2, metal reverb 3, metal ricochet 1, metal ricochet 2, metal ricochet 3), depending on the "soundMix" properties of the shader. For a piece of iron, you would want a ping-y impact, followed by a clang-y reverb (how's that for scientific? :D). For a piece of tin, you want a clang-y impact with a ping-y reverb.

This brings up another interesting point - we can stretch our sound samples out by recording, say, one or two sounds per material, and just messing with the pitch and duration.

Once we've got our "sample" generated, we can use a few simple distance calculations to position the sound (stereo, or 3D surround) and adjust the volume for distance. Add a muffle filter if it's behind something.

In conclusion, it should be entirely possible to dynamically generate a broad range of sounds, without completely pissing away processor cycles. And it might even make games more interesting.

- GNU/Anonymous Coward

Re:Where are the "Sound Acceleration" cards?

[Directrix1]

Samples are a good analogy to Textures. 3-D positional audio is a good anology to the rendering of textured meshes, where EAX or something like what would compare to Pixel Shaders.

I don't think they need many textures/samples though. What they need is a more generic way to reproduce uniqueness using a small subset of samples. Such as having multiresolution layered textures or procedural textures, and increasing the number of simultaneous voices to allow for the combination of several samples to simulate several phases in an action at different intervals/frequencies/intensities.

Re:Storyline!

[MistaE]

I agree with ya dude, but one thing that must be taken into consideration is that developers can't be carried away with a game focusing on narrative too much. I really don't think that gamers would like to play "interactive movies" like Xenosaga (I enjoyed the game, but geeze, the cutscenes were way too long and plentiful).

The trick is going to be balancing the amount of graphical detail with story lines and such. We know that a game that combines the two in just the right amount is pretty damn rare, but I look forward to the days developers get it right.

Personally, tho, I feel that one of the more important aspects is game play. You can have a beautiful game with an interesting story, but if you can't even stand to work inside of the world in terms of control and rules, then what's the point?

Re:at the rate PC games are pushing the market

[ols22]

This is right on. Of course we should expect computing power to grow according to Moore's Law (although there was a recent article indicating this may fail soon for laptops: Wired).

But if one looks into Moore's Law for Software (Googled) you find a different analysis. In short it looks like algorithm development has lagged sufficiently behind the computational power.

So what does this mean for gaming? It seems developers are hanging onto old ideas and relying on the growth in proc speeds (and bus speed and etc. etc.) to enhance their graphics/rendering. Thus the improved visualization comes to the cost of the consumer by forcing them to purchase faster and faster computers. Now we can run the newest games on the tricked out settings only on the fastest/cutting edge computers. This trends seems to be getting worse recently indicating that Moore's Law for Gaming Software is indeed lagging Moore's Law for computing power. IMO...

Re:Please Don't

[onion2k]

But 10 years from now, when someone wants to create something that captures a similar subset of cool features (maybe a fun arcade-y dogfighting game a la Crimson Skies, maybe the new and revolutionary fighting game that introduces some unique quirk to make things fun), they're going to have a hell of a time competing visually in a market where 1,000,000 poly models require a single artist to work for almost a month to make a single character look halfway decent.

Most game models these days are modelled as high-poly originals and then reduced to low-poly in game assets. The time to make a 50 poly model or a 5000 poly model is roughly the same. Whats more, as the graphic power of the target machines is improving, so are the tools people use to make the games in the first place. We've got 3D scanners already.. why not scan the real car for Gran Turismo 29 and use that as a game model?

Re:And AI!

[Saville]

What do you want from your AI? Do you want it to have feelings? Do you want the shop keeper's child to have been murdered by goblins the night before so that he won't give you the discount on your sword that your charismatic attributes you spent your level-up points getting are useless? Or do you want predicable AI?

What do you mean by wanting better AI? If you mean you don't want your Peons to creat traffic jams for themselves when harvesting resources I agree because if I'm playing I told them to do something and they aren't doing it, but beyond that I'm not sure what you mean. Can you give an examples of where you want smarter AI to make the game more fun?

What games need...

[Anonymous Coward]

1. Better physics. An explosion should produce
dust and this dust should float in air and follow
air currents and obscure light. Me having a
crowbar means I should be able to make dents in
walls and (with enough persistence) climb those
walls. When I pick up a heavy object, I should
get tired and not be able to run as fast. The list
goes on.
2. Better AI. Many FPS games keep their baddies
dumb but make them harder to see to make it
challenging. This is no good. Do the opposite
and it gets more fun. The persistent universe
concept really isn't realized very well in games
that try, again for lack of good AI.
3. More realistic sound. It doesn't have to be
phonon-level realistic but it does have to
reflect the media it passes through AND where it
comes from. If I hit a guy in the neck it should
sound different from hitting his stomach. I am
not aware of any game that features realistic
echoes.
4. Much much higher polygon count. This is
especially evident when a game features vegetation.
Take Far Cry. It is supposed to be the newest
engine with all the goodies but you know what,
you'd never mistake any of the plants in that
game for a real plant. The leaves don't move with
the wind - that's physics lacking. Me not being
able to see twigs and identify trees by shape of
their leaves - that's polygon count lacking.
5. One of those days, someone will do smell. That
will revolutionize gaming if done right.
Unfortunately to do smell right you need to get
physics right first because smell diffuses and
rubs off.

So to conclude, wake me up when the above five
are taken care of. Until then, it'll be same
crappy experience and still everyone will go
oooh and aaah with littlest steps forward
(photon counting anyone).

Re:We already *have* Toy Story graphics

[moneymatteo]

We don't have real-time Toy Story graphics yet because there is no Real-Time Motion-Blur. Pause a dvd of toy story and look at a frame. Compare this to a paused frame on a video game and notice the difference. Until this technique is incorporated you will always notice the difference between CG for the cinema and video games.

Babble

[Caedar]

They talk about the next generation having 'Cinematic Graphics'. Isn't that what THIS generation of graphics cards claimed to have?

This seems bogus in many ways.

[Zobeid]

The hype this article lays down in the first page turned me off.

The next generation of console and home computer hardware is going to bring a revolutionary leap in available computing power; a teraflop (trillion floating-point operations per second) or more will be on tap from commodity hardware.

How much power will it draw? How are you going to cool it? The laws of physics appear to present certain obstacles, these are starting to become real problems. But even if you can make this kind of power happen in a game console -- will it make the game drastically better? Will it even make the graphics drastically better? I have doubts.

It looks to me like we've reached a point of diminishing returns with 3D graphics. Each new generation of hardware is resulting in less dramatic improvement to the images we're seeing. Continuing to throw more hardware at games and calling it a "revolution" will lead only to disappointment.

Our stories will have the potential for the same depth and sophistication as is expected today in a film or television show.

Ha, I say! Ha! This is the kind of drivel I've heard from game industry pundits going all the way back to the mid 1980s. Somehow it never seems to happen. We've got plenty powerful enough hardware today, and advanced enough AI algorithms, if only there was a serious push to use them. Yet, this article seems to be implying that a deeper and more sophisticated story is somehow tied to better graphics.

I was recently looking at screenshots from upcoming games: Everquest 2 and World of Warcraft. EQ2 definitely has highly advanced graphics, from a technical standpoint. Tons of polygons, massive detailed texturemaps, advanced lighting effects, yadda yadda. . . So why does WoW often look more attractive? I think it's because Blizzard focussed on art with a sense of style rather than flogging the technology.

Blizzard are also working hard to create a well-designed, well-balanced game that's fun to play. Sony, on the other hand, are bragging about their voice acting and how cinematic everything is. Is it a game, or is it a movie? I'd like to play a game, please. Maybe I'm just old-fashioned that way?

After the introduction the article dives into a lot of technical subjects that I'm not qualified to comment on. At the end it wraps it with a surprising admission. . .

The graphics revolution that is upon us will be a creative one; present work methods are too labor intensive to scale to the volume of data that we will need to create to support the medium.

The author implies that this is a problem to be overcome -- probably by borrowing techniques from film and television. I'm thinking instead: Maybe this is the point where we should take a step back and ask if we're even on the right path, if this is the direction videogames (and computer games) should even be going? Is this real progress?

Re:Storyline!

[a rabid platypus]

Good points! Perhaps I should clarify what I mean by "dynamic". Dynamic as in there is a war going on, the actions of my armies influence the direction of the war and the types of missions I can under take. Note there is a general plot: war; however, I can work within the war to influnce and change it's outcome. Just becasue the game changes according to my actions doesn't mean it doesn't have a theme or "plot". It just means I have more control over the eventual outcome. Games that make me run the same missions, over and over are quite frankly, boring. Replay value is nill, and there is no real sense of accomplishment. In the type of game I just described replay value would be pretty good, as there are many ways to win a war. An example of a game that is too well defined would be Mech Commander 2. Pretty and a good premise, but the same missions. It would be better if the missions were generated according to my actions, and if I could do things inside of missions that would affect future missions. Just becasue a game changes, doesn't mean it doesn't have a theme or an overall plot. It just means I can reach those end goals MY way.

Re:No thanks

[nothings]

He already commented on where he thinks game graphics are going at this year's GDC -- in short, I believe he's saying that 10 years more improvement should get us technologically there, but that there are severe problems with how we create the more detailed and larger quantity of content necessary cheaply enough.