Twilight of the GPU — an Interview With Tim Sweeney

cecom writes to share that Tim Sweeney, co-founder of Epic Games and the main brain behind the Unreal engine, recently sat down at NVIDIA's NVISION con to share his thoughts on the rise and (what he says is) the impending fall of the GPU: "...a fall that he maintains will also sound the death knell for graphics APIs like Microsoft's DirectX and the venerable, SGI-authored OpenGL. Game engine writers will, Sweeney explains, be faced with a C compiler, a blank text editor, and a stifling array of possibilities for bending a new generation of general-purpose, data-parallel hardware toward the task of putting pixels on a screen."

Comments

For once ...

[Qbertino]

For once I'm reading an 'xzy is going to die' article that doesn't sound like utter rubbish. Could it be that, for once, the one stating this actually knows what he's talking about?

My last custom realtime GPU was a Geforce Ti4200. I'm now using a Mac Mini with GT950. Mind you, Blender *is* quite a bit slower on the 950, even though it runs with twice the sysclock, but I'm not really missing the old Geforce. I too think it highly plausible that the GPU and the CPU merge within the next few years.

Re:For once ...

[Anonymous Coward]

On a day when Lehman Brothers and Merrill Lynch, the backbone of our economy, "died", to make these kinds of heartless statements is just pandering to the prejudices of the liberal elite. Instead of buying a Mac you should buy a Dell (which is far more "American" in the truest sense of the word) and instead of irresponsible talk of combining CPU and GPU you should keep them well separated for the good of our nation.

Our country is built on a strong consumer market; it is "progressives" like you who are causing this crisis, which might even result in a president with an Afro. Shame on you.

Re:For once ...

[Antitorgo]

Except that you like everyone else reads "CPU" in the article to mean the Intel/AMD CPU and not think of it as current gen GPUs that are almost capable of massive parallel execution of general purpose code. With the advent of Shaders, more processing was able to be offloaded to GPUs and over the next couple of GPU generations. So his idea is that we'll see less of the OpenGL/DirectX specific API calls and everything being done in CUDA/Shaders. That way the folks who write graphics engines aren't limited to the current SGI implementation (here's a set of vectors describing my object -- draw it) and we'll see different rendering engines based on Ray Tracing for example (or whatever other methods the engine writers want to do it).

This isn't anything new here -- he's basically saying what Intel has already said... You'll see less OpenGL/DirectX and more CUDA/Shader based implementations for rendering engines.

Re:For once ...

[vux984]

No, that can't be it. Know why? Because...why would you put more processing and thus more heat in one place that already has problems with that?

You mean how floating point units used to in a separate coprocessor?
Or how L2 cache used to be on external chips? (And in some cases was even upgradable.)
Or how modems used to have their own signal processors? But now most use the CPU.
Or how we're moving the memory controller into the CPU right now.

Hell, we've even stuck the majority of complete additional CPUs into the the CPU with our modern dual and quad core chips.

Apparently the author doesn't know much about computers.

Apparently you don't know much about computers either.

The entire history of the personal computer is been one long slide of functionality moving towards the CPU. Sure every now and then something new comes along being done by an add-on processor - like the numeric coprocessor for example.

Sure before the coprocessor you could accomplish the functionality of what a coprocessor does with an 'integer cpu', but a hardware optimized numerica coprocessor was a new feature, one that added tremendous floating point performance in dediated hardware. Within a couple CPU generations the coprocessor had been completely absorbed into the CPU.

The author is speculating that the GPU will see the same fate eventually. And he's probably right.

And why install an overkill graphics processing unit inside the processor if most people won't use it anyway?

Once upon a time people said that about numeric coprocessors. "Only a research scientist would need that!"

Re:For once ...

[ZorbaTHut]

Man, you've got some awful, awful arguments here.

Because...why would you put more processing and thus more heat in one place that already has problems with that?

For the same reason that your CPU isn't spread up among thirty chips distributed throughout your laptop: efficiency and cost. Making one chip is generally cheaper than making two, and the amount of bandwidth inside a single chip is massively higher than what you can do with a northbridge.

And why install an overkill graphics processing unit inside the processor if most people won't use it anyway?

Every latest-generation operating system provides a 3d accelerated desktop. Every latest-generation computer provides the hardware to use it. Programs are going to be taking more and more advantage of that feature.

And why attach it to a part that's waaaaaay harder to upgrade and usually either requires a reactivation or reinstalled of your OS?

See question 1.

And how much harder would graphics hardware driver updates be?

Not at all. For one thing, there wouldn't *be* graphics hardware - it'd be more of a vectorized coprocessor. For another thing, why *would* it be any harder? It's not like people are having horrible trouble updating their USB drivers, even if the USB controller is part of another, larger chip.

And would it overheat laptops a hell of a lot faster by putting more heat in one location? (spoiler alert: yes)

Obviously, if they took existing laptop designs, and slapped a bigger heatsource in the CPU, yes. I'm assuming that computer manufacturers aren't functionally retarded, and they wouldn't do that. (Well, maybe some would, but their computers aren't going to be stable anyway.)

And where would the VGA/DVI output go if there's no graphics card?

The same place it already goes on motherboards that have integrated graphics? It's not like "computers without dedicated graphics cards" is a new concept, unless you've been living in a cave for the last decade.

If you put it somewhere else then why move the graphics processor further away from the outputs?

See question 1. Also, "why not?" - it's not like that extra four inches is going to be a serious problem.

As near as I can tell, your argument comes down to the common logical fallacy:

"They *could* do X. But if they do it the *stupidest way possible*, X is a bad idea. Therefore, X is a bad idea."

When determining whether something is a good idea or not, you have to assume it's going to be done well. If the person in charge of integrating CPUs and GPUs is anything less than a complete unalloyed moron, they'll have come up with solutions to all of those issues of yours.

Re:For once ...

[Goldberg's Pants]

what's a few more inches?

The difference between "Is it in yet?" and "Dear god you're ripping me in two!"

Re:For once ...

[Mad Merlin]

So wait, your integrated graphics is *slower* than a low end discrete graphics card that's now 6 generations (or ~6 years) behind... and you find that acceptable?

At the current pace, it'll take them a decade to match a high end card from today with an integrated card, and video cards aren't standing still in the mean time.

You can point at the current integrated market for video cards, but that's not terribly interesting due to the fact that a) current integrated video cards barely do more than send signals to monitor(s), and (as a corollary to a) b) current integrated video cards aren't very fast.

Wake me up when integrated video cards are faster than discrete video cards.

Re:For once ...

[Miamicanes]

> Or how modems used to have their own signal processors? But now most use the CPU.

Welllll... I'd say the move to HSP modems took place more because the ascent of DSL and cable internet relegated modems to the status of, "nice to have if I happen to need it once in a blue moon to send a fax or dialup in the middle of nowhere at some point over the next 2-4 years." Remember all those articles 3-5 years ago about how host signal processing absolutely DESTROYS CPU performance because it demands constant attention from the CPU, and the software overhead of having to keep stopping to service the modem caused the computer to run at least 20-30% slower? Well, not much has changed, except now with a multi-core CPU it can kill the performance of just ONE core instead of bringing the whole computer to its knees. But even with multicore CPUs, I can guarantee that if modems were still the primary way people got online, there would definitely be a thriving market for "performance" modems that offloaded at LEAST the signal-processing functions to a real DSP (like the Lucent "semi-Winmodems", that actually gave users the best of both worlds... offloading the stuff that really dragged the CPU down to its own DSP, but doing things like compression and error-correction that could be handled in discrete batches faster than even dedicated hardware could achieve).

There's another thing to remember about discrete chips... in the early 3dfx days, the mainstream CPU makers (Intel, AMD, and Cyrix) had ZERO interest in giving even the slightest attention to 3D graphics. Unless you're IBM (who wasn't interested in 3D, either), building CPUs is probably way beyond your company's capabilities. HOWEVER, designing a 3D graphics chip with the complexity of the first ones used by 3dfx IS within the capabilities of a well-funded design company with the connections to get it manufactured. It doesn't even need a fab with the capabilities of one owned by Intel, AMD, etc. So discrete 3d cards were an elegant way to sidestep the deadweight lack of interest on the CPU side by shifting it to a chip that smaller companies could design and build. Now that "the big guys" have turned their attention to it, the smaller players don't have a prayer (ergo, the merger mania among CPU/mobo chipset makers and graphics chip makers).

The same observation can be made regarding cache and memory controllers. In the First Pentium Era, volume manufacturers like Compaq (and their comrades at arms, Intel & AMD) regarded cache as a luxury the unwashed masses could live without, even if it only saved $5 and cut the effective performance in half. Hey, consumers only look at that "Mhz" number, anyway... Fortunately, performance-oriented mobo makers were able to take matters into their own hands, and once again do an end run around the CPU vendors' sloth and put cache directly on the mobo. Once CPU makers decided cache mattered, and put lots of it on-die, the marginal benefit of putting more, relatively expensive tertiary cache on the motherboard diminished. As for memory controllers, they got moved into the CPU because it was the only way to reliably achieve increased memory bandwidth (designing a 32-bit parallel interface for ANYTHING that has to run at 400+ MHz and communicate across traces on a circuit board is a hardcore engineering challenge; Serial is cheaper to implement and can be faster overall than a simpler parallel solution, but there's a point where you can't shove the bits any faster, and the only way to increase bandwidth is to go parallel. It's not a coincidence that PCI Express video cards communicate 16 bits at a time, but even the fastest fibre-channel disk or network interface is happy with a single bit.

The sad irony, though, is that 5 years from now, games will probably have graphics about as good as you can get from the best and most expensive SLI solutions money can buy today... but overall performance will probably be less consistent (ie, if Windows decides that it might be a good time to reorganize its temp directory while y

Re:For once ...

[kestasjk]

The key difference is that L2 cache, memory controllers, FPUs, they all need to interact closely with the CPU.
GPUs generally just take data from the CPU and get rendering, so unless they start sending data back to the CPU much more there's no real reason for them to merge.

Another difference; graphics cards benefit from being updated every now and then. Another; GPUs use a lot of transistors, and because of their parallel nature the more transistors the better, it's not something that will get so small that it can be tacked onto the CPU as a nice extra. There are probably more.

Re:For once ...

[dnoyeb]

Not if AMD can help it. Obviously this is not their path or they would not have purchased ATI. They sell two products, one of them is enormously expensive, the other is reasonable. I cant see why any graphics card manufacturer would give up all that profit!?

Also consider that GPU upgrades are much more frequent that CPU upgrades. I don't think the dollars favor integration of the two. I don't think there is enough competition for force it either.

The only company that may achieve this is VIA. For completely different reasons.

Re:For once ...

[juiceboxfan]

It's very slightly (pennies) cheaper to put one chip on the motherboard rather than two. It's MUCH more expensive to merge two big CPU/GPU type chips into one. Manufacturing flaws become more common fast with bigger chips.

I don't think your estimate is correct for packaging and placing a single chip vs. two chips but in high volume manufacturing even pennies make all the difference. What about the cost of the second fan and other infrastructure for the GPU? There is also the issue of real estate - two chips take up more room than one so your etch routing becomes more of a challenge requiring smaller etch geometries resulting in a more expensive PCB.

If we could break CPUs into pieces now we'd do it, both for that and heat reasons. We can't because all the parts currently located in a CPU need to talk to each other very fast. The GPU is something that usually doesn't need to talk to the CPU much. So it's separate.

No, it will always be cheaper to integrate at the chip level. Look at system prices, dual core is cheaper than dual CPU (if you can even find one these days) of similar performance. The trend in embedded systems (even more cost sensitive than PCs) is to integrate everything into a single package (SOC). As others have pointed out cache and the FPU were once separate chips from the CPU they are now all integrated into one package.

Re:For once ...

[crhylove]

Your mom's basement sounds WAY more fun than mine.

Wrong summary

[Anonymous Coward]

He talks about the impending fall of the fixed function GPU.

I hope not!

[jhfry]

I just browsed the article and it looks like what he's saying is that as GPU's become more like highly parallel cpu's we will begin to see API's go away in favor of writing compiled code for the GPU itself. For example, if I want to generate an explosion, I could write some native GPU code for the explosion, and let the GPU execute this program directly... rather than being limited to the API's capabilities.

So essentially, we will go back to game developers needing to make hardware specific hacks for their games... some games having better support for some cards, etc.

API's are there for a reason... lets keep em and just make them better.

Re:I hope not!

[MarkvW]

I read it differently than you did. He's projecting a world where everything is standardized and faster (less 'bus plumbing' GPUs). In such a world you won't need APIs because you'll have libraries that you can include in the compile process.

APIs reduce code bulk at the cost of reduced code speed, don't they?

Re:I hope not!

[Anonymous Coward]

In such a world you won't need APIs because you'll have libraries that you can include in the compile process.

A library you include in the compile process is an implementation of an API.

APIs reduce code bulk at the cost of reduced code speed, don't they?

No.

Re:I hope not!

[GleeBot]

I read it differently than you did. He's projecting a world where everything is standardized and faster (less 'bus plumbing' GPUs). In such a world you won't need APIs because you'll have libraries that you can include in the compile process.

No, you won't need APIs because you'll have a standard instruction set. Current graphics APIs are oriented around things like pushing triangles and running shaders on them. Why bother with the complexity of the graphics APIs, if you can create a standardized instruction set for GPUs?

We essentially already have that, but it's wrapped in the paradigm of shader programming. Stop treating the GPU as a graphics processor, and as just this really parallel coprocessor, and you get what Sweeney is talking about.

The basic point is that GPUs aren't really for graphics anymore, and as we move away from a graphics fixation, we'll come to realize that they're just specialized computers. Using graphics APIs to program them would be as stupid as using OpenGL to write a word processor. Sure, with modern capabilities, it's actually possible, but it's a bad conceptual fit.

Software rendering on the GPU

[KalvinB]

Somehow I don't think there's going to be lack of a standardized API much like OpenGL or DirectX even if it's possible to write code for the GPU as easily as the CPU.

The APIs at the most basic level allow Joe and Bob to build their own system, throw whatever graphics card they can find in it and have the same game run on both systems.

As soon as you start coding for a specific GPU you're going to be treating PCs like consoles. I don't care to have to buy multiple graphics cards to play various games.

APIs are for compatibility and general purpose use. The option of flexibility is great for industry use where you're running rendering farms all running identical hardware and custom software.

Re:Software rendering on the GPU

[Hortensia Patel]

You're missing the point entirely. GPUs in the near future will essentially be general-purpose massively-multicore processors. (Not entirely so, but close enough for "GPU" to become a misnomer.)

You don't need separate binaries for Intel versus AMD processors, and you don't have to write to an API to achieve portability between such different CPUs. Given sufficiently general hardware, there's no inherent reason why GPUs should be any different. There will still be a place for libraries, but their role will be more like C libraries than like GL/DX.

If we are going back to the "old" days...

[Brynath]

If we are going back to the "old" days...

Why can't we skip all this OS nonsense, and just boot the game directly?

After all that will make sure that you get the MOST out of your computer.

Re:For once Ivan Sutherland Wheel of Re-invention

[elwinc]

See Ivan Sutherland's Wheel of Reincarnation. The idea is that CPUs get faster and graphics move there; then busses get faster and graphics moves to dedicated hardware; rinse and repeat. http://www.anvari.org/fortune/Miscellaneous_Collections/56341_cycle-of-reincarnation-coined-by-ivan-sutherland-ca.html

Re:Um, says who? I don't see it at all

[GameMaster]

And, I would argue that even that part of his statement is absurd. Most game developers don't even want to have to write their own game engines much less do it from scratch. Even the ones that do write engines (id, Epic, etc.) will, for the most part, never give up their graphics APIs. APIs don't just give programmers access to specialized hardware, they also provide the developer with a raft of low level tools that he/she has no, reasonable, justification or wanting to spend the time re-developing. What will happen, is that APIs like Direct3D and OpenGL will change (as they have been all along) and simplify, as they won't have to handle all the quirky hardware anymore. The idea that engine writers will completely throw out APIs and will go back to writing everything from scratch in C/C++ is almost as absurd as suggesting that they'll go back to writing everything in Assembly language.

Re:Obvious

[Hortensia Patel]

gpu's aren't really parallel in that [traditional multithreaded] sense, they are parallel in the SIMD sense.

Actually, they're somewhere in between. Some current hardware can reallocate individual processors between fragment and vertex processing depending on the current workload profile. Even at the level of an individual processor lots of "threads" may be running simultaneously; this is to hide latency when a shader program blocks on memory (texture or framebuffer) access.

If you look at NV's descriptions of their 8xx-series drivers, they talk about *hundreds* of threads in flight at any given time. These aren't threads in the classical sense - there's no preemption, for a start - but they're much, much more advanced than SIMD-style "apply this instruction to all these values" parallelism.

Re:Tim sweeny said the same thing 10 years or so..

[blahplusplus]

"Take a 1999 interview with Gamespy, for instance, in which he lays out the future timeline for the development of 3D game rendering that has turned out to be remarkably prescient in hindsight:

2006-7: CPU's become so fast and powerful that 3D hardware will be only marginally beneficial for rendering, relative to the limits of the human visual system, therefore 3D chips will likely be deemed a waste of silicon (and more expensive bus plumbing), so the world will transition back to software-driven rendering."

Nuff said.

As Good as it Gets

[Sponge Bath]

...a C compiler, a blank text editor, and a stifling array of possibilities

Assuming he means Emacs, then this is the way God intended it to be.

Re:Tim sweeny said the same thing 10 years or so..

[blahplusplus]

"Ok so you state that memory bandwidth requirements for GPUs are off the charts. Where do you propose to get more memory bandwidth than on the CPU itself? Seems to me if you want memory bandwidth there is no better place to be than on the cpu die..."

Again you're missing the point, "the jack of all trades, master of none" problem, not to mention the space requirements. GPU's complexity is nothing like the old style co-processor units that were integrated into the core. They require ridiculous amounts of cutting edge ram to get that kind of performance, and they need a lot of ram to output the results of those calculations.

I don't see CPU's integrating 512MB to 2GB of ram in the near term future given heat and die size considerations, and we haven't even touched the extremely low bandwidth between modern cpu's and main memory in PC's (which is much much less then a modern GPU).

The GPU will play it's part for as long as is necessary. I don't rule out that perhaps one day it will be technically feasable but it is nowhere near that day, it's at least a decade or more away.

We've seen this time and time again, processors go through evolutions of integrating and seperating. We went from mainframes to PC's and with the net 'back to mainframes' but notice how each device play's their role, each one didn't totally obsolete the other, they just have become more specialized at their tasks.

Tim Sweeney has vested interest for this

[jokkebk]

It's also interesting to note that the guy being interviewed is in the business of making 3D engines.

Now ask the question: would 3D engine makers perhaps have something to gain if OpenGL and DirectX would be scrapped, as the interview suggests?

Most game dev labs wouldn't have the resources to build their own engines from the scratch using a C++ compiler, making them to - wait for it - licence a 3D engine like the one this guy is selling.

So in summary, the article paints a picture from the future which would be very beneficial to interviewee, so I'd take it with a grain of salt. Either we'd get some de facto 3D engines replacing OpenGL and DirectX, or the game developers will waste time recreating each new graphics technology advancement into their own engines.