Twilight of the GPU — an Interview With Tim Sweeney

cecom writes to share that Tim Sweeney, co-founder of Epic Games and the main brain behind the Unreal engine, recently sat down at NVIDIA's NVISION con to share his thoughts on the rise and (what he says is) the impending fall of the GPU: "...a fall that he maintains will also sound the death knell for graphics APIs like Microsoft's DirectX and the venerable, SGI-authored OpenGL. Game engine writers will, Sweeney explains, be faced with a C compiler, a blank text editor, and a stifling array of possibilities for bending a new generation of general-purpose, data-parallel hardware toward the task of putting pixels on a screen."

Comments

For once ...

[Qbertino]

For once I'm reading an 'xzy is going to die' article that doesn't sound like utter rubbish. Could it be that, for once, the one stating this actually knows what he's talking about?

My last custom realtime GPU was a Geforce Ti4200. I'm now using a Mac Mini with GT950. Mind you, Blender *is* quite a bit slower on the 950, even though it runs with twice the sysclock, but I'm not really missing the old Geforce. I too think it highly plausible that the GPU and the CPU merge within the next few years.

Re:For once ...

[Anonymous Coward]

On a day when Lehman Brothers and Merrill Lynch, the backbone of our economy, "died", to make these kinds of heartless statements is just pandering to the prejudices of the liberal elite. Instead of buying a Mac you should buy a Dell (which is far more "American" in the truest sense of the word) and instead of irresponsible talk of combining CPU and GPU you should keep them well separated for the good of our nation.

Our country is built on a strong consumer market; it is "progressives" like you who are causing this crisis, which might even result in a president with an Afro. Shame on you.

Re:For once ...

[Facegarden]

For once I'm reading an 'xzy is going to die' article that doesn't sound like utter rubbish.

Look, over there! An iPod killer!
*giggles*
-Taylor

Re:For once ...

[ILuvRamen]

No, that can't be it. Know why? Because...why would you put more processing and thus more heat in one place that already has problems with that? And why install an overkill graphics processing unit inside the processor if most people won't use it anyway? And why attach it to a part that's waaaaaay harder to upgrade and usually either requires a reactivation or reinstalled of your OS? And how much harder would graphics hardware driver updates be? And would it overheat laptops a hell of a lot faster by putting more heat in one location? (spoiler alert: yes) And where would the VGA/DVI output go if there's no graphics card? If you put it somewhere else then why move the graphics processor further away from the outputs? These are all questions that pretty much make his ridiculous idea incorrect. Apparently the author doesn't know much about computers.

Re:For once ...

[aliquis]

Not that weird considering that's how it all was done before ... Even on the Amiga you don't have that much to help you get things done once you don't draw windows in Workbench.

The obvious benefit is of course that you do only what you want/need to do and not a bunch of other stuff.

I doubt we'll see specialized chips (in the same physical chip or not remains to be seen I guess) removed totally though. It's not like someone is saying "oh well let's remove this special purpose stuff which can run a few other instructs really fast and only focus on x86", no matter how it's done we'll probably see much better performance for useful instructions. So I don't know if it makes sense to say that the GPU will go away, get integrated, reimplemented, transformed for more general work, blended with a general purpose processor, transformed, whatever, yes, of course it will. But why getting rid of specialized units which do their job very well?

Re:For once ...

[juiceboxfan]

No, that can't be it. Know why? Because...why would you put more processing and thus more heat in one place that already has problems with that?

Cost. It's cheaper to build and install, on the MB, one chip with one die than it is to build two chips/dies on two boards (not to mention bypass caps, fans, RAM, etc.). With combined functions it may be that the one chip produces less heat than the total of the two, although it is all in one spot.

And why install an overkill graphics processing unit inside the processor if most people won't use it anyway?

Maybe two CPU grades? One with full graphics capabilities and the other with basic graphics (could be the same die one passes all graphics tests at full speed the other doesn't).

And where would the VGA/DVI output go if there's no graphics card? If you put it somewhere else then why move the graphics processor further away from the outputs?

You are already going to put a (relitively) long cable from the connector to the monitor, what's a few more inches?

Re:For once ...

[Antitorgo]

Except that you like everyone else reads "CPU" in the article to mean the Intel/AMD CPU and not think of it as current gen GPUs that are almost capable of massive parallel execution of general purpose code. With the advent of Shaders, more processing was able to be offloaded to GPUs and over the next couple of GPU generations. So his idea is that we'll see less of the OpenGL/DirectX specific API calls and everything being done in CUDA/Shaders. That way the folks who write graphics engines aren't limited to the current SGI implementation (here's a set of vectors describing my object -- draw it) and we'll see different rendering engines based on Ray Tracing for example (or whatever other methods the engine writers want to do it).

This isn't anything new here -- he's basically saying what Intel has already said... You'll see less OpenGL/DirectX and more CUDA/Shader based implementations for rendering engines.

Re:For once ...

[vux984]

No, that can't be it. Know why? Because...why would you put more processing and thus more heat in one place that already has problems with that?

You mean how floating point units used to in a separate coprocessor?
Or how L2 cache used to be on external chips? (And in some cases was even upgradable.)
Or how modems used to have their own signal processors? But now most use the CPU.
Or how we're moving the memory controller into the CPU right now.

Hell, we've even stuck the majority of complete additional CPUs into the the CPU with our modern dual and quad core chips.

Apparently the author doesn't know much about computers.

Apparently you don't know much about computers either.

The entire history of the personal computer is been one long slide of functionality moving towards the CPU. Sure every now and then something new comes along being done by an add-on processor - like the numeric coprocessor for example.

Sure before the coprocessor you could accomplish the functionality of what a coprocessor does with an 'integer cpu', but a hardware optimized numerica coprocessor was a new feature, one that added tremendous floating point performance in dediated hardware. Within a couple CPU generations the coprocessor had been completely absorbed into the CPU.

The author is speculating that the GPU will see the same fate eventually. And he's probably right.

And why install an overkill graphics processing unit inside the processor if most people won't use it anyway?

Once upon a time people said that about numeric coprocessors. "Only a research scientist would need that!"

Re:For once ...

[ZorbaTHut]

Man, you've got some awful, awful arguments here.

Because...why would you put more processing and thus more heat in one place that already has problems with that?

For the same reason that your CPU isn't spread up among thirty chips distributed throughout your laptop: efficiency and cost. Making one chip is generally cheaper than making two, and the amount of bandwidth inside a single chip is massively higher than what you can do with a northbridge.

And why install an overkill graphics processing unit inside the processor if most people won't use it anyway?

Every latest-generation operating system provides a 3d accelerated desktop. Every latest-generation computer provides the hardware to use it. Programs are going to be taking more and more advantage of that feature.

And why attach it to a part that's waaaaaay harder to upgrade and usually either requires a reactivation or reinstalled of your OS?

See question 1.

And how much harder would graphics hardware driver updates be?

Not at all. For one thing, there wouldn't *be* graphics hardware - it'd be more of a vectorized coprocessor. For another thing, why *would* it be any harder? It's not like people are having horrible trouble updating their USB drivers, even if the USB controller is part of another, larger chip.

And would it overheat laptops a hell of a lot faster by putting more heat in one location? (spoiler alert: yes)

Obviously, if they took existing laptop designs, and slapped a bigger heatsource in the CPU, yes. I'm assuming that computer manufacturers aren't functionally retarded, and they wouldn't do that. (Well, maybe some would, but their computers aren't going to be stable anyway.)

And where would the VGA/DVI output go if there's no graphics card?

The same place it already goes on motherboards that have integrated graphics? It's not like "computers without dedicated graphics cards" is a new concept, unless you've been living in a cave for the last decade.

If you put it somewhere else then why move the graphics processor further away from the outputs?

See question 1. Also, "why not?" - it's not like that extra four inches is going to be a serious problem.

As near as I can tell, your argument comes down to the common logical fallacy:

"They *could* do X. But if they do it the *stupidest way possible*, X is a bad idea. Therefore, X is a bad idea."

When determining whether something is a good idea or not, you have to assume it's going to be done well. If the person in charge of integrating CPUs and GPUs is anything less than a complete unalloyed moron, they'll have come up with solutions to all of those issues of yours.

Re:For once ...

[Goldberg's Pants]

what's a few more inches?

The difference between "Is it in yet?" and "Dear god you're ripping me in two!"

Re:For once ...

[Mad Merlin]

So wait, your integrated graphics is *slower* than a low end discrete graphics card that's now 6 generations (or ~6 years) behind... and you find that acceptable?

At the current pace, it'll take them a decade to match a high end card from today with an integrated card, and video cards aren't standing still in the mean time.

You can point at the current integrated market for video cards, but that's not terribly interesting due to the fact that a) current integrated video cards barely do more than send signals to monitor(s), and (as a corollary to a) b) current integrated video cards aren't very fast.

Wake me up when integrated video cards are faster than discrete video cards.

Re:For once ...

[Anonymous Coward]

A also stands for asshole. D stand for dick. And dicks want to fuck all the pussies, but assholes just want to go around shitting on everybody. But you know what? Dicks also fuck assholes.

Re:For once ...

[Grishnakh]

The same place it already goes on motherboards that have integrated graphics? It's not like "computers without dedicated graphics cards" is a new concept, unless you've been living in a cave for the last decade.

I don't think integrating the graphics into another chip is even a new concept; isn't that what a lot of low-end Intel chipsets do? Integrate an Intel GPU into the chipset (northbridge, etc.)? It's not that big a step to go from there to integrating it into the CPU itself, just as they're already moving the memory controllers into the CPU (AMD has done this for quite a while now).

Re:For once ...

[onefriedrice]

"Only a research scientist would need that!"

Only a research scientist does need that. Meanwhile, uninformed consumers are being suckered into buying way more than they need to check email and type their documents.

Nevertheless, your point is taken. Personally, I'm not adverse to this type of change, as long as it is a real improvement. I just hope that there will still be options (Celeron) for those who don't need the extras, but I'm confident the market will make sure this is the case.

Re:For once ...

[springbox]

That would be great. Funny to see "software renderers" making a comeback in the future.

Re:For once ...

[Miamicanes]

> Or how modems used to have their own signal processors? But now most use the CPU.

Welllll... I'd say the move to HSP modems took place more because the ascent of DSL and cable internet relegated modems to the status of, "nice to have if I happen to need it once in a blue moon to send a fax or dialup in the middle of nowhere at some point over the next 2-4 years." Remember all those articles 3-5 years ago about how host signal processing absolutely DESTROYS CPU performance because it demands constant attention from the CPU, and the software overhead of having to keep stopping to service the modem caused the computer to run at least 20-30% slower? Well, not much has changed, except now with a multi-core CPU it can kill the performance of just ONE core instead of bringing the whole computer to its knees. But even with multicore CPUs, I can guarantee that if modems were still the primary way people got online, there would definitely be a thriving market for "performance" modems that offloaded at LEAST the signal-processing functions to a real DSP (like the Lucent "semi-Winmodems", that actually gave users the best of both worlds... offloading the stuff that really dragged the CPU down to its own DSP, but doing things like compression and error-correction that could be handled in discrete batches faster than even dedicated hardware could achieve).

There's another thing to remember about discrete chips... in the early 3dfx days, the mainstream CPU makers (Intel, AMD, and Cyrix) had ZERO interest in giving even the slightest attention to 3D graphics. Unless you're IBM (who wasn't interested in 3D, either), building CPUs is probably way beyond your company's capabilities. HOWEVER, designing a 3D graphics chip with the complexity of the first ones used by 3dfx IS within the capabilities of a well-funded design company with the connections to get it manufactured. It doesn't even need a fab with the capabilities of one owned by Intel, AMD, etc. So discrete 3d cards were an elegant way to sidestep the deadweight lack of interest on the CPU side by shifting it to a chip that smaller companies could design and build. Now that "the big guys" have turned their attention to it, the smaller players don't have a prayer (ergo, the merger mania among CPU/mobo chipset makers and graphics chip makers).

The same observation can be made regarding cache and memory controllers. In the First Pentium Era, volume manufacturers like Compaq (and their comrades at arms, Intel & AMD) regarded cache as a luxury the unwashed masses could live without, even if it only saved $5 and cut the effective performance in half. Hey, consumers only look at that "Mhz" number, anyway... Fortunately, performance-oriented mobo makers were able to take matters into their own hands, and once again do an end run around the CPU vendors' sloth and put cache directly on the mobo. Once CPU makers decided cache mattered, and put lots of it on-die, the marginal benefit of putting more, relatively expensive tertiary cache on the motherboard diminished. As for memory controllers, they got moved into the CPU because it was the only way to reliably achieve increased memory bandwidth (designing a 32-bit parallel interface for ANYTHING that has to run at 400+ MHz and communicate across traces on a circuit board is a hardcore engineering challenge; Serial is cheaper to implement and can be faster overall than a simpler parallel solution, but there's a point where you can't shove the bits any faster, and the only way to increase bandwidth is to go parallel. It's not a coincidence that PCI Express video cards communicate 16 bits at a time, but even the fastest fibre-channel disk or network interface is happy with a single bit.

The sad irony, though, is that 5 years from now, games will probably have graphics about as good as you can get from the best and most expensive SLI solutions money can buy today... but overall performance will probably be less consistent (ie, if Windows decides that it might be a good time to reorganize its temp directory while y

Re:For once ...

[GleeBot]

"Only a research scientist would need that!"

Only a research scientist does need that. Meanwhile, uninformed consumers are being suckered into buying way more than they need to check email and type their documents.

Or, say, play Quake.

The nice thing about functionality moving into the CPU standard is that it opens up that functionality for a lot more applications than originally intended. FPUs may have mainly been for scientific work early on, but because it's essentially ubiquitous, now a vast array of programs do floating point computation... probably including your word processor.

The main reason I might have my doubts about this is that graphics performance has been advancing faster than CPU performance for a while. If that trend continues, I'm not sure people would want to be tied down to a single unit for a while. People on the cutting edge replace their CPUs a lot less often than their GPUs. In the long run, it's probably inevitable.

Re:For once ...

[kestasjk]

The key difference is that L2 cache, memory controllers, FPUs, they all need to interact closely with the CPU.
GPUs generally just take data from the CPU and get rendering, so unless they start sending data back to the CPU much more there's no real reason for them to merge.

Another difference; graphics cards benefit from being updated every now and then. Another; GPUs use a lot of transistors, and because of their parallel nature the more transistors the better, it's not something that will get so small that it can be tacked onto the CPU as a nice extra. There are probably more.

Re:For once ...

[dnoyeb]

Not if AMD can help it. Obviously this is not their path or they would not have purchased ATI. They sell two products, one of them is enormously expensive, the other is reasonable. I cant see why any graphics card manufacturer would give up all that profit!?

Also consider that GPU upgrades are much more frequent that CPU upgrades. I don't think the dollars favor integration of the two. I don't think there is enough competition for force it either.

The only company that may achieve this is VIA. For completely different reasons.

Re:For once ...

[MindlessAutomata]

Why is he being modded down? It's a joke, people.

Re:For once ...

[LoRdTAW]

He does make a good point and its a throwback to the old days before GPU's existed. Look back to the days of Doom or Duke3D, You had to hand code and entire rendering engine for the game. No API's or 3D hardware was available, only a single CPU and a frame buffer and compiler. With current and upcoming technology like multi-core and stream processors, there will be plenty of new options for developers to take advantage of. IBM Cell, Intel's Terrascale and Larabee are three of those technologies but I still highly doubt the GPU is done for yet.

3D rendering isn't the only thing going on in games or other programs. You have physics which is used in both games and simulators. We also have AI, which in games is still severely lacking. 3D sound is still simply how far is player from object and attenuate the sound as necessary. So those areas could definatly use those multi core CPU's or Stream processors.

We cant forget about really exciting stuff like real time ray tracing that is well suited for multi core and stream processing but its ways off. I don't think the GPU will disappear in the next five years but I do see it evolving to adapt to new rendering technologies. Instead of a very discreet GPU we will have a very fast accelerator chip that can do more general work. It not only will handle the 3D rendering pipe but also lend itself to tasks like video processing, DSP, audio processing and other compute intense tasks like physics. It is already happening with Nvidia's CUDA and ATI's Stream SDK. We still need a general purpose CPU to manage the OS and I/O but like the IBM cell it could very well move on die. Multi-core X86 CPU's are not the future. Instead we need one or two cores to manage resources and run legacy code.Then you let the stream processors (for lack of a better term) do all the compute intense dirty work. Hell we could even get rid of X86 and go with a more efficient and compact CPU like ARM or something else entirely.

Funny as I am typing this I realize I am pretty much describing the IBM cell processor and Intel Larabee. But still I doubt developers will be the ones holding the ball for interfacing with this new hardware. Hardware vendors and developers will (or rater should) come together and standardize a new set of API's and tools to deal with the new tech. If developers have a breakthrough that is better than ray tracing, they will definitely have the hardware to do it. Welcome to the future.

Re:For once ...

[twotailakitsune]

The AMD 780 chipset does a good job with decoding video (blueRay, and h264), and run most games with there mid settings. It runs better then my high end, at the time, 2004 video card. It is not that they can't do it right. People have not been pushing them to. Now with the high use of video, people who never not use motherboard video are asking for more, and they are getting it.

Re:For once ...

[Anonymous Coward]

Apparently the author doesn't know much about computers.

From Tim Sweeney's wikipedia page:

Tim Sweeney is a computer game programmer and the founder of Epic Games, and is best known for his work on ZZT and the Unreal engine.

Sweeney established Epic as a shareware company while he was a student majoring in mechanical engineering at the University of Maryland.

From your wikipedia page:

No content.

Hmmmmmmm.... A mechanical engineer versus a guy named after a noodle. I wonder who knows more about computers...?

(Seriously, mods, the parent got modded Insightful? The parent seems to know nothing about electrical engineering. Mods: if you know nothing about a subject, don't mod posts about that subject!!!)

Re:For once ...

[ceoyoyo]

It's very slightly (pennies) cheaper to put one chip on the motherboard rather than two. It's MUCH more expensive to merge two big CPU/GPU type chips into one. Manufacturing flaws become more common fast with bigger chips.

If we could break CPUs into pieces now we'd do it, both for that and heat reasons. We can't because all the parts currently located in a CPU need to talk to each other very fast. The GPU is something that usually doesn't need to talk to the CPU much. So it's separate.

Re:For once ...

[Dahamma]

I'm sorry to be blunt, this post is almost entirely inaccurate! Informative, jeesh.

1) There is no GT950 - Mac Minis use the Intel GMA 950.

2) the GMA 950 has NOTHING to do with merging the CPU and GPU - it merges the motherboard chipset with the GPU.

3) the GMA 950 is the same old special-purpose GPU concept as anything from NVidia or ATI (er, AMD) - just slower and using system memory instead of dedicated RAM.

The article is talking about rendering graphics on a high-performance, parallelized general purpose processor, not a crappy GPU embedded on the motherboard. Think a future generation of the Sony Cell architecture, not the Intel GMA!

Re:For once ...

[amdpox]

Yes, it's much better to be told "you're not allowed to do that" than "you are not being allowed to be doing that".

Re:For once ...

[Bill, Shooter of Bul]

double total_profit = profit_per_item * number_of_items_sold;

Re:For once ...

[TwistedSymmetry]

That would be great. Funny to see "software renderers" making a comeback in the future.

Ugh, software rendering? Seriously? *Eyes begin to bleed at the mere thought* ;)

Re:For once ...

[famebait]

Doesn't matter. There are basically 2 big point here:

1) The special-puropose GPU will morph into a more generalized co-processor for handling all sorts of massively parallel stuff.
This is just bleeding obvious, and I can prove that, because it is already happening. As more and more of it becomes programmable, it only makes sense that the built-in rendering microcode is replaced with libraries that ou may or may not chose to hack yourself.

2) Once the generalization is done, it will make sense to merge the two processors. Might sound weird, but we are already doing multicore designs in stead of separate CPUs, which would be the logical choice if density were such an issue. Turns out the benefits outweigh the cost of advanced cooling.

You are right, though, that it will not be a small "nice extra" tacked onto the CPU. It will be a very large part of thte CPU, and of its total working capacity.

Re:For once ...

[Guspaz]

If the performance degradation 5 years ago was 20-30% because the things were pumping out a ton of interrupts, one would expect the resurgence of multithreaded CPUs to further reduce the impact. They should help with context switches. When you've got eight logical cores, that 30% is suddenly 4%, and much less important.

US-Robotics still sells "hardware" modems, although they're not all that cheap. But back in the day, they did cut 50-100ms of latency off a dialup connection when compared to a winmodem (or controllerless).

Re:For once ...

[Guspaz]

You're exaggerating. Modern Intel integrated graphics fully support DirectX 10, and full onboard video decoding. They're comparable to discrete GPUs from about three years ago.

Do these things scream when it comes to games? No, but they don't need to. For the vast majority of people, serious gaming isn't in the cards. These GPUs need to display a 3D accelerated UI with decent performance, and help out with media decoding. Casual gaming is a plus, although businesses aren't interested in that.

By these accounts, integrated graphics are more than sufficient for the vast majority of users.

Besides, from reading most of the comments above, people are missing the point here; this article isn't talking about merging the GPU into the CPU, it's talking about how discrete graphics are becoming increasingly CPU-like, and how we're going to eventually reach the point where GPUs are so general-purpose, they're actually specialized CPUs. This is, infact, exactly what Larrabee is going to be; a very simple massively multicore CPU sitting on an add-in board. Sweeney's prediction is that AMD and nVidia are both going to take their GPUs to similar places eventually.

Re:For once ...

[ZorbaTHut]

Graphics cards were pretty much required in 1998. 3d graphics cards, sure, those were rare and high-end, but there's really not anything about 2d vs. 3d that makes the concept different (for the purposes of this conversation, at least.)

Re:For once ...

[Weedlekin]

It got modded down because there are vast numbers of Slashdotters who are congenitally incapable of recognising any humour that isn't a minor on a known meme.

Re:For once ...

[juiceboxfan]

It's very slightly (pennies) cheaper to put one chip on the motherboard rather than two. It's MUCH more expensive to merge two big CPU/GPU type chips into one. Manufacturing flaws become more common fast with bigger chips.

I don't think your estimate is correct for packaging and placing a single chip vs. two chips but in high volume manufacturing even pennies make all the difference. What about the cost of the second fan and other infrastructure for the GPU? There is also the issue of real estate - two chips take up more room than one so your etch routing becomes more of a challenge requiring smaller etch geometries resulting in a more expensive PCB.

If we could break CPUs into pieces now we'd do it, both for that and heat reasons. We can't because all the parts currently located in a CPU need to talk to each other very fast. The GPU is something that usually doesn't need to talk to the CPU much. So it's separate.

No, it will always be cheaper to integrate at the chip level. Look at system prices, dual core is cheaper than dual CPU (if you can even find one these days) of similar performance. The trend in embedded systems (even more cost sensitive than PCs) is to integrate everything into a single package (SOC). As others have pointed out cache and the FPU were once separate chips from the CPU they are now all integrated into one package.

Re:For once ...

[Molt]

OpenGL and DirectX both have shaders, but that isn't the same as shaders needing one of these. It's equally possible to do the shading code in more conventional language such as C++ and then just draw the pixels on screen with a much lower-level graphics API, possibly as low as simply accessing the framebuffer as a large array. The line about adding new fixed-function features did confuse me a little too though, I ended up guessing it was about the Geometry Shaders which were introduced in DX10. Whilst these are shaders, and so programmable, they do fit into a very specific point of the fixed DirectX pipeline and so can only be used to do things which boil down to 'Add more geometric detail' which when you think about it is quite a small category of task. I could be entirely wrong about what he meant though.

Re:For once ...

[richlv]

my very vague understanding is not a 'merge' as such of two separate technologies as more these two units evolving so that they become more & more similar with each generation. in the end these units become so similar that there is no specific merge to do, you just really calculate all gpu inside the new cpu, or run the os on the new gpu.
at least that's how i understood the idea, which to me also seemed a bit silly at the first moment ;)
though i don't see that happening in the mainstream in at least 10 years.

Re:For once ...

[Molt]

A modern GPU, such as the current flagship GeForce GTX 280, is a relatively specialised piece of hardware. It has optimised support for things such as matrix operations which are used a lot in graphics and yet has no real support for integers which aren't. It has a very fast and entirely dedicated memory bus capable of pushing 140GB/s at maximum for processing geometry and textures. It's highly parallel having two hundred and forty stream processors to process the millions of seperate shader runs which make up a single frame of a high-resolution 3d scene as quickly as possible. It is also a very complex piece of technology having 1400 million transistors compared to a Core 2 Extreme QX9650's 820 million processors.

There's no way that a current CPU could replace that in anything other than a very slow emulation mode even if fully dedicated to the task. If you do want to see the difference download either nVidia's FX Composer or ATI's RenderMonkey which are both shader authoring tools. These will allow you to load a shader and preview it using your 3d card and then drop down to software rendering. Watch as a smoothly-animated complex shader suddenly grinds to a halt, often taking literally seconds to render a single frame. It's not a pretty sight.

In the future with CPUs getting more and more cores and graphics programming getting more generic to support different algorithms without needing to work against the silicon's basic design it's likely we'll see them being more suitable to the rendering task as the article covers but for now trying to substitute the CPU for the GPU would be like entering a bulldozer into a Formula 1 race on the basis that it has a large engine.

Re:For once ...

[crhylove]

Your mom's basement sounds WAY more fun than mine.

Re:For once ...

[Molt]

Since this is the developer who wrote the Unreal 3d engine, which is one of the best in the world at the moment, you'll have to excuse me if I think he knows a little bit about graphics APIs.

Both DirectX and OpenGL are entirely dedicated to the "Here is a triangle-based polygon mesh, some textures, and a few shaders- go draw me a pretty picture won't you?". This is a good approach to a lot of general 3d work but it is not the alpha and omega of 3d rendering.

As a couple of examples it doesn't have good support for voxel-based volumatic rendering which is perfect for cloudy scenes and also could give better support for fractally-defined terrains, it also doesn't support any rendering method which actually uses the equations of curves to render them properly as opposed to turning them into a series of small triangles. Many of these can be shoe-horned into the existing APIs, but when you do so you find you're actually going round the API much of the time and so losing any advantage they give you. There are a number of other examples of rendering methods which won't fit into the existing APIs in the article.

If the article is right and this did come about then there probably would be a series of APIs coming out for the different methods of 3d rendering. Those such as the article's author who're writing the graphic engines for the AAA games almost certainly won't be using them though- they'll be still be hand-crafting their own APIs specific and optimised for the current engine's features and now they'll be building them on top of very low-level graphics APIs with limited features like setting screen resolution and drawing a single pixel as opposed to the polygon-mesh with shaders level of complex APIs today.

Re:For once ...

[ShakaUVM]

>>No API's or 3D hardware was available, only a single CPU and a frame buffer and compiler.

Of course there were APIs and 3D hardware in the pre-Quake days. I know; I wrote 3D arcade games back in the era before Quake 1 even came out, and our company got demo hardware for something like 20 different 3D graphics accelerators for the PC at the time. This was all before the Voodoo came out and turned the industry on its head, mind you. OpenGL was the standard, and you could run OpenGL software with or without hardware acceleration. We ended up using a high end ($20k) graphics accelerator for the PC at the time that had its own API.

>>You had to hand code and entire rendering engine for the game

Sort of. The main reason why you couldn't just render an entire 3D world using a naive sort of OpenGL implementation is that you'd get crap for a framerate. The engines, even those that used OpenGL, were mainly culling algorithms that cut down on the number of surfaces being processed by the engine. Quake did a pretty good job at this, actually. It was quite playable as a true 3D engine (Doom was 2.5D) on a 486 processor.

Modern Quake1 clients have added bloom and a bunch of other stuff that would cause machines from that era to cry, but it's quite telling that if you turn off these culling optimizations on a modern machine, the modern machine will still slow down to a few fps. At least, it does on my 4800X2 and 8800GTS.

I don't think your idea to simply have geometry that gets rendered in a naive fashion would work, to be honest, and especially not a realtime raytracing engine. You still need smart guys like Carmack in there optimizing the hell out of a handcrafted rendering engine.

Re:For once ...

[Goaway]

Yes, hardware rendering looks so much better!

Re:For once ...

[amn108]

Apparently you do not read very well. He put emphasis on generic computing as opposed to specialized hardware. It is not about cramming it all in one silicon package. For all Sweeney cares you can distribute silicon chips all over inside a computer case, as long as they do not limit programmers ability to eh.. program them.

Re:For once ...

[mdwh2]

This isn't anything new here -- he's basically saying what Intel has already said... You'll see less OpenGL/DirectX and more CUDA/Shader based implementations for rendering engines.

Indeed, this has been happening for years - there's less reliance on using fixed function pipeline commands, and instead it's all done in shaders.

I'm not sure that APIs will disappear. There'll be perhaps less usage of them, e.g., with a single CPU/GPU there's no need to transfer data between them using vertex buffers and so on.

But you still want an API - the whole point of using APIs rather than banging the hardware directly is that the same software can support hardware from different companies, and that the hardware can be upgraded more easily without breaking compatibility. This is surely a better situation than with CPUs, where software for one CPU won't run on another, unless it's specifically been made with the same instruction set.

APIs are also useful to avoid reinventing the wheel - things like setting up the screen, setting up the multithreaded code (it's much easier to just provide the vertex and pixel shaders, and pass them to the API) and matrix transformations.

Re:For once ...

[Creepy]

Part of the issue is that if the GPU and CPU are merged, they would also attempt to keep the same die-shrink, while traditionally are a generation behind in GPUs (though this gap has been shrinking). By keeping one generation behind there is a significant expense savings, which is why integrated CPU/GPUs are usually 2-3 generations behind (since that is the "value" segment of the market and usually production issues are worked out by then).

The grandparent is correct that the more you stuff on a single piece of silicon, the more bad chips you get, but you (parent) are also correct that the more you jam on a piece of silicon, the cheaper the overall costs if you can effectively build them in volume. And while dual core is cheaper than dual chip (due to the extra wires, socket connectors, and infrastructure such as controllers), jamming more on the chip is not always the answer - Intel quad-core chips are two dual-core chips in a single package (saving money by avoiding requiring extra stuff like socket connectors as well as benefiting from a larger volume of dual core processors). Intel has no intention to build a true quad that I know of - they rolled out a 6 core die yesterday, so I expect the next step to be 12.

The goals of SoCs are different than those of typical PC parts - the main motivators there are reduced power consumption and a smaller package. That means they're willing to trade performance for size.

Re:For once ...

[Just Some Guy]

But even with multicore CPUs, I can guarantee that if modems were still the primary way people got online, there would definitely be a thriving market for "performance" modems that offloaded at LEAST the signal-processing functions to a real DSP

My wife didn't understand why I wouldn't let her parents have my SupraExpress 56k serial modem. Granted, I may very well never use it again, but if I needed it and didn't have it, I would be very grouchy.

Re:For once ...

[Antitorgo]

Umm... what? Why the namecalling?

There's CUDA and FireStream which are programming the shaders w/o going through DirectX/OpenGL.

As far as D3D11 -- I think the only support there which is similar is the compute shaders which I'm unsure if it will apply. There's also the Apple OpenCL initiative which aims to accomplish a similar thing. AFAIK, none of the GPGPU bare-to-the metal APIs allow you to render to a texture so I think it might not be possible to accomplish a pure Stream based rendering engine (yet).

In any case, I think his original point stands -- that the monolithic SGI based APIs are going to still be used (OpenGL/DirectX), but the hardware under the covers will be more exposed to allow programmers to be more creative with how they want to utilize the highly parallel processing that the new chips provide. Thus allowing programmers to do things in software that were previously done with dedicated hardware.

Re:For once ...

[Creepy]

You're on the wrong track - Fixed function features are actually features that do not use shaders at all. For example, you may have a new fixed function for hardware particles (that's an older one, but the one that popped into my head) - you pass the arguments to it and run it, but you don't actually program it (there is no script like there is for shaders).

    Shaders REPLACE the fixed function pipeline, so if you use them, you actually need to rewrite any fixed function feature you want for the object/polygon being rendered. Many commonly used features have shading language definitions, such as fog and lighting, but you are also free to write your own. This is why he later goes on to say that CUDA may be able to do something similar - general purpose shaders are essentially a set of programmable vector units.

The simplest explanation of Geometry Shaders is that they sit between the Vertex Shaders and Pixel shaders and can move or generate vertices in hardware. There are a few uses for this, such as instantiation (creating copies of a model), tessellation (converting a primitive such as a sphere into polygons), and single pass Cube Mapping (a texture mapping technique often used for scene geometries difficult to render in polygons), but in general, the usage is fairly limited (first generation hardware had a 1024 vertex emission limit and runs slower the more vertices that can potentially be emitted).

Re:For once ...

[Stooshie]

I don't know, your mum made it fun when she took me there! ;-)

Re:For once ...

[colmore]

You're right! The floating point unit will never merge into the main CPU, it can't be done!

Wait...

So when GPUs first came out, they were cheaper than high-end CPUs. This is no longer the case (at the high end). GPUs are doing little more than a bunch of floating point ops. If you have a many-core setup where one or a few of the cores are set up for fast floating point operations, it makes sense to do graphics on the main CPU, rather than taxing the system bus for the intercommunication.

Nobody is saying to run go build this right now. But this guy (who sounds a lot more informed than the Slashdot naysayers) is saying that the performance gain to be had by the current architecture is getting slimmer, and now that we want fully programmable (not just shaders) GPUs it doesn't even make all that sense.

I'm really looking forward to it. We've been stuck with one rendering algorithm for way too long.

Re:For once ...

[Tetsujin]

It got modded down because there are vast numbers of Slashdotters who are congenitally incapable of recognising any humour that isn't a minor on a known meme.

I'm a minor on a known meme, you insensitive clod!

Re:For once ...

[mrchaotica]

Would't 1 or 2 cores match up against a gpu?

No, not even slightly. Just do some simple math: 1 core, running at (say) 2GHz, vs. 64 (or more!) cores running at 500-1000MHz. Which seems faster now?

Re:For once ...

[juiceboxfan]

Part of the issue is that if the GPU and CPU are merged, they would also attempt to keep the same die-shrink, while traditionally are a generation behind in GPUs (though this gap has been shrinking). By keeping one generation behind there is a significant expense savings, which is why integrated CPU/GPUs are usually 2-3 generations behind (since that is the "value" segment of the market and usually production issues are worked out by then).

The grandparent is correct that the more you stuff on a single piece of silicon, the more bad chips you get, but you (parent) are also correct that the more you jam on a piece of silicon, the cheaper the overall costs if you can effectively build them in volume...

True, and even though I did mention single die I would expect a first generation CPU/GPU to be two dies in one package. You get the option of mixing generations as well as an improved yield through prescreening the individual pieces. This would still give the advantage of fewer pins to deal with, compared to two packages.

Intel has no intention to build a true quad that I know of - they rolled out a 6 core die yesterday, so I expect the next step to be 12.

A hex core processor that tests out with two bad cores could either be called scrap or a good quad core.

Can't wait to welcome our new duodeca-core overlords;-)

Re:For once ...

[mrchaotica]

Both DirectX and OpenGL are entirely dedicated to the "Here is a triangle-based polygon mesh, some textures, and a few shaders- go draw me a pretty picture won't you?"

That's not entirely true: OpenGL can also handle quad-based meshes. ; )

Re:For once ...

[Weedlekin]

In Soviet Russia, known meme is minor on you.

1. Minor on known meme.
2 ???
3. Profit!

In my day, we only had minors on unknown memes, and we thought we were luck.

All your known memes are belong to us.

Re:For once ...

[Fred_A]

But the more die you roll out on your board, the more chances one makes its saving throw in canse of failure.
Oh, wait, "die"...
Never mind.

Re:For once ...

[Creepy]

Well, he also guessed GPU and CPU convergence by 2006 and 2007, and in my 1996 computer graphics class I predicted ray tracing would be viable by 2011-12 and surpass rasterization by 2017 (yes, I wrote a paper - buried somewhere in college stuff, but 2011 is pretty close to the late 2009-early 2010 target for Larrabee), so I could say I was prophetic as well. Of course, I have to ignore all the things I was wrong about, such as display size (resolutions greater than 1024x768? - who needs that?), single CPU (parallelism? what's that?), no vector processing, etc. I did my prediction entirely based on math - pixels, CPU doubling and ray tracing vs rasterization in Big-Oh analysis.

I guess computer graphics and science fiction share the fact that you can predict some parts of the future with some degree of accuracy, but you will always have some error, either with timeline or technology, no matter how much you base your analysis on real world metrics.

Voxel based rendering is making a comeback mainly because Volume Ray Casting can be done in parallel which makes it very suitable for either a vector unit or any other general purpose GPU (like DX10-generation graphics cards). I wouldn't say it's a be-all end-all rendering technique, but yes there are some good uses (nuking terrain, for instance).

I think his point for the lack of an API was not suggesting everything will be written from scratch by AAA game devs - he meant techniques can be patched into the code without the restrictions imposed by fixed functional hardware or even fixed functional hardware mixed with shaders.

    Let's use a "for example" case - say you are using ray tracing and want to add soft shadows. Ray tracing by itself assumes point lighting, and if this ray tracer were built into, say, a fixed hardware pipeline, you would be force to use hard shadows or some vendor defined extension that gives you a specific type of soft shadows. If you used a "shader" to do soft shadows, you would have to rewrite any fixed functionality used in the ray tracer since the shader replaces any fixed function code. If this were entirely in software, however, you would have the flexibility to keep the "fixed" code and tack on new code without restrictions, so if you wanted soft shadows in your ray tracer you might write a photon mapping implementation to get it and you don't have to rewrite other code.

Re:For once ...

[rtechie]

You're exaggerating. Modern Intel integrated graphics fully support DirectX 10, and full onboard video decoding. They're comparable to discrete GPUs from about three years ago.

Feature set isn't the issue. It's pushing raw polygons, and the memory bandwidth constraints are killer for integrated graphics.

Do these things scream when it comes to games? No, but they don't need to. For the vast majority of people, serious gaming isn't in the cards.

Which has been true for a long time, the Voodoo launched at $250 and ALL it was useful for was 3D games.

By these accounts, integrated graphics are more than sufficient for the vast majority of users.

For about 15 years now the market has been basically divided into low-end "business" GPUs and high-end "gamer" GPUs. You're proposing a change to the status quo that doesn't seem to be in evidence.

Business NEEDS 3D graphics work. That market will always be there and ATI and NVidia will always produce new technologies for them. This is the reason PC gaming is unlikely to die anytime soon. The world MUST HAVE high-end discrete GPUs for 3D graphics processing (Do you think Pixar is going to close up shop?), there's billions to be made there. And it's only a little extra effort to develop low-yield versions of those chips for the consumer marker.

Video gaming is, right now, the largest entertainment market in the world. It beats film, music, theater, etc. Much of it is centered on the PC. People don't seem to grasp that, globally, PC gaming sales far exceed console gaming sales. The notion this is going to disappear overnight is pretty laughable.

Re:For once ...

[gd2shoe]

He got modded down because he's an AC troll. It only sounds like a joke until you realize that it's the same type of drivel that we've become accustomed to seeing from ACs. He's either being serious, or trying to be provocative. Either way, I'm not laughing. You may laugh at him, if you like.

Re:For once ...

[ceoyoyo]

In embedded system-on-a-chip stuff you're talking about putting a few components on a chip. The feature count is WAY smaller than a CPU (or a GPU). That can be done fairly cost effectively because the CPU makers have already led the way. Even then, many system-on-a-chip products are really system-in-a-package, where different bits of silicon are put together inside a single package. Here's a conference paper that talks about it: http://www.acreo.se/upload/Publications/Proceedings/OE00/00-COLLANDER.pdf

Merging a modern CPU/GPU is a whole different ball game. The various bits of the CPU has been integrated because that's the only way we could keep making them faster. The parts that are actually on the same piece of silicon with the CPU are there because they need high bandwidth to the CPU. The GPU does not, at present.

Why do we have 8 processor computers made up of two quad core chips? Because it's not yet practical to put 8 cores on one chip. We will, but we don't now. CPUs and GPUs might merge, but they'll be low power, slower versions of both. Nvidia and AMD won't be putting their latest and greatest GPU on a chip with the fastest processor out there.

Re:For once ...

[ceoyoyo]

Why wasn't the FPU integrated with the CPU right away? Because it didn't make sense. Then, as clock rates went up, the FPU had to be integrated to keep up.

The GPU and the CPU don't talk to each other that way. The GPU needs MEMORY bandwidth. It doesn't really care what's in the CPU registers.

I don't think this guy ever says that the GPU and CPU are going to merge. What he does think is that the GPU is going to go away, to be replaced by a general purpose vector/stream/dsp type processor. When or if that happens, THAT kind of coprocessor would probably be worth integrating into the CPU. In fact, it's already been done with Cell.

But a GPU? There's no reason to integrate a high power GPU with a high power CPU, and lots of reasons not to.

Re:For once ...

[LordKazan]

GPUs are also very expensive to make due to their large die size.

In fact die size may be the single biggest thing that SHOULD prevent the combination of the GPU and CPU. As the mm^2 of the die grows, the cost grows even faster.

In fact the biggest differences (other than a few tweaks to correct errors) between the disaster Radeon HD2k series and the vastly superior Radeon HD3k series was a die shrink - they're all R600 series units. It went from a total bomb (HD2k series) to a very competative card - the die shrink increased yields, decreased cost per unit, decreased voltage required and heat output. The HD2900 was 420mm^2 while the HD3800 was 192mm^2.

Simply shrinking from 80nm to 55nm turned an non-profitable card into a profitable one. Their new flagship product - the HD4870 based on the R700 core is also 55nm but has 50% more transistors than the HD3800. The differences in the core architecture resulted in more than 2x the performance for only a 50% increase in transistors. Saving their butts from the disaster that was the 80nm R600 by shrinking the R600 to 55nm gave them the time to develop the R700 without being buried by their competition.

If die size is that important and can be THAT expensive I doubt die size issues will EVER allow them to combine the CPU and GPU.

Re:For once ...

[amn108]

HyperTransport from AMD, and QuickPath from Intel are designed among other things to interconnect different facilities on the motherboard with more than adequate speeds and latencies. Did you take that into account when writing what you wrote about "us not being able to break CPUs into pieces"?

GPU not talking to CPU to the extent programmer wants is one of the very problems that obviously trouble Sweeney as he advocates for a more generic approach to graphics programming. The whole interview and his answers revolve around being able to free ourselves from being "force fed those triangles down that pipeline" approach so to speak.

Re:For once ...

[amn108]

I like the last sentence of your post. That is exactly how I feel, having done some hobby graphics programming before, reading Michael Abrashs weird but highly functional perky algorithms.

Also it seems that a common dedicated GPUs value per dollar ratio is much lower than for a CPU, given the fact it sits idle most of the time when you don't use DirectX or OpenGL, and even if it does something "useful" again it's only DirectX or OpenGL which are hardly considered usable for general purpose computing for the masses. Except for gamers, any buy like that is a waste of money. All those billions of transistors and horsepower that could put CPUs to shame, but it comes with a peekhole size programming potential.

Re:For once ...

[amn108]

Yes but FPU functionality is a very desirable generic functionality that almost every application may need, because the FPU unlike the CPU is made to work with REAL numbers, the numbers real world uses. It is not like FPU is hardwired to operate some highly specialized pipeline and that only, the way modern GPUs are. Modern GPUs are like lollipop factories, churning out wrapped candy by billions, about as fast as you could ever do that with a machine, but that's all they can do as factories. An FPU is a calculator, and calculators are a pretty generic tool that is used in all natural sciences. An FPU is a necessity for any modern computer, because it is among the very few examples of GENERAL PURPOSE DEDICATED hardware.

Re:For once ...

[ceoyoyo]

The first time I'm aware of that an FPU was integrated into a common CPU was the 486 DX. In 1989. Hypertransport was introduced in 2001. Incidentally, 486 DX parts that had defective FPUs were sold as 486 SX processors to recover some of the cost that would otherwise have been lost. Later 486 SX processors were made without the FPU, to save die area (and therefore cost).

AMD claims that integrating the memory controller into the CPU greatly reduces latency (improving performance). Everybody seems to agree with them.

So no, HyperTransport is not designed to interconnect things on the motherboard with the speeds and latencies that are available on the CPU die itself. No external bus can do that, now that CPUs are running at speeds where clock pulses can barely traveling across the die fast enough.

Yes, the article advocates moving away from GPUs to something much more general purpose and much more programmable. That will not be a graphics processing unit. Integrating such a complementary processing unit with the CPU might make sense, though it will be expensive. Integrating a high performance GPU does not.

Re:For once ...

[Guspaz]

PC Gaming is indeed a large market. However when compared to the entire PC market, it's a tiny part. Sure, millions of gamers need powerful 3D graphics in their PCs. And the billions of non-gamers don't. For those non-gamers, integrated stuff is good enough. Almost overkill, these days, really.

Re:For once ...

[Guspaz]

The limits of human visual perception aren't only limited to their resolution, or how fast they can discern different images. If that was all that mattered, you could run DooM at 2560x1600 at 60FPS and pretty much have that covered. It's the content of the image that's important.

I think we're decades away from achieving true photorealistic graphics in games. Even the best offline-rendered CG isn't close. Final Fantasy: The Spirits Within was the closest we've probably gotten, and that didn't even manage to get to the other side of the uncanny valley.

For all we know, integrated graphics might go in a completely different direction. For example, the evolution of the GPU is clearly towards the CPU; Larrabee is just that, a 32-core Pentium with a big die shrink and modern extensions and other tweaks tacked on. If that's the way we're going to go, and all 3D rendering is done via software rendering on what is essentially a simplified CPU, I rather suspect that integrated graphics will before long simply cease to exist.

I mean, we'll already be running everything on a software renderer. And there won't be any particular reason why that software renderer need be limited to the graphics CPU. I think that, relatively soon after graphics CPUs replace GPUs, it will simply make more sense to just ditch integrated graphics and run those software renderers on the system's main CPU. There would still be some sort of motherboard-mounted graphics hardware, but it would probably just consist of a DAC/DSP hooked up to the CPU that enables the hardware to take the framebuffer from system memory and convert it to VGA/DVI/DisplayPort/etc.

And I think that this will start to happen within a year of two of graphics CPUs taking over. Which might be as soon as 2010 or 2011, since Larrabee is launching in late 2009. So I'll make that prediction now; integrated graphics will become irrelevant by 2013 at the latest.

Re:For once ...

[Undead NDR]

And why install an overkill graphics processing unit inside the processor if most people won't use it anyway?

No one's talking about installing a processing unit inside the processor. The whole point is that the CPU will simply be powerful enough to take care of all the graphics.

Re:For once ...

[Graywolf]

Interesting read, thanks.

Re:For once ...

[amn108]

About your first paragraph: I am not sure what does it have to do with anything? So yes, FPU merged with CPU with 486DX in '89, and HT was introduced in 2001. Defective FPUs sold as 486SX processors. So? What does it have to do with anything?

Yes, it is a known fact that inside a silicon die, latencies are substantially lower and speeds are substantially higher. It does not mean you should not distribute multiple CPUs with their own memory controller and cache across a motherboard. They do not have to talk to each other all that often, this is what parallel processing is about among other things - being able to perform a task independently from counterparts.

HyperTransport IS designed to interconnect things on the motherboard. Read on it again. I never said it does it "with the speeds and latencies that are available on the CPU die itself", you misquoted me. It does not need to for what it is used.

And I never said a GPU will stay a GPU if integrated on-die with the CPU.

Re:For once ...

[ceoyoyo]

"HyperTransport from AMD, and QuickPath from Intel are designed among other things to interconnect different facilities on the motherboard with more than adequate speeds and latencies. Did you take that into account when writing what you wrote about "us not being able to break CPUs into pieces"?"

I stated that the various bits that were integrated into the CPU were integrated primarily for the purposes of speed, NOT for the convenience of fewer chips on the motherboard. Your paragraph, quoted above, was offered as a counter argument.

To answer explicity: no I did not consider hypertransport as relevant to integration of the FPU since the two were separated by more than a decade. No, hypertransport is not relevant to most other parts because (like the memory controller) because they do benefit from being on the silicon with the CPU, communicating faster than hypertransport can allow.

So you were agreeing with me in your last post? Why didn't you say so!

Re:For once ...

[amn108]

Yes I agree with you on all but one important point - We CAN split the CPU somehow, by taking the cores out for instance and giving each its own cache and memory controller. This is what multiprocessor systems do as opposed to multicore systems. It is essentially the same as a GPU plugged in a PCIx slot. I basically reacted on you saying that "we can't because". That is misinformation, because we can and we will have opportunity to do so many times in the future. Unless the whole system is put onto a single die, which will present us with essential monopoly over parts we can choose for our systems, the motherboard remains relevant, and various bits of the system will be put outside a die, also parts that can be put into a CPU. I hope this clarifies my opinion, and exactly where it differs from yours. I did not mean to be offensive for the sake of it ;-) I also expressed my opinion so it is known that it is fully possible and relevant to make a generic GPU (which will be more like a G'eneric'PU then) and put it BESIDE the CPU on the motherboard, not inside it. For most application, given its own cache and memory access facility, it will do just fine and the rest of the system will do too.

Re:For once ...

[vux984]

Yes but FPU functionality is a very desirable generic functionality that almost every application may need, because the FPU unlike the CPU is made to work with REAL numbers, the numbers real world uses.

I know what a floating point number is. ;)

It is not like FPU is hardwired to operate some highly specialized pipeline and that only, the way modern GPUs are. Modern GPUs are like lollipop factories, churning out wrapped candy by billions, about as fast as you could ever do that with a machine, but that's all they can do as factories.

Not quite.
1) GPU's are becoming rather like generic processors. I'm sure you've heard of the folding@home GPU edition, or the mersenne prime GPU edition -- efforts to use the GPU for highly parallel processing. (And successful ones at that, often significantly faster than the CPU based versions.)

GPUs as they've advanced and become increasingly more programmable are useful for performing common and somewhat generic tasks; audio/video encoding/decoding, compression/decompression, encryption/decryption.

2) In the not too distant future EVERYONE will be running a fully 3d desktop environment; not in the literal sense where you can move around in it first-person style; but in the sense that the 2D desktop metaphor will be rendered as a 3d-space. We already see it now with windows that cast shadows and have transparency, you see it in OSX's expose where dragging a gadget causes ripples, or windows Vista's flip-3d task switcher, or Compiz's rotating cube to move between workspaces.

Some of this stuff is just gratuitous shiny eye-candy and ultimately distracting or ineffective, but over time we'll refine it, and get it to the point where its natural and clean and each effect genuinely enhances the usability of the OS.

OSX has already made video acceleration an official requirement of the OS. Vista is practically there. Linux will probably always have it as an "option", but the modern desktop environments will certainly use it.

An FPU is a calculator, and calculators are a pretty generic tool that is used in all natural sciences. An FPU is a necessity for any modern computer, because it is among the very few examples of GENERAL PURPOSE DEDICATED hardware.

A GPU is pretty well a necessity for any modern desktop computer too. A server can get away without one, but that's about it. But if GPUs get integrated into the CPU and software can use them easily using a consistent instruction set and can count on them being there, the then we'll see them get used for encryption/decryption and other tasks for which that sort of parallel architecture is suitable and they will be perhaps even more important on a server than on the desktop.

Wrong summary

[Anonymous Coward]

He talks about the impending fall of the fixed function GPU.

Re:Wrong summary

[sketerpot]

Of course you can run OpenGL or DirectX on a software renderer, and I'm sure they can both be efficiently parallelized. The point is that soon we won't have to. Or want to.

Re:Wrong summary

[Eladith]

Indeed, the summary however sounds pretty accurate to me.

...a stifling array of possibilities...

Software rendering has gone quite far in terms of image quality in the last two decades. The images artists can create with modeling applications coupled with mental ray for example are absolutely stunning. It will be interesting to follow these techniques to find their way into real-time applications.

Pixar's Renderman has proven quite succesful for rendering somewhat complex sequences of moving images. I'll be waiting to see real-time implementations of REYES-style micropolygon approaches in the coming years. Few advantages would be high-order surfaces, high quality sampling and shading, displacements, motion blur and depth of field. Like said in the article:

So it sounds like right now is a really good time to be a graphics programmer.

Isn't "inflection point" two words?

[hartze11]

Or do I need a GPU to calculate that?

I hope not!

[jhfry]

I just browsed the article and it looks like what he's saying is that as GPU's become more like highly parallel cpu's we will begin to see API's go away in favor of writing compiled code for the GPU itself. For example, if I want to generate an explosion, I could write some native GPU code for the explosion, and let the GPU execute this program directly... rather than being limited to the API's capabilities.

So essentially, we will go back to game developers needing to make hardware specific hacks for their games... some games having better support for some cards, etc.

API's are there for a reason... lets keep em and just make them better.

Re:I hope not!

[MarkvW]

I read it differently than you did. He's projecting a world where everything is standardized and faster (less 'bus plumbing' GPUs). In such a world you won't need APIs because you'll have libraries that you can include in the compile process.

APIs reduce code bulk at the cost of reduced code speed, don't they?

Re:I hope not!

[phanboy_iv]

You don't need and "API" to code for a Intel CPU versus an AMD CPU. You just write it, compile it, and run. No API needed. That's what he's talking about, I think.

Re:I hope not!

[Emperor Zombie]

Actually, he's saying the opposite:

TS: I expect that in the next generation we'll write 100 percent of our rendering code in a real programming language-not DirectX, not OpenGL, but a language like C++ or CUDA. A real programming language unconstrained by weird API restrictions. Whether that runs on NVIDIA hardware, Intel hardware or ATI hardware is really an independent question. You could potentially run it on any hardware that's capable of running general-purpose code efficiently.

Re:I hope not!

[BagOBones]

No what he was saying is that currently you have to with a different set set of explosion code depending on your app running on a DirectX system an OpenGL system or a console. If they they all just had CPUs with lots of cores you could write a general explosion with a few minor tweaks needed for consoles with specific CPUs.

More cores means more general CPU instead of some GPU that has does a few limited things fast.

Re:I hope not!

[Anonymous Coward]

In such a world you won't need APIs because you'll have libraries that you can include in the compile process.

A library you include in the compile process is an implementation of an API.

APIs reduce code bulk at the cost of reduced code speed, don't they?

No.

Re:I hope not!

[jd]

I'm more reading it that if you have a heterogeneous set of ultra high-performance specialized CPUs, you don't need to separate them out into CPU, GPU, or whatever. You have a cluster, where sub-tasks are migrated to the CPUs best designed for those sub-tasks, with no artificial designation of what goes where. There is no central control or central data store, and no transfer of thread control. Messages containing data would be passed, but you wouldn't be looking at a traditional API in which flow control passes from caller to callee, it would be much closer to a MIMD-style high-performance computing architecture.

However, I could be being influenced strongly by the fact that I've used such architectures in the past and happen to think that they're usually superior to sequential software designs or over-generalized processors, provided you can shift the data fast enough. (There have been some designs which reversed the entire concept: as code is generally smaller than data, there have been projects in which the functions are migrated to where the data is, only duplicating the data if absolutely necessary to support multiple read/write threads. If you had a high-enough performance RISC core, a good enough compiler, a very efficient microcode layer and something analogous to Transmeta's software instruction set, you could do this at the CPU level, but I suspect it's only beneficial if different threads almost always work on different data.)

Given the increasing performance and increasing availability of kernel-bypass and even CPU-bypass technologies, data-driven hardware is bound to eventually supersede purely procedural hardware - at least in some things. Whether graphics would be included is unproven. As the PCI bus is designed to work from host nodes to peripheral nodes, it is NOT designed to work host-to-host or peripheral-to-peripheral, such a design might require a really perverse PCI architecture or even force the eventual abandonment of PCI in order to support a more fluid layout rather than a pure hierarchy. That might happen - over the next decade or two. Until PCI Express 2.x (double rate) becomes the standard - or is superseded - on home desktop machines, this would not seem to be even a remote possibility. You don't have the functions, the multi-mastering, the virtualizing or the data rate to move to a wholly distributed internal architecture otherwise.

Re:I hope not!

[cmdotter]

No, what he's saying is that in this day and age the GPU is becoming programmable. He then states that he wants to be able to program it. In itself, this is fine. The first thing anyone does with highly powerful and low-level hardware instructions is to wrap it up into a nice little API

Boost, STL, and C++ itself are tools that we use to make life better. Of course OpenGL and DirectX will change with the times. If I were on the teams writing those APIs I'd be looking at ways to do realtime ray tracing and the like on next gen (programmable) hardware. The APIs will be here for a while, although admittedly, there is always opportunities for other APIs to come to the fray

Re:I hope not!

[the_greywolf]

We tried that. The company went out of business and got bought by NVIDIA. (Not that I'm opposed to Glide4.)

Re:I hope not!

[jhfry]

There is no such thing as "hardware that's capable of running general-purpose code"... in fact there is no such thing as "general-purpose code". Code is nothing but a language that is either interpreted by a processor, or can be compiled such that can be interpreted by a processor (or virtual machine). Either way, all code is highly dependant upon the capabilities of the hardware.

It's easy to throw Intel and AMD out there as though they provide an example of how it might work... however that analogy only works because they both use the same instruction set. If you consider all of the non-x86 processors out there, and you realize that all of the "code" needed to render scenes now must be provided in binary form for each manufacturer's gpu, perhaps even multiple different binaries as the gpu's feature set changes, it starts to look bleak.

Sure, right now with just Intel, AMD, and Nvidia in the market, you can imagine that they might be able to keep it pretty simple. Each manufacturer could provide a compiler for different languages for their gpu's that game developers would use. But what happens when Nvidia releases their next great processor that isn't binary compatible with their previous ones? Do the game developers need to provide 2 binary versions?

I think API's are a great way of allowing hardware and software to advance as quickly as possible. I also think that the current graphics API players, GL and Directx, completely fail in their responsibilities. OpenGL is nearly, for lack of a better word, obsolete, while DirectX isn't open, and develops much too slow. Give us a cross platform, open, powerful, and well managed graphics API and let the hardware makers truely innovate.

The alternative discussed in the article will give us what x86 does now... a complete lack of innovation in the hardware's design that reduces improvements to smaller transistors, more cache, and more cores.

Re:I hope not!

[Mike610544]

I was thinking a similar thing. Moving from a standard API to general purpose code would create many more possibilities for quirky/unpredictable behavior across different hardware. The increased flexibility is appealing, but it would clearly be more work for engine developers.

On the other hand, Sweeney's thoughts about the possibility of game development jumping from $10M to $30M based on this type of thing seemed weird to me. If I'm not mistaken, content creation dominates the budget for most modern games (regardless of whether the engine is licensed or in-house.)

Re:I hope not!

[GleeBot]

I read it differently than you did. He's projecting a world where everything is standardized and faster (less 'bus plumbing' GPUs). In such a world you won't need APIs because you'll have libraries that you can include in the compile process.

No, you won't need APIs because you'll have a standard instruction set. Current graphics APIs are oriented around things like pushing triangles and running shaders on them. Why bother with the complexity of the graphics APIs, if you can create a standardized instruction set for GPUs?

We essentially already have that, but it's wrapped in the paradigm of shader programming. Stop treating the GPU as a graphics processor, and as just this really parallel coprocessor, and you get what Sweeney is talking about.

The basic point is that GPUs aren't really for graphics anymore, and as we move away from a graphics fixation, we'll come to realize that they're just specialized computers. Using graphics APIs to program them would be as stupid as using OpenGL to write a word processor. Sure, with modern capabilities, it's actually possible, but it's a bad conceptual fit.

Re:I hope not!

[jhfry]

yeah the the binary "program" that generates the explosion, must then be compiled for each different processor... or processors must standardize on one instruction set... the first way makes it difficult for game developers... the second slows hardware innovation because it limit's hardware designers (make a processor that doesn't conform and it fails because no older software can use it).

API's solve both weaknesses with very little trade off. As long as the API is upgraded frequently to support new features, and is kept backward compatible to support old software, neither the developers nor the hardware designers are limited. One gpu can be really good at raytracing, while another might support all of the same features but excel at 2d video presentation (scaling, denoising, mpeg, etc...).

Sure there is a cost for an API... it does limit developers to what is allowed by the API, it does mean that code is not necessarily optimized for the gpu, it does provide another layer of complexity and potential points of failure, and there is always the chance that API's are mired in corporate muck or hardware vendors fail to support them fully. But those are political and social issues, not technological ones. API's are the best way for graphics to work... the current API's are the problem.

Re:I hope not!

[jhfry]

This all assumes that all of the different players' PU's are binary compatible... at which point there can be very little innovation in hardware except to make things smaller or add more of them. Think x86.

Re:I hope not!

[dgatwood]

I just browsed the article and it looks like what he's saying is that as GPU's become more like highly parallel cpu's we will begin to see API's go away in favor of writing compiled code for the GPU itself.

Translation: in the future, OpenGL will be supplanted by OpenCL or similar. Okay, maybe I might buy that in theory, but... not soon. I would think that the abstraction in things like OpenGL provides some benefits over writing raw C code in terms of making it easier to do graphical tasks.

Re:I hope not!

[Disoculated]

The article says you would be creating your API, not restricting you to the APIs of Direct X or OpenGL. Software houses would probably build/trade/sell CUDA implementations much like they do engines these days.

Re:I hope not!

[jd]

At the deep RISC level, they probably wouldn't be. In fact, they'd certainly not be, or you'd simply have an SMP cluster with some emulation on top. If you're going for the migrating code, you'd need binary compatibility at the emulated mode (think Transmeta or IBM's DAISY project) but the underlying specialization would give you the improvement over a homogeneous cluster. If you're going for the totally heterogeneous design - basically the Cell approach but on a far, far larger scale - you need endian compatibility and bus protocol compatibility but nothing more.

This Cell-like approach gives the greatest room for innovation but also imposes the greatest development costs and greatest purchase costs. It also makes ABI backwards compatibility extremely hard or impossible, so you'd end up with a proliferation of builds of the same code for any binary packages (including all closed-source) and a far more complex build and optimization process for all source packages (gentoo users beware). It also makes bus design far more complex, as the more specialized the decentralized processor units (DPUs) get, the more synchronization headaches you will get.

A DPU cluster should logically give the best performance, for the same reason pure RISC outperforms pure CISC - fewer overheads, tighter logic and also (in consequence) more real-estate for optimizations, parallelizations, cache and other goodies. Distances would be greater between processing units, which will have an impact, but so long as the mean gain across the DPUs exceeded the mean loss due to extra distance and extra communication layers, you'd gain overall. This means a DPU computer cannot be flat beyond a certain scale. As SMP clusters cannot exceed 16 processors due to locking issues with shared resources, DPU computers cannot exceed 16 DPUs for a single resource and would probably avoid sharing resources if at all physically possible. This means a DPU computer must be heavy on duplicate resources. But for duplication to beat the deadlocking issue, bus bandwidth needs to be extremely high and bus latency needs to be almost non-existent.

Cell processors are much too basic to run into these sorts of problems, but if you wanted to scale the concept up by, oh, an order of magnitude and beat the design limitations in the Cell processor, you'd need to be spending serious time and money. I expect further "specialist" *PUs to be developed for some time, but the truly RISC, truly distributed DPU is unlikely to exist outside of theory or maybe a research lab or two for at least a decade and I don't expect DPU home machines for at least 30+ years.

Re:I hope not!

[Goaway]

Try paying a little closer attention to who exactly was talking, OK?

Re:I hope not!

[syousef]

Try paying a little closer attention to who exactly was talking, OK?

Don't care when he's talking shit.

Try being a little less condescending, OK?

Software rendering on the GPU

[KalvinB]

Somehow I don't think there's going to be lack of a standardized API much like OpenGL or DirectX even if it's possible to write code for the GPU as easily as the CPU.

The APIs at the most basic level allow Joe and Bob to build their own system, throw whatever graphics card they can find in it and have the same game run on both systems.

As soon as you start coding for a specific GPU you're going to be treating PCs like consoles. I don't care to have to buy multiple graphics cards to play various games.

APIs are for compatibility and general purpose use. The option of flexibility is great for industry use where you're running rendering farms all running identical hardware and custom software.

Re:Software rendering on the GPU

[mistahkurtz]

why not standardization on the hardware level, instead? then, almost all code written for the hardware would be "generic" and "native".

on another note, seems like taking the quadro and firegl lines to the consumers, and then going the next step. assuming that the GPU in fact, does not go away.

Re:Software rendering on the GPU

[mapsjanhere]

Could you get around this by "compiling" the game for individual PUs? Looking at most games, the .exe is relatively small, with lots and lots of level/zone/equipment/sound files. The later should be independent of the PU, and only a small subset would need to be PU specific. I'm aware that "compiling" is the wrong word here, since they won't distribute source code without a breakthrough in DRM technology. But it could be as simple as buying a game, and on install getting a PU specific file either from disc, or for newer iterations from a download server.

Re:Software rendering on the GPU

[Anonymous Coward]

Somehow I don't think there's going to be lack of a standardized API much like OpenGL or DirectX even if it's possible to write code for the GPU as easily as the CPU.

Generally, the whole point of standards is that there should be a lack of them.

Re:Software rendering on the GPU

[Pedrito]

As soon as you start coding for a specific GPU you're going to be treating PCs like consoles. I don't care to have to buy multiple graphics cards to play various games.

I got the impression that they're expecting C++ compilers for all the GPUs, eventually, so then they'd simply have rendering libraries for each GPU. I also got the impression that they'd be waiting until most gamers had one of the compatible GPUs. Let's face it, most gamers usually don't buy the cheapest graphics card and now the two major players, ATI and nVidia have GPUs that are easily accessible. It won't be too long until you can't buy a video card from them that doesn't support CUDA and whatever the ATI equivalent is.

I think it's a pretty safe gamble for video game developers, certainly the 1st Person 3D game developers.

Re:Software rendering on the GPU

[Hortensia Patel]

You're missing the point entirely. GPUs in the near future will essentially be general-purpose massively-multicore processors. (Not entirely so, but close enough for "GPU" to become a misnomer.)

You don't need separate binaries for Intel versus AMD processors, and you don't have to write to an API to achieve portability between such different CPUs. Given sufficiently general hardware, there's no inherent reason why GPUs should be any different. There will still be a place for libraries, but their role will be more like C libraries than like GL/DX.

Re:Software rendering on the GPU

[Firehed]

As soon as you start coding for a specific GPU you're going to be treating PCs like consoles. I don't care to have to buy multiple graphics cards to play various games.

And yet somehow, there's a tremendous amount of GPU-specific optimization code present in every game out there. It may not be a complete codebase rewrite, but it's not as simple as $quality = ($gpu == 'fast') ? 'high' : 'low'; regardless of brand, generation, speed, etc.

Re:Software rendering on the GPU

[Antitorgo]

Actually, no.

Currently you could write to CUDA/FireStream or GLSL/HLSL which leaves it up to the driver to determine specifics for the GPU.

Re:Software rendering on the GPU

[MostAwesomeDude]

In case you didn't know, this high-level code has to be compiled *at runtime* for the particular GPU.

Re:Software rendering on the GPU

[Antitorgo]

True, but typically only once so really not a big deal.

Actually, this is preferable so that any "tweaks" can be implemented by the driver writers without recompiling every program that uses those drivers.

Re:Software rendering on the GPU

[Dolda2000]

His point was not that standardized APIs would disappear, but rather than graphics APIs, more specifically, would. He meant that it would be replaced by, simply, an API to load compiled code (shader code, that is) into the GPU and letting it run. The actual code could be freely written by the developer, and would not even necessarily be constrained to doing graphics processing.

Re:Software rendering on the GPU

[jabjoe]

Don't get to hung up on the GPU as hardware OpenGL/DirectX. Think of it as a collection of stream processors. That collections of stream processors are good for any stream processing task, why limit it to OpenGL/DirectX? So, what you end up with is something like OpenMP extension for C/C++ allowing you to hand out programs or fragments of programs to available stream processors. There is no reason at all there can't be a standard API for handing out to stream processors, maybe even the compiler can notice a large loop of little code that doesn't need to be done in sequence and make a stream program of it. Maybe that could even be done at runtime if the instruction set of the stream processors is the same/similar.

Obvious

[meatmanek]

Is C the best language for parallel processing? C is designed for an iterative, single-process system. When massively parallel processors (whether general purpose or for graphics specifically) become popular, there should be a paradigm shift in the popular medium-level language. Something more along the lines of APL might be appropriate.

Re:Obvious

[jacquesm]

gpu's aren't really parallel in that sense, they are parallel in the SIMD sense.

Re:Obvious

[Hortensia Patel]

gpu's aren't really parallel in that [traditional multithreaded] sense, they are parallel in the SIMD sense.

Actually, they're somewhere in between. Some current hardware can reallocate individual processors between fragment and vertex processing depending on the current workload profile. Even at the level of an individual processor lots of "threads" may be running simultaneously; this is to hide latency when a shader program blocks on memory (texture or framebuffer) access.

If you look at NV's descriptions of their 8xx-series drivers, they talk about *hundreds* of threads in flight at any given time. These aren't threads in the classical sense - there's no preemption, for a start - but they're much, much more advanced than SIMD-style "apply this instruction to all these values" parallelism.

Re:Obvious

[Z34107]

C may not be the "best" language for parallel processing, but it's not like it's impossible, or difficult, or not recommended. You ARE dependent on your operating system for multitasking, but it's not like POSIX processes are that horrible, and Windows threads are actually pretty stupidly easy.

Developing an algorithm that can be parallelized (did I invent a word?) is harder. But, if you can describe something in terms of a Nondeterministic Finite Automaton, you already have something that can be made massively parallel, so win!

Re:Obvious

[tepples]

C is designed for an iterative, single-process system.

But new languages based on C++ introduce primitives that compilers can more easily invert into SIMD, such as C++/CLI's for each() and C++0x's new range iterator syntax.

Re:Obvious

[phantomfive]

I think that's what he was actually talking about, my understanding is that APL was a SIMD sort of a language....it did amazing things when working with arrays. Not that you could read it.

New market opportunity for render engines?

[compumike]

For the last decade or so, it seems like the rendering side was abstracted away into either DirectX or OpenGL, but if the author is correct, those abstractions are no longer going to be a requirement.

While I don't know a lot more about the various other rendering techniques that the article mentions, it seems like there might be an opportunity emerging to develop those engines and license them to the game companies.

I suspect that game companies won't want to get into the graphics rendering engine design field themselves, but there's real possibility for a whole new set of companies to emerge to compete in providing new frameworks for 3D graphics.

--
Hey code monkey... learn electronics! Powerful microcontroller kits for the digital generation.

Re:New market opportunity for render engines?

[mikael]

Before the advent of 3D accelerators, game companies wrote their own low-level renderers. These did the vertex transformation, lighting, clipping and texture-mapped triangle and line rasterization (some companies even explored the use of ellipsoids). Wolfenstein 3D, Quake and Descent as examples.

The low-level graphics rendering part is a very small part of the game engine - rasterizing a textured primitive with some clipping, Z-buffering and alpha blending. But getting this as fast as possible requires a good deal of profiling and analysis to get it as optimized as possible (Brian Hook did a software version of OpenGL for SGI).

3D chip makers gradually took over this area by designing hardware that could do this task far faster than the CPU. First they took over the rasterizing part (3Dfx piggyback boards), then took over the vertex transformation, lighting and clipping through the use of high performance parallel processing hardware (Nvidia TNT). There are other optimisations such as deferred rendering which optimise the order of rendering primitives to save on framebuffer writing.

Initially, all stages of the pipeline were fixed functionality, but this was replaced by programmable vertex transformation (vertex programs in assembler, then vertex shaders in a shading language) needed for matrix-blending in character animation. Fixed functionality for pixel processing was replaced by register combiners (for baked shadows), then by fragment programs and fragment shaders. Geometry shaders were also introduced to handle deformation effects.
There are also feedback features where the output of the rendering can be made to a texture, and thus used as an input texture for the next frame.

All the latest DirectX and OpenGL extensions relate to setting up the geometry/vertex/fragment shaders for this functionality.

That is what Intel and software renderers have to compete against. They would be to implement a set of 3D CPU instructions that allow textured triangles, line and points to be rendered with fully programmable shaders from one block of memory to another (specified by pixel format, width, height, etc...). They could use the memory cache for this purpose, but would have to replace the FPU with hundreds of DSPs to achieve this. Otherwise they would have to provide free access to the framebuffer with hundreds of threads or cores.

Re:New market opportunity for render engines?

[Panaflex]

Otherwise they would have to provide free access to the framebuffer with hundreds of threads or cores.

I'm only just now starting out in CUDA development... but that is what the CUDA developers are aiming for next. Right now they're just allocating chunks of GPU memory for vectors/structs/etc.. but I'm fairly sure we'll get FB access in the near future.

Re:New market opportunity for render engines?

[Hal_Porter]

I think something like DirectX will be needed as an abstraction layer over the differences between NVidia and ATI. But also over different generations of one vendors graphics chips.

But I can accept that a DirectX that abstracts two very different general purpose architectures will be a very different API from the current DirectX.

But then DirectX, unlike OpenGL, was always supposed to be the thinnest abstraction layer possible over hardware differences. That's the reason it's such a hairball to program.

you wish

[heroine]

GPUs are going to be driven by the same things that drive game consoles & set top boxes. You can copy pure software, but you can't copy dedicated chips. You can copy video rendered by a CPU but not video rendered on a dedicated chip. Dedicated chips are going to stay cheaper than CPUs for many years. Just because you can decode HDTV in software doesn't mean there still isn't huge momentum behind set top boxes and DTCP enabled GPU's.

abolish standards based programming? Never

[heroine]

Standards based programming isn't going anywhere. That's crazy. We need Direct X, OpenGL, JMF, & MHP if only to outsource large chunks of the programming to cheaper divisions. How great it would be if everyone could base their career on hand optimizing ray tracing algorithms for SSE V, but the economy would never support it. These things have to be outsourced to select groups, firewalled behind a standard & higher paid programming done to the standard.

I read it differently

[Jabbrwokk]

I have a bit of trouble with tech-speak, but I read the article as suggesting the GPU was on its way out entirely, meaning on a multi-core system one (or more) of the cores could be used to render a game's graphics while the rest of the cores do the rest of the work.

If that's correct, I wonder if this would make the PC platform attractive to gamers and programmers again? Eliminating the need for a $500 graphics card - and one of the hurdles to programmers - all of a sudden makes a cheap, but processor-powerful PC pretty attractive for gaming.

Also, it would eliminate the "runs best on" nonsense.

Re:I read it differently

[Ilgaz]

I suggested my cousin to buy a "Macbook" (non Pro) trusting to its dual core Intel CPU and Intel X3100 integrated Graphics.

You should see how it doesn't perform on OS X and performs a little on Windows XP installed via bootcamp. The idea of installing XP via bootcamp to a Apple computer just to get game support and a bit performance shows the danger of GPU vendors death.

I don't think anyone can optimise better than Intel for Intel CPU, that should be even worse on other integrated graphics chip vendors.

"runs best on" has a far more evil alternative, "runs best on DirectX xx.xx on windows". At least with OpenGL, vendors such as Blizzard, ID Software can ship multiplatform games. Integrated graphics, no matter what Intel or their friends say, doesn't perform. Just look at the scandal system requirements of recent Intel only games for OS X. They keep saying "Integrated graphics not supported", I don't think they love the situation. I have seen a single game demo and its 3 FPS, right 3 FPS performance and became glad that cousin likes RTS kind of games :)

Re:I read it differently

[R4nneko]

At 3 frames per second I don't think he will be playing Real Time Strategy games, I think any game he tries will qualify for Turn based.

Re:I read it differently

[rho]

That's funny, I know somebody who plays WoW on a Macbook. Giant raids can get slideshowy, but most of the time it works okay.

I think you're expecting a lot from a $1000 notebook.

Re:I read it differently

[Ilgaz]

There are very talented developers at Blizzard who can actually code for OpenGL. They are also a legend for supporting any hardware. They are the ones who implemented "multi threaded" OpenGL right after OS X supported it.

WoW is really a rarity just like any other Blizzard game. I don't expect too much from a $1000 notebook, I just wanted to tell the experience of integrated graphics even if it was developed and supported by CPU vendor (Intel) itself.

Re:I read it differently

[rho]

Again, I think you have an expectation problem, rather than that Intel has a design problem. I use a first-edition Macbook all day, every day. I heap amazing abuse on it. Granted I don't play any games on it other than the (very) occasional Flash game. But the graphics are exquisitely adequate for my needs. And as WoW shows, a game can certainly fit inside the graphics chip's bailiwick. The problem lies elsewhere and is not inherent to integrated graphics.

Um, says who? I don't see it at all

[spoco2]

I just don't get how they can be saying this at all.

Ok... so from the article we have:

Take a 1999 interview with Gamespy, for instance, in which he lays out the future timeline for the development of 3D game rendering that has turned out to be remarkably prescient in hindsight:

        2006-7: CPU's become so fast and powerful that 3D hardware will be only marginally beneficial for rendering, relative to the limits of the human visual system, therefore 3D chips will likely be deemed a waste of silicon (and more expensive bus plumbing), so the world will transition back to software-driven rendering. And, at this point, there will be a new renaissance in non-traditional architectures such as voxel rendering and REYES-style microfacets, enabled by the generality of CPU's driving the rendering process. If this is a case, then the 3D hardware revolution sparked by 3dfx in 1997 will prove to only be a 10-year hiatus from the natural evolution of CPU-driven rendering.

Which they say is remarkably true.

HUH?

So, all these new GPUs really don't speed up your machine? So I can take my nvidia 8800 OUT of my box, leave it to an onboard graphics chipset and I'll be getting the same performance will I?

Yeah.

Right.

I don't see AT ALL how they're getting this right?

Please someone enlighten me, as I'm not seeing it from where I'm sitting.

Re:Um, says who? I don't see it at all

[Antitorgo]

It says it right there in the article.

Current generation GPUs are to the point where you can basically run arbitrary code on the GPU (as shaders) in high vectorizable code. So instead of the monolithic APIs such as OpenGL/DirectX, we'll see custom rendering engines that take advantage of the parallel cores running in GPUs and future CPUs (if Intel has their way).

So in a sense, he was correct. If you take CPUs in his 1999 interview to mean current GPU processors. Which is becoming somewhat of a misnomer nowadays with the CUDA/Firestream SDKs out now.

In the end, his point is that OpenGL/DirectX will probably go away in favor of custom rendering engines based on other techniques that will, in the long run, result in much better/different rendering techniques. (Ex: Voxel based rendering, Real-time ray tracing etc).

Re:Um, says who? I don't see it at all

[Ilgaz]

No, don't you dare to do such a thing buying Intel's claims.

Forget 8800, my Quad G5's NV6600 which is a scandal at Apple's part outperforms any 2008 integrated graphics.

I know you are being a bit sarcastic, just in case if anyone tries to do such thing :)

Looking at game boards really explains a lot about how integrated graphics (doesn't) perform. Even "integrated sound" itself is a huge joke taking down entire system performance.

Re:Um, says who? I don't see it at all

[GameMaster]

And, I would argue that even that part of his statement is absurd. Most game developers don't even want to have to write their own game engines much less do it from scratch. Even the ones that do write engines (id, Epic, etc.) will, for the most part, never give up their graphics APIs. APIs don't just give programmers access to specialized hardware, they also provide the developer with a raft of low level tools that he/she has no, reasonable, justification or wanting to spend the time re-developing. What will happen, is that APIs like Direct3D and OpenGL will change (as they have been all along) and simplify, as they won't have to handle all the quirky hardware anymore. The idea that engine writers will completely throw out APIs and will go back to writing everything from scratch in C/C++ is almost as absurd as suggesting that they'll go back to writing everything in Assembly language.

Re:Um, says who? I don't see it at all

[Antitorgo]

Actually, I think he says in the article that you'd see APIs like Direct3D and OpenGL staying around and leveraging the CUDA/FireStream APIs instead of relying on hardware specific features (at least, that is what is already happening at the driver level).

I think he envisions that new engines/APIs will come about (such as those that could be Voxel based or based on Ray Shaders, etc.) once the CUDA/FireStream APIs become more mature and the hardware gets faster.

Also, by breaking out of the hardware specific implementations of the SGI-based implementations (OpenGL/Direct3D) it would provide more flexibility for programmers to insert their own customizations into the rendering pipeline. With the hardware designed around one specific rendering design, programmers were limited. With the evolution of GPUs turning into highly parallel GPGPUs and CPUs turning into 80+ core beasts, it opens a whole realm of possibilities.

Re:Um, says who? I don't see it at all

[FuturePastNow]

But integrated graphics aren't about gaming. They're about driving a desktop display with minimal power, heat, and cost. Believe it or not, those three things are actually pluses for non-gamers.

So no shit, a Geforce 6600 (which is what, three years old?) outperforms Intel GMA. No one at Intel cares.

Re:Um, says who? I don't see it at all

[ClosedSource]

"The idea that engine writers will completely throw out APIs and will go back to writing everything from scratch in C/C++ is almost as absurd as suggesting that they'll go back to writing everything in Assembly language."

I agree that it's unlikely that APIs will be thrown out, but if they were, using assembly might actually be a better idea than C/C++. Were there really state-of-the-art games written in C in the pre-API era?

Re:Um, says who? I don't see it at all

[jjohnson]

Part of Sweeney's point that you're missing is that programmers are already writing raw shader code for DX10, and making more general use of CUDA. But more than that, DX and OpenGL are APIs to a particular model of 3D rendering that was, ten years ago, a good match to the benefits of the 3D hardware presented. Now that the GPU is a general purpose processor, the triangles pipeline model for renderers need no longer limit programmers, and they can implement different engine models (like Voxels) because it's all being done in software on a general purpose chip, whether that chip is called a CPU or a GPU.

Re:Um, says who? I don't see it at all

[Ilgaz]

Guess which vendor uses OpenGL functionality extensively, offloading whatever possible to GPU when possible? Even including text drawing. Apple OS X, since early 10.2

I have a serious question of the real power benefit of running a complete CPU powered junk rather than a low power Nvidia/ATI optimised for portable devices. If your machine draws a window causing 40% percent of CPU peak, you aren't saving power, you are wasting it since a GPU could do it almost requiring fraction of power.

Re:Um, says who? I don't see it at all

[shiftless]

You are correct in the short term, but the point is, turning the GPU into a more general purpose massively parallel number crunching maching and integrating it into the CPU will allow us to do AMAZING things in the long term.

In the short term people will continue using APIs like OpenGL and DirectX, whose designers will basically have to add a lot of code to perform all the functions that the high end GPUs used to do. Or I could see NVidia/ATI releasing a new closed-source "driver" that looks like a GPU to the OpenGL API, but is actually a software implementation that performs all the work using the new CPU instructions.

Then in the medium term, this functionality will gradually be reimplemented into OpenGL/DirectX, which will allow the API designers to add new, interesting and amazing capabilities relatively easily. Bear in mind that these new features will work exactly the same on every CPU that contains the now-standardized "GPU" functionality, and there is no need to upgrade your CPU to use them.

Perhaps at some point in the future your GPU-integrated CPU won't be fast enough to run the latest and greatest game with its special new rendering features, but all you have to do is upgrade to CPU v2.0 with its new and much faster integrated GPU to enable those capabilities. Intel, AMD, doesn't matter, you pick whatever manufacturer you like and have one less part to upgrade.

In the long term, expect to see totally new APIs (now regular ole libraries, to replace OpenGL/DirectX) based around this new way of doing things, and allowing basically unlimited possibilities. There could be entire new rendering "paradigms" aside from the standard vertex/triangle/triangle patch deal (i.e. raytracing) that could be implemented at full speed practically effortlessly. They all have the same basic requirements, which is the ability to crunch a lot of numbers really fast.

Let's not also forget that this now-general-purpose "GPU" can also be used to process things such as physics calculations, etc. It's an all-in-one solution to tackling any sort of math/processing problem of that nature.

This is the direction we are currently headed. More general purpose = more flexibility = more possibilities. Combine this with advances that are happening elsewhere, and the future is looking very exciting.

Re:abolish standards based programming? Never

[Bill, Shooter of Bul]

I think he was implying that it would be rolled up for most games in their engine. So developers would pay a licensing fee for the engine which sort of defined an api for the developers to use.

If we are going back to the "old" days...

[Brynath]

If we are going back to the "old" days...

Why can't we skip all this OS nonsense, and just boot the game directly?

After all that will make sure that you get the MOST out of your computer.

Re:If we are going back to the "old" days...

[crhylove]

I expect some intensive games to be bundled with their own custom linux and be DVD bootable soon enough. Especially if the new multicore architecture of the world invalidates directx!

Re:If we are going back to the "old" days...

[BigJClark]

This type of device sir, would be a console.

Re:If we are going back to the "old" days...

[TriezGamer]

Not any more. Consoles now have an OS as well.

Re:For once Ivan Sutherland Wheel of Re-invention

[elwinc]

See Ivan Sutherland's Wheel of Reincarnation. The idea is that CPUs get faster and graphics move there; then busses get faster and graphics moves to dedicated hardware; rinse and repeat. http://www.anvari.org/fortune/Miscellaneous_Collections/56341_cycle-of-reincarnation-coined-by-ivan-sutherland-ca.html

Re:Tim sweeny said the same thing 10 years or so..

[Antitorgo]

He is talking about custom rendering engines that would run on dedicated processors close to the video memory (such as on current generation GPUs). You could probably write a custom rendering engine in CUDA right now -- might not be all that great, but bandwidth would be a non-issue since it is running on the video card.

I think too many see his reference to CPU and think of CPUs running on the motherboard. Whereas he is looking at CPUs as any processor in the system, including the GPU cores running on current generation video cards.

Not sure why Tim Sweeney gets so much flack, he is the lead developer for a pretty popular 3d rendering engine...

Re:Poor title

[David Greene]

I think the point of the article is that computing paradigms are merging. You won't have a CPU and a GPU. You'll have one thing that looks like both. In other words, you'll have a multicore, parallel, vector machine.

And that, absolutely, positively, will happen. Larrabee, or something like it, is the future. If you hold AMD stock, sell now, because Fusion doesn't sound anything like Larrabee and is going to seem positively draconian by the time it comes out.

In some ways, these new processors will look like a Cray YMP on your desktop. It's a rough analogy but suitable for illustration. Of course there are all sorts of differences in the way the memory systems will work and that's a huge part of the performance equation.

It seems to me that Tim puts a bit too much faith in compilers. He talks about language extensions but only in CUDA-like terms of "where things will run." A compiler needs a lot of information to be able to vectorize. The user often has to provide that information in a language like C because of its loose typing, aliasing and side-effect rules.

My prediction is that some APIs will go away, but many of the low-level ones will stay because it's often faster to call into a hand-coded library than rely on the compiler to have enough information to automatically optimize the code. Eventually compilers will start pattern-matching to these APIs. Higher-level APIs will be developed to save developer time, not CPU time. They will exist almost purely for code reuse purposes.

I disagree with Tim that hardware vendors will differentiate on performance. At least, in the way he's thinking. It won't be hardware gadgets, vector length or number of pipes that matter. It's going to be the compiler, programming environment and libraries. To the extent that the hardware supports those in its ISA, hardware will matter. But the bulk of the muscles of the chip won't matter so much as their placement and utility (by the compiler). The inflection point is leading to a world where software is king.

Er, no

[Renraku]

I disagree.

Direct hardware programming has always been the best in terms of performance. However, it is the worst in terms of compatibility. If you're programming consoles, this is just fine. If you're programming for PCs, not so much.

It will never go back to programming for specific pieces of graphical hardware. I'd say that each vendor MIGHT make a major chipset, and that those chipsets would be coded for, and everything else gets API'd, but even this is unlikely. If a company had to have two or three sets of programmers for their graphics, each team for a different major chipset, we'd see more expensive games or prettier games with crappier gameplay.

Even the OpenGL/DirectX split takes a heavy toll on programming resources for game developers.

Re:Er, no

[Narishma]

You missed the point entirely. You don't write different code depending on whether the CPU is from AMD or Intel. In the same way, you won't be writing different code depending on whether the processor (CPU or GPU) is from AMD or Intel or nVidia. You will be writing in a high level language like C++ and it will be compiled to run on the CPU/GPU, not unlike CUDA. So you won't need API's like OpenGL or DirectX anymore.

Re:Er, no

[crunchy666]

You're thinking about this the wrong way. What you would be programming for is a general purpose language, with extensions (C/C++), then a compiler would generate the necessary binary code specific to the hardware. The compiler would handle the chip specific portions, leaving you to contend with creation of the graphics routines to render your scene (handling parallelism, vectorization, optimizations, etc.), in any way you see fit (rasterization, ray tracing, etc.) There is no reason why this would be any more difficult to support than the current DirectX/OpenGL APIs.

Re:Er, no

[starfishsystems]

It will never go back to programming for specific pieces of graphical hardware.

I agree with you. Contrast with the article, which seems to be trying to make approximately the same point by overstatement: "Graphics APIs only make sense in the case where you have some very limited, fixed-function hardware underneath the covers."

But of course graphics APIs continue to make sense - in the same way as operating systems continue to make sense - regardless of how rich or lean, slow or fast, or standardized or funky the underlying hardware may be. Last time I checked, there was still tremendous value in being able to offer appropriate and consistent abstractions for expressing algorithms and data, whatever the domain.

Consider the alternative. Taking the article at face value, each application would have to code all its own graphics abstractions in some lowest-common-denominator base language, "a language like C++ or CUDA," as it claims. If you discard the notion of an API, that means you make no provision for a higher-order abstraction above the base language, hence no reusability and no extensibility. That seems pretty retrograde to me.

If, on the other hand, you follow the standard software engineering paradigm, you will have a well defined API with a set of useful base classes and stable functionality for common operations. It will include full access to primitives, which may be the real issue that the article is trying to raise, though it fails to do so. And it will be extensible, so that the API can become richer as techniques evolve.

What's not to like about this? Well, it's not really newsworthy or controversial, is it? It's just software engineering as usual. Ho hum.

Re:Tim sweeny said the same thing 10 years or so..

[blahplusplus]

"Not sure why Tim Sweeney gets so much flack, he is the lead developer for a pretty popular 3d rendering engine..."

Just because he's a good programmer doesn't mean his statements about other things will be true, each statement must be taken individually.

Tim sweeney said 10 years or so ago the GPU would be integrated into the CPU, it hasn't happened.

Not only that the bandwidth requirements are off the charts for modern GPU computing. Sometimes I wonder if these programmers are even aware of wtf it is they are saying. I know lots of programmers who know dick all about the relationships in hardware. Tim sweeney borders on being one of those types of programmers. It's like he's so focused on development he's not seeing the forest from the tree's.

Also game engine's are many man projects, tim sweeny would be just one single dude on a team, nothing notable IMHO.

A rose by any other name...

[Ostracus]

"There are really countless possibilities. I think you'll see an explosion of new game types and looks and feels once rendering is freed from the old SGI rendering model. "

Hmmm. That reminds me. When are we getting rid of the Von Neumann architecture?

Anyone bought a hardware modem recently?

[gelfling]

Anyone bought a hardware encryption processor?
Anyone bought a Dialogic voice board?
No probably not. You see eventually all hardware collapses into the CPU(s).

But the problem is that Games and Graphics are the Lamborghinis of computing. Performance uber alles. So I'm not convinced that anything short of the biggest quad core machines has the general purpose in software performance sufficient to away with these big honking graphics cards. Maybe not and smaller machines can do it but it's just as likely you buy a nanoITX daughterboard for your PS3 (or whatever the analog is to that next year) to handle all the 'other' computing requirements too.

Poor Intel

[Ostracus]

"It seems to me that Tim puts a bit too much faith in compilers."

Itanium

Re:Poor Intel

[David Greene]

Actually, the analogy doesn't quite fit. Vectorization is quite a bit more comprehensible than the contortions compilers have to go through to get performance out of IA-64.

The user directives are easier, too. It's much easier for a user to say, "this loop has no cross-iteration dependencies" than for the user to mark which branches should be if-converted, which loads should be speculatively hoisted, etc. Thus for IA-64 the compiler ends up having to try to figure all this stuff out. In other words, it's much easier for the user to help the compiler understand the general parallelism available in the code than it is for the user to try to convey minute detail.

IA-64 is in many ways a house of cards. The compiler has to do a bunch of things (predication, static speculation, VLIW packing, scheduling, etc., etc., etc.) and do them all well to get performance. If it fails at one of them the code tanks. For something like Larrabee where there are multipel paradigms available (serial, vector, and multicore), you don't have to use all of them perfectly. You just have to use them well enough.

While it's easier to express parallelism, what's missing in Tim's responses is the acknowledgment that the user will have to do this.

You can read the Larrabee paper to get a good idea of the kinds of things Tim is talking about (software rendering and why it may just work).

Would be nice

[moniker127]

I hope that it goes back to programmer controlled rendering. I'm not a programmer by trade, but I like coming up with my own systems, and I cant tolerate the rules set in place by an API, which is why I havnt done much hobby graphics programming in a while. I predict that GPU's will die, and that we will just expand on multi core processing by automating it, so that programmers wont have to consider how many cores their application runs on, the load balancing will just be done under the hood, automatically. As a side note, why is it that source runs so well? I'm not really up to speed on it, but dosnt source use some unique method that runs it primarily off of the GPU?

Re:Would be nice

[moniker127]

Primarily off of the Cpu I mean.

And the question is "when?"

[The Living Fractal]

Assuming this does in fact happen in the future... when will it happen?

First let's be clear that by "it" I mean the complete disappearance of the GPU market.

To answer, first ask: who will be among the last ones to be willing to part with their GPU?

I think the answer to this question is: gamers.

Which leads me to the condition of the 'when' (not the date of course).

And that condition is: when the integrated GPU/CPU is capable of performing under the JND (just noticeable difference) threshold of what the separate GPU itself could do.

In other words, once they start to output graphics which aren't discernibly different (which are so realistic that you cannot tell the difference) then at that point the separate GPU will perish because there is no longer a motive for it to continue to exist, as no human perception can tell them apart.

 

Fall of DirectX

[zrq]

What I got from the article wasn't so much the 'fall of the GPU' as a move away from the fixed APIs e.g. DirectX and OpenGL.
In which case, would this be a huge bonus for Linux ?

A lot of people say they use Linux for work but have Windows installed on the same machine so they can run games. If the games programmers are going to be writing C/C++ code that runs on the CPU or GPU, then in theory this cuts out the proprietary graphics APIs and makes the games much more portable. Could this mean that more of these games will be ported to Linux ?

Re:Fall of DirectX

[mzechner]

well we already have opengl (es) for that which is remarkably cross platform. somehow microsoft was able to get 90% of gamedevs to use directx and we know how that went. were it not for directx games could be ported without hazzle. berkley for networking ( only a single function name different on windows ), openal for sound, a couple of lines of os dependant thread, input and window handling that can be hidden away ( sdl ) and some caution when doing file handling. sadly this is not the case ( except for id ) most people i ask about their preference of directx over ogl answer something along the lines of "better documentation and samples". however, imo the direct3d docs are a mess and a lot less helpful than the equivalent ogl man pages or specs. there's probably more samples for direct3d out there which makes copy & pastin easier i guess... so on the software end there's really no excuse not to write crossplatform games. i rather think that it's a matter of quality assurance and support cost. also, microsoft is supporting gamedevs a lot whereas that's impossible in case of linux in that form. apple probably also gives a rats ass on that matter. it won't be a bonus for linux imo. actually i don't believe it's going to happen as predicted. that would be going back to the old dos times where you wrote everything from scratch. i don't believe that'd be cost effective and as others stated those low-end parts will be outsourced and hopefully a new standard will evolve or we'll really be back to the dark age. people seem to underestimate the huge amount of work the hardware does these days without us graphicsprogrammers noticing.

Re:Who the hell is Jim Sweeny?

[Wescotte]

John Carmack's evil twin.

Re:Tim sweeny said the same thing 10 years or so..

[modemboy]

Ok so you state that memory bandwidth requirements for GPUs are off the charts. Where do you propose to get more memory bandwidth than on the CPU itself? Seems to me if you want memory bandwidth there is no better place to be than on the cpu die...

Failure to make PU specific files available

[tepples]

Could you get around this by "compiling" the game for individual PUs?

And having it failing to run on newly released architectures? GPU instruction sets change much more often than even the Mac platform's m68k to PowerPC to x86 to x86-64.

But it could be as simple as buying a game, and on install getting a PU specific file either from disc, or for newer iterations from a download server.

Until the publisher goes out of business or just decides to discontinue porting an older game's PU specific files to newer video cards in order to encourage users to buy^W license the sequel. And then you get into microISVs that don't have the money to buy one of each make and model of video card against which to compile and test a game's PU specific files.

Re:Tim sweeny said the same thing 10 years or so..

[blahplusplus]

"Take a 1999 interview with Gamespy, for instance, in which he lays out the future timeline for the development of 3D game rendering that has turned out to be remarkably prescient in hindsight:

2006-7: CPU's become so fast and powerful that 3D hardware will be only marginally beneficial for rendering, relative to the limits of the human visual system, therefore 3D chips will likely be deemed a waste of silicon (and more expensive bus plumbing), so the world will transition back to software-driven rendering."

Nuff said.

You are quite wrong

[Anonymous Coward]

You are very, very wrong. The history of computer hardware has been one where extra functionality is moved from the cpu for speed, folded back in a few years later for efficiency, and farmed out to an add-on card for speed some time later...

See http://catb.org/jargon/html/W/wheel-of-reincarnation.html for details.

Re:You are quite wrong

[gullevek]

and what would be an example for this?

I haven't seen FPUs, external cache, memory controllers as a comeback on any new mainboard.

I think that with so many core CPUs and so fast speeds that extra GPUs will be gone at some point in the future. There might *other* additional cards come, for other functionality. But the current GPUs are just something that is here for now.

Re:You are quite wrong

[Anonymous Coward]

The Wheel of Reincarnation has ground decisively to a halt, now that CPUs are going massively-multicore. The idea of special-function processing hardware is as dead as the idea of buggy whips.

Within a generation or two you'll find that multicore CPUs will do a better job of utilizing all of their silicon than dedicated GPUs. At that point there will be no turning back. nVidia and AMD are both in a world of shit, technically speaking.

Re:You are quite wrong

[sricetx]

The last PC I had that had a slot for external cache was a 486. This was around 1994, and even then COAST modules http://en.wikipedia.org/wiki/Cache_on_a_stick were a little difficult to find -- It's not like you could just walk in to the local Futureshop and pick one up.

It will be good riddance to video cards if that functionality moves to the CPU as far as I'm concerned. Especially if Intel and AMD continue with their recent trend of developing open drivers for their chips. Unlike other companies in the market, who only release binary blob drivers and deny serious problems with their current generation of laptop graphics chips http://www.theinquirer.net/gb/inquirer/news/2008/07/09/nvidia-g84-g86-bad.

Re:You are quite wrong

[Goaway]

and what would be an example for this?

How about, say, specialized graphics hardware being implemented in separate chips for home computers, and then later being discarded in favor of using faster CPUs to do the rendering instead?

Like what happened in the 80s and 90s.

Re:You are quite wrong

[amn108]

No he and the man who is quoted on the page he links to is very correct. If desired, a much more capable FPU then we already have may be implemented pretty soon that will put the triple-scalar AMD FPUs found in Ahtlons and whatever FPU designs are put to use in Intel CPUs all to shame. This is exactly the point of the quote. Intel only does not make a faster FPU because they do not need to (they only need to have it marginally faster each time to remain competitive) unless it becomes very important, which it will when GPU functionality is migrated back to CPU.If you consider that a dedicated modern GPU has FPU performance orders of magnitude faster than the CPU FPU, it would make perfect sense for NVidia and ATI to produce add-on FPU cards. But these are graphics companies and their chips call for more than a FPU unit, they make profits by keeping the users convinced that you need a separate chip to not only do FPU stuff, but draw triangles with textures. Perhaps it is because NVidia and ATI know that they would never be able to compete with Intel if a marked for dedicated FPUs would present itself, given the fact that Intel has been designing PUs since like 60s. They try to play it like we need dedicated graphics hardware, and they know it is not Intels horse, and with all the knowledge they have accumulated since it all started with GPUs, they are at the exact place they would like to be. They just want to play it like a board game, one dice throw at a time. But it is still a matter of demand. If Barack Obama decides we need to find aliens pretty damn soon because we're lonely and a law is written where every American is paid 10$ for every piece of processed data packet they send to SETI via the SETI@Home software, I assure you a dedicated FPU that shows what dedicated off-die hardware can do will be on the market very soon after. But a reasonable demand has to be there. It will be easier than before with HyperTransport, QuickPath and all. Before it was propriatary slots for each add-on for the system and an ISA bus. It was a special slot for the FPU f.e. I am not sure why they would not make ISA cards with FPUs THEN (though I am sure they did that too), but today, even if PCIx would not do, I am sure it is doable with some other existing means without reinventing whole motherboards.

As Good as it Gets

[Sponge Bath]

...a C compiler, a blank text editor, and a stifling array of possibilities

Assuming he means Emacs, then this is the way God intended it to be.

Re:As Good as it Gets

[Brynath]

Well only cause Emacs can control the butterflies...

Real Programmers use Butterflies

Re:As Good as it Gets

[StormReaver]

"Assuming he means Emacs, then this is the way God intended it to be."

Don't you mean GNU/God?

Re:abolish standards based programming? Never

[tepples]

So developers would pay a licensing fee for the engine which sort of defined an api for the developers to use.

Right now, computer graphics programming students use OpenGL. What would these students be able to afford should OpenGL become irrelevant?

Biggest problem is memory bandwidth

[Chris Rhodes]

If the CPU and the GPU are one, they compete for memory access. That's why the GPU is separate, and has its own memory. Putting them together is a bizarre idea.

Re:Biggest problem is memory bandwidth

[Chris Rhodes]

Yes, but usually only as an extension to their main graphics memory. Typically, GPU systems using system memory run slower than GPU systems with dedicated memory. Vista capable machines even have to have dedicated GPU memory to have a rating above 2, I think.

Also, current designs reserve a block of main system memory for use by the GPU, and prevent the CPU from accessing it. If the GPU was on the CPU, the same would have to be done, and there would still be a performance detriment, as they would both access memory over the same lines. I.e., it would slow down the GPU whenever the CPU had to access memory.

So really, sharing the same memory lines would be a huge detriment to systems used to dedicated memory. Of course, they could always increase the number of pins on the CPU, and block off some system memory for the GPU. Then they'd have to implement a different section of cache on the same chip.

That is why most graphics systems, even on laptops, especially under Vista, have at least a small amount of onboard memory. Sharing memory, even with separate, synchronized access, is a major performance detriment to a GPU.

Typically, FPU instructions are executed in their own block inside other compiled code. The program waits for the FPU to finish before retrieving and using the results. With a GPU, the paradigm is different, the program expects the GPU to work on its own while the CPU is crunching numbers or whatever.

There's really no comparison between moving the FPU onto the CPU vs. moving the GPU onto the CPU.

Re:Biggest problem is memory bandwidth

[Penguinoflight]

It's working pretty well for the xbox360. Granted, doesn't have anything similar to a PC memory bus.

Re:Tim sweeny said the same thing 10 years or so..

[blahplusplus]

"Ok so you state that memory bandwidth requirements for GPUs are off the charts. Where do you propose to get more memory bandwidth than on the CPU itself? Seems to me if you want memory bandwidth there is no better place to be than on the cpu die..."

Again you're missing the point, "the jack of all trades, master of none" problem, not to mention the space requirements. GPU's complexity is nothing like the old style co-processor units that were integrated into the core. They require ridiculous amounts of cutting edge ram to get that kind of performance, and they need a lot of ram to output the results of those calculations.

I don't see CPU's integrating 512MB to 2GB of ram in the near term future given heat and die size considerations, and we haven't even touched the extremely low bandwidth between modern cpu's and main memory in PC's (which is much much less then a modern GPU).

The GPU will play it's part for as long as is necessary. I don't rule out that perhaps one day it will be technically feasable but it is nowhere near that day, it's at least a decade or more away.

We've seen this time and time again, processors go through evolutions of integrating and seperating. We went from mainframes to PC's and with the net 'back to mainframes' but notice how each device play's their role, each one didn't totally obsolete the other, they just have become more specialized at their tasks.

Re:Hello world without OpenGL or DirectX?

[Narishma]

Then why aren't commercial games already distributed as C++ source code so that they can be recompiled for AMD or Intel CPUs?

This doesn't make any sense. I didn't say they would release the source code. Only that they would write directly to the CPU/GPU, without using API's like OpenGL, just like they currently do for everything that is not graphics rendering.

Then what will computer graphics programming students use to write a demo of rotating Platonic polyhedra textured with "Hello World"?

The same things they used before OpenGL/DirectX became prevalent. The same things they used during the DOS era.

Developer time is still valuable

[tepples]

I didn't say they would release the source code. Only that they would write directly to the CPU/GPU

Then how can publishers assure users that a title still on store shelves is compatible even with a PC manufactured after the game went gold?

The same things they used before OpenGL/DirectX became prevalent. The same things they used during the DOS era.

Which took an order of magnitude longer to get going from scratch than a typical GLUT or AllegroGL app. Because developer time is still valuable, software rendering won't make the OpenGL API go away, even if only because of OpenGL in software.

Re:Developer time is still valuable

[Narishma]

Then how can publishers assure users that a title still on store shelves is compatible even with a PC manufactured after the game went gold?

Did we read the same article? In the interview they talk about future processing units (either CPU or GPU doesn't really matter) that consist of a lot of small general purpose processors that you access like a regular CPU. Developers will use them just like they use CPU's at the moment.

Which took an order of magnitude longer to get going from scratch than a typical GLUT or AllegroGL app. Because developer time is still valuable, software rendering won't make the OpenGL API go away, even if only because of OpenGL in software.

OpenGL won't go away. It'll still be around for backwards compatibility but it will be obsolete. What software rendering will afford you as an engine developer is flexibility. I don't have time to repeat what's in the article anyway but the gist of it is that it will allow you to take advantage of the hardware more optimally than if you use a fixed function API like DX or OpenGL.

Re:abolish standards based programming? Never

[Eskarel]

Unfortunately, for a number of reasons, OpenGL is already relatively irrelevant.

If it weren't, linux gaming wouldn't be in the situation that it is.

DirectX is better established, better run, and better performing.(Some of this is because Microsoft didn't upgrade versions of OpenGL for a long time, but a lot of it is the fact that the OpenGL folks, like most standards bodies, spend so much time deciding on anything whatsoever that they get left in the dust).

If what this guy is suggesting becomes the case, rather than forcing every little guy out of the market, it might break the back of DirectX(which given that it's established, well known, and at the moment actually superior) which might mean more portable games, which would of course be good for Linux, Apple and everyone else.

Re:Tim sweeny said the same thing 10 years or so..

[binarylarry]

I think one thing is clear... the man owns a lot of Intel stock.

FYI he's talking about girth, not length.

[Anonymous Coward]

If they're not curling their toes and mimicking a seizure, you're doing it wrong, or she's got some issues.

http://www.goaskalice.columbia.edu/2524.html

Re:Who the hell is Jim Sweeny?

[binarylarry]

I think you mean John Carmack's retarded half brother, from his mother's second marriage.

What architecture?

[tepples]

Developers will use [future coprocessor cards] just like they use CPU's at the moment.

Which means they'll have to both be x86, or both be ARM, or both be some other architecture that NV and ATI can agree on. goodluckwiththat.

Re:What architecture?

[Hal_Porter]

I predict that NV and ATI will both invent their own mostly secret architectures and their legions of rabid fanboys will claim that 'their' company's architecture is superior. Microsoft will keep adding stuff to DirectX to work as an abstraction layer over the two.

Re:What architecture?

[shutdown -p now]

Which means they'll have to both be x86, or both be ARM, or both be some other architecture that NV and ATI can agree on.

You only need to standardize on the format of some kind of low-level (register-based etc) bytecode that's uploaded into those things; what it gets translated to before it's run is doesn't matter, and can be something very different in different cards.

Re:abolish standards based programming? Never

[spitzak]

I think you missed the words "graphics programming students" in the parent post.

They use OpenGL, whether or not it is better or worse than DirectX.

Mr. Carmack are you still around?

[icedcool]

I want to hear what John Carmack thinks about this.
Does he agree/disagree and why?

I always like seeing two giants in the industry debate on high level topics. It gives some insight into trends... and I just plain dig gaming anyway...

Re:Winmodem woes

[spitzak]

Winmodems are not really a good example of this. The CPU does not have special instructions or hardware added to it to make it act like a modem, it's just that it got fast enough that this was possible.

I think the best example is the FPU. That used to be a seperate chip and it was then incorporated into the CPU.

Hardware that disappeared because the CPU got faster is a different story and not really relevant.

Not yet

[Stan Vassilev]

What Tim Sweeney predicted back in the 90-s was that by 2006-7 CPU's would overpower GPU-s. It's the end of 2008, and we're at least 5 years away from CPU-s catching up with the future GPU hardware, and at least 10 years away from standards and engines emerging on mainstream hardware, which will challenge DirectX+OpenGL.

Now he's at it again, and I could understand him, as we often get caught predicting what we want to happen, for what we think will happen. General purpose vector computing power is far better and more interesting for a game engine developer, but he doesn't take any of the remaining economics into account.

For a new piece of technology to be picked up by most people, it has to provide substantial improvement over the current status-quo in some area that matters, while providing good DirectX/OpenGL drivers that people can run (at that point) over two decades of CAD/games/rendering software that uses those technologies.

Not to mention that those CPU-s with general purpose instruction sets, should somehow match the efficiency, cost and performance of GPU-s with fixed pipelines and limited instruction sets. Not a trivial task by any means.

Don't forget where the current growth is: smart phones, ultra mobile laptops, and laptops. Those devices tend to use far more dedicated hardware than desktops, because it's far more efficient (for ex. video/audio playback on smart phones is hardware accelerated).

Re:Not yet

[sznupi]

Oh I don't know...one of the reasons I could afford the smartphone I currently have is that it runs on SymbianOS EKA2 kernel, which has "sufficiently-fast real-time response such that it is possible to build a single-core phone around itthat is, a phone in which a single processor core executes both the user applications and the signalling stack... This has allowed SymbianOS EKA2 phones to become smaller, cheaper and more power efficient"

And most netbooks/laptops have integrated gfx of "classic" type; I'd guess those devices will be the first to be taken over by AMD Fusion/Intel Larabee style integrated CPU/GPU.

Enough with this Larrabee fear-mongering

[billcopc]

Every other graphics blog talks about Larrabee like it's going to take your baby away. It won't. It will suck, just like every other Intel graphics chip. The problem isn't that Intel can't make fast chips, it's that they don't have the graphics experience to come close to ATI and NVidia.

Intel is good at cheap, "good enough" graphics for office machines - the freebie that does the bare minimum, which satisfies 95% of all users out there. That's where they should stay. It's a good market for them, an excellent candidate for chipset/CPU integration, and an area the other players don't put too much focus on. Yeah, you can get NVidia/ATI graphics onboard, and they're decent, but it's not their bread and butter.

If anything, I fully expect the GPU to become more important over time, but it would be nice to have 100% complete software fallback, allowing any game or app to run on any hardware. You might get a slideshow with onboard video, but it should still be allowed to run. None of this DirectX hardware versioning - it defeats the point of having a middleware API in the first place! It would also let developers code whatever effects they want, allowing hardware to accelerate them if present, while still being every bit as stunning on software emulation - just slower!

It would mean an end to "DirectX 10 effects", which I think is a damned good thing! If anything, DX10 has shown us its tricks are nothing special, all of them feasible with DX9 primitives, just a little more of them. That sort of artificial feature bloat would be a thing of the past with the adoption of a general-purpose graphics programming language.

Re:abolish standards based programming? Never

[Bill, Shooter of Bul]

My understanding is that the majority of games are already building off of an existing engine, rather than the raw api frameworks. I would guess that computer graphics programming students would actually program in the native language, learning how to implement efficient graphics algorithms. Really creating a simple ( albeit insufficient) software 3d graphics engine isn't that difficult.

Re:abolish standards based programming? Never

[Eskarel]

Yes, my point is that OpenGL is already irrelevant, so the fact that it may become irrelevant in the future is really rather immaterial.

My machine language course in University was MIPS/RISC, and while a few more systems are using that now, it was pretty irrelevant then.

So long as the general approach to rendering doesn't change it won't matter all that much, and there will always be a cheap, but largely inadequate and irrelevant system for students to work with.

What about audio?

[serodores]

How long has audio been around? Have you ever seen an audio chip integrated into the CPU? Most of them are done by onboard chips, not on the CPU.

Re:What about audio?

[vux984]

How long has audio been around? Have you ever seen an audio chip integrated into the CPU? Most of them are done by onboard chips, not on the CPU.

They've moved from being standalone cards to being predominantly integrated into the mainboard and using the cpu for processing... rather like HSP modems, really.

Re:What about audio?

[mrchaotica]

Have you ever seen an audio chip integrated into the CPU? Most of them are done by onboard chips, not on the CPU.

That's because it's not performance-critical enough to need to be in the CPU. It has been integrated, though: into the southbridge, along with the disk controllers, USB controllers, modems, ethernet, and all the other random crap.

Re:What about audio?

[default luser]

Right now, with on-board sound you have an external amplifier + DAC/ADC chip. The CPU does all the controlling/processing.

Anything analog is tough to do on a CPU, so they typically don't integrate it. As mentioned in another post, this is also why the MODEM analog interface is not integrated on the chip, and it only uses the CPU for digital signal processing. Analog components today are as "integrated" as they're ever likely to be.

The things that have been completely integrated on the chip in the last two decades (MMU, FPU, cache, memory controller, SIMD/DSP, multiple cores) have all been fully digital. Further, the integration happened after the devices became "good enough" for most users. This bodes well for the integration of the GPU, for there are already on-board GPUs that are "good enough" (e.g. AMD 790), and DVI/Displayport mean that a completely digital video system is possible in the future.

Re:Utter nonsense

[ToasterMonkey]

RTFA, idiot. You didn't read it, you accused someone of being payed off, and you made assumptions based on the context provided by a bunch of tween posts on an Internet forum. You are either playing Slashdot bingo or you're a tool.

OpenGL, and DirectX force you into rendering a specific way.

If the hardware was fully programmable, you wouldn't need to be forced into doing things the OpenGL/DirectX way, and we might see a resurgence of past software rendering techniques. OR, you could still do things the OpenGL/DirectX way... He says this in the God-damned article, you choad.

This guy has no clue at all or has been paid off.

Right, so you are probably too young, or too stupid to know what past software rendering systems were. Why do I fucking bother with you? Because it's fun and you make me laugh.

You use OpenGL so ... that you have a portable interface to the graphics system

For Christ's sake, Tim Sweeney makes it perfectly fucking clear that he feels when CPUs are powerful enough or GPUs programmable enough, we'll be able to do all our rendering in C++... again. You know, that PORTABLE language that's older than you are? Holly shit balls, Batman, how does Doom run on dos/windows/mac/linux/game boy/cell phone/my toaster without being written to one of these 'portable interfaces to a graphics system'?!???!111
OH, because all rendering was written in C for a single processor? Yes, and even some fugly non-portable assembler, dick boy. Damn, code 'libraries' aren't going away, just the narrowly focused window through which we access a certain category of hardware.

The whooooooooole point this guy was trying to get across was you won't need a portable little window onto the graphics system when the hardware is fully programmable, and general purpose. At that point a portable language will do, and yes, we already have those, been there, and done that.
*WOOSH*

MOD PARENT DOWN (flamebait)

[shentino]

nt.

Re:Tim sweeny said the same thing 10 years or so..

[breeze95]

Tim sweeney said 10 years or so ago the GPU would be integrated into the CPU, it hasn't happened.

Isn't Intel working on integrating the GPU and CPU? Hence, the monthly statements by Nvidia that they are not afraid of Intel. With the development of multi core CPU it is now possible to economically integrate the GPU into the CPU. I expect in about 2 years we will see the first multi core CPU with an integrated GPU from Intel. This would make Sweeney prediction about 10 years early. I guess he is a futurist.

Imagine That

[nick_davison]

In other news...

A man whose company makes its money writing game engines says, "APIs are going to go away. It's going to be very, very hard to build a game engine in the future when you can't rely on the APIs anymore. So everyone'll have to switch to the few companies that build game engines instead. Like mine. I recommend you start now and save yourself the headache."

Hmm, I detect no bias whatsoever.

Well, almost as little as when nVidia tells the world that they have seen the future and it's in GPGPUs replacing CPUs. Amazing how everyone has seen the future, it supports their business model and the rest of us can save ourselves a lot of pain if we jump on what pays them well.

Re:Imagine That

[Goaway]

Except, you know, he was saying the exact same thing ten years ago when he wasn't making his money that way.

Special vector unit?

[marcovje]

Call me stupid, but from what I saw from Larrabee it centers around a new specialized very wide vector unit to do most of the work. So far for a any plain old C compiler

Re:Tim sweeny said the same thing 10 years or so..

[TheThiefMaster]

It's not uncommon for lower-end graphics cards to use ordinary DDR2 (the GeForce 9500 GT has a DDR2 version), and you can get some surprisingly powerful GPUs integrated into a motherboard chipset, which don't have the 100W+ coolers a cpu or dedicated cards can (there's an integrated GeForce 8300 in an AM2+ motherboard).

I wouldn't be at all surprised to see the lower end GPUs being embedded in the CPU instead of the chipset.

Tim Sweeney has vested interest for this

[jokkebk]

It's also interesting to note that the guy being interviewed is in the business of making 3D engines.

Now ask the question: would 3D engine makers perhaps have something to gain if OpenGL and DirectX would be scrapped, as the interview suggests?

Most game dev labs wouldn't have the resources to build their own engines from the scratch using a C++ compiler, making them to - wait for it - licence a 3D engine like the one this guy is selling.

So in summary, the article paints a picture from the future which would be very beneficial to interviewee, so I'd take it with a grain of salt. Either we'd get some de facto 3D engines replacing OpenGL and DirectX, or the game developers will waste time recreating each new graphics technology advancement into their own engines.

Re:Tim sweeny said the same thing 10 years or so..

[blahplusplus]

"I wouldn't be at all surprised to see the lower end GPUs being embedded in the CPU instead of the chipset."

You're still ignoring the memory bandwidth issue, the bandwidth to main memory is still at least 10x worse then on card DDR2 with wide memory interface. Integrated on CPU means that the memory that will be used for the gpu will be main memory which is infinitely slower then on card memory, this is lost on the integrationists.

Re:Tim sweeny said the same thing 10 years or so..

[TheThiefMaster]

Infinitely slower? Even with the memory controller in the CPU as well (like all modern cpus), and using DDR2 (or even DDR3) ram?

What about my comparison with graphics chips integrated into motherboard chipsets? Especially the example I gave, which has to go to the CPU for memory access anyway, as for that CPU socket the CPU has the memory controller, not the motherboard chipset? That one seems to work well enough to me, and could only work BETTER by being integrated into the cpu, with access to a better power rail and being closer to the memory controller and ram.

Re:Tim sweeny said the same thing 10 years or so..

[blahplusplus]

There are other problems like the latency issues. If you have the GPU on the CPU, you still have latency to and from ram, along with other programs, not just the GPU. You have bus competition between two sets of devices in which the memory is shared between programs and the gpu itself which effectively cuts the bandwidth by a decent percentage when the program is running.

Look what happened to dedicated soundcards

[ciderVisor]

I think the point of the article is that computing paradigms are merging. You won't have a CPU and a GPU. You'll have one thing that looks like both. In other words, you'll have a multicore, parallel, vector machine. And that, absolutely, positively, will happen. Larrabee, or something like it, is the future. If you hold AMD stock, sell now, because Fusion doesn't sound anything like Larrabee and is going to seem positively draconian by the time it comes out.

In the '80s and '90s, if you wanted decent sound, you had to fork out for a dedicated soundcard. Something like the Creative AWE32 springs to mind. These days, all the sample replay, effects processing and mixing is done by the CPU and a buffer's worth of data is placed in memory waiting for a DMA transfer to a pair of DAC's.

From what I could gather from TFA, Sweeney is predicting that the processes which are currently handled by a dedicated 'Graphics Processor' will instead be executed by the main general-purpose CPU(s) and fed to a frame buffer for final display on the monitor.

Re:Tim sweeny said the same thing 10 years or so..

[master_p]

Couldn't the hardware remain relatively the same (i.e. dedicated fast VRAM and bus) but the programs sent to it be simple C++ programs that are parallelized with the help of the compiler?

I don't think Sweeney is that away from reality. He did not say that all the graphics will be run from the main PC bus. He said that the fixed function processors will be a thing of the past, and since they will be general purpose, there would not be a need for an API between the applications and the hardware.

Well yeah, but....

[crhylove]

And that word processor will STILL run better than Microsoft Word!

Wishes can come true.

[crhylove]

Ah, but with a multicore chip, you can HAVE a bunch of copied dedicated chips. :) Particularly as we scale up to 16 core or 32 core chips.... Suddenly you could have the equivalent of 30 Geforce GPUs and still retain a dual core CPU for the rest of the game code.

Unlikely.

[crhylove]

I think a more likely scenario is an open source ray tracing or other rendering technique engine running most games and 3d applications. Of course there will be some competitors and clones, but they will eventually fade away as the GPL code improves inevitably over time being worked on by so many game companies and other 3d programmers constantly. You might even see a scenario where a single graphics engine runs 90% of 3d apps, games or otherwise! I'm extrapolating 10 years, I don't expect any of that to happen over night.

A decade or more away....

[crhylove]

A decade away from now we will likely have massively multi core quantum chips, running at 256 bits or more. To even pretend that that kind of CPU wouldn't easily run ray tracing with 20 of it's cores tied behind it's back is pretty silly. I would shorten the time scale to 5-7 years. But I'm not profit, either, it could happen even sooner! The trend to more and larger multicore chips is pretty inevitable. When everyone's cell phone is a beowulf cluster of 128 bit x86 architecture cpus, a lot of this whole discussion is going to be completely invalid.

Re:A decade or more away....

[blahplusplus]

"A decade away from now we will likely have massively multi core quantum chips, running at 256 bits or more. To even pretend that that kind of CPU wouldn't easily run ray tracing with 20 of it's cores tied behind it's back is pretty silly. I would shorten the time scale to 5-7 years. But I'm not profit, either, it could happen even sooner! The trend to more and larger multicore chips is pretty inevitable. When everyone's cell phone is a beowulf cluster of 128 bit x86 architecture cpus, a lot of this whole discussion is going to be completely invalid."

Note people like you said the same thing in the Pentium 2 era, and even long before that in the mainframe era, etc, it hasn't happened yet. Then there was the big Pentium 4 slowdown and moore's law slowed to a crawl for a while. We'll see what the future brings, but I really doubt it will be in 7 years, you're forgetting the cost factor and the amount of traces needed.

It will definitely be at least 10 years before integration is necessary, what you're forgetting is the bandwidth requirements and software requirements are going to keep increasing along with CPU power and companies like Nvidia aren't just going to sit on their asses.

Re:Tim sweeny said the same thing 10 years or so..

[crhylove]

OK, so he was optimistic about 3-4 years. So?

OpenGL renders on the CPU just fine

[s_p_oneil]

Even if GPU's go away, there's no indication that libraries like OpenGL will go away. As a 3D graphics programmer who uses OpenGL and shaders, I know that no one will want to reinvent the wheel and rewrite all that low-level rasterizing, perspective correct texture mapping, and TnL code. Even when using shaders for as much as possible, you don't have to do all that by hand.

Re:Poor title

[mikael]

The main interest from the scientific computing community is that the graphics card can both run a numerical simulation and render the results both at the same time without having to bung all the data across between different processors or computers. In the past programmers would have to run their simulation on a supercomputer (maybe be allocated 64 nodes for a couple of hours) and then pipe the data into a SGI workstation to render. Now, they can have a desktop PC in their office with 2 Gigabytes of GPU memory, 1000+ stream processors and a nice high resolution plasma monitor to visualize everything.

End of the GPU...

[justleavealonemmmkay]

Will it be replaced by the NKVD (iN Kernel ViDeo) ?

Sweeney is clairvoyant ;-)

[amn108]

Sweeney is right IMO. It is also one of the more important interviews and I recommend it to all that are currently doing computer graphics. This is essentially a RISC vs CISC debate again, and it shares all the common arguments with it. Given most of the industry today is a RISC design abstracted to a CISC interface, I agree with Sweeney and think that the true maturing will happen when the CPU and GPU will merge. Intels battleplan is many core approach. Specialized hardware is always faster clock for clock, but programmers appreciate generics everywhere, and since we do not pay for our GPUs per computation result they produce, an idle GPU is a waste. And it is idle half the time, being specialized as it is. Except of all the folks who play WOW 24-7.

breaking up function still makes sense

[Dan667]

I think the problem with this argument is that it still makes sense to break up the functionality into separate hardware. Then those that want to go hog wild with some aspect of the hardware can (like graphics). Putting all of the functionality into the CPU seems like it would lead to stagnation of technology, because only a couple of players would be able to make any real advances.

Death of the GPU != death of the API

[DrVomact]

I don't understand why Sweeney thinks that the "death of the GPU" would entailthe death of all graphic APIs. By definition, an API is a programming interface that helps the programmer do certain jobs without having to code low-level stuff over and over again, right? So if graphics rendering starts being done by the CPUs instead of a dedicated GPU, how does this change the situation?

Yes, I understand that the APIs will have to be rewritten to accommodate the underlying hardware, and that some restrictions that were imposed by GPUs may go away if we use a general-purpose CPU instead...but why in the world does that mean the game programmers of the future will be confronted by nothing but a blank screen and a program editor? Don't we have APIs now for writing programs that run on CPUs? I'm pretty sure that in addition to the program editor, they'll still have graphics APIs—just newer and better ones.

In fairness, I have to say that this is a pretty short and shallow interview. Maybe Sweeney meant to say something like, "people who write game engines will have to start over from scratch, and rethink their basic tools". Maybe.

Power matters.

[argent]

One comment dropped almost at random in the middle: "if it consumes far less power when you're running old DirectX applications".

Or if it consumes less power, period. And power is money. These guys are looking at the $600+ PC market. When they made the prediction about the death of the GPU in 2006 they were making games that took $2000 PCs to run well... and I'm sure a $2000 PC in 2006 would have been happy without a GPU. Now they're making games that take $600 PCs to run well, and those are high end. You can get a laptop for what, $400 now? You're not going to get a CPU or a GPGPU getting that kind of performance in a $200 PC or a $400 laptop for a few years, and by that time what's the low end going to be?

Re:Tim sweeny said the same thing 10 years or so..

[Grishnakh]

Again you're missing the point, "the jack of all trades, master of none" problem, not to mention the space requirements. GPU's complexity is nothing like the old style co-processor units that were integrated into the core. They require ridiculous amounts of cutting edge ram to get that kind of performance, and they need a lot of ram to output the results of those calculations.

I don't see CPU's integrating 512MB to 2GB of ram in the near term future given heat and die size considerations, and we haven't even touched the extremely low bandwidth between modern cpu's and main memory in PC's (which is much much less then a modern GPU).

The GPU will play it's part for as long as is necessary. I don't rule out that perhaps one day it will be technically feasable but it is nowhere near that day, it's at least a decade or more away.

Don't forget, integrating different functions onto one chip is much easier said than done. Putting RAM on a CPU die isn't that easy, and doesn't work that well, because the process technologies used between them are different. A lot of people think you can just put all kinds of different circuits on a single die (logic, memory, analog, etc.), and that just isn't the case. It can be done, but there's huge tradeoffs involved.

Agree. Next OpenGL: OpenRT?

[argent]

So what if the kinds of things that OpenGL and DirectX have implemented can now be done "efficiently enough" in high level code instead of dedicated hardware (albeit by making the whole system more expensive and power hungry... and that matters), that doesn't mean there isn't room for dedicated hardware, it just means the dedicated hardware needs to be doing something more sophisticated.

The raytracing acceleration in Slusallek's RPU was able to get 2008 CPU performance out of a low density FPGA implementation in 2005... with about the same hardware budget as a late '90s Rage II GPU, using an API called OpenRT - Open Ray Tracing.

Re:Tim sweeny said the same thing 10 years or so..

[blahplusplus]

Don't forget, integrating different functions onto one chip is much easier said than done. Putting RAM on a CPU die isn't that easy, and doesn't work that well, because the process technologies used between them are different. A lot of people think you can just put all kinds of different circuits on a single die (logic, memory, analog, etc.), and that just isn't the case. It can be done, but there's huge tradeoffs involved.

Thanks for the heads up. Everybody thinks it's as easy as shrinking the die, and circuits = same everywhere.

The thing they are not understanding is the architecture and actual structural geometry. Things are made of stuff, which has to be shaped properly, then the data has to circulate and be stored somewhere within a given period of time, and routed around, etc, etc.

It doesn't exist on the xbox 360

[Chris Rhodes]

The graphics controller on the xbox is separate from the cpus. The whole point of the article is merging the cpu and gpu onto one chip, requiring memory access to go over the same lines.

Re:It doesn't exist on the xbox 360

[Penguinoflight]

The xbox360 actually uses GPU memory for the whole system. Kind of the opposite of a on-board video card on a PC, the xbox360 cpu must connect to the GPU to access memory.

Since the memory is fairly fast, the system still remains resonsive, most of the time.

It still doesn't exist on the xbox 360

[Chris Rhodes]

It is not the architecture referenced in the article, where GPU commands are integrated with the CPU commands. Please examine the specs from Microsoft:

• HC17.S8T4.pdf

There is a dedicated GPU, with a memory controller on-die. The CPU cores are on a separate die, and contain a separate instruction set. The CPU's memory access is tightly linked with the GPU so that the CPU can send geometry data to the GPU with no delays.

If anything, this represents a third architecture, more different from that discussed in the article than it is from a PC shared-memory architecture (except in the PC, the north bridge mediates the memory access, instead of the GPU.)

My point is entirely that if you merge the CPU and GPU command set into the same core, you will slow the system down. The XBOX360 gets speed from linking the CPU's geometry data send to the GPU's memory access, in essence allowing it to get a free pre-fetch. That's not what the article is about.

Re:It still doesn't exist on the xbox 360

[Chris Rhodes]

Even further, the article states the the increase in speeds of CPU's will enable CPU-driven rendering. However, this seems to be illogical, since any dynamically generated graphic first requires processing a large amount of memory, then generating the data (be it pixel color data or geometry data), which has to be stored somewhere.

Ultimately, the article's premise would require a reversion to old-school pixel color data written to frame buffer memory. Each subsequent stage in translating the original 'concept data' to the dynamically generated graphic would require the CPU to constantly read and write memory. Since current designs trend towards multi-core systems not utilizing 'shared memory' (unlike systems like the XBOX or the PCI bus,) memory accesses are staged over the same FSB.

This demands that the CPU be fast enough, per frame, to handle both the graphics processing and whatever traditional logic programming games require or will require. Given the fact that each new generation of games tends to stress both GPU and CPU to the limit, and simple tasks like path choosing can't be sufficiently addressed by the clock cycles remaining to the CPU, this whole thing strikes me as a pipe dream.

Re:Utter nonsense

[gweihir]

The problem is not the hardware. It is the interface. You think people will design a new graphics library with a new interface easily? Think again. OpenGL is an interface first and an implementation second. Same for DirectX. Ever heard that there are software implementations of OpenGL? I wonder what they are for? Maybe so you do not have to start from scratch and do not have to start from scratch and so that you do not have to work with some hiome-coocked interface?

I re-iterate: This guy has no clue.