Slashdot Mirror
John Carmack Discusses 360's Edge, Considers DS
Via a Gamasutra post, John Carmack's comments on upcoming id choices. Game|Life has a few quick comments on Carmack's hope to bring Orcs and Elves to the DS. This would be id's first game on a Nintendo platform in some time. Likewise, he makes it clear that he considers the 360 the dev platform of choice due to the ease of development on the console. From the article: "the honest truth is that Microsoft dev tools are so much better than Sony's. We expect to keep in mind the issues of bringing this up on the PlayStation 3. But we're not going to do much until we're at the point where we need to bring it up to spec on the PlayStation 3. We'll probably do that two or three times during the major development schedule. It's not something we're going to try and keep in-step with us. None of my opinions have really changed on that. I think the decision to use an asymmetric CPU by Sony was a wrong one."
Nope, Carmack was just responsible for Doom... Doom II... Quake... Quake III... Quake III... Doom III...
You're after Romero's head. And not the one in map30.
At least no one can say that he doesn't know what he's talking about. It's kind of hard to disprove that man's intellect when it comes to software development.
I think you're wrong about id's dedication to writing good 3D engines for the hardware of the times, regardless of complexity. Quake3's engine, for instance, allowed for multithreaded rendering when nobody else was even considering multiple CPUs.
It wasn't completely stable -- and I wonder how many people actually turned the feature on -- but it was multithreading *way* ahead of its time on the gaming front.
C
The Sun is proof that we can't even do fire properly.
Most americans who see VB also say "wtf are they doing it this way for?".
"their tools are more based on Asian design and logic"?
What a bizarre concept. When I worked for a software company in Japan, the only way that culture intruded into software development (aside from an organizational structure that makes Dilbert's seem positively effective) was the fact that programmers who could used English manuals. Japanese manuals and other reference/instructional materials were so vague and information-poor that the language barrier was easier for my co-workers to overcome than reading materials in their native tongue!
I will grant you that Asians (East Asians, at least) seem much more comfortable with interfaces that we find incredibly cluttered. But that's not programming as such.
And with a funny accent.
No reason to lie.
The hardware designers at Sega probably thought much like the way you do. That's the idea behind the massive failure called Saturn. Keep in mind, Sega's arcade boards used to have multiple processors. If their coders could deal with that, why wouldn't everyone else?
Now, don't get me wrong, I love the Saturn. Good developers could do amazing things with it -- Radiant Silvergun, Panzer Dragoon Saga, NiGHTS, Powerslave, Virtua Fighter 2, Astal, Guardian Heroes... but most chose to develop for the slightly less powerful and far more developer-friendly PlayStation.
And why? Because it made sense. It's not just a matter of developers being lazy or unskilled; if it is too hard to develop for a system, that also means doing so will take longer and cost more.
Circumcision is child abuse.
http://www.ps3forums.com/showthread.php?t=52467
This thread has been one of the funniest things I've ever seen. All the PS3 fanboys are bashing Carmack for his comments about Cell, despite the fact that it's quite clear none of them program at all, let alone program on asymmetric CPUs.
Hilarity ensues as people who would have been lauding Carmack to the skies if they'd seen only his gripes about the 360 CPU attempt to prove that he's totally irrelevant and afraid of learning about technology.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
The trend towards and away from specialized processors comes and goes.
I think Carmack's view of the PS3 is a lot more realistic than yours. He's talking from the perspective of someone who has to ship a product or his friends don't eat. You're talking about what would be nice if programmer time were free.
FWIW, I have a PS3, which I am using to do Cell development. It really is very impressive... And it also really is a lot more work than a more traditional multicore system. The decision to specialize an extra time here reflects Sony's PS2 design (crappy CPU with two very impressive and non-interchangeable vector processors to make up for it), and I think it also reflects Sony's arrogance; they simply assume that, of course, people will be willing to spend twice as long developing software on their system to get a noticeable but not earth-shattering improvement in performance.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
You would have to read my post about software engineering vs. development to understand a bit more about where I am coming from. I don't like that Sony has changed architecture in every console release, this has been a detriment to what they can achieve, but now they have IBM on board so I doubt the Cell Architecture is going away anytime soon. For now it is harder to program and takes a little more time, but that will change, as it does with every new architecture. As more development tools are available the programming will become easier, as more libraries and engines become available it will become even easier. Once you start thinking parallel you actually see that it is easier than not as it allows better division of code and work. If the next generation of servers, consoles, and consumer electronics are powered by the Cell Processor, as IBM, Sony and Toshiba claim, then you will see the necessary tools and mind shifts happen at a very rapid pace.
As a Software Engineer I grow more an more disgusted with the x86, and to a lesser extent Power, architecture every day. Ah if only I could go back to the CBM days.
True, however that was many years ago, when Carmack still had some ambition and imagination. I'm not saying Doom III was bad, but...
At any rate, it's trendy to bash the PS3 lately. It really is a leap in console capabilities (and I don't even like/buy/use consoles period), but all the negative comments about developers avoiding the PS3 are creating a self-fulfilling prophecy. The problem is people viewing it as a hurdle rather than an opportunity.
The fact of the matter is that multi-threading is here to stay, and if developers ignore it on the PS3, they're only delaying the inevitable. It's a safe bet that the XBox 1080 (or whatever) will have multi cores, and of course the PC industry is full-steam ahead on that front. Granted, the architectures may be completely different, but the principles involved are the same.
Oh well, I'm tired of listening to myself bitch about all the bitching. I guess we'll see what happens.
https://www.eff.org/https-everywhere
I don't think you quite follow the essence of the complaint.
With the absolute best tools conceivable, even with tools that we don't even have the technology to build, Cell will still be harder to develop for than a more conventional processor.
Yes, it's more powerful; probably a LOT more powerful. It's still mork work to get anything done.
Your claim tht it will be "easier" as it allows "better division of code and work" is just plain nonsense. Any division of code and work I can do on a multi-core system, I can do on a single-core system, too. That division is already available to me. The extra work Cell imposes is that I have to divide it asymmetrically; I can't just partition the task in whatever chunks the task makes sense in, I have to partition a lot of it in terms of the very specific requirements of the SPEs.
That work won't go away, even with perfect tools. It's harder, and it will always be harder.
I think it's probably worth it for supercomputing. I'm less sure that it's worth it for consoles, because game development costs are a plague upon the industry, and making them worse won't help.
Will it get easier than it is now? Yes, but the underlying fact that it is "harder to program and takes a little more time" won't. What might change is that, right now, it takes a lot more time; that might be plausibly reduced. But, in the end, making Cell just as easy as a multicore SMP system is in the same bin as lossless compression that is guaranteed to compress ALL possible inputs.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
oremor nhoj em llik tsum uoy emag eht niw ot
The enemies of Democracy are
I wonder if they would do Doom for Wii so that I can really jump around and smash my wiimote into things it's never smashed into before.
Australian running a company that does C# / C++ / Java / SQL / Python / Mathematica
True, however that was many years ago, when Carmack still had some ambition and imagination. I'm not saying Doom III was bad, but... ... but you're still going to imply that Carmack is a luddite who can't adopt to new technology. What-fucking-ever, he is still what he's always been -- a pusher of new technology, and a great determiner of what technology actually works in practice and what is theoretical pie-in-the-sky wishfull thinking. Just because he is no longer the only man on earth who is serious about pushing the envelope on 3D technology doesn't mean he has lost his ambition or imagination.
The problem is people viewing it as a hurdle rather than an opportunity.
It's a hurdle and an opportunity. Practical people see this, theorists don't. Hell, even IBM admitted that it was a hurdle for programmers in an architecture talk about Cell that I attended at UT.
It's a safe bet that the XBox 1080 (or whatever) will have multi cores, and of course the PC industry is full-steam ahead on that front.
Are you mental? The 360 is already multi-processor, and multi-core is just a performance optimization to reduce communication overhead between processors (while making DRAM access more expensive). It already requires multi-processor programming which Carmack is an early adopter of. You think he doesn't know how to write a multi-threaded application? Please. His point, and a very good one at that, is that it is harder to write multi-threaded code when some of your processors have drastically different capabilities than others. Like I said even IBM, the creator of Cell, agrees with this assesment, so it is nothing but bald-faced denial of reality to pretend otherwise.
The enemies of Democracy are
True, however that was many years ago, when Carmack still had some ambition and imagination.
Blame id's game designer for the game design, Carmack is a coder and making the game fun is not his department. He makes it run and easy to modify, the game designers make the gameplay work as intended after that.
Doom 3 has a great engine but most of its capabilities aren't in plain sight. Someone made bots for Doom 3 and just by using the AAS (with the AAS data automatically calculated for the MP maps) they were quickly able to navigate levels better than many bots that were in development for years, no waypoints necessary.
Justice is the sheep getting arrested while an impartial judge declares the vote void.
I think maybe you misunderstand the issue here... it's not that multithreading is hard. In fact, the XBox 360 has three cores if I remember correctly... any game for the XBox 360 is multithreaded. The problem is that the Cell processor uses heterogeneous cores... that is, there is one main core which is basically a regular PowerPC core, as well as 8 additional cores (1 of which is disabled on the PS3) which are completely different, designed for number crunching on large amounts of data, and are radically different to program for. Having actually done a good bit of programming on this architecture, I find that it is not actually that difficult to get a program to take advantage of a reasonable amount of the processing bandwidth in the Cell processor... however, it is VERY difficult to utilize a larger amount of the available computing capability of the processor. Therefore, most developers have found it to be much easier to get X amount of performance out of the XBox 360 than it would be to get X amount out of the PS3, even though in theory the PS3 is capable of X+Y.
Not only is there the issue of 1 main core with 8 additional cores, the 8 additional cores don't have random access to main memory and are really only efficient at doing stream processing - they are tuned for vector operations. I'd agree that the GP(and GGGP) missed the point. The issue is not single core vs. multi-core - it's standard multi-core shared memory architecture vs. whack-ass-we-designed-it-ourselves-DSP-style-multi -core. Developers get the idea of multithreaded programming - it's hard, but they understand it. It gets much harder when you have to manually handle memory transfers for each of the threads to ensure they have the data they need to process.
So, it sounds to me like you haven't DONE any Cell development.
There's a huge difference between Cell and "PPC with vectors". It's called local store. Each SPE has 256KB of local storage. You have to have your code in there, and then stream data through. That means you have to do a fair amount of setup and partitioning specifically around that 256KB limit, which wouldn't apply on a multi-core PPC. That's a real issue, and you can't just paper it over for real projects.
And yes, in theory, the tools should be able to hide some of that from you. That's why Carmack's comments about the tools are so damning. If there has ever been an architecture which desperately needs polished and mature tools, this is it.
Also, I don't know where you get the idea that SPEs can do scalar code. They are 100% vector-only. The closest they can get is to emulate scalar code by ignoring the rest of a vector while manipulating only its first slot. That can be done, but it leaves you with a very slow processor spending a lot of its time masking things out and merging vectors together. (or, if you just omit those slots, and use only one slot out of each potential vector, your data takes much more space; 4x as much for 32-bit objects, 16x as much for bytes.)
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
> At any rate, it's trendy to bash the PS3 lately. It really is a leap in console capabilities (and I don't even like/buy/use consoles period), but all the negative comments about developers avoiding the PS3 are creating a self-fulfilling prophecy. The problem is people viewing it as a hurdle rather than an opportunity.
I'm a PS2 programmer; as "bad" as the PS2 is to program for, the PS3 is FAR worse; getting any sort of decent performance out of the PS2 involves utilizing each processor, and now our job is (at least) 7 times harder on the PS3?! Only a masochist would want to program on that thing. Give me ram, lots of it, and not fragmented into tiny little pieces. And few, but faster processors, then many, and slower. There are only so many tasks that can be parallized.
Even on the XBox, MS's tools were miles ahead of Sony's. Most of Sony's PS2 tools haven't been updated in ~4 years, and you wonder why developers are avoiding the PS3?!
And you have the gall to tell me and others it's a self-fulfilling prophecy, when people, like me, are saying the PS3 is "hard, dam hard" _based on past experience_, when you've never even programmed the PS2? Boy, are you naive!
Maybe _you_ want the "opportunity" of staying up late chasing down DMA bugs, trying to figure out why the hell your streaming engine isn't loading some data fast enough (guess what -- streaming is even MORE troublesome 'cuz we have 4 times the memory to fill, but yet the DVD transfer rate has only increased by 2 on next-gen consoles), trying to debug VU code when one of your models isn't skinning properly, trying to figure out where you're going to fit all the game assets in memory, etc, but I've jumped through enough hurdles, that I don't want or need any more then necessary, because I have better things to do (such as implementing the game), then fighting broken, and limited hardware. The principles are indeed the same, but the devil is in the details, and frankly, we're getting tired of having to spend such insane amounts of time on them.
And yes, I do actually love programming the PS2. The risk/reward ratio is very fulfulling. The XBox (1 or 360) even more so. But people aren't bitching when they are stating facts -- "Programming the PS3 is hard. Period." The risk/reward ratio is out of line compared to other consoles -- and we have to ask "Why? Why does it have to be so difficult?"
Maybe Sony will wise up, and realize that "when you make it -easy- to develop on your system, people will -want- to, and be are more then happy to spend the time expirementing. It's all about minimizing the cycle: code new feature - compile - link - export assets - convert to native format. Make it easier on the developers and we will love you -- make it harder and we will hate it. It's not rocket science, only computer, and social science.
Anyways, I've rambled on long enough.
Cheers
No, Carmak is NOT using DX9. He's using an Xbox 360 - that's something completely different. Granted, the API looks a lot like Direct3D 9, but it's not actually compatible, and it exposes 100% of the hardware features directly with minimal (virtually zero) driver overhead, no stupid driver bugs, no emulation, no possibility of missing features...
On a PC, OpenGL and Direct3D 9 are almost exactly equivalent. On a modern engine, 70% of the code is (or could be, if you put a little effort into the design) API-independent. Things like scene traversal, resource management, batching, loading and generating geometry and so on. In the API-specific part, you have a small piece of code that does useful work, and in an ideal world this is all you'd need. It simply manages resources for you, so you can (for example) tell it to load a mesh onto the video card and it does it, or give it a triangle mesh and tell it to draw it.
The larger part is dealing will all kinds of stupid garbage. For example, in Direct3D 9 you can lose any data you have stored in VRAM at any time, so you have to detect this condition, re-load (or regenerate) all the data, and fix it before you can do any more rendering. It's not difficult, but it's more work. In OpenGL (and on games consoles) you do not need to do this. With OpenGL, you need to detect and load appropriate extensions, which you don't have to do as much in Direct3D 9 (you do have caps bits, which serve much the same function). So in the OpenGL code, you might need a couple of extra paths to deal with hardware / drivers that don't support specific extensions. You could have four separate paths for things like render-to-texture, or several different texture formats for floating point textures, you might have to deal with drivers that don't support S3TC compression, and so on. You've got to deal with the operating system, other software components you're using, and the whole thing's a huge mess.
That entire part is unnecessary on a console. You have one piece of hardware, which maps directly to the available API, with 100% of all features exposed, including things that PC APIs deliberately prevent you from doing. It makes writing code so much simpler - you write some basic, low-level rendering code which interacts directly with the hardware, then get on with the useful task of writing the game engine. You never have to worry about whether or not you've hit a slow path (happens in OpenGL a lot) or a bug (happens in Direct3D a lot, and OpenGL if you're using Intel's rather crappy drivers) in the driver, because the drivers don't actually do anything more than queueing command packets for sumbission to the video hardware.
The same applies to the Xbox, but also to the GameCube, the Wii, the Dreamcast, and to a lesser extent the PS2 (because you have to write all rendering code in VU assembly to do anything useful).
I am well aware of the software cache, and auto-SIMDization. I was involved in getting the tutorials on those topics prepped last fall, and I've done a bunch of other writing on this. I am a big fan of Cell, and I spend more time than is probably healthy immersed in reading or writing about it.
The software cache imposes a fair number of cycles of latency per access even when the datum in question is in the cache; it's even slower when the datum isn't in the cache. Is it useable? Certainly. However, it's going to give you a serious performance hit, especially when used heavily. You can't just use it and get full performance from the SPE without worrying about local store; you have to use it sparingly only for access to large chunks of data, while keeping the bulk of the data you're working on in local store all the time. It does not solve the problem; it mitigates it.
The auto-SIMDization stuff does not make scalar code efficient. What that does is, in some cases, automatically convert scalar operations on arrays into vector operations on the same arrays. It doesn't solve the more general problem of operations that aren't being performed on whole arrays; those stay slow. what's impressive is that they handle the alignment problems involved in vectorizing, e.g., a[i] = b[i + 1] * c[i + 3];, not that they have magically cured the problem.
So we're back to where we started. The cell development support and compiler can give you comparative ease of use, but you lose a lot of the performance potential of the processor. All those theoretical numbers you see are based on the assumption that you are doing 90% of your calculations with no latency; not waiting for DMA, not accessing a cache, and so on.
So, really, you do have to do a lot of extra work to get the theoretical performance, or even very close to it. The software cache and other techniques mitigate this, so you can get enough performance from the SPEs to benefit from them somewhat without having to do anything exceptionally elaborate, but they don't give you the theoretical performance. If you want that performance, you have to do a lot of fiddly little management of, for instance, the tiny local store available on the SPEs. If you want to have that handled automatically, you take a very noticeable performance hit.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
I think its more of a total pipeline issue. When you look at software development tools coming from a largely US based company (MS for example) you've got a defined pipeline for development process. You've got IDE project templates. You've got command line tools that you can batch process files on as part of your build process. It's just more integrated. Looking at the solutions that come from Sony or Nintendo (though Nintendo is way better at this than Sony) you tend to get a loose collection of tools that aren't really integrated. This means you have to create all of the glue in-between each layer. That takes time and engineering effort. Sometimes this means greater flexibility, but again, it's all about how much time do you want to spend for your flexibility?
Honestly, developers don't want to re-create the wheel each time for their build processes, but these kinds of things end up forcing that on them. The first chance they have to jettison it, they're going to.
- Sighuh?
It just pains me to see possible innovation being killed because it's difficult!!! Is the PS2 tough to develop for, heck yes...PS3...even harder...remember when this was a good thing? The 360 is great, easy, straight forward and BORING!!!!!! Don't get me wrong, I like writing code for the thing, because it makes my job darn easy and I pretty much know what I am going to get on the back end.
...
You're an idiot!!!
I developed games for years before I 'burnt out' on the deadlines and schedule
The fact is no one wants to program for hardware that is 'interesting and challenging' they want to produce software which is 'interesting and challenging'. It is very difficult to produce an advanced 3d engine on a piece of hardware when you're fighting for adequate performance and as you add complexity you're constantly fighting with the hardware to get stable performance.
Essentially, an architecture which is 'difficult' to program for make creating high performance applications like building a sand castle in the rain.
You're a fool if you think developers only want "the most powerful box". There's a lot more to the industry, heck software programming in general, than power. Also, you completely ignore the fact that if you were right, then you'd have developers flocking to the Xbox, since it was the most powerful console last generation.
-- jchenx