Slashdot Mirror
The Technical Difficulty In Porting a PS3 Game To the PS4
An anonymous reader writes "The Last of Us was one of the last major projects for the PlayStation 3. The code optimization done by development studio Naughty Dog was a real technical achievement — making graphics look modern and impressive on a 7-year-old piece of hardware. Now, they're in the process of porting it to the much more capable PS4, which will end up being a technical accomplishment in its own right. Creative director Neil Druckmann said, 'Just getting an image onscreen, even an inferior one with the shadows broken, lighting broken and with it crashing every 30 seconds that took a long time. These engineers are some of the best in the industry and they optimized the game so much for the PS3's SPUs specifically. It was optimized on a binary level, but after shifting those things over [to PS4] you have to go back to the high level, make sure the [game] systems are intact, and optimize it again. I can't describe how difficult a task that is. And once it's running well, you're running the [versions] side by side to make sure you didn't screw something up in the process, like physics being slightly off, which throws the game off, or lighting being shifted and all of a sudden it's a drastically different look. That's not 'improved' any more; that's different. We want to stay faithful while being better.'"
You realize the PS3's hardware WAS exotic right? That's exactly why it's hard! Write code optimized for multiple SPE units, and see how well you can get it to run on x86.
I don't know about that. Sometimes there is an inherent trade-off between being machine-friendly and human-maintainer-friendly. Tuning for machine performance sometimes gets in the way of high-level abstractions that make porting to a different architecture easier.
Reminds me a bit of the Story of Mel:
http://www.cs.utah.edu/~elb/fo...
Table-ized A.I.
That's easy! I wouldn't try to run SPE units code on x86 but x86 code on x86. The "optimization" shouldn't have affected the Game or it's portability and that's why it's their own damn fault.
Optimized PS3 code is not what you think it is. Its not taking some general C code and rewriting it a bit to be friendlier to the underlying CPU architecture or rewriting it completely in assembly. The PS3's Cell processor is, in simplified terms, a general purpose CPU and six special purpose coprocessors. So optimization is really figuring out how to pull major pieces of code out of the general purpose and move it to the appropriate specialized coprocessor, and then add the control code necessary to coordinate the two. Or to put it even more simply, optimizing for the PS3's cell processor is really an exercise in parallelizing code 7 ways.
Mainly because the PS3 was a radically different architecture, and when the company had to do assembler-level optimisations to get the game working well, and now they're trying to recode those optimisations for x86, while ensuring the shaders, physics, lighting etc all work perfectly, it's an impressive project.
Its not about assembly code, this mischaracterizes the problem.
Optimizing for the PS3's Cell architecture is not simply rewriting some critical code in assembly. Its more of a parallel processing effort. To greatly simplify things the Cell has a general purpose CPU and six specialized coprocessors. The trick to Cell optimization is moving code from the CPU to the coprocessors and keeping those coprocessors as busy as possible. Now add the complication that the coprocessors are not interchangeable.
Assembly has little to do with it. Its architecting the code to keep 7 parallel processors going, whether the underlying code that implements this software architecture is C or assembly doesn't matter a whole lot.
You realize the PS3's hardware WAS exotic right? That's exactly why it's hard! Write code optimized for multiple SPE units, and see how well you can get it to run on x86.
There's actually some validity to the GP's post.
Ideally, you would write the game portably, knowing that you will need to potentially take it to market on a lot of platforms, if it ends up being a popular title, and so as a result, you'd have a minimal porting set that could just be compiled and run, with additional optimizations on top of that tuned for the platform on which it's going to run.
Although not done a lot recently, the implementation of the original libc had C versions for all the code contained therein, and then had hand optimized assembly versions that would replace the C versions on a specific platform.
The intent was to be able to get it up and running on a new platform fairly quickly by having a small *required* assembly language footprint in the context switch, bootstrap, and CRT0 code, and then optimize the C code to assembly on a platform specific basis once you wer up and running self-hosted on the platform. This also gave you the opportunity to check assembly optimizations in user space first, without breaking everything by trying something that wouldn't work because of some mistake, and ending up with a lot of work to back the changes out (this was back in the RCS/SCCS days, where source code control systems weren't as capable as they are today).
It makes sense to do the same thing for games; minimally, the complaints they had about shaders should have been totally workaroundable, given that Direct X doesn't allow indefinite termination shaders, and requires the code to be fully unrolled compared to, say OpenGL, where there's no guarantee that a shader terminates (one of the reasons a game can crash a Mac or Linux using OpenGL, but can't crash Windows, using the OpenGL compatibility layer -- if it won't unroll, then it's discarded by DirectX).
In any case, it does show that there were at least some corners cut, and just because the host library is similar, you shouldn't expect the hand tuned code to be at all similar, especially going from a Cell architecture PS/3 (essentially, a data flow processor) to Von Neumann architecture on an AMD processor on the PS/4. It's obvious that all the hand tuned pieces would need to be rewritten, just as if you were porting to Windows or XBox or some other platform that wasn't also Cell-based. You'd think if they had planned ahead for ports to other platforms other than the PS/3, that that planning would be directly applicable to geting the code running on the PS/4 as well.
Alternatively they're really good programmers who got explicitly told "make this run like shit off a shovel and don't worry about portability - this will only ever be on PS3". You can say "but we should really write portable code", but if SMT still tell you to ignore portability then you're left with either doing what you're told or quitting.
Yeah, I had a sig once; I got bored of it.
Well, I'm not so sure about that. They designed their game for the horribly quirky PS3, which means that they'd either end up wasting much of the console's power or they'd twist the code until their video game works like a streaming application - which is the only thing the Cell can do efficiently. Since portability was not an issue (they knew the game was a PS3 exclusive) they decided to go with a PS3-specific design in order to get the most out of the hardware, thus making their game more appealing and thus more profitable.
They could decide between "write this platform-specific program in a suboptimal way so that it's easier to port to another platform" or "heavily optimize for the sole target platform in order to increase market success". I don't think the latter was an invalid choice. If anything, this story illustrates just how bad an idea it was to put a Cell in a game console.
USE HOT GRITS WITH STATUE OF NATALIE PORTMAN (NAKED AND PETRIFIED)
Is it a good practice for cases like these? I argue not.
Let's say that we do create a reference version, then optimize. Since we are intending to push the hardware to its maximum, we have to assume that we will hit the occasional performance wall. How do we deal with that? We change the behavior of the program to fit within the limitations. This means that our definition of correct behavior has changed and the reference version is no longer correct. So, we update our reference version to match the version we plan to publish. This involves backporting the changes, then carefully testing to verify that the behavior is exactly the same as in the optimized version.
We're left with the question: Why do we call it a reference version if it is derived from the version that is supposedly derived from it?
The optimized version is the real reference version. The "reference" version is really just a port to a hypothetical platform. And, rather than just porting the final version, we are porting every bit of wasted effort along the way.
We get all the cost of the PS3 to PS4 port, dragged out over the whole path of development, with no target platform to sell it on. Sure, porting "reference" to PS4 will be cheaper than PS3 to PS4; however, PS3 to reference to PS4 will be much more expensive than directly porting PS3 to PS4.
So, best case, we spend more money to save time porting to a platform we never intended to support. Worst case, we spend a lot of money on a port that doesn't go anywhere. It's a lose-lose.
Had they followed the practice, they would have a version of the source code that runs correctly (but slowly) that they could optimize for different target platforms.
I expect that when they started, they had no intention of porting to other platforms.
Naughty Dog is Sony these days. They only make games for Sony platforms. So they targeted only the PS3. I'll bet when development started, Sony hadn't finalized PS4 plans.
Now the PS4 is out and desperate for games (go ahead, name a PS4 exclusive), so Sony is having them port it to PS4. And since the game was never intended for anything other than the PS3, they're running into difficulties.
I wouldn't blame the programmers for optimizing for the only platform they were told to target, I'd blame the managers for suddenly springing a new platform on them after the game was done.
You are in a maze of twisty little relative jumps, all alike.
Oh, yeah ... that is the other mistake. "No one's ever gonna look at this code again, so it's okay if it's an incomprehensible, unmaintainable mess. Ship it!".
No... You're not talking about some code that's designed to be portable, you're talking about code that was targeting exactly one piece of hardware, and trying to make the output look better on that exactly one piece of hardware than anyone else developing for that same exact one piece of hardware. Because of this, games for consoles have an insane level of optimisation for that one piece of hardware. More so in this case, you're talking about a piece of hardware that had one of the most exotic CPUs in decades. Porting from that to x86 is a huge challenge.
I'd suggest that the PS3 is radically different enough that if your *reference design* wasn't PS3 specific, you've probably already failed.
You can't "optimize" a bubble sort into a quick sort.
(One interesting effect of both the PS2 and PS3 was that their design was so bizarre that it took years for programmers to be able to optimize effectively. This meant that games were consistently better every year even without any changes in the hardware. With a more conventional architecture, what you buy in the first year will essentially be state of the art for the life of the console. Makes it much harder to get people excited about "this year's version of xxx".)
Because the PS3's hardware was stupid. It never paid out the kind of performance dividends they claimed it would, and to even approach them took insane amounts of effort.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Now the PS4 is out and desperate for games (go ahead, name a PS4 exclusive)
Umm wait, I think I can name one..ummm. Resogun? Ummmm Infamous Second Sun? And there's a Brown Shooter or two I can't remember the name of. [looks over at the PS4 sittingmostly idle because of the lack of games in genres I want to play....maybe I should have got a Vita instead. I do play DCUO on it now and again....but that is not exclusive the PS4]
Because the PS3's hardware was stupid.
No, it was not.
Yes, yes it was. I didn't argue that cell had no purpose, but it was stupid to use it for gaming. And sadly, cell was only ever widely used in scientific computing in PS3s, because the only other way to get it was madly overpriced. Then Sony removed OtherOS and the supply of PS3s usable for scientific computing was constrained, although by that time the cell had been far surpassed.
Sony gained dominance in part because development for the Playstation was simpler than development for the Saturn or the Jaguar. Microsoft gained a foothold in part because development for the PS2 was a super bitch. Then Sony went on to make another console for which development was difficult, and the 360 became the dominant console of its generation. Microsoft probably wouldn't even be in gaming today if Sony had adopted a more conservative architecture for the PS3. It took them until the PS4 to figure that out, and they did indeed finally manage it — by making the same console Microsoft was making, or vice versa.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
*sigh*.
I see my analogy failed.
If my job is to write a program that does a sort, and I write a bubble sort, then if I really need it to work better, I don't tweak it, I redesign from scratch. Most people won't call that an optimization - they'll call it a redesign. Hence my original comment. (And yes, if we're being pedantic, *if* the sort is part of a larger program, then I would call rewriting the sort an optimization.)
My point was that if you don't *start* by taking all the strengths and weaknesses of the PS3 into account in the initial design phase, influencing or even dictating many of the the large scale design decisions (including the actual game play), you are unlikely to make maximum use of the machine.
You could call rewriting the design an optimization, but I don't think that's how most people think about it.