Slashdot Mirror
Doom 3 Source Code: Beautiful
jones_supa writes "Shawn McGrath, the creator of the PS3 psychedelic puzzle-racing game Dyad, takes another look at Doom 3 source code. Instead of the technical reviews of Fabien Sanglard, Shawn zooms in with emphasis purely on coding style. He gives his insights in lexical analysis, const and rigid parameters, amount of comments, spacing, templates and method names. There is also some thoughts about coming to C++ with C background and without it. Even John Carmack himself popped in to give a comment."
..also nice.
I'll be here all week.
It's just a personal point of view about coding style... some things like vertical spacing using the braces (at the section "Doom does not waste vertical space") are just the opposite of a readable source code (just in my opinion, of course, as someone that makes a lot of source code reviews of other people)
A developer needs to make an application for the hardware people have, not the hardware he wishes people had. Otherwise, he's likely to end up limiting his market to a subset that's not big enough to turn a profit.
In some ways, I still think the Quake 3 code is cleaner, as a final evolution of my C style, rather than the first iteration of my C++ style, but it may be more of a factor of the smaller total line count, or the fact that I haven’t really looked at it in a decade. I do think "good C++" is better than "good C" from a readability standpoint, all other things being equal.
I sort of meandered into C++ with Doom 3 – I was an experienced C programmer with OOP background from NeXT’s Objective-C, so I just started writing C++ without any proper study of usage and idiom. In retrospect, I very much wish I had read Effective C++ and some other material. A couple of the other programmers had prior C++ experience, but they mostly followed the stylistic choices I set.
I mistrusted templates for many years, and still use them with restraint, but I eventually decided I liked strong typing more than I disliked weird code in headers. The debate on STL is still ongoing here at Id, and gets a little spirited. Back when Doom 3 was started, using STL was almost certainly not a good call, but reasonable arguments can be made for it today, even in games.
I am a full const nazi nowadays, and I chide any programmer that doesn’t const every variable and parameter that can be.
The major evolution that is still going on for me is towards a more functional programming style, which involves unlearning a lot of old habits, and backing away from some OOP directions.
One might suggest that every good programmer, if they spend enough time improving, eventually moves toward a more functional programming style.
"First they came for the slanderers and i said nothing."
I know some situations else-if statements are necessary, but my understanding is that case statements are far faster. So this "beautiful code" perhaps should have focused on code that could process faster as if this is one mistake in efficiency, wonder what else ID missed?
The only reason to take *screenshots of code* is if you somehow broke copy/paste.
I've developed for large game and non-game projects, and each needs a different approach. Console games especially have serious problems with dynamic memory allocation (they don't typically have swap files and can die due to heap fragmentation) so you have to avoid a lot of convenience libraries like STL.
STL, however - especially in newer compilers that support C++0x - is actually quite good and is very, very robust. It's a good way to avoid a lot of the memory management bugaboos that happen when you *are* doing lots of dynamic/heap allocation. So I would very much endorse a sane amount of STL use in desktop code.
The other thing that rubbed me the wrong way here was public member variables. Since inlining and move semantics make getters and setters essentially free, there is no good reason to expose bare, public variables on anything but the simplest, most struct-like objects. The biggest source of weird, hard to trace bugs in our code at the game studio were often due to people modifying public members of other objects in unexpected ways or at unexpected times.
Having public, non-const member variables actually hurts a principle the author supports, which is "Code should be locally coherent and single-functioned". This means that an operation should do one thing and put you in one of several known and easily discoverable states, even on failure. That is, if I say, make this guy do X, then either he does X or he fails and ends up in a known state. If that state is available in the form of modifiable public data, then his state can get messed with at any point along that path by some other code, and the final state (in cases of success and failure) is not fully known. At the very lest, making data private means that only certain code paths can modify the data, and it's much easier to keep state coherent.
Anyway, that's just my $0.02.
No end users give a fuck how 'nice' your code looks if the end result is a pile of shit in shades of brown and black.
So is this why they turned out such garbage? they were only looking at the code and not the end result?
Really this story sounds like some excuse... "well at least the code looks nice!"
Years ago I had to have this fight with a co-worker. His electric mode in Emcas was writing out everything as a single tab char, and doing the indenting on its own. The problem is, not everything obeyed this style -- not even a little.
I had to explain to him that since we weren't all using Emcacs, he needed to be sure to leave the code in a usable state because his commits were leaving the whitespace as complete useless crap to everything else.
I don't care what editor you use, but it needs to leave the files in a usable state for everybody else. Having special formatting to work with your editor just pisses everybody else off.
Lost at C:>. Found at C.
I really liked this bit, because it's something I've been really focusing on for the last year or so, and I think it has significantly improved my code:
Comments should be avoided whenever possible. Comments duplicate work when both writing and reading code. If you need to comment something to make it understandable it should probably be rewritten.
Comments can be useful, IMO, but primarily only for generating documentation (think Javadoc or doxygen, etc.). Other exceptions include bits of code that perform highly-optimized mathematical calculations, in which case I think the best solution is to write a proper document and then add a comment linking to the document, and bits of code that do something which apparently could be done differently but for some other reason must not -- assuming that explanation doesn't belong in the doc-generating comments.
Other than that, I find it makes my code a lot better if every time I find myself wanting to write a comment to explain some bit of code's purpose or operation, I instead refactor until the comment is no longer necessary. Often it's as simple as taking a chunk of code from one method/function and pulling it out into another with a well-chosen name, or else introducing a variable to hold an intermediate value in a calculation, with a well-chosen name. Sometimes the fact that a bit of code is hard to explain is a strong indicator that the design is wrong, that stuff is mashed together that shouldn't be.
The bottom line is that I've found eliminating comments does more for improving the readability of my code than anything else, and I've gotten similar feedback from colleagues whose code I critique by pointing out that they can eliminate their comments if they refactor a bit.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Indeed the code is beautiful and well organized. I think it took me 10 minutes to locate and disable the copy protection, after spending 30 minutes just marveling at the organisation and targeting out code i'd like to steal later. (mostly physics and maths!) Another 5 minutes cleaning up something i don't remember to have it compile. Source download -:> Ready executable ~ 45 minutes. (compiled on an x6).
Can I light a sig ?
And a note on the relative evil of comments; bad or not, well placed comments have saved me an awful lot of time when taking on maintenance of code bases in the past. Most of the time they can't present a design document to you, or if they do it covers the design at the start of the project, a decade and a half earlier. Code is a method of communication between two programmers, but if the code doesn't suffice to illuminate the design the original programmer had in mind, I'd really appreciate a comment explaining his thoughts. Especially if the particular section of code is complex, and especially if I'm the guy writing it and end up being the guy maintaining it a couple years later.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Lets face it, its 8yo code.....
I found both the article and what JC wrote to be highly informative and rather validating of my approach to things. In software, we usually get little validation because of the wide variety of opinions of who we work with. We've all seen the extremes: The hard core C programmer who can't be bothered with any OO nonsense, and who advocates inspecting the assembly of every method you write. The C++ hippie who sees everything as some kind of exercise in getting the compiler to write the code for you.
I'm sure most of us follow a more balanced approach. C++ has to be about performance over anything else, otherwise there are plenty of other languages that accomplish much greater degrees of expression, but can't cash the performance check. But, expressibility is important, too, because performance goes out the door once we stop understanding what the code is doing. It's nice to have a language that lets you express things somewhat functionally, yet gives you the flexibility to wring out serious performance.
It is pointless to make f(const float parm) const because it cannot change whether it is const or not. C++ passes by value. If you really want to be purposeful say f(const float& parm). This is a basic review item. Read Lakos. The author doesn't know what he is doing.
"Things seem wiser when you become older and senile"
In the name of all that is unholy, I summon the King Troll by uttering his vile name three times:
APK! APK! APK!!!!!!
I read this article yesterday when it was on Kotaku. I have some issues with some of the statements, several of which (such as comments and {} formatting) really depend on the editor being used.
I think the author believes that the Doom 3 code is beautiful because it matches his own coding style preference. Truly beautiful code is beautiful not because of style, but because the code is extensible, robust, and maintainable. Many of the arguments in the article touch on several of those points, but don't really tie in strong enough to definitively prove the code to be beautiful. For me, the most important of those three is the last mentioned, maintainable. A programmer is happy when the code they must maintain (perhaps written originally by another person or team) is easy and straight forward to maintain. Robustness and extensibility tie into maintainability. I say code is beautiful when I can spend minimal time fixing errors and maximizing the number of new features in that code.
after the first three conclusions, and i stopped reading so i can't speak for the rest. should be: 1.) const as appropriate, not "const everything possible". const can fuck you hard in OOP if you use it wrong, 2.) you can never have too many comments, and 3.) tight vertical spacing is archaic and stupid, unless absolutely necessary for some display reason
if this guy was interviewing here and mentioned all the things in his article, i probably wouldn't hire him. too much "religion", as it were, which is a huge red flag for me because it's usually masking something...
The people that worry about coding style don't write good code. It might look nice, but is fundamentally they don't spend enough time on functionality.
I have found consistently that the big office blowhard developer that sends out numerous emails and writes documents about the company's coding style practices is typically the person who is the worst programmer in the bunch. He only writes beautifully coded bugs.
I haven't thought of anything clever to put here, but then again most of you haven't either.
This is a perfect example of why code beauty is subjective. On the one hand the author loves saving vertical space by not putting parentheses on their own lines, then he hates it when you remove them entirely, even though it saves even more space. He finds conditionals without parentheses to be less readable. I find non-horizontally aligned parentheses less readable, and have no problem following conditionals without them.
I think where it becomes less arbitrary (but still subjective) is in the actual design. I don't see the point of not using STL. Being able to write 1 hash (or use someone else's) without needing multiple copies in the code simply to deal with different types seems great to me. Yeah it's a little harder to read, but you only need to read one of them, and you know all the hashes are the same unless there are methods overloaded for different types. Checking that each part of 2 classes that look identical except for types, are actually identical is tedious.
Sometimes there is a tradeoff between code readability and design. I tend to favor design over readability. Inheritance makes code less readable, but if done correctly, it makes the design much better by reducing redundant code. Redundant code may be perfectly readable, but it makes it much harder to "read" the design of the application.
As an engine programmer at one of his competitors, I can assert that I completely agree with all of his opinions, especially "if it needs comments, it should probably be re-written" and "const everything as possible". Also we use our own STL-like containers, we don't use an off-the-shelf STL implementation because most of them are more bloated than we would like on consoles (e.g. too much inlined code)
always give enough comments to ensure that a drunken idiot programmer could understand what is going on
(of course in 4 months you might BE said DIP ...)
Any person using FTFY or editing my postings agrees to a US$50.00 charge
How do you represent that as a case statement in C++? Come on, I'm waiting. Oh, that's right. You can't.
switch (dot < -LIGHT_CLIP_EPSILON ? 1 : dot > LIGHT_CLIP_EPSILON ? 2 : 0) {
case 1:
sides[i] = SIDE_BACK;
break;
case 2:
sides[i] = SIDE_FRONT;
break;
default:
sides[i] = SIDE_ON;
}
Yeah, yeah, I know, that's totally ridiculous (although I did see things as bad and worse as a CS instructor's assistant whose job it was to grade Pascal students' programming assignments back in the day - that was very interesting to say the least).
On a side note, why can't > and < characters be used in a code element? Um, that's lame, especially for a site that discusses programming so much.
Better known as 318230.
Half the 'beautiful' styling mentioned in the article (minimal use of templates, getter/setters, etc) leads to code that executes faster, without relying on compiler optimization to fix it up later.
"False hope is why we'll never run out of natural resources!" - Lewis Black
if you had any understanding of what you are talking about, you would say that the difficult and important things are to choose the data structures for you problem, to design the function that renders the model in multiple threads, to design the interthread communication and so forth... you would say, the beauty lies in the architecture... instead you just come with your sad shit about what language is better then what other... you are just poorly programmed bots, made by a second rate god apprentice...
to bad it runs like shite..
I remember trying to use solaris once - this was at the end of sun's lifecycle. The updater told me there was like 20 critical patches I needed to install post first install. Every single time I tried updating the critical security patches Solaris would just up and die. A complete reinstall of the OS was required just because the buggy security patches not working. I did this 3 times thinking it was a fluke. (solaris 10 out of the box). Eventually I just said fuck it and ran solaris without any updates. I come to later find out to even get patches I needed a 'support contract' - yes they were charging for critical security patches. All of this pain just to use ZFS.. (at the time)
It can look wonderful. Hell I can write a program to generate a bunch of random code that "looks good".. it's so beautiful... But how the fuck does it run? In doom's case the game ran great AND the code looked good. In solaris's case, the code probably just looked good..
Most game companies write engines to make games. The game is the product, so it's not too important how it looks under the cover. iD makes games to publicize engines. So it really does matter for the code to look nice.
Its easy to go back over a finished codebase, run some static analysis and refactoring tools and clean it up. Chances are this is what was done before releasing it to be open source. I can guarantee this source coded didn't look anything like this during development.
Of course this is maybe why the game took longer to develop, spending more time making the code look purdy then, say, making the game a superlative gaming experience.
I haven't thought of anything clever to put here, but then again most of you haven't either.
The Doom 3 code is quite ugly.
If you want to read good code, try the Linux kernel.
If that's beautiful, I really don't want to read his "ugly" code...
Oh my god, this is the worst programming advice I've ever heard. Is this a joke? Maybe some clever attempt at creating job security?
There is a terrible dearth of commented code in the world -- especially in the lower-level languages like C and C++ -- and this guy is telling people we need fewer comments in our code?
Modern copyright is theft of culture from everyone and it retards the progress of the useful arts and sciences.
He loves the lack of white space, I hate it. Cramped code is irritating to read. If you want to take up less vertical space, reduce your font and increase the whitespace. You have a better sense of the separation of statements, stronger scoping and less room for error.
He also loves the lack of comments. I remain firmly in the camp that if you eschew comments as common practice, you're an idiot and you should stay away from programming on big teams.
It's not a clarity of code issue. I expect your code to be clear, too. But even after 20 years of programming, I read English faster than I read code. A description of an algorithm in English is going to be more terse than the code that implements it. Your code has to account for edge cases, but I probably just want to know what the code does and how the code does it at a high level so I can get a sense of the system and architecture. A descriptive method name only tells me WHAT the method does, not the manner in which it's done.
English (any natural language, really) is a powerful language with extraordinary expressive power. I don't understand why programmers are constantly trying to sweep it under the rug. Don't fill your code with useless comments like // increments the counter by 1, but if you're doing a non-trivial mathematical calculation that takes a whole method to encapsulate it, let me know what I'm getting in to.
Code comments--especially system level comments--should include the name of the author or current maintainer, as well. I tag my methods with my name and the date that the code was put in so people know where to go if there's trouble. They don't have to hunt through perforce time-lapses to see that I checked it in, they just email me.
And have some consideration for the new guy on the team, or the team that has to use your code 5 years in the future. They can't ask you questions, the context of the situation is lost, the code-base might be in the middle of being re-purposed (common in the game industry--which is where I am); comments are essential to maintainability. Man, I do code reviews and people often manage to forget exactly what they were trying to do, and it's only been a few hours. We always work it out, but if there were a comment, we wouldn't even have to spend THAT time.
Use comments. Use them wisely. It makes you a better programmer because you're wasting less of OTHER people's time.
Came hoping to learn what's so beautiful about iD's code, left convinced that the author (Shawn McGrath) and I have rather different opinions on that... iDs code is certainly not an example of poor code, in a previous job I had the opportunity to view code from around 20 different AAA game studios, its definitely in the top quarter (but that's not saying a great deal); mostly the article is 50 paragraphs of cooing "iD does what I do, guys!" Analysis of what makes said style "beautiful" is subjective at best, and furthermore the author describes himself as "not a coder". For what its worth, IMHO, the best code that I've seen came out of Remedy.
I normally hate most of the garbage that comes out of Kotaku, but this is a really good article. That said, it reminds me a lot of Yossi Kreinin's C++ FQA. A good chunk of the article is spent talking about how Doom 3's source is good code despite being bad C++. What kind of language is best written in a way frowned on by the C++ community? Absurd!
One of my early CS professors for sure. Not one number could be found in your code. If you wanted to divide by 2, you'd better have a const up top defined as 2 and then use it.
Code should be locally coherent and single-functioned: One function should do exactly one thing. It should be clear about what it's doing.
AGREE, though applications with much scoped behavior can become a mess.
Local code should explain, or at least hint at the overall system design.
AGREE, it needs to push you to learn the design or hint that there's a design behind it.
Code should be self-documenting. Comments should be avoided whenever possible. Comments duplicate work when both writing and reading code. If you need to comment something to make it understandable it should probably be rewritten.
NOT AGREE, you miss the whole encapsulation point. Hence blackbox testing of code and modularity will suffer--yes it will suffer.
"All id games before Quake III were written in C."
Fool! Quake III Arena was also written in C.
It seems like a lot of contributors here are missing the point. Discussions like this--and particularly the analysis that provokes them, regardless of how perfect it may be--are highly valuable and productive, for the same reason as project post-mortems. Too often, debates about development (where it's styling, methodologies, or rules of thumb) are done before coding begins, and as a result, several things invariably happen:
This, of course, is a reflection of the principle that a good engineer learns from his mistakes, while a great engineer learns from other people's mistakes. Both pictures and hindsight are worth a thousand pre-emptive, ill-informed words. I'm particularly pleased with the focus on coding quality, which (in post-mortems and hindsight analysis like this) takes second place to project and management practices. I will take, and have learned infinitely more from, 10 /.-ers debating an informed and evidence-based review of a quality codebase over 1000s of discussions about programming theory, the latest development paradigm fad, or some wannabe game designer's blog post.
Once Carmack was a genius. Now he is a bad joke, and the world is filled full of game engines infinitely better than anything Carmack could dream of coding today. The why of this is interesting.
Doom was a masterpiece of an engine, although it was surpassed in every possible way a little later by the 'build' engine (see 'duke nukem'), the work of another young programmer.
Quake 1 had a very interesting engine (with Abrash doing much of the important work) but had the misfortune to be badly timed- an engine optimised for software rendering on the cusp of the PC GPU revolution.
Abrash, of course, went of to do the wonderful software renderer for Unreal (sadly obsolete, since real fans had 3D hardware by the time it was released).
Quake 2 and 3 (now Carmack's exclusive show) were horrid disappointments, although 3 had an interesting, if very limited, tessellation system for curved surfaces. However, it was Doom 3 that convinced most sane observers that the age of iD was over.
Rage was an abomination, but also a project that took a ludicrous number of years to complete. The engine is so putrid, Bethesda (the suckers that ended up buying iD) can neither use it internally (on non-iD projects) nor license it out for others to use.
My point- well it was around the time Carmack started mithering about his code, and issuing public statements about how iD needed to move to C++, and write code according to the methods dictated in Computer Science text-books, that iD went down the drain. When Carmack was a natural coder, assuming his ideas and methods were good, he produced good engines. When he felt embarrassed and uncomfortable with his self-taught methods, and needed to impress 'academic' types with his source (not the finished applications), his output went to hell.
Now some of you will try to tell me 'academic' code is essential in this case, because others needed to use and support his code. WRONG! It was when Carmack started to code this way that his engines became less and less desirable for others to license. Contrast this with Epic, and their Unreal engine. The Unreal engine ended up inheriting the entirety of iD's early success.
Carmack's last two 'technologies' were useless. The dreadful shadow system in Doom 3 set back shadows in games for years, because the method was so ill conceived. The constantly streaming textures in 'Rage' gave us some of the worst texture work ever seen in a supposedly AAA title. While Carmack's streaming made midground images look great (mostly) foreground and background textures looked like total lo-res crap. Worse, the production pipeline for artists on this engine was the least efficient ever invented, and led to artists simply replicating the same texture over and over into the megatexture, and then smearing a few pixels so Carmack could claim that ever texture was unique.
One could argue that Gearbox developed and released Borderlands 1 + 2 (same concept as Rage but far far better execution) to mock iD, and prove how quickly, cheaply and how much better such games could be executed on the Unreal engine.
Sorry, "My source looks great, and a Computer Science teacher at a third rate university would give me top marks for that" is not a selling point in the games industry, when it is your output that counts.
Carmack's engines cannot do interiors above average, do exteriors very poorly, cannot handle large regions, cannot handle realistic vegetation, have poor lighting models that are years out of date, and lack good physics. Affection for iD's later work is actually largely affection for the Quake 3 licensed engine, after it had been extremely improved in every respect by other software teams. The work iD couldn't be bothered to do itself, no matter how much money they were raking in from the days when people did license their engine.
And the Doom3 code we are talking about here. I think only one non-iD team licensed that engine (at a time when iD was desperate for business) for that game with the 'Native American' that is captured can space invaders that some of you might just remember. Yes, that sure was some beautiful code.
Much of code is a matter of taste:
if(x==1){bla();}
if(x==1){
bla();
}
if ( x==1 ){
bla();
}
if(x==1)
{
bla();
}
Are taste. Comments and variable/function names are functional and thus more arguable (but still generally religious). Any review of this sort that talks about code formatting is wasting our time(unless they went way overboard with something stupid) with religious nonsense so I wish he would stick to the benefits of how they pass parameters etc.
The only time anyone ever "Wins" the code formatting argument is when something else is brought into the argument such as "Format it my way or get fired." or "Format it my way or I quit"
Nearly every place I worked had someone who always began the argument about coding standards with "I don't care which standard we use as long as we all stick to it." But then they relentlessly argued for their standards and wouldn't give an inch with well structured arguments for every space, comma, and return. Often these standards had all kinds of specific metrics like a certain ratio of comments to lines of code. This way they could point to other people's code and mathematically prove that they sucked. Although the worst were the passive aggressive sorts who would reformat any block of code they touched on to "their" standard which was wildly different from the entire rest of the programming team.
I work at Google. I find it interesting to compare/contrast the style points in this article with the very thorough style guide at Google. Generally, I agree with Google where they differ.
Const and Rigid Parameters: At Google, all reference/pointer inputs and all member functions must be const where possible. Local variables do not need to be. I personally find it more hassle than I care to bother with on local variables, but the rest is all or nothing, and I (and Google) agree with him that all is much better than nothing.
Minimal Comments: Urgh. Yes, there are useless comments out there. There are also way too few useful comments. At the bare minimum, there should be a top-level comment for every class. No exceptions. Ever. And to suggest you should rewrite every piece of code that does something non-obvious or solves a non-obvious problem? Good luck with that.
Vertical Spacing: Google also uses same-line braces, but it's pretty arbitrary - consistency is the important thing. Anyone who says otherwise is a zealot =). In any case, it's not an excuse to remove all vertical white space.
Minimal Templates: You should not add templates just for the heck of it, but forcing a hash table to have a char* key is NOT a good thing.
Get/Set Methods: Public member variables are forbidden at Google. A little boilerplate is deemed a very acceptable tradeoff for controlled mutation.
Streams: Also not allowed at Google, except for logging. This is not because the syntax is "ugly" though. It's because (a) streams are poorly implemented and are inefficient, and (b) standardization is required. Interesting that John Carmack mentions that misaligned printf formatting is a big cause of bugs - in g++, the compiler catches that. Alas not in VS.
Operator Overloading: Also forbidden at Google. Honestly though, for all the bugaboo made about these guys, I don't see anyone really abuse them anywhere.
Horizontal Spacing: Ew? As for tabs, they are outlawed at Google. Editors are configured to replace them with spaces.
Method Names: Meh. Google generally agrees except for pure accessors. He doesn't have those, so I guess that's agreement.
I bet the author did not read that book.
My other signature is a car
unsigned long i = someMethodThatReturnsA32BitBitfield();
...
...
switch(i)
{
case 24: DoThis(); break;
case 385: DoThat(); break;
case 4812334: DoSomethingElse(); break;
}
How exactly is that going to be evaluated in constant time?
Hello!!! everybody, Fashion,low price,the good shoping place, click in. ===== ( http://www.sheptrade.com/ ) ===== Discount Air Jordan (1-24) shoes $35, Air max shoes (TN LTD BW 90 180) $36, Nike/shox (R4, NZ, OZ, TL1, TL2, TL3) $35, Handbags ( Coach Lv fendi D&G) $36, T-shirts (polo, ed hardy, lacoste) $20, Jean (True Religion, ed hardy, coogi)$35, Sunglasses ( Oakey, coach,Armaini )$16, Watches(Rolex BREITLING IWC) New era cap $12, Discount (NFL MLB NBA NHL) jerseys, free shipping, Accept credit card payment! ===== ( http://www.sheptrade.com/ ) =====