Slashdot Mirror


The Technology Behind ID's Games

orac2 writes: "The current issue of IEEE Spectrum has an article on the groundbreaking technology behind iD Software's games, from the days of Commander Keen through to Return to Castle Wolfenstein. Graphics technologies covered include the original 2-D buffer trick that made side-scrolling games on the PC feasible, as well as the more modern Raycasting and Binary Space Partition Tree techniques. Carmack is quoted extensively."

30 of 323 comments (clear)

  1. Quake I / II by Shamanin · · Score: 3, Informative

    I read this in the IEEE spectrum. They kept mixing up the advances in Quake I with Quake II. They even refer to a picture of Quake II as Quake I. But, there was some interesting history nonetheless.

    --
    come on fhqwhgads
  2. Super Mario Brothers 3 didn't run on the SNES by Ted_Green · · Score: 3, Insightful

    "First, they decided to see if they could recreate on a PC the gaming industry's biggest hit at the time, Super Mario Brothers 3. This two-dimensional game ran on the Super Nintendo Entertainment System, which drove a regular television screen."

    If they can't even get that right, how am I to believe what they say about frame buffering, Hmmmmmmmmmm?????

    I'm going to go get some pie instead.

  3. Carmack IS God! by casings · · Score: 5, Interesting

    I mean, hes not only a very good programmer, he looks like a computer dork, has a phat car, and actually cares what the community thinks about his games. As was the case when there was such a backlash about fixing the bug in the engine of quake2 that produced the strafe jump, he changed it, uproar ensued, he changed it back.

    Carmack embodies what every programmer and any kind of computer company should strive to be.

    Carmack has embraced the platform-generic opengl, and even coded his engine to be compatible on every major os. I love you carmack, please have my love child.

    I MEAN C'MON hes the one responsible for such things as the infamous railgun, and the hilarious warnings about piracy on my copied version of wolf3d, which i still play on my 386 laptop.

    1. Re:Carmack IS God! by Pfhreakaz0id · · Score: 5, Funny

      Crap!, I've got mod privs right now, but I can't find my -1, Fawning!

  4. Excellent diagrams by jukal · · Score: 4, Informative

    This one is particularly good: about binary space partition tree.

    1. Re:Excellent diagrams by jtdubs · · Score: 5, Informative

      Binary space partitioning trees proper have fallen out of use in games.

      All BSP-based games now-a-days use Solid Leaf BSP trees, which are a variation of the original and have many more useful properties for games.

      It is these trees and the sectors that they create that are used for determining portals.

      If two sectors have the same portal then those two portals connect and can see eachother. If a portal has no pair, then it is a portal out in to infinite space and is hence a leak in your map. Assuming, of course, a map based on a single BSP tree.

      Once you have portals, PVS generation is easy. And there you go, BSP and PVS.

      A lot of modern research is shying away from BSP and PVS. They limit you to indoor spaces. Entirely different technology is required for outdoor scenes. And then special hacks and logic are needed to allow for seamless transitions between the indoor and outdoor worlds.

      Some think that octrees with a form of occlusion culling might very well be better as they can represent both indoor and outdoor geometry without making any distinction between them. They are much harder to cull though as no handy PVS information is present and only more difficult methods exist such as Z-pyramids.

      Others stick with BSP and PVS and use the one-sided portals that would normally represent leaks but instead in this case represent windows from the BSP into the terrain, and vice-versa. This combined with some extra lighting and shodow-volume information and you can have your lighting and transitioning between indoor and outdoor be seemless.

      Anyway, just the ranting of a bored man at work.

      Justin Dubs

    2. Re:Excellent diagrams by jtdubs · · Score: 3, Informative

      No problem. This is a common question.

      Here's the thing.

      BSP trees work by breaking up the world into arbitrary convex hulls. This is good because you can render a convex hull that you are inside without worry about some of the triangles overlapping other ones.

      In other words, take two cubes connected by a recangular hallway.

      Theoretically the BSP tree could divide this into three sectors. One for each room, and one for the hallway. Each of these individual sectors is a convex hull.

      So, if you are in the sector for a room you can just render that entire sector, without using a z-buffer or any kind of depth testing, and you don't have to worry about things overlapping. I mean, there is nowhere in a room where you can stand where the left wall and the floor will overlap.

      See what I mean?

      Now, while you are rendering the room you render it's portals. It has a portal connecting it to the hallway sector. So, you render the hallway's sector, clipped to the polygon that is the portal.

      Then, while drawing the hallway you draw the sector connecting to the second room. So, you draw the second room, clipped to the portal leading from the hallway to that room, which is already clipped to the original portal.

      So, we just drew the two rooms and the hallway, with no traingles overlapping.

      This is exactly how the original portal-based renderers worked.

      Now, why doesn't this work for outdoor scenes?

      Imagine a large field with rolling hills. Now, try and find a convex hull. It's tough.

      Imagine a hill shaped like a dome.

      This dome forms a conCAVE hull. You can't break it down in to ANY convex hulls. So, every single triangle has to be part of it's own, unique, convex hull.

      So, now every triangle in this hill is it's own sector. Our BSP tree is huge. And we have MILLIONS of portals connecting every triangle to the ones next to it.

      This is HORRIBLY ineffient and slower than just brute force rendering.

      So, to summarize. Indoor scenes tend to be easily decomposable into convex hulls. And hence work well with BSP trees and PVS. Outdoor scenes aren't. They tent to have a lot of concave areas.

      So, hope that helps.

      Justin Dubs

  5. Truly impressive by aengblom · · Score: 5, Informative
    The truly breakthrough technology is that ID made it possible to spontaneously induce vomiting without noxious fumes! ;-)

    ARRG (offtopic)
    Editors please (as in pretty) fix this:

    User types in comment and submits it (without subject). Is told to type in subject, but then is told "you have submitted to quickly." User loses comment because it is cleared from browser cache and slashdot doesn't put it in the error page.

    ( /offtopic )

    --


    So close and yet so far from the world's perfect ID number
  6. Re:Man... Carmack is 31 by EvilTwinSkippy · · Score: 5, Insightful
    You get a lot better. (Speaking as a 27 year old programmer.)

    C code is C code. No new languages, techniques, or processes will ever replace an experienced architect. Crap passes through an IDE every bit as well as the good stuff.

    I have a volunteer who works with me. The kid is brilliant, and has programming mojo pouring out of his eyebrows. But there are so many debugging techniques, algorythems, and habits that he doesn't have. (Yet.)

    I'm not saying older in neccissarily better. Experience is the key. 20 years of experience is 20 years of experience whether you start at 7 or 27. In my case it's 7.

    --
    "Learning is not compulsory... neither is survival."
    --Dr.W.Edwards Deming
  7. What I read... by Tall+Rob+Mc · · Score: 5, Funny

    Basically, what I got out of this article was that John Carmack is almost single-handedly responsible for all of my non-productive time over the past 10 years. Thanks John!

  8. The Evolution of the id Engine by death00 · · Score: 4, Interesting

    "There were critical points in the evolution of this stuff," Carmack says, "getting into first person at all, then getting into arbitrary 3-D, and then getting into hardware acceleration....But the critical goals have been met. There's still infinite refinement that we can do on all these different things, but...we can build an arbitrary representational world at some level of fidelity. We can be improving our fidelity and our special effects and all that. But we have the fundamental tools necessary to be doing games that are a simulation of the world."

    This article highlights how far we have come as game developers. id has been the "poster child" of the game development community, with the majority of other game developers following their lead. Doom III will continue this trend.

    The next generation of games is going to be outstanding!

    This article gives a great view of where we can be going with new technology. How realistic will games be in 10 years? My guess is that the graphic reality will become nearly indistinguishable from real life, but the greatest innovations will be in game-play. Interfacing with a keyboard/mouse/joystick isn't realistic. Voice control and force-feedback-like technologies are the way of the future, if our computing power can support it.

    Kudos to Carmack on 10 years of FPS game design. Here's to the next 10!

  9. Id didn't develop the Keen trick by Junks+Jerzey · · Score: 4, Informative

    Good article, but the Commander Keen scrolling trick was old news by then. Lots of Apple II, Atari ST, and Amiga scrolling games did the same thing. Impressive? At the time, yes. But let's not get too carried away with giving Carmack credit for everything.

    1. Re:Id didn't develop the Keen trick by drivers · · Score: 3, Insightful

      Good article, but the Commander Keen scrolling trick was old news by then. Lots of Apple II, Atari ST, and Amiga scrolling games did the same thing. Impressive? At the time, yes. But let's not get too carried away with giving Carmack credit for everything.

      The article doesn't say that The Carmack invented smooth scrolling full stop. It said that he figured out how to do it in EGA mode on the PC. The market for EGA cards was much larger than any of those closed (but optimized for cool graphics and sound unlike the PC) platforms.

      Kind of like how someone figured out how to [kind of] play digital sound through the standard PC beeper. Of course the Amiga, etc. could do that with dedicated hardware but that's not the point.

  10. Age has nothing to do with it ... by AHumbleOpinion · · Score: 4, Insightful

    Age has nothing to do with it, it's all about experience and continuing to learn. Older programmers are better as long as they continue to learn and maintain curiosity about "new stuff". Experience + current knowledge > current knowledge. The myth of older programmers not being as good is really that many older programmers stop learning. For a while they are able to outperform younger programmers due to experience but eventually their laziness catches up and they being to underperform.

  11. Re:Not new or groundbreaking by Anonymous Coward · · Score: 3, Interesting
    You mean where it says in the article "The answer was a technique known as binary space partitioning (BSP). Henry Fuchs, Zvi Kedem, and Bruce Naylor had popularized BSP techniques in 1980 while at Bell Labs to render 3-D models of objects on screen. "


    The point is that Carmack brought it to the world of PC games,

  12. The Gamer's Prayer, according to ID by matastas · · Score: 5, Funny

    Our Carmack, who art in Texas,
    hallowed be thy textures.
    Thy software come, thy games be done,
    on my b0xen as it is at E3.

    Give us this day, our daily FPS.
    And forgive us our camping,
    as we gun down those who camp against us.

    Lead us not into a spawn site,
    but just give me the damn BFG.
    For the gaming market, the GeForce,
    and the booth babes are yours, now and until payday.

    1. Re:The Gamer's Prayer, according to ID by Simon+Garlick · · Score: 4, Funny

      Hilarious :)

      My personal taste would have been to replace the line

      "Thy software come, thy games be done"

      with

      "Thy software come, when it is done" :)

  13. Softdisk by Sean+Clifford · · Score: 5, Interesting
    For those interested:

    I joined Softdisk in 1995, a few years after the id guys left. The company was stunned by the success of Wolfenstein and Doom, and by Duke Nukem - also born of Softdisk alumni. It was basically a subscription software company, selling a package (card games, screen savers, etc.) on disk monthly. It was a good model for the 80's.

    Softdisk tried to produce a couple of games, one called Greed (later In Pursuit of Greed) which was basically a 3D Doom-clone shooter. There was some neat technology (e.g. curved surfaces), but the art was...uh, well weak. The gameplay was decent, but there were some bugs to stomp and the ship date slipped...and slipped...and slipped. It was released, but didn't live up to the hype. The game was torn to shreds in the reviews. There was a second 3D shooter - developed totally in house, though it was basically a one-man project. The lead (only) programmer left, so it was shelved.

    Softdisk finally shut down its on-disk-monthly subscription software and became an ISP/web development company. It was a necessary move, but sad since the company kicked a lot of ass in the 80's with LoadStar and Big Blue Disk.

    For those interested, I ran Softdisk's online download software stores on CompuServe, Prodigy, AOL while another dude took care of eWorld. We were selling Commander Keen, Dangerous Dave, and a host of other early games the id guys produced at Softdisk. Last I checked, they were still being sold (at $19.95 a pop, even).

    1. Re:Softdisk by John+Carmack · · Score: 5, Informative

      Here is an interesting bit of history:

      Greed was built on the engine I wrote for Raven/Origin's Shadowcaster game, while the other Id guys were working on Spear of Destiny, the commercial Wolfenstein game.

      The reason softdisk got the technology was that they were still making lots of noise about suing us for doing Keen while we were working at softdisk. Our original parting deal was that we were going to continue doing the Gamer's Edge games for a while, basically for free, as penance. We weren't savvy enough to get anything binding down on paper, so even when it was all wrapped up, there was room for twisting our arm a bit. (another trivia bit -- George Broussard at Apogee arranged to have Apogee produce one of the Gamer's Edge titles for us, so we could focus more on Wolfenstein).

      We finally arranged a technology transfer of the latest engine code for free and clear severing of our ties. After they showed that just having the technology was not a guarantee of success, they had the nerve to come back and ask for more, but by then we were able to just tell them to go away.

      BTW, Duke Nukem does not have a Softdisk heritage, it was by Todd Replogle (sp?), who was strictly Apogee-grown.

      John Carmack

  14. Re:Man... Carmack is 31 by robson · · Score: 4, Funny

    Not to troll or anything -- but it did seem quite amazing. I keep thinking he was like 27 or some such.

    Well, it's not as if 31 is that old... I mean, I'm 31, and I'm not "old"...

    ...right...?

    :)

  15. Carmack's age by ragnarok · · Score: 3, Funny

    Carmack is only 31? Damn, I better get to work.

    --
    Search first, ask questions later.
  16. Carmack's real innovation - 2 3/4 D by Animats · · Score: 4, Interesting
    Carmack's real innovations were in ways to do something that looked like general 3D but wasn't, quite. So his games did things that are easy now, but were really tough on the hardware he did them on.

    Flight Simulator pioneered this sort of limited 3D. Bruce Artwick did the original Flight Simulator on machines that didn't have enough power to fill the whole screen with a solid color in one refresh. He wrote a book about how he did it in 1985. The pain, the agony...

    Artwick seems to have dropped out of game development, but Carmack keeps pushing what's possible with available hardware.

  17. Re:Not new or groundbreaking by mckayc · · Score: 5, Funny

    Oh really, is that so? Hmm, where did I see that, oh right, IN THE ARTICLE.

    Profiting from improvements in computer speed and memory, Carmack began working on how to draw polygons with more arbitrary shapes than Wolfenstein's trapezoids. "It was looking like [the graphics engine] wouldn't be fast enough," he recalls, "so we had to come up with a new approach....I knew that to be fast, we still had to have strictly horizontal floors and vertical walls." The answer was a technique known as binary space partitioning (BSP). Henry Fuchs, Zvi Kedem, and Bruce Naylor had popularized BSP techniques in 1980 while at Bell Labs to render 3-D models of objects on screen.

    (emphasis mine)

    Perhaps READING THE ARTICLE would have saved you the trouble of trying to show us how smart you are.

  18. Re:Not new or groundbreaking by LMCBoy · · Score: 5, Funny

    Everything's stolen these days. Take the FAX machine. Why that's nothing but a waffle iron with a phone attached!

    </abesimpson>

    --
    Liberal (adj.): Free from bigotry; open to progress; tolerant of others.
  19. Re:he is a C hacker, that's it by WasterDave · · Score: 4, Funny

    Why people worship him, especially now, is beyond me.

    Well, ummm, you have a go then. Particularly at the 1996 stuff - you have a P75 and 8Mb, render a 3d texture mapped scene at 20fps.

    Dave

    --
    I write a blog now, you should be afraid.
  20. Re:Amiga??? by mindriot · · Score: 3, Informative
    My theory is that while the Amiga scrolled by changing the pointers to where the screen was in memory, the PC dumb-copies entire screens.

    Well, that's what Id avoided on PC... (did you rtfa? :)) When EGA cards came up, they had enough graphic memory and functionality to change pointers... back then I tried this a little on VGA (320x200 tweaked mode giving four screens, and changing a pointer to scroll/switch screens for double-buffering).

    The Amiga did of course have much better graphics hardware, including a blitter for fast graphics data transfer and accelerated drawing functions, (at that time) lots of video memory, and further hardware acceleration such as sprites support. And, also, support for bitplanar graphics modes, easing smooth scrolling.

  21. Say it with me now...! by aztektum · · Score: 4, Informative

    It's *id* Software. Not ID. Not iD. id. as in the psychology term.

    id
    Pronunciation: 'id
    Function: noun
    Etymology: New Latin, from Latin, it
    Date: 1924
    : one of the three divisions of the psyche in psychoanalytic theory that is completely unconscious and is the source of psychic energy derived from instinctual needs and drives.

    --
    :: aztek ::
    No sig for you!!
  22. Softdisk v. Id by Sean+Clifford · · Score: 4, Informative
    Unsubstantiated company rumor on the Duke thing, I guess. :)

    I remember Al talking about the lawsuit and the source code. One poker night (which I played badly) Dan Tobias went on a long rant (suprise) about the whole ordeal. I share his opinion that moonlight code belonged to the programmer, not the company.

    Absolutely nothing came of the source code. It sat in Jim's office unused.

  23. Originality, creativity, etc by John+Carmack · · Score: 5, Informative

    I was fairly pleased with how that article turned out - when I first heard about it, I dreaded seeing a trivialized simplification of the issues, but it turned out as representative as you can be in that space.

    However, I really dislike discussions of the attribution of techniques to a particular programmer. Everything is derived from things before it, and I make no claims of originality. I would say that one of my talents is the ability to be aware of what sources are feeding into my work, and be able to backtrack to them. Also, there are always lots of other possible answers for any given problem that can be made to work. BSP vs sector list, Portals vs PVS vs scan line occlusion, tilted constant Z rasterization vs block subdivision vs background divides, etc. Looked at in the proper perspective, individual techniques just aren't all that important. Sometimes it sounds like "Dude, he INVENTED needle nose pliers!!!"

    Heck, I somewhat deride the very concept of originality. Creativity is just synthesis without the introspection. Lots of people will catch on that and start a rant about how Id games aren't original, but they are missing the point - it is possible to set out and develop something that will be received as "original" without ever having an "original" idea spring into your mind.

    The best way to get answers is to just keep working the problem, recognizing when you are stalled, and directing the search pattern. Many of the popular notions of innovation and creativity are in some ways cop-outs that keep people from being as effective as they could be. The little document I wrote about developing a part of the shadow algorithm for Doom that Nvidia has on their website was a pretty good example of my process. Don't just wait for The Right Thing to strike you - try everything you think might even be in the right direction, so you can collect clues about the nature of the problem.

    John Carmack

  24. Re:On Creativity... by Paul+Komarek · · Score: 4, Insightful

    To add another perspective:

    The mathematicians I have met (I'm one of them) by-and-large feel that new math ideas are *discovered* instaed of *created*. The distinction is important. Truth and algorithms already exist, we're just trying to *find* them and sort through the crap. Just because no human has previously written down some piece of truth or an algorithm before you do, doesn't mean you invented that truth or algorithm.

    We're all standing on the shoulders of reality, trying to decode what we see. John Carmack's comment about struggling with a problem in order to understand it seems very much in line with this view, and very much inline with the academic research process. Academics don't get research done just by sitting around, trying to be creative. We do research by repeatedly struggling with a problem until we figure out which defects in our brain prevented earlier understanding.

    -Paul Komarek