Slashdot Mirror
The Struggles of Developing StarCraft
An anonymous reader writes "Patrick Wyatt led production efforts for several of Blizzard Entertainment's early games, including Warcraft 1 & 2 and StarCraft. Wyatt has just published an in-depth look at the development of StarCraft, highlighting many of the problems the team encountered, and several of the hacks they came to later regret. Quoting: 'Given all the issues working against the team, you might think it was hard to identify a single large source of bugs, but based on my experiences the biggest problems in StarCraft related to the use of doubly-linked linked lists. Linked lists were used extensively in the engine to track units with shared behavior. With twice the number of units of its predecessor — StarCraft had a maximum of 1600, up from 800 in Warcraft 2 — it became essential to optimize the search for units of specific types by keeping them linked together in lists. ... All of these lists were doubly-linked to make it possible to add and remove elements from the list in constant time — O(1) — without the necessity to traverse the list looking for the element to remove — O(N). Unfortunately, each list was 'hand-maintained' — there were no shared functions to link and unlink elements from these lists; programmers just manually inlined the link and unlink behavior anywhere it was required. And hand-rolled code is far more error-prone than simply using a routine that's already been debugged. ... So the game would blow up all the time. All the time.'"
Wyatt also has a couple interesting posts about the making of Warcraft 1.
If Blizzard's #1 priority wasn't 'obscene profit' these days, it might make the job a little easier/fun.
... congratulation... what a fresh piece of news !
This is so pefect for Computer Science courses! Someone might actually look up from their laptop at the lecture slides if this gets mentioned!
Thanks to Blizzard's new and idiotic direction with WoW, I discovered private servers and can finally enjoy the game again either on: vanilla, BC, or Wrath servers. Cataclysm and on, can suck my balls.
Obviously the Zerg.
Thus demonstrating my long-time assertion (ever since my stint in the video game biz) that video game developers are crap programmers. No common routines to manipulate a heavily-used data structure? Not even a set of macros? This kind of story belongs over on The Daily WTF.
Chelloveck
I give up on debugging. From now on, SIGSEGV is a feature.
{{Citation Needed}}
Thus demonstrating my long-time assertion (ever since my stint in the video game biz) that video game developers are crap programmers.
Broad Generalisations are almost always wrong. There are some game developers, especially those who have experience in other fields of software, which aren't "crap programmers". Video Games are some of the most hardware intensive programs, utilising every major feature and capability that the system supports. Sometimes it's acceptable to break the 'best practices' rules to get a little performance (if the profiler says it's warranted), but I agree there's no reason to be manually managing linked lists.
Sometimes a 'Lead' programmer will be the worst of them all, and the others must follow the bad example or be made an example of. Let's not lump all game programmers into the horrible coder pile. As you've pointed out, TDWTF has plenty of examples of nightmares from all sectors, not just video games. That the game developers allow studios to get away with "crunch time" consistently instead of firing the dumbasses in charge who purposefully gork the schedule is both a large source of bugs and proof we need a union or better work conditions at least. I can forgive crunch for one project. Maybe even two in a row... but every single game? Yeah, that's either abuse or incompetence. Either way, it's why I only buy (and work on) indie games now. They tend to have better working conditions, even if some of the beginners in the indie market churn out even worse code -- At least they have an excuse in not knowing any better; What's that say about the "pros"? That they're all crap programmers, or that the programmers are forced to write crap code? I put it to you that it's the latter, not the former.
I'm a scientist with a CS minor and some experience doing SW engineering (probably badly) on small projects. Even *I* know you abstract the actual storage and provide some simple accessor/insertion functions to prevent people from actually touching the storage implementation. For a couple of reasons: 1) to prevent goofballs from creating pointer messes, and 2) so that, if you want to, you can completely rip out the actual storage implementation and replace it without breaking anyone's code. I'm sure there are also reasons 3) - Eleventy, but those two are pretty obvious. Not to mention which, it only adds maybe a couple hours (if you're diligent and paranoid) over the short term, and probably saves weeks over the long term.
Not to mention which, aren't there standard libraries to do this stuff? Did STL not exist back then?
There are certain types of software development that seem to attract people who have a combination of significant ignorance and huge egos. Game development is one such field, and Ruby on Rails web development is another.
These people often are quite young, and as a result of this often have little to no academic background or formal training of any sort. Although they're quite ignorant, they don't realize this fact. This prevents them from keeping their egos in check, which in turn makes them think they know everything. Believing that they're all-knowing, they never both to seek further education or investigate alternatives, and continue to make the same mistakes over and over.
Having worked in both the game development industry and doing web development more recently, I do have to say that the RoR programmers are much worse than the game programmers. At least the game programmers usually have some understanding of algorithms, and many have a solid understand of mathematics, even if it's all self-taught. Compare this to the RoR programmers, many of whom refuse to learn about databases, even though they're generally developing database-driven applications. You wouldn't believe how many times I've encountered people like this who don't know any SQL, and who don't even know what database indexes are.
Given the freely available sys/queue.h from FreeBSD, which provides macros for several different list types and the common operations on each, this is inexcusable.
These days nobody should be writing there own containers, except in VERY atypical situations that don't arise for most people. There are perfectly good implementations of both intrusive and unintrusive containers in STL and Boost, and if you aren't using them, you are being incompetent.
I was working 'down the street' if you will at the time, as a programmer on one of their direct competitors (Age of Empires), and it's easy to forget the circumstances we all were working under at the time.
People weren't using Templates in games for a couple of reasons - Familiarity being one, but the state of the code was new, and especially the asm outputted by compilers was often very inefficient which it was not outright BUGGY if you pushed it. Templates were still very new then.
Also, you tend to forget how slow and limited PCs were then -- Your phone today probably runs circles around not just the machines that games were run on, but the PCs used to develop them.
The System Specifications "On the Box " for Starcraft were - A Pentium 90 (or equivalent - that could be a 486-133) , 16MB of RAM and Windows 95 or NT 4.0, and a SVGA video card with 512kb or 1MB of VRAM. Think about that for a minute. These were 2d video cards, not 3D. Age had almost identical specs. A full rebuild of AoE on our Dev machines (Pentium Pro) took 15-20 minutes to make.
It was very normal to worry about saving 2 bytes, or just a few cycles of CPU time back then. So you did everything bespoke by hand, and didn't genericsize much.
And to be honest. We didn't know then what we know now. The programming practices most of my peers just do automatically today -- we hadn't developed/learned them yet. We did what we could with what we had in knowledge and tools, and shipped complete AAA games for costs in man-hours and dollars that seem ludicrously small today.
Don't get me wrong, Besides being a lot of work, It was a lot of fun too. One thing I remember was our companies putting each other on the Beta Test list for our upcoming games.
Myth is still superior in every department.
CUnit In case you couldn't tell, a CThingy is a sprite that can be drawn, a CFLigny is some kind of particle engine, a CDoodad/CUnit is some artificial separation of concerns. This is in SC1, btw.
"First they came for the slanderers and i said nothing."
I call it Smartest Motherfucker in the Universe Syndrome. You see it all the time on Slashdot: People who seem to think they are gods gift to development. They'll talk about how amazingly stupid Company X is for not doing thing Y, when it is clear they have NO idea of all the shit that would be involved in that. They berate others for bugs in code, yet their own code is bug ridden, and so on. They are just convinced they are WAY smarter than all those other dumb programmers (and IT people, and managers, etc) out there.
It's a pretty common affliction and leads to more than a few problems. You are correct that it seems to congregate in some areas. I'd expand the RoR thing (though they are pretty bad) to just "web development" in general. Probably because of the massive market and low barrier for entry, it attracts a lot of people who's knowledge does not match their talk.
If only you had a 5 digit UID to lend any credence to your history in the industry.
A low UID means that you know about slashdot a long time ago. It does not mean a damn thing on any other topic.
The GP has point. There was a balance, even in the mid to late 90s. The GGP overstates things, if you were worried about the overhead of calling a common function you made that function an inline or macro. And the fact that my current slashdot account has a 7 digit UID in no way changes the fact that I actually worked on Starcraft and know Pat Wyatt and his partner in crime Mike O'Brien.
Yet here you are.
Well, I'm surprised linked lists were being used. For my in-progress game (Java combat flight simulator) I will almost never iterate over lists. Using an (associative) 'map' structure is vastly more efficient at runtime. You can still access the complete set as you would an unordered list if you need to, but most of the time you are trying to find a specific object based on some criteria (which the map is good for). Yeah, sounds like having a green programmer work on a bit of code that was heavily used affected everyone in the team. It is a surprised one of the more senior devs didn't see the problem coming and rectify it - but then most commericial games have insanely ambitious development timelines. Fortunately, mine game does not (since, as an 'indie', I'm not relying on it for my main income; and I'm not burning much capital on it either; slower to market, but guaranteed to be a better base for a long lived product line :) re-using the good code in later versions is where I'll make the real money).
That is all.
Blizzard was never any good when it came to programming. Their games are always extremely slow, so you can tell in the back the code is doing some pretty funky inefficient stuff, and the bugs are a glimpse at the underlying spaghetti code. Blizzard's coding should be taught in classes as: What not to do. They're still horrible programmers: UT3 renders ultra-realistic scenes 60fps flawlessly with all the settings on max, while Diablo 3 struggles to draw some top-down view... and that's just the optimization! Looking at bugs, they have some weird funky bugs, for example pressing two keys simultaneously in Diablo 3 made the wizard completely invulnerable. What kind of crappy coder lets that get in there?
Hmm strange, I've played the shit out of Starcraft and Brood wars. Single player and LAN, too. And not once it crashed. I think it was stable enough.
All of these lists were doubly-linked to make it possible to add and remove elements from the list in constant time — O(1) — without the necessity to traverse the list looking for the element to remove — O(N).
Please explain how a doubly linked list now makes add & remove O(1). The rest of us shlums have been relying on hashtables when we needed O(1) and assumed linked lists of any variety were always going to be O(n) at best...
So basically, game developers are some of the worst developers out there. Gee...like anyone really needed to read this article to know that.
With twice the number of units of its predecessor — StarCraft had a maximum of 1600...
Nope. Starcraft's unit limit is 1700.
associate hash maps were some exotic thing found in unversities. your main tools, turbo pascal and microsoft C++, had no such thing. you would have had to write the whole hashmap from scratch. considering they couldnt wrap their heads around writing linked list functions, i dont think a hash map would have worked out any better for them.
but people should remember the times. this was when a few MB of RAM was considered pretty damn good, dialup was normal, you went to something called a 'software store' to buy your games, which were stocked right next to books about how to write compression algorithms, and the latest version of PC Paintshop.
if you wanted to open a graphics context, you had to write the routine yourself, by calling interrupts into DOS and then use hand coded address of the graphics screen (A000:0000 iirc). There was no 'windowing system', there was no Standard Template Library, there was no boost, there wasn't really Python, it was just hack hack hack and get it out the door.