Slashdot Mirror
Designing Multiplayer Game Engines?
"Lag is not really critical, but I still want things to be responsive and it must scale up well with the
number of clients. The size of the map data, the complexity of the
objects and bandwidth constraints rule out sending the complete game
state, so only incremental updates will work. The situation is further
complicated by the need to limit updates to just the areas of the map
that are visible to a given player/team - this is clearly necessary to prevent client-side hacks such as gaining full map
knowledge.
I understand the theory well enough, but I'm interested in practical
advice on how to implement a solid architecture. What should the
object model look like? How do I propagate events that are only
partly within a client's field of view? Are there any novel features
in C# that might make my life easier? How can I make the networking code
as transparent as possible so I don't have to write SendUpdate()
after every assignment?"
Your comments, insights, hints and flames are eagerly awaited."
Having been on the dev-team of a popular MMPOG, I have to say that while your ideas are fundamentally correct, but now is not the time to learn c#, just to add another language to your CV. In addition, C# is too OS limited as it stands and you need to be thinking outside the box.
Thanks
Sigs are dangerous coy things
Sounds like the architecture you're planning is quite close to FreeCiv's. You might want to take a look at that.
there is an old (1999 ish) article at www.gamasutra.com/features/19990903/lincroft_01.ht m (free reg required) that talks about all the net play problems that the Xwing vs Tie Fighter team faced.
I remember reading it a while ago before you had to register, it had alot of insights as to saving game states, lag, updates, and "late joiners" to a game.
I'll avoid most of my comments about your choice of language because most of it is of a political nature, rather than practical one; however, I really wouldn't suggest trying to make a massively multiplayer game with a language you're unfamiliar with. It's quite an undertaking even with a language you know. I know; i'm working on one.
As for the networking code transparency, this one seems fairly obvious to me.. Just keep a datastructure containing all the changed or "tainted" objects as you go. Make mutator functions of your classes set objects as tainted. Then, just do the networking updates once or twice every time through the main loop (assuming it's in the same thread. Otherwise, you can implement something that might end up being a little more efficient).
As for updating only what the player needs to be updated on, this seems like a question of algorithm efficiency. I don't know the specifics of your game, but with most massively multiplater games, transmitting the entire world state, or even the entire list of changes to every client, every cycle would be insane. So, you have to only update the section of the world that the player can see. How to do this well depends on the internal structure of the world, and what sort of stuff the player can "see". If the game is room-based, then this is easy. If the player can always just see a specific size circle or rectangle around him, this is also easy (each event can check distance to all players in its regeon). If it works like most RTS with arbitrary viewing areas, then you might have to be a little more clever. Whether this is even much of a concern is really a question of the number of people supported, and the expected hardware this'll be running on.
Hope that helps,
ben.c
AOP might be a great way to handle the problem of dealing with the update issue. You can program as though they are on the same box and code your aspects to deal with what to send and when. It's not an easy thing to learn (think when you went from procedural to OOP) but it can pay huge dividends.
I use AspectJ (www.aspectj.org) but that's for Java...I don't know of any AOP toolkits available for C# since it's such a new language.
Good luck.
"All I ask is for a chance to prove that money can't make me happy."
If you've never written a client-server game at all, you might check out Netrek for some basic ideas. It isn't massive, but it is client-server, and it's where I learned most of the important things I know about network programming in general (and I do, in fact, get paid to write network code for games now.) Quake or Quake 2 source are probably also good things to look at, though I haven't seen their code personally.
None of those solve the major problems you're really asking about though: how to decide who gets to know about what. Worldforge is the only open source project I can think of to point you to in that area. Perhaps some MUDs might be useful as well, but they tend to be based on rooms, not areas, which doesn't translate well at all to most other games.
Probably the biggest problem is avoiding N^2 operations as much as possible. At some level, there's no way around it: N players in an area generate events that have to be propagated back out to N (or at least N-1) players. This obiously makes scaling to arbitrarily large populations difficult. On the other hand, if you can guarantee a set maximum number of players in an area/server/whatever, you can target that maximum and not worry about it a whole lot.
A slashdot post isn't really the right medium to answer this question. There aren't any quick and easy answers, you need to figure out what makes sense for your game on your own. I've spent a large portion of the last year thinking about these problems myself, and I wish you the best of luck on it. I'm having a blast myself, but it's also the most challenging work I've ever done.
I am interested in hearing your reasons for using C#. That's actually somewhat shocking, but maybe it's just ignorance on my part. I can't imagine why you'd want to bet the farm on such an immature system, regardless of the strengths you've perceived in it. What does it do for you that C++ doesn't, and is it really worth it?
First off let me say that you are way in over your head, don't go into this thinking you will actually make a everquest. Everquest took 4 years to develop with a full staff of experienced programmers and artists, DAoC only took 2 years, but they had a very complete engine to work with. However i do not want to discurege you, attempting this, while foolish is a noble task where you will certainly learn a great deal. that being said:
The only communication from clients would be commands to units, which is simple enough
Generally the way this is handled is setting up a TCP communication for critical information, such as the stats of the player, text communication etc.. then send small UDP (connectionless) messages for non-critical data, such as player and monster movments. Be warry of using TCP for everything, this is something Anarchy Online did, which caused some pretty massive problems on the server side.
The situation is further complicated by the need to limit updates to just the areas of the map that are visible to a given player/team - this is clearly necessary to prevent client-side hacks such as gaining full map knowledge.
There are two approches for this, one is a moving 'buble' of information, where you update the client with all data within a certain radius, another is the zone approach (i.e. everquest) where you send the an entire zone worth of information. Note that you are not sending everything, only monster,player movment and state (are they fighting).
What should the object model look like?
I have no idea, i imagine it would be far too large and complex to even describe in a slashdot post. However i recommend you look at it from a top down view, then try implementing from the bottom up. For example, you know that you have a 'world' and the 'world has players, monsters and items', 'players and monsters' could be concedered 'actors' with certain command antributes. 'actors interact with items' etc.. you should start seeing how to construct a object model. also don't worry about what this looks like at first and you will no doubt be revising it as you go a long, make things the work and compile so you have some possitive feedback for you hard work. When things seem out of hand, step back, make some coffee and draw out what you have and try to thing of a way to break it apart into managable sections.
ok that is all, good luck.
-Jon
this is my sig.
It sounds like you're taking a lot of the stuff off the client's hands to add security. While this is a good idea, I can't stress enough the importance of the client predicting the positions of players and objects. Objects especially are easy to predict (an arrow arcing through the air, something rolling, etc.) Movement prediction can make or break a game; you'd be surprised how little tolerance players have for people/things jumping around with no movement in between.
Heck, players can be pretty predictable objects themselves, if you want to get fancy. Most games have pretty simple algorithms predicting the future state of a player character (if player X is moving in a straight line, he'll probably continue to do so.) But I think you can get fancier than that. What about curves? (circle strafing) Or if the player is repeatedly hitting a button every half-second. (chopping wood?)
It'd take a lot of coding, but the master server could conceivably take note of each player's input characteristics and compile some sort of "personality" profile for each one, which it could transmit to clients every time they start up, to help with their local prediction.
Please note that I have no idea how much CPU time this would take up,
[PowerPoint] is a tool for capitalist presentation
In all seriousness, in my opinion (unless you're doing this solely as a personal learning experience), you are starting with two critical strikes:
1. You're trying to do a major project in a language you don't know (and an immature one at that).
2. You're trying to do a major project in a genre with which you have no experience.
Either one could cripple the project. Put them both together and you're doomed before you start. You may eventually make it work - sort of - but it will never work well, and it will be riddled with bugs.
I encourage you to start by developing a small multi-player game in a language in which you are already proficient. This will let you focus on the design and structure without fighting the language. Keep it simple, manage the scale, but incorporate the kinds of capabilities you want in the final version.
When you've got that working, throw it awy and develop it again in C#. Since you're starting with a working design , you're now free to focus on the mechanics of the language. You need time to learn its limitations and idiosyncracies, and to become proficient. (I will let others debate the wisdom of C# - I'm skeptical of all proprietary languages, especially until they're field-proven.)
Once you have succesfully finished a small project in C#, you can begin planning your real game. Based on your experiences, you may decide to scrap C# entirely. If you choose to stick with C#, then throw away ALL of your original code and start over. No matter how good you think your first code was, by the time you finish the big project you will know that it's crap. Might as well get it out of the way up front to reduce re-work and improve the overall quality.
Of course, if this is a project you've been assigned as a commercial effort, you won't be given the luxury of doing it well. You probably already have a deadline pulled out of thin air, and you're probably already behind schedule. Speaking as a pointy-haired boss who actually has significant coding experience (a long time ago, in a galaxy ... etc.), most PHB's have no clue when it comes to software development. They work with the suits to draw up pretty little Gantt charts, and haven't the foggiest notion as to why they are complete fantasies. You can see some of the results in the bargain bin of your local Best Buy, or in the "still delayed" list of your favorite gaming magazine.
In any case, good luck.
The important thing is that I really like the The WorldForge Project (in fact, it's one of my favourite projects (if not the favourite project), promoting the idea of free software to broader audience.
They're doing amazing work and I simply can't imagine what will they achieve in few years, but I'm sure all of current proprietary games won't even compare with those developed as a part of The WorldForge Project
So yes, I think promoring WorldForge is worth losing all of my karma.
Now back on topic...
You may want to announce your plans on the cpptraining at worldforge.org mailing list. It's originally meant to learn C++, not C#, but it's read by people who want to learn how to write MMORPGs, and those who want to and really can learn, like Bryce Harrington, so it's a good place to find people who can help with your project.
Also check out the other WorldForge mailing lists, especially Protocols, Server, Client and General.
Read the Development Area on WWW. Read about servers and clients. Use WorldForge protocols and libraries. Download games and read the source. You'll find there everything you need.
~shiny
WILL HACK FOR $$$
I made a 3d game engine and in my cleverness, I figured I'd reduce load on the server by offloading the physics asynchronously onto the clients. I learned the hard way that THIS WONT WORK. The reason is that different floating point processors get slightly different answers in some instances. Indeed, if all processors are the same stepping of the same intel processor, you'll be fine, however for example an Athlon might in some rare circumstance be different by the very last decimal point from an Intel. The butterfly effect will eventually catch up to you and the Athlon machine will for example detect a collision where the intels didn't.
Using emulation or fixed point is either too slow or too inaccurate, so I ended up just doing all the work on the server and doing continual sychronization.
To be more precise, in order to smooth out 'lag' time, the clients would do their own emulation, but would resychronize all decisions on the 'heartbeat'. Using interpolation, this worked out to have great apparent lag response, even lag times of 3 seconds were smoothed out. The only problem then was when a client's lag was unstable, fluctuating a lot. I smoothed this out by emulating a 1 second lag in all circumstance, so everyone has a smoothed out lag response which isn't too bad to play. Only unstable lag of 2 seconds or more caused a problem, where that client would see his character jump around everytime synchronization kicked in.
If you're really trying to develop a massively multiplayer game, make sure that you're ready for a collossal failure.
:-)
I can already see the debate about C# heating up over this -- but that's tangential to the real problem. It's not that C# will doom your project, it's that you wouldn't choose C# if you knew what you were doing (and your project weren't already doomed).
First, my personal suggestion (and I say this as a developer with 2 years commercial MMP development experience at this point): EITHER you want to write an infrastructure, OR you want to write a game. Writing both by yourself will take you the better part of 10 years. Having another programmer around who is doing the other part is handy -- but making sure that they are separate tasks is important. I recommend ditching C# to use Python, and my personal infrastructure project, the Twisted network framework (http://twistedmatrix.com), but if you're not going to use that, then find another high-level language with good asynchronous networking support and the ability to load code at runtime. Other good possibilities are Common Lisp and Scheme.
If you don't have any experience in the area, and this is for an Open Source project, join an existing project and learn some things from there. I can also highly recommend getting involved with a failed project in the game industry to see how difficult the whole thing really is
Be prepared to fail at least once. The number of failure points in an MMP project is astronomical: client code, server code, internet latency, even the community itself is a potential "bug". If your technology is great and your game is fun, but it attracts really mean-spirited people for some reason, you might see your servers empty out over the course of a few months, or never even get to a real "massively" multiplayer state.
Glyph Lefkowitz - Project leader, Twisted Matrix Labs
Writer, Programmer - Not a member of the TSU
Not exactly true.
Java has a lot of overhead. But that overhead is there for a reason. Java code gets faster over time as it runs (both due to optimizations done at run-time and the fact that class loading is still an expensive operation). However, in the case of a massively multiplayer environment, Java may be (and probably is) the best choice for the server environment. In a controlled environment where the application is expected to run for a significant amount of time, Java can actually perform as well or better than natively compiled code.
The "Java is slow" argument is soooo tired. Sure, there are many instances where Java is slower than other languages (even C#). But if you're going to knock Java's performance, explain why or shut the f@#* up.
"Don't blame me, I voted for Kodos!"