Slashdot Mirror

← Back to Stories (view on slashdot.org)

Building Scaleable Middleware for MMORPGs

Posted by simoniker on from the complexity-plus-plus dept.

CowboyRobot writes "ACM Queue has an article exploring the challenges of developing a reliable platform for an MMORPG, specifically looking at Wish by Mutable Realms. From the article: 'A common scalability problem for distributed multiplayer games relates to managing distributed sets of objects... A player may not be a member of more than one guild, or a guild may have at most one level-5 mage (magician). In computing terms, implementing such behavior boils down to performing membership tests on sets of distributed objects.'"

23 of 163 comments (clear)

  1. MMORPGs need better real-time characteristics by Bryan+Ischo · · Score: 5, Insightful

    I've tried a few MMORPGs and have found them all to be lacking in the same key area: one's control over one's character is not real-time. This is a generic description of a problem which surfaces in many ways in MMORPGs, most notably in the combat system. I haven't found one yet that allows real-time combat; it's always "click on the guy you want to fight and press the 'attack' button", then sit back and watch. Typically can do things like cast a spell or use a buff or otherwise make strategic changes to the way that your character is fighting, but you can't aim, run around, swing at the monster, etc, as you can with first person games.

    The game that comes closest to the combat system I would want is Jedi Academy, in which the multiplayer mode works just like the first-person real-time perspective of the single player game. You do have to aim, you do have to run around and avoid shots, you do have to swing your light saber yourself. I find this to be infinitely more enjoyable than the MUD-like "you hit the spider for 10 points, it hit you for 5 points" back-and-forth that is common on all of the MMORPGs that I have played.

    One gets the feeling in playing these MMORPGs that your client view of the world only loosely approximates what is happening on the server. You can make your character run from here to there and find that other people are "sliding" by or popping in and out as you get only sporadic notification from the server of what's really happening. It all gives a very disconnected feel that I really find unappealing about MMORPGs.

    There must be some kind of scaleability limitation though because Jedi Academy only supports about 30 players or so at a time in an area that is far smaller than a play area in an MMORPG. I think that if someone could design an MMORPG that played like an FPS, but had all of the depth and breadth of one of these not-so-real-time MMORPGs, it would be ideal.

    As an aside, has anyone beta tested Worlds of Warcraft? It like an excellent execution of the MMORPG genre, but I have yet to read any comments from beta testers on whether or not the fighting is real-time or "faked" like other MMORPGs is ...

    1. Re:MMORPGs need better real-time characteristics by Dodger73 · · Score: 5, Insightful

      This burns down to the scalability problems the article is mentioning. Real-time characteristics always mean more frequent transfer of potentially larger data packages, and the more frequent processing of those packages. While you may be able to run Jedi Academy with 30 players on a cable connection, the same is not necessarily true for 300 players.
      There are ways to at least make bandwidth and processing requirements scale less than linearly with the numbers of players, but the actual problem persists. The more players, the more data. The more data, the more bandwidth requirements and the more latency. The more latency and bandwidth requirements, the more the realtime characteristics suffer. Needing halfways reliable security (read: hack protection) methods doesn't make it any easier.

      It is not only the reason why MMOs aren't realtime like an FPS, but also why FPSs aren't MMP like MMOs ;)

    2. Re:MMORPGs need better real-time characteristics by Surlyboi · · Score: 5, Insightful

      There must be some kind of scaleability limitation though because Jedi Academy only supports about 30 players or so at a time in an area that is far smaller than a play area in an MMORPG.

      You hit the nail on the head with the scalability issue. Unless you're playing a game like Planetside where there's no significant penalty for dying, (other than just having to respawn and grab more gear) you're going to have a lot of unhappy players who get 0wned by the LPB twitch freaks.

      I think that if someone could design an MMORPG that played like an FPS, but had all of the depth and breadth of one of these not-so-real-time MMORPGs, it would be ideal.

      I agree, it would be spectacular. But as it is, there're a ton of people playing SWG who'll just spam damage on players as they load into new zones. Unless everyone in the world is on the same footing connection-wise and the ganeworlds are seamless; a real-time implementation of a combat system would only compound this kind of grief play.

      --
      Mod me down and I will become more powerful than you can possibly imagine...
    3. Re:MMORPGs need better real-time characteristics by Bryan+Ischo · · Score: 4, Insightful

      Well you are lucky that there are DOZENS of MMORPGs that satisfy your gameplay requirements. I am just looking for ONE which satisfies mine.

      For what it's worth, the MMORPGs that I have played are pretty weak in the strategy area anyway. Really there is no reason for a fight to last more than 1 second anyway. It might as well work like this: you click on the spider, the server pre-calculates how much damage it would do to you and you would do to it, and the server does the damage and it's done. There is no reason to have to sit and wait while your avatar hack smindlessly at the spider at a pre-determined rate and the spider does the same to your avatar. If there is no skill involved in the actual fight, then just skip it and go to the results!

      Yes, it is true that you can cast spells and such, or switch to a different weapon normally. But I've found that it just leads to a formula which you use over and over again when fighting. You click on the spider, you say attack, when it hits you you heal, you watch the attack while you want for your mana to recharge so you can heal again, etc. You might as well just code all that up into a script that you run whenever you want to attack a spider.

      I like real time fighting because it brings a fun arcade-y aspect to the game. It also makes it feel like you're more "in" the world and actually controlling your avatar, instead of just sitting back and watching what could have been a MUD anyway if it weren't for the 3d graphics.

    4. Re:MMORPGs need better real-time characteristics by cbreaker · · Score: 4, Insightful

      Exactly. A 20 person UT match requires a surprising amount of bandwidth alone to make it as "realtime" as it can get, and a lot of CPU usage. If they were to try and accomplish the same thing on a scale of 5,000 players per server cluster at any given time, it would require too much bandwidth and resources to be profitable.

      I see this becoming better in the future as CPU power and bandwidth get more and more available, and the prices of these games get higher and higher.

      --
      - It's not the Macs I hate. It's Digg users. -
    5. Re:MMORPGs need better real-time characteristics by harlows_monkeys · · Score: 4, Insightful
      I've tried a few MMORPGs and have found them all to be lacking in the same key area: one's control over one's character is not real-time....Typically can do things like cast a spell or use a buff or otherwise make strategic changes to the way that your character is fighting, but you can't aim, run around, swing at the monster, etc, as you can with first person games.

      That's because of the RPG in MMORPG. In an FPS game, it is supposed to be a contest of your skill and reflexes vs. mine. In an RPG, on the other hand, if I'm a 20th level Fighter and you are a 10th level Fighter, I should be able to always beat you on physical skill. The only way you should be able to win is if I do something strategically wrong. Hence, the lack of detailed control over the physical aspects of combat.

    6. Re:MMORPGs need better real-time characteristics by SnappleMaster · · Score: 3, Insightful

      Have you tried Dark Age of Camelot? Combat is "turn-based" like most MMORPGs but because of the way the combat system is designed, competitive player-versus-player combat is really more like FPS than "traditional" MMORPG.

      It may not be quite what you're looking for, but combat in DAoC is certainly not "click attack and wait for someone to die".

      --
      Be happy. Nothing else matters.
    7. Re:MMORPGs need better real-time characteristics by Anonymous Coward · · Score: 1, Insightful

      *cough*Planetside*cough*

  2. Re:Scalability and joining guilds by ackthpt · · Score: 3, Insightful
    Surely this is a classic example of the Manager pattern.

    I dunno, I've seen groups where there are 5 managers and one peon. Managers seem to accumulate.

    You have a bunch of objects [Avatar] (all alike, at least programmatically :-) who want to perform operations on other objects. If the system has a [GuildManager] class, then access to this distributed network of avatars can be forced through the choke-point of 'can this avatar join this guild'.

    Uh. By the time you're an Avatar you shouldn't be joining guilds. You should be heading them. Unless, and I may be mistaken here, Avatar is now an inflated title for someone who's really the assistant to the vice grunt.

    --

    A feeling of having made the same mistake before: Deja Foobar
  3. Scalability problem? by Anonymous Coward · · Score: 1, Insightful

    I don't see a scaling problem in the example.

    Why not just keep a variable on each character associated with each guilds he's in. And then, each guild keeps a list of members.

    Thus, a query on the data is always fast and local.

  4. They should focus on a game that has shipped. by Anonymous Coward · · Score: 1, Insightful

    Everybody has their own MMO now, but unless they've actually shipped one they really probably don't understand the true challenges in building one.

  5. Its not fake, its a dice game by rufusdufus · · Score: 4, Insightful

    The RP in MMPORG means Role Playing ala Dungeons and Dragons. Thus, the focus is on character development, not first person shooter style twitch. RPGs are based on a dice game, and is really about mathematics. The people with powerful characters are the ones who can do math, not the once with a cable modem in the same town as the server.

  6. Nothing To See - Move Along by Anonymous Coward · · Score: 1, Insightful

    Wake me up when you have something written by someone who actually has written and shipped a commercial MMORPG.

  7. Do some of the work client-side... by mooman · · Score: 4, Insightful
    Given their monstrous system requirements:
    * P4 2.0GHz; P4 2.8GHz recommended (or Athlon equivalent).
    * 512MB RAM; 1GB recommended.
    * 64MB DX 9.0 Video Card (GeForce 3/4 Ti; ATI 8500+); 128 MB GeForce FX or Radeon 9600+ recommended.
    * 16bit Sound Card; 24bit recommended.
    * 8 GB free disk space; 7200+ RPM recommended.
    * Connection to the Internet; 33 Kbps modem minimum; broadband recommended.

    ...maybe they should find a way to send datasets to the client machines and let them do their own manipulations.. Needing 8 Gig of disk on a 2+ GHz machine has to imply that the server doesn't handle all the real-time work... They are prime candidates for middleware that does some distributed computing and let all the customers' beast machines do the grunt work...

    --
    In the Portland, Ore area and like card games? Check out: http://groups.yahoo.com/group/portlandgames/
    1. Re:Do some of the work client-side... by Dodger73 · · Score: 3, Insightful

      That is one of the things to be done to improve on scalability issues. However, to prevent hacking, most of the data has to be at least verified by the game servers. Otherwise you end up with players using speed hacks to outrun you at 150MPH, fight four times as fast, or cast fifteen devastating uber-magic-spells at a time. So, again, you end up with the game server having to know everything, even if the frequency of processing here doesn't have to be quite as high.

      Another problem, and this is the major issue I see with Wish's MSRQs, is, the target market. To date, and I don't see that one changing anytime soon, Asia is by far the largest market for MMORPGs, Korea being the largest in the bunch, with some 15 million people being involved in one or more MMORPGs (on a side note, interestingly enough, the 2D MMO Lineage being the most successful with supposedly over 3 million (!) subscribers. Let that one sit in your head for a while. Then imagine the continuous revenue stream at, say, $4 a month per subscriber. Scream. Read on once your antidepressants have kicked in).

      The majority of asian MMO-players play in gaming cafes. Machines like, say, a P3/800 with a GeForce1 or Radeon, or equivalent systems, is the predominant amount of gaming horsepower you'll find in Asia. This is one of the big problems a developer of an MMO has to deal with, and it is one of the big reasons, why there are so many MMOs that either are not even released for the western market, or simply don't make it. The asian gaming landscape is quite different from the US and european ones. With its current requirements, Wish has little of a chance of wide spread adoption in Asia (unless everyone there suddenly decides to perform major upgrades on all of their machines). With accumulated development and infrastructure cost for a big MMORPG at launch time exceeding $8 Mio. more often than not, it is quite a gamble to not give the asian market major consideration in both gameplay and requirements.

    2. Re:Do some of the work client-side... by Stray7Xi · · Score: 2, Insightful

      I disagree... I know everyone will say "Never Trust the Client" but it shouldn't be considered a rule, but taken as very sound advice. If it was to be considered a strict rule multiplayer FPS and MMORPG's just wouldn't be viable. In the mudding community there's been a few games that have tried to move to graphical versoins using a dumb client, they always suffer awfully from lag, very tilebased movement or slow movement.

      There are certain things that are still appropriate to be done on client, in fact some are pretty much required. Most (all?) mmorpg's do movement prediction and collision detection (if it's in the game) on client side. This is where distribution could help the most.

      Most games players CAN speed hack, it's possible and most of the security mechanisms are on client (so the client turns you in as a cheater). The GM's take a reactive approach, banning anyone getting caught doing it (and it's quite easy to detect a speed hacker)

      So how can this be distributed? Make player's clients police eachother, every client observes the movements of X (3?) other nearby players. If they go over the speedlimit or break collision rules (walk through trees? other players?), flag them for the server. Get enough flags and the server will do calculations. If there's false flaggings (hacked client reporting others as cheaters) coming from someone, take less heed from their reportings (or investigate them).

      This doesn't need to be instantaneous (because this is reactive not proactive, just like how mmorpg's are done now). This will work on huge datasets since it's being broken down into small problems (the bandwidth requirements may be a bit higher, but should be negligible). Finally, it puts less trust then the current model.

      The problem with this is it still allows collusion (not to be confused with collision) of players with hacked clients. But they can't cheat near other players at the cahnce they'll be policed (which they pretty much have to do with current model anyways). So it'd work well in adversial (pvp/pk) games but maybe not in EQ-type where 3 people can go to a corner of a world and cheat out of sight (all 3 with hacked clients).

      Distributed clients would be perfect for integrity checking of others. Since these games require you to trust the client to some extent. Furthermore it may not be game related, but its about time these patch servers took a BT approach.

  8. Re:Scalability and joining guilds by Anonymous Coward · · Score: 5, Insightful

    This problem is in no way limited to MMORPG, the problem of authenicating and managing objects across multiple servers/clients is central to all online games. As a hobby games developer with a pretty good understanding of this I suggest you read Policing Online Games and then compare the conceptual pitch to issues in digital cash and online money transfers etc.
    These ideas also overlap with the much hated and draconian 'trusted computing' models.

    Enforcing a set of rules across a network of untrusted hosts is a fascinating problem. For example Gnunet and Freenet forgo a centralised trust agent and allow trust to emerge from the interaction, and recorded past behaviour, of individual nodes.

    Digital 'trust' is sure to remain a huge area of interest. However it will also continue to be an area dominated by soothsayers, witchdoctors and charlatans because it contains a numer of fundamental logical problems which are not solved in the traditional human way of appeal to authourity.

  9. Re:That sounds complicated by Great_Jehovah · · Score: 2, Insightful
    Lets all go play Savage on Linux instead.

    Is it actually available for Linux? The site seems to imply that it is currently Windows Only.

  10. Re:"Authors thought of everything". by Anonymous Coward · · Score: 1, Insightful

    I might be wrong, but isn't Nethack text based? It's a lot easier to write a description of an action being carried out, than to actually simulate the action in a virtual world (with physics etc). Animating a towel being dipped into a fountain (an animation which must be carried out by all sorts of different player types and models), is hardly comparable to just writing "you dip the towel in the fountain" in some data file.

  11. UT Design Is Not MMOG by meehawl · · Score: 3, Insightful

    A 20 person UT match requires a surprising amount of bandwidth alone to make it as "realtime" as it can get

    The large quantity of bandwidth exchanged in a UT (or similar peer-based FPS game) is an artifact of a design as single object view game with no distributed Server-side processing. Instead of waiting for bandwidth and CPU nirvana, there are smarter ways to maximize Server-side entity state updates while optimizing Server-Client bandwidth and delivering only environmentally-relevent data. Also, using multiple, distributed Servers enables you to multiplex Server-Client entity state updates using multiple pipes so you don't get a blocks or racing on a single message broker.

    --

    Da Blog
  12. Math is not the point, though by Shoggoth+of+Maul · · Score: 3, Insightful

    "RPGs are based on a dice game, and is really about mathematics. The people with powerful characters are the ones who can do math, not the once with a cable modem in the same town as the server."

    Basically true (about D&D and other RPGs, I mean), although it's not the ideal. The Holy Grail of both P&P (Pencil & Paper) and MMO- RPGs is a system that conforms to common sense, so that math enters your gameplay only a little more than it enters the processes by which you live your everyday life.

    That way you can really focus on *CHARACTER* development, rather than *STAT* development.

  13. Re:Truly a Dilbert moment. by humankind · · Score: 2, Insightful

    I agree with you and this is what really, really disturbs me about programmers and development projects these days.

    In the classic sense, you select the tools/language best suited for the job. When you're dealing with huge amounts of data and lots of simultaneous usage, performance is going to be the difference between success and failure.

    Yet, what do these people do? They choose a set of tools because development would be less problemmatic? Catering to the narrow specialties of their development team is a priority over the long-term performance and stability of the application? Or better yet, let's offload the responsibility for the game being "playable" upon continued development of new technology, which at some point might actually provide the resources necessary to make our badly-designed application perform adequately.

    I understand the importance of selecting tools that are well-supported and easy to use, but some applications, such as MMORPGs really require hard-core performance. As a programmer, I've always felt it was a cop-out to force the companies I develop software for to throw more hardware at the application until it stops burping.

    On the other hand, Slashdot is using Perl in real time, so maybe I should just shut up.

  14. Re:This project is going to fail by Anonymous Coward · · Score: 1, Insightful

    Web services were never designed to solve this sort of problem. They were never focused on the needs of a performance-critical, real time system like a MMORPG, but rather on doing things like swapping data between corporations and driving Web sites. It's probably more worthwhile to compare with things like MPI (used in supercomputing applications) and CORBA than XML-RPC or SOAP layered over HTTP. I think it'd take a lot of hubris to decide that the latest buzzword of the day (Web Services) is going to magically solve all the problems of the "old" models (CORBA, application-specific protocols).

    Oh, and not to reopen that old debate, but Web services is basically CORBA ten years ago, with a different implementation (XML, SOAP, etc.) that should hopefully work better than CORBA's baroque design-by-committee industry standards. But behind all the buzzwords, we're still talking about the same basic sorts of distributed technology.

    I don't imagine you want to suggest to John Carmack that he implement the Doom 3 network code using Python and XML-RPC, do you? :) MMORPG middleware is probably one of the few applications where it still makes sense to have a lot of C/C++ code.