Slashdot Mirror


On the Feasibility of Single-Server MMOs

GameSetWatch takes a look at the issues involved in creating an MMO that does not split its users among many different servers. They suggest that running a single "shard" is the next step in the evolution of MMOs, since it better allows player choices to have a meaningful impact on the game world; supporting different outcomes across multiple shards is a technical nightmare. They estimate, from the hip, that the cost to develop the technology required to support a massive amount of players (i.e. far more than EVE Online) on a single server to be roughly $100 million. Another recommendation is the strong reliance on procedural and user-generated content creation to fill a necessarily enormous game world.

28 of 316 comments (clear)

  1. Lag. by Becausegodhasmademe · · Score: 5, Insightful

    TFA misses out one very important point.
    Lag.
    Lag is the primary concern amongst many EVE Online players. Certain areas of MMOs are more popular than others. Major Cities, battlegrounds, etc are places where large numbers of players congregate. Until we find a way to elminate the lag caused by sheer population density, single server MMOs are going to be strangled in what they can offer in terms of 'multiplayer'.

    1. Re:Lag. by zwei2stein · · Score: 4, Interesting

      "Half Sharding" like Guildwars has solves that.

      Is town too busy? Boom, new set of districts is spawned! They will be probably located on different server too, making players overcrowding non issue. Players can switch them at will (as bonus, they are grouped by geolocation and laguange, but can still switch to different ones.

      The only way to handle rush of thousands of players to one area when special events happen. 5 thousands of players want to participate in xmass feestival? no problem, just spawn 50 districts in that town.

      --
      -- Technology for the sake of technology is as pathetic as eschewing technology because it's technology.
    2. Re:Lag. by Canazza · · Score: 4, Informative

      Warcraft has this problem, especially in towns. It can take between 30 seconds to two minutes to fully load Dalaran the first time you log in as it has to not only download everyones armour, but then read/decompress that off your disk. My friend - who has 48Gb of RAM - created a RAM Disk and loaded WoW's game files onto it (WoW's game files come to around 13Gb). It loads fast, and I mean lightning fast, since the RAM has a faster throughput than the HDD.
      Most of the 'lag' in Warcraft comes from the user end, as opposed to any server creaking.

      This has led to some humorous bugs in the game. For example, "The Naked Bug" - which tends to happen either in Cities or when going into an instance (25 man instances mainly) Where characters armour fails to load and your buddies are all running around in the nip. Anyone already in the instance is naked to you, and anyone joining after is normally fully clothed. And the only way to fix it is for you to exit and re-enter the instance, or for each naked person to exit and re-enter (for example, by dying).

      --
      It pays to be obvious, especially if you have a reputation for being subtle.
    3. Re:Lag. by fractoid · · Score: 4, Interesting

      The mantra that I read somewhere and which I think describes the solution perfectly is "Fragment your world, not your playerbase". You can support any number of players in the same world without problems as long as player density doesn't rise too high. Just avoid centralising features from the game (the auction house in World of Warcraft was a prime example until they replicated it to all capital cities), or instance them off (invisibly, using something similar to WoW's zone phasing, but forcing parties to share the same zone to avoid "I'm standing right on top of you, where tf are you?" situations).

      Sharding's just the easiest solution, and hence most common. It's really not necessary any more, and detracts a lot from the social side of sharded MMO games.

      --
      Rampant carbon sequestration destroyed the Dinosaurs' tropical paradise. I'm here to help repair the damage.
    4. Re:Lag. by Canazza · · Score: 4, Informative

      What i'm saying is that a single server, or multiple servers, can't do anything for over-zealous reading from the users disk. That the Lag problem isn't solely related to the server.

      --
      It pays to be obvious, especially if you have a reputation for being subtle.
    5. Re:Lag. by Hurricane78 · · Score: 3, Interesting

      Well, you can easily solve this, if you allow some more strong MIP mapping technologies for everything, including the transferred packets, models, etc.

      For example you could just transmit the data of the most relevant people and objects, until the pipe is X percent full, then use some "group" model, with only simple coordinates and very simple models etc for every player outside of that relevance radius. And so on... in a curve that is somewhat between quadratic and cubic (depending on the dimensions of your relevance space).
      Do not forget to include the main viewing targets of your player into the calculation, so they can still target something far away, and see it in a good quality.
      You could even make it work like interleaved JPEGs, where you only load the roughest details at first, and then become more and more detailed, the more you want to view it (=the more you wait).

      I think people would prefer that to having every single object that is visible at all to be transferred and rendered in some kind. Nobody cares about the quality of stuff he does not care about. ^^

      --
      Any sufficiently advanced intelligence is indistinguishable from stupidity.
    6. Re:Lag. by KahabutDieDrake · · Score: 5, Interesting

      EVE's problem isn't really Lag. That is what everyone calls it, but that isn't accurate. EVE's problem is their architecture wasn't build with the scale of play they encourage today in mind.

      In EVE a solar system is a discrete "zone", there are many thousands of solar systems. Each one is assigned a node on the server (blades actually) and that node may and probably does host more than one solar system. They have had limited success in beefing up big fights by moving solar systems with expected fights to a node by itself. Thereby offering as much power as their architecture will allow. However, they can only do this at "down time" which is one hour daily, so if a big fight breaks out in the middle of the day, there is no chance of it going well. Most of the REALLY big fights happen in systems that often have very low average traffic, so they are assigned to a shared node. Then for 8 hours, that system has 400 people in it, and the node is past it's limits very quickly. If they had coded the game so that a single solar system could use more than one node at a time, they could brute force the problem away entirely. But that isn't possible the way it's built.

      Even so, the EVE cluster is/was on the top 500 list of super computers. You can't say it's not for lack of trying.

      Why yes, I WAS an EVE player. From Beta till about a year ago. I finally gave up after countless fleet encounters were destroyed by CCP's clever, but impotent load balancing. The breaking point was when I realized that even when we had a dedicated node for every solar system in our territory, we still couldn't have a full out fleet battle without crashing the node. I'd have been happy to get half our fleet into combat, but we couldn't even do that. Granted, we had 800 ships or so and our opposition had at least 1000. I've yet to see any game that can put nearly 2000 players on a battlefield and still function.

      CCP does get credit though for effort. 3 years ago you'd be LUCKY to pull off a 200 man fight. Now you can put 500 or so into a system and get your fight on without major game breaking things happening. It won't be silky smooth, but you can get it done. Ironically, 3 years ago a 200 ship fleet fight was a rare and wondrous spectacle. While a year ago, I could assemble a 200 pilot fleet in 20 minutes. So what was a major event is now a typical saturday night. The servers got better, a lot better, but they aren't keeping up with the players.

    7. Re:Lag. by Antique+Geekmeister · · Score: 3, Funny

      Well, that's not 'excessive reading', per se. That's 'too complex of a world'. That's inherent with such sophisticated graphical images, but is gracefully avoided by one of my favorite games, Kingdom of Loathihg. That's a stunning example of gameplay, social interaction, humor, and creativity reducing the need for complex graphics and sophisticated local computer capabilities.

    8. Re:Lag. by 3p1ph4ny · · Score: 3, Insightful

      I don't think that's the whole picture.

      For example, if I play WoW and meet a real life friend who plays WoW, we can only play together if we happened to sign up on the same server.

      If I play Guild Wars, we both sign in, switch to the same district, and we're in game together. It is possible for any player to go to any district in the game (if they have unlocked the content, obviously). So, you can always play with new people you haven't connected with before.

    9. Re:Lag. by argStyopa · · Score: 4, Interesting

      "I've yet to see any game that can put nearly 2000 players on a battlefield and still function."

      Then you have to look for games designed from the GROUND up to support such play. Typical 'sword and sorcery MMOs' that are built for perhaps up to a 'raid' of 50 players are simply not going to scale (without ridiculous hardware requirements) to 1000 players impact-free.

      OTOH - there ARE games that were built to simulate the actions of hundreds of players in a single area, so scaling to thousands is logically a lower burden. www.battlegroundeurope.com (formerly ww2online.com) catches a lot of crap for a graphics standard that doesn't approach the bells & whistles you see on today's (shoebox-arena) shooter, but I've played in many battles that (in a multi-square km area) with at least a thousand players including multi-crewed vehicles, hundreds of infantry, and a swirling aircraft battle overhead fighting for hours for a strongly-defended city. It doesn't happen often (playerbase is now only about 12k) but I just use it as an example of a system that WAS designed to handle such a load.

      --
      -Styopa
    10. Re:Lag. by N1ck0 · · Score: 3, Informative

      Fracturing a game based on location is not that difficult design wise. The main problem is cost effectiveness I hate pulling out business models and cost benefit analysis, but unfortunately centralized systems usually take either large centralized shared storage, RDBMS, schedulers, resource allocation management, etc; or the proper development of a distributed data server farm. This ends up costing a lot of money and a decent amount of development & operations manpower...in the MMO world your profits are not that high to begin with.

      Even on some of the grids that I've built that have gone google-style (map-reduce, column based DB, true distributed file system, on cheap hardware) the costs quickly hit a few million per location. On our larger GPFS, oracle, vmware, on HP based clusters the software site licenses alone will eat most game companies.

      Your best bet these days would be to try to develop something that uses Amazon's elastic compute cloud, apache's messaging system, run something like greenplum to distribute the DB load, and don't skimp on developers...cheap ones usually don't understand that in a distributed environment that every little chunk of code from the network/disk/etc up has to scale and be massively efficient cause you can't afford to 'just throw faster hardware at it'

    11. Re:Lag. by nschubach · · Score: 4, Insightful

      Depends on if you want a game or a game world. I prefer the game world concept. I hate instanced areas that segregate the community. Guild Wars always felt empty because there was never anyone playing in my areas. I remember holding instances in another game. I think it was EQ2. (Join a zone during peak hours and goof off or go AFK for a few hours while the players leave and the zone counts drop.) Why would you want this? Because you have a whole zone to yourself that no players would likely join. You had a better chance of getting rare mobs, collection items, etc. But it killed the game for me.

      Instancing allows people to play their own game and not get involved with the community. Essentially, you are paying for a multiplayer game with friends instead of playing an MMO with a huge community of people.

      Of course the publishers like this. (Less piracy, monthly income...) but it defeats the point of MMOs. Instancing, teleportation, and all the new "features" being put into MMOs are just breaking down the community and world feel. There's no point working with other players toward a goal. There's no point sticking with a group and completing a dungeon. You can just leave when your quest is done.

      --
      Every time I start to have faith in humanity, I ruin it by driving to work between 7 and 8 am.
    12. Re:Lag. by N1ck0 · · Score: 3, Insightful

      Also where you do have large open parts of the world you can dynamically allocate location clusters based on groups/concentrations of people, hand off the processing of those groups to various processing nodes. Usually you need to add the ubiquitous 'fog of war' to make this work graphically. Yes the management is complex, but it is very doable.

      Another major problem most MMOs make is they build the client protocol with way to many "global" broadcasting, and forced acknowledgment of updates. This tends to get out of hand very quickly. And from a design perspective while its cool to have things like every client triggering the play of the same sound and graphic event at the exact same split second, such events come at a high cost. A lot of games deploy these entire world updates for way too many basic changes....do I really need to ensure that every client withing 500 ft knows that X Player's avatar twitched his left hand, and then wait for an acknowledgment before moving on? You really need to prioritize what events are important and enforce that pretty heavily on the development team.

    13. Re:Lag. by cml4524 · · Score: 3, Insightful

      not get involved with the community

      Have you played an MMO lately? The "community" is usually the worst part of the game...

  2. I want to see all MMORPG on one server by linzeal · · Score: 5, Funny

    Think of how much fun it would be to crash one of those Eve Online ships into the shire!

    1. Re:I want to see all MMORPG on one server by Canazza · · Score: 3, Funny

      NPC: Hello Bodo Gaggins, we need you to deliver some pies to various places in the shire

      Deliver pies to:
      Bree
      Stormwind
      and The Crashed Ammar Frigate that used to be Hobbiton

      --
      It pays to be obvious, especially if you have a reputation for being subtle.
  3. Impact? by American+Terrorist · · Score: 5, Insightful

    "since it better allows player choices to have a meaningful impact on the game world"

    Am I the only one here who doesn't want the collective impulses of 1 million 15 year olds impacting my game experience? Instead of theorizing about how awesome it would be to have a server with 5 million people on it at the same time, why don't they try to design a game that would actually be fun to play with 5 million other people on your server. I can't think of any, but if they can, I'd at least be willing to listen to their ideas.

    Since the authors of this worthless article don't have any new ideas other than "WoW with tons of people on the same server!!!", I don't know how this thing got out of the firehose.

    1. Re:Impact? by Yvanhoe · · Score: 4, Interesting

      I was very surprised to discover the concept of shards when I first got interested into MMORPG. Such games are not MMO but simply multiplayer games... That makes the whole thing sound like a crooked deal : "yeah, there are 5 millions people connected but don't worry, you will be alone in the most popular dungeon. Oh, and don't worry you won't have any impact in the world since the place where you are is just instanced and will be respawned identically once you finish it."

      EVE is a very good example of the interest of having a single universe where players actions do have an influence, but it is not for everyone, some people will get intimidated by it. However, I don't understand why they think that the EVE model won't scale. They already have 300 000 players and their system is quite simple : different stellar systems can run on different servers. You cannot interact with something that is not in the same solar system, for that you have to "jump". It is easy to add more systems by adding more servers. It does scale up.

      --
      The Wise adapts himself to the world. The Fool adapts the world to himself. Therefore, all progress depends on the Fool.
    2. Re:Impact? by Anonymous Coward · · Score: 4, Insightful

      Am I the only one here who doesn't want the collective impulses of 1 million 15 year olds impacting my game experience?

      For one, you'd be mistaken about the "it's only 15-years old playing" cliche. Granted, a lot of oldsters act like they're immature teens, but still. If you're interested, head to Charles Stross' presentation at Login 2009 where he speculates about MMOs in 2030.

      Anyway, you won't be rolled over by a 1-million strong wave of players trying to kill-steal your camp. There's a difference between total numbers and player density in the game world. You need to increase the numbers, but keep the density low. And that requires a lot of social systems in place. Regardless of whether you're thown in Smallville (WoW-ish model, complete with its contingent of sociopathic über-powered peeps ready to faceroll you for the fun of it. Like Smallville. Minus Clark) or Metropolis (the uber-EvE model), you probably know only the few people in your guild, your guild alliance, some iconic local figures, and that's it. What makes interesting a large server are:

      1) The large choice: no matter what you're interested in, there's people interested in the same (which, on smallville-type servers, is an iffy proposition)
      2) The economy. It's impossible to build a workable complex economic system when there's less than 10k people interacting with the economy. The economy remains ludicrously simple (and easily manipulated). Compare to a 250k-players economy of EvE, and it's not even comparable. Go to million-players economies, and you end up with mechanisms that work.

      But the two points above should tell you what kind of game it's interesting to design. It's not a game about being the Hero and going to Slay the Dragon. It's a game about social interactions, player-based activities, with a major economical facet. That's what you need. Not, like the article implied (it's a general public article, after all) WoW with a single shard, but something different. EvE shows it's possible (why do you think the vignettes for the article show that game?).

    3. Re:Impact? by Bored+Grammar+Nazi · · Score: 3, Informative

      Not, like the article implied (it's a general public article, after all) WoW with a single shard, but something different. EvE shows it's possible[...]

      It's possible on EvE because the place is basically empty. A space station here, an asteroid belt or two there, a couple planets... In the end, it's just a bunch of generic models thrown around in what otherwise is a lot of empty space.

      Now imagine doing the same on a WoW-style game. You would have to do a huge work of world design. Surely there are algorithms that can generate assloads of terrain, cities, etc, but your designers still have to tweak things in the gameworld to make it interesting.

      Not to mention the amount of disk space that would be required for the data files making up this non-empty gameworld.

    4. Re:Impact? by montyzooooma · · Score: 3, Informative

      Technically they have 300k user accounts, and probably about half that many players. Multi-accounting is encouraged on Eve.

    5. Re:Impact? by Sobrique · · Score: 3, Informative

      Actually I think that's one thing that EVE does quite well - because of the 'nature of the game' the annoying wombats don't tend to get all that far - skill training being real time for example, is one of the best things ever - because it means you _have_ to be patient, you cannot grind level 60 in 2 weeks.
      It also means that there isn't really 'levelling up' - anything you choose to do, doesn't mean you fall behind said 15 year olds. And also, in EVE, the other players are targets, resources and obstacles, which means having more annoying idiots, just gives you that much more of a warming glow when you exploit them mercilessly.

  4. Re:Ah, but what I'M interested in is.. by American+Terrorist · · Score: 4, Insightful

    how long until we see a MMO that is run _entirely_ server-side - that is, a game that does all graphics and game logic calculations server-side, OnLive style, and simply streams HD-video to each client. If this considerable hurdle could be jumped, we might be able to see a lot more in the way of a dynamic MMO world, with actual physics, terrain deformation, collision detection, and a bunch of other features that are tough to do with the current client-server system.

    Just.... no. There are many reasons graphics are done client side. Let me know when you figure them all out. Sure, something like this is theoretically possible, but only if you want to pay upwards of $200 per month to play that game. Since it's a lot cheaper for me to buy a really nice graphics card, why not render client side?

  5. Actually, I see an even bigger problem by Moraelin · · Score: 4, Interesting

    Actually, I see another even bigger problem, at least for more traditional (WoW-type) MMOs. How big should your world be?

    Too little player density => people start complaining that it's pretty much a single-player game like Oblivion, except you occasionally see another player. Many games ended merging up shards more for that sensation of empty space than because of costs. (It's equally easy to just merge the physical servers inside a shard, to support a lower population per shard, if you're only concerned about hardware costs.)

    Too many players on too little surface => lag (think: landing in Ironforge, back when it had the only auction house for Alliance), routinely having 5 players camping the same mob, and generally it just starts feeling cramped. Again, you have players starting to complain.

    Basically if you want to be single-shard, you have to essentially guess how much population you'll get. Maybe just within the right order of magnitude, but guess nevertheless. It's not that trivial. On one side of the guessed-wrong spectrum you have WoW which got launched with only a handful of servers and had massive queues, on the other end of the spectrum you have more than one game who thought they'll be teh WoW-killer and then had to merge 4 servers in 1.

    Merging or splitting shards is an easier way to deal with that problem than having to physically add or remove new areas, to fit the population.

    Additionally, world size influences other things, like travel times, exploration, etc. There is an ideal apparent size where people don't feel like they're being packed like sardines and running around a back yard, but don't go "fuck it, I'm not spending another hour just running back to the quest giver" all the time either. It's easier to fine tune that if it's its own problem, orthogonal to everything else, than when it also has to fit the population numbers.

    Basically if EVE's game type was well suited for that kind of one-shard world, more power to them, but for other types of MMOs it might actually be a bad idea.

    --
    A polar bear is a cartesian bear after a coordinate transform.
    1. Re:Actually, I see an even bigger problem by Opportunist · · Score: 3, Insightful

      "RP servers? Oh, you mean the ones where I have to choose a "in game style" name, but get rewarded with not having to compete for the surface mobs because everyone else is sitting in the inn, but at my beck and call when I need them for an instance, all I gotta do is ask akin to 'mighty warriors needed to unearth the rock of awesome...'? Yeah, they're cool!"

      *sigh*
      This is unfortunately what RP-Servers are, essentielly, like. You don't really get any good RP there either. Simple reason: No reward for good RP.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    2. Re:Actually, I see an even bigger problem by Xelios · · Score: 4, Informative

      EVE solved this problem by creating a big world to start with then artificially cordoning off certain regions. You literally couldn't go there, even though the regions existed in the database and showed up on the galaxy map. As the population density grew they gradually started to open up more regions to the players.

      --
      Murphey's fighting Occam, and we're in the stands.
  6. 15 year olds? by Moraelin · · Score: 5, Insightful

    Actually, while I'll agree with your main idea that it's undesirable, I just have to wonder about the 15 year olds quip.

    From my limited experience -- and fully aware that the plural of anecdotes isn't data, but I have to start somewhere -- the vast majority of children I ended up grouped with were actually nice people and played the game well. Conversely, most of the more annoying trolls I've known, were middle-aged men. I guess mid-life crisis goes "I can still gank newbies" instead of "I can still get a car with a wing, and teenage hookers" in some people.

    The thing that got me to start thinking about it all -- and bear in mind, I'm not saying it's the worst, just that it was a shock at the time -- was discovering that a (now ex) boss, a respected middle-aged, mid-level manager, was talking l33t in an MMO. I get a tell that, really, makes me wish I had a Rosetta stone to decrypt that garbled nonsense, and wonder who the fuck is that retarded kid? Lo and behold, it's the boss. I _know_ he can type very fast, so he doesn't even have the excuse of not having the time to type the "y" and "o" in "you."

    Another midle-aged guy I know gets his jollies ganking newbies. That's his idea of showing how great he is, apparently.

    One was literally the most retarded player I've ever grouped with. He managed to reach level 70 (at that time, the max) while still believing stuff like that if he takes a step back when an enemy slashes at him, the enemy will miss. 'Cause that sword doesn't reach to his new position, see. Geesh. Or he still thought that it's a good idea as a hunter to run backwards when he gets aggro, 'cause, see, he manages to squeeze in another ranged shot now and then that way. And generally, I mean, not just as in "hadn't figured out the game yet", but as in, "had the most ridiculous ideas and insisted that that's how the game works." He actually was proud of his "footwork", lemme tell you.

    After a wipe or two I actually wished we had a 15 year old in his place. At least those tend to be good at figuring out a game.

    One was not just a complete CS-head, but actually proud of his spewing the most offensive sexist remarks at anyone who had a female name in the game. There was stuff he was telling me (and you know you can't stop them from talking about CS even if you tried) that made _me_ cringe, and I'm a guy. And he's standing them beaming proud of how witty he was.

    Etc.

    So, 15 year olds? I can deal with 15 year olds. It's the older retards that I fear a lot more.

    --
    A polar bear is a cartesian bear after a coordinate transform.
  7. Re:EVE ONLINE IS NOT SINGLE SERVER! by Sobrique · · Score: 3, Informative

    Marketing hype? That EVE is a single universe, with a peak concurrent user count of >50k? How's that 'hype' - it's true. Yes, they do multiprocess by switching you onto different servers when you jump system (sometimes). They also multi-process by having a proxy tier, a solar system tier, and a database tier. But that's not 'sharding' either.
    When most MMOs have too many users, they end up firing up a new copy of the code on a standalone server, and let there be another 'copy' of the universe, for someone else to be 'king' of.
    The only place that's happened with EVE, is the China implementation, and that was solely because China has rules regarding MMOs that ... would have been generally negative to implement on the Tranquility (original) cluster.