Slashdot Mirror

← Back to Stories (view on slashdot.org)

Load Balancing Heavy Websites on Current Tech?

Posted by Cliff on from the more-hits-thru-heavier-iron dept.

squared99 asks: "I have just delved into some research on a set up for very high traffic websites. I'm particularly interested in how many webservers would be needed at minimum and the type of technology powering them. Slashdot seemed like a good sample site to check out, so I went to Slashdot's technology FAQ to get a starting point. This setup seems to be from 2000, is most likely a bit out of date, and I'm assuming the same number of webservers would not be needed with current server technology. What would experts in the Slashdot community recommend as a required setup to handle Slashdot-like volumes, if they had to do it today using more current hardware? How many webservers could it be reduced to, while maintaining enough redundancy to keep serving pages, even under the heaviest of loads?"

6 of 63 comments (clear)

  1. Prime Example: wikipedia by dyftm · · Score: 5, Informative

    http://meta.wikimedia.org/wiki/Wikimedia_servers This is how they do it.

    1. Re:Prime Example: wikipedia by FooAtWFU · · Score: 3, Insightful

      Well, few sites of that popularity are quite as 'read-write'. When you have people submitting edits to articles every second, things get a little trickier.

      --
      The World Wide Web is dying. Soon, we shall have only the Internet.
  2. Test, test, test... by PaulBu · · Score: 3, Funny

    ... and you've just missed your greatest opportunity for this by not providing a link to your website! ;-)

    Paul B.

  3. Pound by slashflood · · Score: 3, Interesting

    Take Pound, a few web server machines, a database server and a NFS server (no Coda, AFS or GFS needed in most cases) and you should be set. This is a setup that I installed for a high traffic website and it is very stable.

  4. Your question cannot be answered by Guspaz · · Score: 3, Insightful

    It is impossible to answer your question unless you define "heavy" traffic.

    Some people might consider a hundred thousand pageviews per day to be heavy. Others might consider a million pageviews per day to be heavy.

    From experience a hundred thousand for a reasonable application can be handled on one server. A million would probably require 2 to 4.

  5. Re:Some more considerations by DrZaius · · Score: 3, Informative

    NFS sucks. Use something like CVS to keep your webroots in sync and have each server host it's own copy of the content.

    You get to get out a massive single point of failure (the NAS) and you get a little closer to linear scalability (adding another webserver doesn't put more load on your NFS box).

    --
    -- DrZaius - Minister of Sciences and Protector of the Faith