Slashdot Mirror

← Back to Stories (view on slashdot.org)

The Design Of The Google File System

Posted by timothy on from the library-of-babel dept.

Freddles writes "This is an interesting paper (PDF) describing the design approach to Google's file system. The design had to take account of requirements for huge file sizes, a highly responsive infrastructure and an assumption that hardware components will always fail."

11 of 210 comments (clear)

  1. Only a file system? by jrrl · · Score: 5, Interesting
    Back in the early days at Lycos, Danner Stodolsky, now at Akamai used so many weird little tricks to make things faster that we used to joke that we'd end up with a custom operating system. The supposed name? LycOS.

    Luckily the world was saved from this possibility.

    -John (now, one of those "why, back in my day..." story telling guys... sigh.)

    --
    Self Serving Sig: Hosting Comparison
    1. Re:Only a file system? by FireBreathingDog · · Score: 3, Interesting
      Nice menu: not alphabetized, and "Use a digital camera" appears twice with two different icons. Then there's the inexplicable and unexplained "scribus" menu item, the only item that is neither a phrase nor capitalized.

      Steve Jobs must be shitting in his pants.

      --
      Shame on Google.
  2. Word processor? by Anonymous Coward · · Score: 2, Interesting

    What word processor/text editor is used to write all of these technical papers? Almost every paper I've seen looks like it's written in the same program.

  3. Various hardware life expectancies? by The+Ancients · · Score: 3, Interesting
    ...and an assumption that hardware components will always fail.

    I think perhaps this is something we could all take a little more seriously. Part of me realises this is a comment on the sheer data being manipulated, but then something else that sprung to mind is the gradual reduction of warranties on HDDs, for example. I wonder what sort of stats an operation of this size could gather on various hardware components, and their varying propensities to wither and die.

    --
    The Mothership
    1. Re:Various hardware life expectancies? by forevermore · · Score: 2, Interesting

      Gradual reduction of hard drive warranties? Didn't Maxtor just bump up the warranty on their drives to 5 years? And WD and Seagate both have 3 year warranties on their drives. Granted, I'm talking about the "good" (SATA, 8 meg cache, etc.) drives, not the cheap ones that most of us users are using rebates to get for really-cheap.

      --
      Do you really need reason for beer? Wingman Brewers
  4. they published it ... by trick-knee · · Score: 5, Interesting

    ... which may not have happened from just any company of google's prominence. I mean, they have highly successful business and technical infrastructure models and they didn't HAVE to share it with anyone.

    I wonder what they believe will protect their business from poaching of these ideas?

  5. RAIC?? by More+Karma+Than+God · · Score: 3, Interesting

    Could we call Google a Redundant Array of Inexpensive Computers?

    What else can it be programmed to do? Could this become the basis for a personal computer where you just add computers seamlessly when you need more power?

    --
    Go here to create your own Slashdot dis
  6. Re:And starting with Linux 2.7... by Anonymous Coward · · Score: 3, Interesting

    Actually this sounds exactly like the sort of file system that would be useful in a render farm.

    How long before ILM or Weta has a GFS disk array?

  7. PC #1782563 by can56 · · Score: 2, Interesting

    See Verity Stobs article -- Cold Comfort Server Farm -- in the August/2003 edition of Dr. Dobb's Journal, for the sad truth about Googles' server farm. Sniff ;-(

  8. This sounds like a GPL violation to me! by CypherDeaz · · Score: 1, Interesting

    On the GNU linux wouldn't under the true GPL licence such deep modifications to the GNU Linux be a GPL violation?

  9. Is there still a Google dance? by harmonica · · Score: 2, Interesting

    I thought the Google dance was history, and the index is now being updated more continuously (how exactly, I don't know)?