Slashdot Mirror

← Back to Stories (view on slashdot.org)

New Linux Petabyte-Scale Distributed File System

Posted by samzenpus on from the check-it-out dept.

An anonymous reader writes "A recent addition to Linux's impressive selection of file systems is Ceph, a distributed file system that incorporates replication and fault tolerance while maintaining POSIX compatibility. Explore the architecture of Ceph and learn how it provides fault tolerance and simplifies the management of massive amounts of data."

5 of 132 comments (clear)

  1. History by Alcoholic+Dali · · Score: 4, Informative

    Ceph was designed by Sage Weil (of WebRing fame), who is also one of the founders of DreamHost. They will likely be using it internally soon, if they aren't already. http://en.wikipedia.org/wiki/DreamHost

    1. Re:History by TooMuchToDo · · Score: 4, Informative
      http://www.dreamhost.com/jobs.html

      FILE SYSTEMS SOFTWARE ENGINEER
      Los Angeles, CA

      New Dream Network has a vacancy for a Senior File Systems Software Engineer in Los Angeles, CA. Minimum requirements – Master’s degree in Computer Science or Computer Engineering, minimum of 2 years experience in storage programming, and background in Linux kernel programming, file systems development, network programming and Operating Systems design.

      Qualified applicants should send a plain text resume to cephjobs@dreamhost.com

  2. Is it ready for primetime? by Meshach · · Score: 5, Informative

    The headline in the Ceph wiki: Ceph is under heavy development, and is not yet suitable for any uses other than benchmarking and review.

    --
    "Maybe this world is another planet's hell"
    Aldous Huxley
  3. Re:Is data integrity really necessary for large da by CoderJoe · · Score: 5, Informative

    Google's BigFile/BigTable architecture is a distributed filesystem. if a node goes down, the data that was on that node gets copied to other nodes to keep the replication count up.

    Facebook is using apache cassandra, which adopts similar designs.

  4. Re:Is data integrity really necessary for large da by ProfMobius · · Score: 5, Informative
    First, Facebook & Google data are not possible to regenerate, as they are personal things, like emails, messages, posts, etc.

    Second, you have other sectors producing large amount of data beside your favourite networking website. One example is the LHC. It is going to produce terabytes of data per DAY (15 petabytes per year). Another are space telescopes. Those data can't just be 'regenerated'. 1 day worth of data is incredibly expensive to produce.

    Distributed file systems are already there, and people use them. Maybe not on your level of computer usage.

    When you don't know what you are talking about, I think it is better to just keep quiet.

    --
    EULA : By reading the above message, you agree that I now own your soul.