Slashdot Mirror

← Back to Stories (view on slashdot.org)

Open Source Highly Available Storage Solutions?

Posted by Cliff on from the RAID-taken-to-the-next-level dept.

Gunfighter asks: "I run a small data center for one of my customers, but they're constantly filling up different hard drives on different servers and then shuffling the data back and forth. At their current level of business, they can't afford to invest in a Storage Area Network of any sort, so they want to spread the load of their data storage needs across their existing servers, like Google does. The only software packages I've found that do this seamlessly are Lustre and NFS. The problem with Lustre is that it has a single metadata server unless you configure fail-over, and NFS isn't redundant at all and can be a nightmare to manage. The only thing I've found that even comes close is Starfish. While it looks promising, I'm wondering if anyone else has found a reliable solution that is as easy to set up and manage? Eventually, they would like to be able to scale from their current storage usage levels (~2TB) to several hundred terabytes once the operation goes into full production."

1 of 46 comments (clear)

  1. GlusterFS by danielcolchete · · Score: 5, Informative

    GlusterFS (www.gluster.org) is just "THE" best cluster filesystem I have ever studied. I'm testing a few of them for a project here at my job and, at least for my case, GlusterFS is the best. It can scale to the petabytes with as many servers as you want. It can also use InfiniBand as interconnect protocol besides the usual TCP/IP/Ethernet.
    It's design is simple is smart. Every feature is a translator that interconnects to other translators. So, you may organize your filesystem they way *you* want it.
    Let-me give-you an example: they have 2 translators: 'unify' to unifying harddrives as one and 'afr' for automaticly file replication. Depeding on the order you use it you have two completly different setups. You can have two cluters replicating eachother or you can have a cluster of replicating servers pair.
    Beside it's features and design, it's development team is *very* friendly. Yesterday someone (user) asked for a feature in the devel list, a get answered saying: good ideia, i'll do it.
    Very good software.
    Take a look: http://www.gluster.org/glusterfs.php