Slashdot Mirror

I won't bother going through comments to see if this has been posted.

This story has some interesting comments from a guy who claims he's CEO of digitalbazaar.com, a company that created a distributed filesystem named Starfish.

Open source and cross platform.

AC because I can't remember my freaking password.

[FULL DISCLOSURE: I am the President/CEO of Digital Bazaar, we created Bitmunk - a legal P2P music trading network that uses watermarking to protect both artists and customers].

This isn't a new concept, Bitmunk has been doing it for three years. We have over 1 million songs on our network - it's something that works for both the artists and the music fans. We're one of the only companies on the net that have empirical evidence that artists are willing to sign up to this setup. We believe it's a fair balance between the two approaches... there is a very long debate that I and Bill Rosenblatt had on this issue over 2 years ago:

http://wiki.digitalbazaar.com/en/Manu_Sporny_vs._Bill_Rosenblatt_DRM_Debate

It is a shame that we have been pushing this concept ever since our company was founded and it is just now reaching the mainstream...

GFS

You can't really compare GFS and Starfish. GFS is a shared-disk clustered filesystem. For example, GFS is useful when you have 10 machines sharing a cabinet of hard drives via FibreChannel or shared SCSI. The disks must be situated very close to the machines.

Starfish is a distributed clustered file storage network. Starfish is capable of using any commodity grade hardware and uses software to ensure high availability and node-level fault tolerance. Hard drives do not have to be shared between machines (a requirement for GFS) and it is meant to scale to 100s if not 1000s of nodes (GFS doesn't scale to that size easily). Starfish is capable of using GFS as its block-level file storage mechanism. Starfish is a file-level network file system - GFS is a block-level local file system. I hope that clears things up... in short, you don't have to pick between GFS and Starfish - they are complementary.

Lustre

The Lustre team is a great bunch of people - we ran their clustered file system software for two years before we needed to create our own. Lustre is run on most of the highest performing supercomputers in the world. The biggest reason we had to create Starfish was because Lustre does not distribute its metadata. There is a comparison between Lustre and Starfish on our website. You should check them out if you're trying to decide on a good distributed clustered file system - Lustre focuses on ultra high performance, Starfish focuses on data redundancy and high availability.

IBRIX

No idea - we haven't had a chance to get some real-world benchmarks from their clusters. As far as bottlenecks in their system, we don't have access to their source code, so we can't do a thorough analysis. It seems that their system design is close to ours, I would expect that they currently perform better than Starfish due to the maturity of their project. You tend to have to export IBRIX filesystems via NFS, which limits the fault-tolerant aspects. Starfish is fully POSIX compliant and can be mounted just like any Linux filesystem. Wish I could tell you more - in short, they probably perform better, have more features and cost far more than Starfish.

GFS and Lustre have free downloads available via their respective websites - it's worth taking them for a spin. You can also download and play around with Starfish, we even provide a quick start tutorial. You can even use it for up to 1TB of storage or 10 machines at no cost.

GFS

You can't really compare GFS and Starfish. GFS is a shared-disk clustered filesystem. For example, GFS is useful when you have 10 machines sharing a cabinet of hard drives via FibreChannel or shared SCSI. The disks must be situated very close to the machines.

Starfish is a distributed clustered file storage network. Starfish is capable of using any commodity grade hardware and uses software to ensure high availability and node-level fault tolerance. Hard drives do not have to be shared between machines (a requirement for GFS) and it is meant to scale to 100s if not 1000s of nodes (GFS doesn't scale to that size easily). Starfish is capable of using GFS as its block-level file storage mechanism. Starfish is a file-level network file system - GFS is a block-level local file system. I hope that clears things up... in short, you don't have to pick between GFS and Starfish - they are complementary.

Lustre

The Lustre team is a great bunch of people - we ran their clustered file system software for two years before we needed to create our own. Lustre is run on most of the highest performing supercomputers in the world. The biggest reason we had to create Starfish was because Lustre does not distribute its metadata. There is a comparison between Lustre and Starfish on our website. You should check them out if you're trying to decide on a good distributed clustered file system - Lustre focuses on ultra high performance, Starfish focuses on data redundancy and high availability.

IBRIX

No idea - we haven't had a chance to get some real-world benchmarks from their clusters. As far as bottlenecks in their system, we don't have access to their source code, so we can't do a thorough analysis. It seems that their system design is close to ours, I would expect that they currently perform better than Starfish due to the maturity of their project. You tend to have to export IBRIX filesystems via NFS, which limits the fault-tolerant aspects. Starfish is fully POSIX compliant and can be mounted just like any Linux filesystem. Wish I could tell you more - in short, they probably perform better, have more features and cost far more than Starfish.

GFS and Lustre have free downloads available via their respective websites - it's worth taking them for a spin. You can also download and play around with Starfish, we even provide a quick start tutorial. You can even use it for up to 1TB of storage or 10 machines at no cost.

GFS

You can't really compare GFS and Starfish. GFS is a shared-disk clustered filesystem. For example, GFS is useful when you have 10 machines sharing a cabinet of hard drives via FibreChannel or shared SCSI. The disks must be situated very close to the machines.

Starfish is a distributed clustered file storage network. Starfish is capable of using any commodity grade hardware and uses software to ensure high availability and node-level fault tolerance. Hard drives do not have to be shared between machines (a requirement for GFS) and it is meant to scale to 100s if not 1000s of nodes (GFS doesn't scale to that size easily). Starfish is capable of using GFS as its block-level file storage mechanism. Starfish is a file-level network file system - GFS is a block-level local file system. I hope that clears things up... in short, you don't have to pick between GFS and Starfish - they are complementary.

Lustre

The Lustre team is a great bunch of people - we ran their clustered file system software for two years before we needed to create our own. Lustre is run on most of the highest performing supercomputers in the world. The biggest reason we had to create Starfish was because Lustre does not distribute its metadata. There is a comparison between Lustre and Starfish on our website. You should check them out if you're trying to decide on a good distributed clustered file system - Lustre focuses on ultra high performance, Starfish focuses on data redundancy and high availability.

IBRIX

No idea - we haven't had a chance to get some real-world benchmarks from their clusters. As far as bottlenecks in their system, we don't have access to their source code, so we can't do a thorough analysis. It seems that their system design is close to ours, I would expect that they currently perform better than Starfish due to the maturity of their project. You tend to have to export IBRIX filesystems via NFS, which limits the fault-tolerant aspects. Starfish is fully POSIX compliant and can be mounted just like any Linux filesystem. Wish I could tell you more - in short, they probably perform better, have more features and cost far more than Starfish.

GFS and Lustre have free downloads available via their respective websites - it's worth taking them for a spin. You can also download and play around with Starfish, we even provide a quick start tutorial. You can even use it for up to 1TB of storage or 10 machines at no cost.

Really? Cause, your download page warns that it's a beta product and should not be used in production.

We do that as a preventative measure for people that don't get support through us. We don't want anybody to assume that the file system is ready for a highly-available cluster without talking to us first. As with all file systems, there are trade-offs to using Starfish. We are being honest - the software is stable as far as we can tell, but it doesn't have a great deal of field use (it was released to the public in March 2006).

We must be especially careful with file systems - data is very important to people. If somebody uses our system and loses data, we can't fix that - not that it has ever happened. We put the beta message as a warning that people should talk to us before thinking about putting our system into production. After all - it doesn't have nearly the amount of testing behind it that EXT3 does. It is common for a file system to remain as a beta product for the first year or two.

On the other hand, there stable ZFS support in OpenSolaris, (non-open) Solaris, and in a few months in FreeBSD, and since it's available fully free and open source, there are no necessary licensing costs. Are there any advantages of Starfish over ZFS?

Hmm... ZFS and Starfish aren't really meant to address the same storage problem. Take a bit of time and read through what ZFS does and what Starfish does. ZFS is a block-level file system. Starfish is a file-level distributed clustered storage system. Those are two very different things - at the end of the day they store files, but in very different ways and for very different purposes. Starfish can use ZFS as it's block-level file system... it can also use Reiser and EXT3.

Here are a couple of reasons to use Starfish (even though we think that ZFS is a fantastic solution for block-level file system problems):

• Starfish is a decentralized, multi-node fault tolerant file storage solution that provides N-way redundancy.
• Starfish is useful when you have a large number of nodes that need to access data in parallel. The ability to perform asynchronous parallel throughput is one of Starfish's biggest advantages.
• Starfish runs in userspace - which means that it is capable of using much smarter algorithms and databases to manage metadata and file placement. For example, a SQL database is used for file system metadata, which means that a variety of optimizations (such as creating metadata indices) to the file system can be performed at runtime.

However, this really isn't a "what is better, Starfish or ZFS?" discussion. You can have the best of both worlds: Starfish as the file-level network storage cloud using ZFS as the block-level file system.

Really? Cause, your download page warns that it's a beta product and should not be used in production.

We do that as a preventative measure for people that don't get support through us. We don't want anybody to assume that the file system is ready for a highly-available cluster without talking to us first. As with all file systems, there are trade-offs to using Starfish. We are being honest - the software is stable as far as we can tell, but it doesn't have a great deal of field use (it was released to the public in March 2006).

We must be especially careful with file systems - data is very important to people. If somebody uses our system and loses data, we can't fix that - not that it has ever happened. We put the beta message as a warning that people should talk to us before thinking about putting our system into production. After all - it doesn't have nearly the amount of testing behind it that EXT3 does. It is common for a file system to remain as a beta product for the first year or two.

On the other hand, there stable ZFS support in OpenSolaris, (non-open) Solaris, and in a few months in FreeBSD, and since it's available fully free and open source, there are no necessary licensing costs. Are there any advantages of Starfish over ZFS?

Hmm... ZFS and Starfish aren't really meant to address the same storage problem. Take a bit of time and read through what ZFS does and what Starfish does. ZFS is a block-level file system. Starfish is a file-level distributed clustered storage system. Those are two very different things - at the end of the day they store files, but in very different ways and for very different purposes. Starfish can use ZFS as it's block-level file system... it can also use Reiser and EXT3.

Here are a couple of reasons to use Starfish (even though we think that ZFS is a fantastic solution for block-level file system problems):

• Starfish is a decentralized, multi-node fault tolerant file storage solution that provides N-way redundancy.
• Starfish is useful when you have a large number of nodes that need to access data in parallel. The ability to perform asynchronous parallel throughput is one of Starfish's biggest advantages.
• Starfish runs in userspace - which means that it is capable of using much smarter algorithms and databases to manage metadata and file placement. For example, a SQL database is used for file system metadata, which means that a variety of optimizations (such as creating metadata indices) to the file system can be performed at runtime.

However, this really isn't a "what is better, Starfish or ZFS?" discussion. You can have the best of both worlds: Starfish as the file-level network storage cloud using ZFS as the block-level file system.

Ever heard of Starfish? It's a new distributed clustered file system:

Starfish Distributed Filesystem

From the website:

Starfish is a highly-available, fully decentralized, clustered storage file system. It provides a distributed POSIX-compliant storage device that can be mounted like any other drive under Linux or Mac OS X. The resulting fault-tolerant storage network can store files and directories, like a normal file system - but unlike a normal file system, it can handle multiple catastrophic disk and machine failures.

I read on the web site that mirroring should only be available in August 2007, is that true?

Also, is it possible to have StarPeers of different size? I have a server with 80 GB in RAID1 and one with 500 GB in RAID1. If I set both of them as StarPeers will both have access to 580 GB of storage?

Thanks,
GFK's

Ever heard of Starfish? It's a new distributed clustered file system:

Starfish Distributed Filesystem

From the website:

Starfish is a highly-available, fully decentralized, clustered storage file system. It provides a distributed POSIX-compliant storage device that can be mounted like any other drive under Linux or Mac OS X. The resulting fault-tolerant storage network can store files and directories, like a normal file system - but unlike a normal file system, it can handle multiple catastrophic disk and machine failures.

I read on the web site that mirroring should only be available in August 2007, is that true?

Also, is it possible to have StarPeers of different size? I have a server with 80 GB in RAID1 and one with 500 GB in RAID1. If I set both of them as StarPeers will both have access to 580 GB of storage?

Thanks,
GFK's

Why is Starfish better than pNFS?

How much does the software cost?

Why is Starfish better than pNFS?

How much does the software cost?

Why is Starfish better than pNFS?

How much does the software cost?

Ever heard of Starfish? It's a new distributed clustered file system:

Starfish Distributed Filesystem

From the website:

Starfish is a highly-available, fully decentralized, clustered storage file system. It provides a distributed POSIX-compliant storage device that can be mounted like any other drive under Linux or Mac OS X. The resulting fault-tolerant storage network can store files and directories, like a normal file system - but unlike a normal file system, it can handle multiple catastrophic disk and machine failures.

And you can build clusters at relatively low cost:

For a 2-way redundant, RAID-1 protected, 1.0 Terabyte cluster: $2,000 (Jan 2007 prices). Per server, that breaks down into around $400 for a AMD 2.6Ghz CPU, 1GB of memory, and a motherboard with integrated 100 megabit LAN connection, SATA support, 350 watt power supply and a commodity server enclosure. Four SATA 500GB hard drives will run you around $600. The cluster would ensure proper file system operation even in the catastrophic failure of a single machine. Hard drive failure rates could even approach 50% without affecting the Starfish file system.

(warning: I work for the company that created Starfish)

Starfish was designed to automatically back data up - HSM was designed in from the beginning. You never have to backup a Starfish storage network. Take another look at Starfish - it does exactly what you're asking for:

Starfish Introduction (mentions file mirroring)

Our first Lustre cluster was 3 servers - it worked just fine. Starfish effortlessly scales down to 2 servers. Here is an example of it doing so:

Just because something scales to thousands of active nodes and disks, doesn't mean it can't scale down gracefully. The Internet is a good example of this concept.

Full Disclosure: I'm one of the author's of the Starfish Filesystem.

Simply not true anymore, lukas84. High-availability solutions don't have to cost "big money". Starfish is the perfect example of such a system. In fact, it is THE reason we wrote Starfish: To provide an in-expensive, fault-tolerant, highly available clustered storage platform that works from the smallest website to the largest storage network. We've based the technology on the assumption that having expensive hardware/software is the wrong way to go about solving the problem.

Full HA environments do not need to be incredibly complex. If your HA solution is incredibly complex, you've done something wrong. Take a look at how easy it is to set up a Starfish file system:

Starfish QuickStart Tutorial

That solution doesn't cost "big money", nor is it "incredibly complex".