Slashdot Mirror

If it's cheap, then Netapp might not qualify... :)

What about technologies - NAS? Host-attached? Gateway/NAS? Grids?

Other companies/products to consider:

EMC (The Celerra is a nice product)

Onstor Bobcat

HP

IBM

Hitachi

Panasas

Exanet

Yotta Yotta

StoreAge

If you want basic raid devices look at Infortrend/Transtec. Their S-ATA offerings now support RAID-6 and are dirt cheap.

It would be nice to see comparisons to RedHat/Sistina's GFS, Lustre (backed by HP), and others listed here.

Also how does this compare to clustered storage that is not run on the hosts themselves like NetApp new Spinnaker based clustering. You also have folks like Isilon, Panasas, and Terrascale.

Anybody have an good data on this?
-Ack

From a Panasas press release:

The Panasas storage cluster can scale from Gigabytes to Petabytes in capacity and still be managed as a single system. Key attributes include: a single virtualized global namespace, dynamic load balancing and Quality of Service attributes. These combine to simplify ongoing operation of the system while maintaining peak effectiveness.

From a Panasas press release:

The Panasas storage cluster can scale from Gigabytes to Petabytes in capacity and still be managed as a single system. Key attributes include: a single virtualized global namespace, dynamic load balancing and Quality of Service attributes. These combine to simplify ongoing operation of the system while maintaining peak effectiveness.

Panasas http://www.panasas.com/products_overview.html has some products which probably fit your requirement of high speed distributed storage.

Panasas is a GPL Violator, as per conversations with them during a demo. They have a proprietary module that is built against only RHEL kernels. They confirmed that this module uses kernel headers and other parts of the kernel, but they refuse to release the source to any customer.

If you are ever in a situation where you need to upgrade kernels in a hurry, switch branch (2.4 to 2.6) or the company goes outta business then you will have just a bunch of nfs file servers with locally attached disks. Makes debugging kernels a pain.

Panasas http://www.panasas.com/products_overview.html has some products which probably fit your requirement of high speed distributed storage.

Right now there are a lot of file systems that do somehing not all that different than what Sun is proposing. The project I am on is evaluating them as we speak for a center wide filesystem. I've had the fun (no sarcasm, honestly) of setting up a number of different onces and helping to run benchmarks and tests against each. All of them have strengths. Every single one of them has some nasty weaknesses.

If you are looking for an open source based cluster file system, Lustre is what you want. It's supported by LLNL, PNNL, and the main writers at ClusterFS Inc. It's a network based cluster FS. We've been using it over GigE. However, we've found that there needs to be a ratio of 3:1 for data server:clients for a ratio. Wehave only used one metadata server. Failover isn't the greatest. Quotas don't exist. it also makes kernel mods (some good and bad) to do a mild fork of the linux kernel (they put them into the newer kernels every so often). It only runs on Linux. Getting it to run on anything else looks...scary.

GPFS runs on AIX and Linux. Even sharing the same storage. It runs and is pretty stable. it has the option to run in a SAN mode or network based FS. In the latter form, it even does local discovery of disks via labels so that if a client can see the disks locally it will read and write to them via FC rather than to the server. It, however, is a balkanized mess. It requires a lot more work to bring up and run: there is an awful lot of software to configure to get it to run (re: RSCT. If you haven't had the joys of HATS and HAGS, count yourself very, very lucky).

ADIC's StorNext software is another option. This one is good if you are interested in ease of installation, maintanence, and very, very fast speeds (damn near line speed on Fibre channel). I have set this one up for sharing disks in less than two hours from first install to getting numerous assorted nodes of different OS's to play together (Solaris, AIX, Linux). It freakin on virtually everything from Crays to Linux to Windows. It's issues seem to be scaling (right now doesn't go past 256 clients) and it has some nontrivial locking issues (righting to the same block from multiple clients, and parallel I/O to the same file from multiple clients if you change the file size).

There are some others that are not as mature. Among them are Ibrix, Panasas, GFS, and IBM's SANFS. All of them are interesting or promising. Only SANF looks like it runs on more than Linux though at this point. Our requirements for the project I am on are to share the same FS and storage instance among disparate client OSes simultaneously. This might not be the same for others though and these might be worth a look. Lustre dodges this because its open source and they're interested in porting.

Having lived here off and on since '83, I can honestly say I like "da burgh". There're good normal and more adventerous restaurants, mainstream and alternative cinema, good Rails To Trails, really excellent public theatre (Pittsburgh Public Theatre) and the Pittsburgh Symphony (site done by a friend of mine, FWIW), low traffic compared to other metro areas (Philly, NYC, Baltimore, and LA from direct experience), nice people, good high speed access in a growing number of neighborhoods, etc., etc. There's also an advocacy group of non-codgers trying to work on relevant issues for geeks and other young professionals.

And to note a few success stories: Fore Systems (now Marconi), Lycos, Free Markets Online (woulda liked to have a piece of that IPO), the Seagate magnetics research center, a mysterious whiz-bang startup, etc.

On the political scene, the local county (Alleghany) just switched from a three-headed-dog-of-county-commisioners system to a single county executive. UPitt, CMU, Pittsbugh, and Alleghany County are starting to cooperate better in attracting business and supporting spinoffs and startups. CMU's actually figured out that the wild-eyed innovator rarely makes the best startup CEO and is trying to support startups/spinoffs with more serios business support. Pitt's learned that lesson as well.

Now that's not to say that we don't have 'issues', like tax structure, crappy roads, a high codger factor, but things are definitely on the mend around here. We haven't gotten to the point of choking on our own success: housing's, food nad clothing cheap, traffic's low (relatively), no more choking pollution. We've got good and growing support for ADSL (which brings you this missive), cable modems, and CPDP support.

All said, I'm glad I moved here and it looks like things are going to be moving in the right direction very nicely over the next 10 years or so. Meanwhile, we don't need to cope with the crap you need to cope with in longer-standing "high tech" areas.

Oh, and the standard disclaimer: "just my 0.02 worth".

- Barrie