Hmm. It would be a challenge to implement a parallel filesystem across your storage appliances (if that is what the netapp/bluearc products are).
A traditional SAN breaks up the path components, so that you could access the same disks with multiple disk controllers, with those controllers pointing to different host bus adapters inside the attached servers. I'm guessing your appliances put all (or at least the majority) of the path--disks, controller(s), network stack, protocols--in a sealed box, so access to those disks is limited to the appliance. Not very scalable from a parallel filesystem standpoint.
There is a nice overview of SANs at Wikipedia. For product specifics, you'll want to check out SAN vendors: EMC, HP, IBM...
That's a difficult question to answer without knowing something of your setup. How are the spindles organized--SAN, individual file servers NFS cross-mounting, or what? Which OSes are you running? Also, how much money are you able to spend to resolve this problem?
If you could rebuild everything from the ground up (and had tons of money to throw at it), you'd most likely want to build a system based on a very expensive vendor solution.
Assuming that you can't do that, your best bet would be to go with some sort of parallel filesystem, the likes of Lustre, GFS, Ibrix, GPFS or CxFS. The architectures of these vary, but the basic principle they share is performance scalability based on increasing the number of data paths to the disk. So if you have, say, 100 nodes on a high-speed network, you take 10 of them and attach them to your SAN. The parallel filesystem spans the entire SAN and so requests from the nodes can reach any bit on the SAN from any of the ten paths. If you need more performance, you add more paths: controllers, HBAs and storage nodes. I know GPFS scales linearly in performance based on the number of paths to the data, and I believe the others scale well also.
I haven't hit 50 TB on disk (I have on tape, but your post suggests that tape wouldn't give you the performance you need), but I have set up several 4-8 TB GPFS filesystems that could easily grow to 50 TB if I had the spindles.
Good luck finding a solution; symlink-based solutions on a convnentional *NIX filesystem are a nightmare; I sympathize.
You might really enjoy the work of Hayao Miyazaki if you're not already familiar with it. Mononoke Hime, Tonari no Totoro and Castle of Cagliostro are particularly good (also Naushika, though haven't seen the whole thing). It's a bit different than the titles you listed (or at least those with which I'm familiar) but the quality is very good.
Slashdot Mirror
A traditional SAN breaks up the path components, so that you could access the same disks with multiple disk controllers, with those controllers pointing to different host bus adapters inside the attached servers. I'm guessing your appliances put all (or at least the majority) of the path--disks, controller(s), network stack, protocols--in a sealed box, so access to those disks is limited to the appliance. Not very scalable from a parallel filesystem standpoint.
There is a nice overview of SANs at Wikipedia. For product specifics, you'll want to check out SAN vendors: EMC, HP, IBM...
If you could rebuild everything from the ground up (and had tons of money to throw at it), you'd most likely want to build a system based on a very expensive vendor solution.
Assuming that you can't do that, your best bet would be to go with some sort of parallel filesystem, the likes of Lustre, GFS, Ibrix, GPFS or CxFS. The architectures of these vary, but the basic principle they share is performance scalability based on increasing the number of data paths to the disk. So if you have, say, 100 nodes on a high-speed network, you take 10 of them and attach them to your SAN. The parallel filesystem spans the entire SAN and so requests from the nodes can reach any bit on the SAN from any of the ten paths. If you need more performance, you add more paths: controllers, HBAs and storage nodes. I know GPFS scales linearly in performance based on the number of paths to the data, and I believe the others scale well also.
I haven't hit 50 TB on disk (I have on tape, but your post suggests that tape wouldn't give you the performance you need), but I have set up several 4-8 TB GPFS filesystems that could easily grow to 50 TB if I had the spindles.
Good luck finding a solution; symlink-based solutions on a convnentional *NIX filesystem are a nightmare; I sympathize.
Micron's lobbying wouldn't have anything to do with Micron posting a loss last quarter, would it? Nah. Of course not.
You might really enjoy the work of Hayao Miyazaki if you're not already familiar with it. Mononoke Hime, Tonari no Totoro and Castle of Cagliostro are particularly good (also Naushika, though haven't seen the whole thing). It's a bit different than the titles you listed (or at least those with which I'm familiar) but the quality is very good.