Slashdot Mirror

← Back to Domains

Domain: ibrix.com

Stories and comments across the archive that link to ibrix.com.

Comments · 7

  1. IBRIX by Midnight+Warrior · · Score: 2 · on Building a Scalable Mail System?

    If you have a lot of data, then you can choose a scalable system like IBRIX and then use stateful load balancers between each of the POP3/IMAP servers. When you get to multiple nodes on the same filesystem, you have two problems: synchronization between nodes and locking.

    Note that the Oracle Clustered Filesystem v2 has now been merged with the mainline kernel.

  2. IBRIX by Wells2k · · Score: 3, Informative · on Building a Massive Single Volume Storage Solution?

    You may want to take a look at IBRIX systems. They do a pretty robust parallel file system that has redundancy and failover.

  3. IBRIX by Anonymous Coward · · Score: 1, Insightful · on Building a Massive Single Volume Storage Solution?

    Check out the IBRIX Clustered Filesystem. http://www.ibrix.com/

  4. The IBRIX file system is a strong runner for this. by schnook · · Score: 2, Informative · on Distributed Storage Systems for Linux?

    Check out http://www.ibrix.com/ This is a perfect solution for your requirements. Pixar uses this.

  5. There are a lot of cluster file systems by anzha · · Score: 5, Informative · on ZFS, the Last Word in File Systems?

    Right now there are a lot of file systems that do somehing not all that different than what Sun is proposing. The project I am on is evaluating them as we speak for a center wide filesystem. I've had the fun (no sarcasm, honestly) of setting up a number of different onces and helping to run benchmarks and tests against each. All of them have strengths. Every single one of them has some nasty weaknesses.

    If you are looking for an open source based cluster file system, Lustre is what you want. It's supported by LLNL, PNNL, and the main writers at ClusterFS Inc. It's a network based cluster FS. We've been using it over GigE. However, we've found that there needs to be a ratio of 3:1 for data server:clients for a ratio. Wehave only used one metadata server. Failover isn't the greatest. Quotas don't exist. it also makes kernel mods (some good and bad) to do a mild fork of the linux kernel (they put them into the newer kernels every so often). It only runs on Linux. Getting it to run on anything else looks...scary.

    GPFS runs on AIX and Linux. Even sharing the same storage. It runs and is pretty stable. it has the option to run in a SAN mode or network based FS. In the latter form, it even does local discovery of disks via labels so that if a client can see the disks locally it will read and write to them via FC rather than to the server. It, however, is a balkanized mess. It requires a lot more work to bring up and run: there is an awful lot of software to configure to get it to run (re: RSCT. If you haven't had the joys of HATS and HAGS, count yourself very, very lucky).

    ADIC's StorNext software is another option. This one is good if you are interested in ease of installation, maintanence, and very, very fast speeds (damn near line speed on Fibre channel). I have set this one up for sharing disks in less than two hours from first install to getting numerous assorted nodes of different OS's to play together (Solaris, AIX, Linux). It freakin on virtually everything from Crays to Linux to Windows. It's issues seem to be scaling (right now doesn't go past 256 clients) and it has some nontrivial locking issues (righting to the same block from multiple clients, and parallel I/O to the same file from multiple clients if you change the file size).

    There are some others that are not as mature. Among them are Ibrix, Panasas, GFS, and IBM's SANFS. All of them are interesting or promising. Only SANF looks like it runs on more than Linux though at this point. Our requirements for the project I am on are to share the same FS and storage instance among disparate client OSes simultaneously. This might not be the same for others though and these might be worth a look. Lustre dodges this because its open source and they're interested in porting.

  6. Check out... by La+Camiseta · · Score: 1 · on Bulk Data Storage For The Common Man?

    Ibrix has a really good clustered FS setup. Plus you can just plug in more systems when you want them.

    Another thing that you may want to check out something liket GPFS if you want to build your own filesystem cluster.

  7. Re:I had a related question by dolphin-brother · · Score: 2, Informative · on Renderfarm Setup Tips?
    That's a difficult question to answer without knowing something of your setup. How are the spindles organized--SAN, individual file servers NFS cross-mounting, or what? Which OSes are you running? Also, how much money are you able to spend to resolve this problem?

    If you could rebuild everything from the ground up (and had tons of money to throw at it), you'd most likely want to build a system based on a very expensive vendor solution.

    Assuming that you can't do that, your best bet would be to go with some sort of parallel filesystem, the likes of Lustre, GFS, Ibrix, GPFS or CxFS. The architectures of these vary, but the basic principle they share is performance scalability based on increasing the number of data paths to the disk. So if you have, say, 100 nodes on a high-speed network, you take 10 of them and attach them to your SAN. The parallel filesystem spans the entire SAN and so requests from the nodes can reach any bit on the SAN from any of the ten paths. If you need more performance, you add more paths: controllers, HBAs and storage nodes. I know GPFS scales linearly in performance based on the number of paths to the data, and I believe the others scale well also.

    I haven't hit 50 TB on disk (I have on tape, but your post suggests that tape wouldn't give you the performance you need), but I have set up several 4-8 TB GPFS filesystems that could easily grow to 50 TB if I had the spindles.

    Good luck finding a solution; symlink-based solutions on a convnentional *NIX filesystem are a nightmare; I sympathize.