Slashdot Mirror

← Back to Stories (view on slashdot.org)

PVFS2 - a High-Performance Parallel File System

Posted by timothy on from the good-nodes-are-still-available dept.

neillm78 writes "As part of the development team, we're announcing PVFS2 version 1.0 here in Pittsburgh at the SC2004 conference! PVFS2 is a GPL/LGPL based parallel file system for cluster-based applications. It logically groups any number of storage servers into a coherent file system for use by client nodes, specifically tailored to handle efficient access to large shared files. PVFS2 supports access via an MPI-IO interface for high-performance parallel applications, but you can still mount it like a regular GNU/Linux file system for traditional serial applications and managment. The PVFS2 project is conducted jointly between The Parallel Architecture Research Laboratory at Clemson University and The Mathematics and Computer Science Division at Argonne National Laboratory. Please feel free to give it a try!"

5 of 26 comments (clear)

  1. Been following it for a while... by brsmith4 · · Score: 2, Informative

    PVFS (in its first incarnation) despite some instability (more so due to the fact that our first cluster was COTS cheap-o hardware), really helped drive down the load on our clusters by removing the need to perform NFS writes to a single head node for scratch space. The set up is extrememly simple and the code base was really small.

    I plan on evaluating PVFS2 for our new clusters along with Lustre and GFS although I have heard nothing about the latter two operating over the MPI-ROMIO subsystem (which would definitely offer a performance increase).

    1. Re:Been following it for a while... by brsmith4 · · Score: 4, Informative

      It's a parallel file system, not a drop in replacement for local FS's like XFS or ext3. It runs across multiple hosts, striping the data on each host. Also, haveing multiple I/O hosts in the array helps to distribute the read/write across multiple nodes, thus reducing the overhead for those operations.

      This is like "Distributed NFS" although that description does it a huge injustice, it should help to get the point across.

  2. Re:I know this is for large clusters..... by brsmith4 · · Score: 3, Informative

    Simple answer: No. This is commonly used for allocated scratch space in cluster environments e.g. beowulf. We use it to reduce the reads and writes that usually bring an NFS system to its knees. It would not help Bittorrent.

  3. Re:I hope the meta-data performance improved... by alecthomas · · Score: 2, Informative
    Has the meta-data server been speed up at all, or made distributed with some kind of coherency-syncro backend?

    From the PVFS2 Guide:

    The new design has a number of important features, including:

    * modular networking and storage subsystems,
    * powerful request format for structured non-contiguous accesses,
    * flexible and extensible data distribution modules,
    * distributed metadata,
    * stateless servers and clients (no locking subsystem),
    * explicit concurrency support,
    * tunable semantics,
    * flexible mapping from file references to servers,
    * tight MPI-IO integration, and
    * support for data and metadata redundancy.
  4. Re:I hope the meta-data performance improved... by rizzy · · Score: 3, Informative

    > * flexible and extensible data distribution modules,
    > * distributed metadata,
    > * stateless servers and clients (no locking subsystem),

    Just to clarify... while we have distributed metadata, we don't have *replicated* metadata. At least, not yet.

    If you have multiple metadata servers they will do load balancing. If you are working with lots and lots of small files, having a couple metadata servers might alieviate a possible bottleneck.