Domain: globalfilesystem.org
Stories and comments across the archive that link to globalfilesystem.org.
Comments · 10
-
GFS
You might wanna take a look at GFS.
I'm not quite sure if it's designed for a WAN environment, but it is a Storage Area Network which can support multiple clients. It's cool even if it won't work for what you want.
While I wait for CODA or some other project to become stable, I've been using rsync to replicate video files between multiple geographically different video servers. Rsync is great, because if a large file is still being uploaded, it will transfer as much as it can to the other servers, and when it runs 1 minute later, it will start where it left off. I whacked up a little script that won't start rsync if a copy is already running, otherwise you'll end up with a whole bunch of copies of rsync running. -
GFS *is* what you really want
Check out The Global File System, it's the heat! GFS is *the* open source filesystem that has the backing of the big storage manufacturers, like EMC Corporation. It blows away AFS, try it for yourself and you will see.
-
Re:No it won't.
This is probably going to be necessary at some stage as there now seem to be at least 5 JFS's for Linux: Reiser, IBM JFS, SGI XFS, Ext3 and TUX2.
There's also GFS (Global File Sytem) which has journaling, but is a successor to NFS.
-- -
Re:But why?Home computers (Linux systems) CAN share disks like this if you want to invest in fibrechannel (which may not be as expensive as you think). Check out:
http://www.globalfilesystem.org
Very cool technology. I have been following this for quite a while and it shows tremendous promise for solving all kinds of disk scalability problems.
-
64-bit Hardware
If at all possible, I'd recommend getting some 64-bit hardware. Probably an Alpha-based system. Next, get a decent filesystem like ReiserFS or Global Filesystem.
If you are running on x86 hardware, there's not telling if the accesses will be capable of reading large files (>2GB).
--
Ski-U-Mah! -
Re:Not "real" HA clustering. (yet?)
This also pretty accurately decribes what the Global Filesystem is addressing.
-
Re:CentraVision's license?
A kernel module falls under the GPL. Yes, I know, binary-only modules are allowed by convention, but it still sucks.
You're going to be out of luck should you find a later kernel gives better performance but breaks binary compatibility. Think about proper async I/O, which is coming and can give a handy boost. If you have the budget for Fibre Channel fabrics at some point, at least look at the Global File System.
BTW, if you're going to compare this cluster with a Cray T3E or IBM SP, actually compare them, don't just say they're comparible. The T3E's network is one-of-a-kind, with large bandwidth and almost no latency. (And I certainly wouldn't comare MPI implementations. Myricom's sucks and is causing no end of problems for some other projects.) You can't compare on that aspect with any commercially available interconnect. And there are much larger SPs around and coming, like San Diego's and the second phase of NERSC's.
Don't take this the wrong way. What you've put together is impressive, especially surviving the procurement process, but there's still a lot of work to be done to catch up with the big boys. You know that, but a good many people reading the iterview may walk away with a good-we're-at-the-leading-edge-now impression. We aren't. We're at the cost-effective edge, but we can make the leading edge...
-
XFS won't be handled like a GPL project by SGIAt the SGI road-tour, one of the things brought up is that while maintaining a commerical XFS and a "GPL" XFS, SGI would prefer to maintain just one source tree. Hence, while being forkable into a true GPL project, SGI won't be the one's maintaining such a GPL-like project. Instead, what they described sounded more similar to Netscape's handling of MPL'd code. For those that don't know, Netscape released Mozilla under the NPL (Netscape Public License) which gave Netscape certain privilages that others didn't recieve. Unlike GPL which should put everyone under equal footing. Since Netscape prefers that all Netscape code be covered by the NPL, it was stated that MPL code submittions would be rewritten by Netscape so it could be added as concepts implimented via NPL code. SGI's handling of GPL will be similar. They will accept *input* on concepts, even the code. But the only code that will be integrated into the "commerical-able/close-able" tree will be what they write themselves. This might work in the short term for minor performance and feature enhancements. But major over-halls and the concept of security patches tend to be hard to re-impliment. Major over-halls tend to be too extensive to replicate without looking into the other's code. Security patches tend to be very specific changes that if reimplimented a different way may produce another security consern that hasn't gotten the level of peer review that the open-source patch has. So, as the Linux community over-halls the code with major changes or impliments security related changes, I believe the relationship between SGI's close-able effort and the Linux community's true GPL changes will end up having to fork. Hence, the support from the original author, SGI, will most likely degrade over time since the stated goals from SGI do not run in parralel with the stated purpose of GPL (everyone is on equal footing--I show you all of my code, you show me all of your's).
There is a couple ways that SGI could attempt to keep this event from occuring (all of which are undesirable but possible). One, and probably the most desirable, is to ask the submittion author for permittion to permit them to also add the code to a closed tree as-is. Second possiblity, is to integrate GPL'd submittions into the closed tree despite such actions being a violation of the GPL. While SGI over-all is a fairly honest company, individual employees have a conflict of interest in that they have their work done for them in GPL form and a method of hiding that they are stealing instead of reimplimenting it by hiding the verbatem copies into a closed tree. It would be all too easy for an individual employee to end up doing this and very difficult for the Linux community to audit for it. The third possiblity is to gain "open source hot-word compliance" while not actual encouraging third party changes. This is the method that nVidea & Intel has provided drivers under and the way Caldera has made their user-space file system kernel extentions available. Put simply, don't document or document the code so poorly that a programmer would prefer to do a ground up rewrite than to try to make sense of the existing code. (There is also the possible change that SGI could give up on maintaining a closed-source tree in parralel which is the whole reason for all of these issues but such a change on SGI's part is not realistic.)
Btw, to be fair to SGI, I feel that IBM's closed and GPL JFS offering to Linux will most likely suffer the same issues.
I believe that despite the speed at which XFS is being ported that ext3 will remain a preferable short term solution and that reiserfs which doesn't suffer the closed & open issues will be a preferable long term solution.
In terms of the closed source CXFS offering that will be coming from SGI, I would encourage people to look at GFS (Global File System) as an open source alternative which may eventually surpass the SGI offering in some areas.
-
Shamless plug.
Get rid of nfs. There are much better ways of distributing filesystems out there. Like GFS. http://www.globalfilesystem.org
-
Re:Linux, Fibre Channel, and SANs - popular linksThere's a lot of good points here; Linux has Fibre Channel Arbitrated Loop (FCAL) support via the ISP2100 Qlogic chipset and the 2.2 kernel, but that doesn't constitute a SAN. A Veritas-style transactional filesystem and volume manager are definitely required fundamental pieces for conventional SANs; they allow you to utilize multiple disk i/o queues and spindles to maximize your i/o throughput. Clustering filesystems like GFS are also an important step necessary to work towards developing a single-system image, allowing multiple machines to work together with a uniform set of available and shared resources. As such SANs are the enabling technology for true clustering. Clustering with shared resources, global process ID's, and process migration between systems are the ultimate in flexibility as processes don't require modification to take advantage of clustering capabilities; this is much more powerful than simple shared-nothing Beowulf clusters.
We are currently running several Solaris systems with a 3rd-party FCAL SAN-in-a-box from XIOtech. We have a second one serving our Netware servers. If you're serious about getting the benefits promised by SANs today, I'd recommend checking them out. They allow you to create virtual disks in various RAID configurations, move the virtual drives between different virtual clusters, and copy and swap virtual drives on the fly.
Lans Carstensen - lans@carstensen.cx
Links
XIOTech press release on LinuxToday
The Global Filesystem Group
The High Availability Linux Project
RSi's RSF-1 high availability software, available for trial download