Slashdot Mirror
Consoldated Network Storage?
bigstupid asks: "Is there anyway to better utilize storage space on your network? I have a home network with about nine permanently attached PCs. A few of these are older PII300 with smaller hard drives (3-10GB). What I want to do is consolidate as much of the network storage as possible. That is: Instead of 2.4GB here, 4.6GB there, 5GB hither, 5BG tither, and 6 GB yon, I would like this storage space to appear to any computer I designate a 'client' to see and use this storage space as one large (in the case above 23GB) volume. I know I can do this within a machine with logical volumes or RAID, but is there a piece of software - client or server side - that will do this on Linux or Windows?"
You could do this on windows 2000 servers... With a DFS (Distributed File System.) You can see some info about it here:
o wi tworks/fileandprint/dfsnew.asp
http://www.microsoft.com/windows2000/techinfo/h
--- "Remember, there's a difference between bowing down and bending over." -Frank Zappa
Look at OpenAFS.
--sdem
... such as SMB, Coda, NFS, AppleTalk and AFS work very nicely. We use a centralized data repository which is backed up daily. Users get access to their data from their stations, finding files takes less time, and backups are a breeze.
Modern IDE drives will handle the traffic that's generated by a decent number of client PCs. This lets you can place a couple of 200GB drives in a machine which will act as a server, and not have to worry about scouring 9 or 10 PCs to find your work! (Note: Some older BIOSes are limited to 45GB, so you might want to check for BIOS upgrades if you run into this issue.)
I haven't tried this, but it would be fun to play with.
Steve
Just be careful when you do this. If one of your 2.4gig HD's craps out, you've lost the whole array (that means all your data/porn)!!
Each node has a spare 20GB partition that is currently doing *nothing*. I would simply love to find a filesystem solution that can handle stripping or mirroring for a nice 32*20/x GB of filespace
That easy. Create a partition on each box and export it via NFS. Then plunk down a NetBSD box on the network and RAID the partitions with RAIDframe. Export *that* partition via NFS as well. Export it via Samba and even the little Windows boxes can play.
FreeBSD has RAIDframe as well, but the NetBSD version is marginally more robust and has worked over NFS are far back as '98.
Moneyed corporations, non-working 'poor' and criminal prisoners are turning productive citizens into tax-slaves.
The first thing to do is expose all of your drives in the "same format". On Windows machines, share the extra disks as normal. In linux, use NBD (network block device) or iSCSI to expose the disks as raw partitions accessable over ethernet to the other linux boxen.
Now, on a special linux machine (the sucka, or Serialization and Uniformity Cache Kludge Administrator) mount all these exposed drives via "mount.smbfs" for the windows boxen. Use loopback filesystems the size of each Windows disk to create virtual devices accessable on said remote winboxen. Use md or LVM to stitch the exported linux box disks and loopbackfs-over-smbfs together into a software RAID disk.
Finally, format this UBER meta disk via your favorite filesystem. Expose it to windows via samba, and linux via NFS.
Of course, this whole setup serializations all the operations through one machine. Everything takes one round trip over the network. And unless you use RAID striping, if a machine goes down, so does the whole disk!
Other method, more complex:
Check out the Parallel Virtual Filesystem. What you do is for each spare linux box that has a disk, you run both the IO server and a client. One machine also has to pick up the slack of the metadata manager (no big deal...) Of course, for each linux machine, you have to pick and mount certain the Windows disks (via mount.smb) and run IO server procs for each mounted volume. Finally, you have to run samba on at least one of the linux machines running PVFS to expose those files back to the Windows machines. If you can tweak the samba source to use larger than normal block transfers, do so, because PVFS suffers when you transfer data between nodes that are too small.
Or you can use OpenAFS. Someone else mentioned it here. But it's not as much fun, and it is a big deal to set up if you haven't done it before.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON