Slashdot Mirror
Consoldated Network Storage?
bigstupid asks: "Is there anyway to better utilize storage space on your network? I have a home network with about nine permanently attached PCs. A few of these are older PII300 with smaller hard drives (3-10GB). What I want to do is consolidate as much of the network storage as possible. That is: Instead of 2.4GB here, 4.6GB there, 5GB hither, 5BG tither, and 6 GB yon, I would like this storage space to appear to any computer I designate a 'client' to see and use this storage space as one large (in the case above 23GB) volume. I know I can do this within a machine with logical volumes or RAID, but is there a piece of software - client or server side - that will do this on Linux or Windows?"
You could do this on windows 2000 servers... With a DFS (Distributed File System.) You can see some info about it here:
o wi tworks/fileandprint/dfsnew.asp
http://www.microsoft.com/windows2000/techinfo/h
--- "Remember, there's a difference between bowing down and bending over." -Frank Zappa
Use NFS and reexport it using samba.
Look at OpenAFS.
--sdem
Just go buy yourself a 23 gigbayte harddrive.
... such as SMB, Coda, NFS, AppleTalk and AFS work very nicely. We use a centralized data repository which is backed up daily. Users get access to their data from their stations, finding files takes less time, and backups are a breeze.
Modern IDE drives will handle the traffic that's generated by a decent number of client PCs. This lets you can place a couple of 200GB drives in a machine which will act as a server, and not have to worry about scouring 9 or 10 PCs to find your work! (Note: Some older BIOSes are limited to 45GB, so you might want to check for BIOS upgrades if you run into this issue.)
Simple. Setup a Linux machine as a server running Samba (http://www.samba.org). Setup a user account on the server and then use the Windows "Map Network Drive" functionality to map the home directory for the user account. My University did this for the CompSci students. Worked brilliantly.
My advice: Search around for a good online samba reference/tutorial.
When one of your drives crash, which files do you want to loose?
Just the one drive, or all files on all drives?
You might try running RAID over something like iSCSI (if it can be done), and re-exporting that filesystem from a central server.
"I assumed blithely that there were no elves out there in the darkness"
is this what you could be looking for... you dont specify what type of OS you need.
http://www.sistina.com/products_gfs.htm
I think that Mosix has something like that. I want to say it's called mosixfs, but it could very likely have a different name.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
I haven't tried this, but it would be fun to play with.
Steve
You see, instead of 2.4 GB hyah, 4.6 GB nyah, 5 GB over hyah, and 6 GB over nyah...
I could be wrong, but I seem to recall that if you're careful, you can share a chain of SCSI devices between multiple machines...
Maybe making a SCSI RAID, or an IDE raid connected to one of those IDE-to-SCSI adaptor thingees (Promise sells such things; they even sell ready-to-go SCSI-connectable RAID cases.. just fill with IDE (!!!) hard drives!) and plugging it into all yer boxes would work? ^_^
Honey, I shrunk the Cygwin
Ian Murdock, of debIan and Progeny fame, had a project called LinuxNOW which causes workstations on the network to share hard drive space as if a single drive, and also share processor cycles as in clustering.
w -2001-03/lw -03-murdock.html
Here is an older article on LinuxNOW.
http://www.linuxworld.com/linuxworld/l
Here's my 2 cents:
I was in pretty much the same boat as you, but I picked up a couple new 60Gb-ers. I thought about combining them using MS's DFS, but then the drives started failing on me one-by-one.
I recommend you do the same. I mean ask yourself: Do I really want to trust my porn collection to a 6-year-old 2Gb drive? NO!!
Hardware's Cheap. Use Wisely.
HURD - Hurd's Under Research & Development
It's theoretically possible to design a distributed filesystem that can handle the failure of one or more nodes.
Kind of an interesting project, actually. I don't recall this being done before.
May we never see th
Just be careful when you do this. If one of your 2.4gig HD's craps out, you've lost the whole array (that means all your data/porn)!!
Before you mod me down, this is serious :-)
I am the admin at our school for a newly installed 32 node linux beowulf. Each node has a spare 20GB partition that is currently doing *nothing*. I would simply love to find a filesystem solution that can handle stripping or mirroring for a nice 32*20/x GB of filespace (where x is the amount of redunancy to be tuned for optimal reliability).
If anyone has the solution, even if it requires work, I am all ears.
-- The morphemes of your disquisition are ascertainable, but they have eschewed an ambit of transpicuous exposition.
OpenAFS. Click that link.
--sdem
What's wrong with setting up 1 of those computers as a server with your RAID tech of choice, then using the hard drives of the others as a small system drive, and just install all your software (or as much as possible) and user data to run from the network? It's not like hard drives are expensive, and the new drives will be faster then the old drives, and you end up with the same result as using the other drives, but without the hassle.
They consolidated the word consolidated.
"And like that
You know, they make 200 gig drives now... I hope I'm not sounding elitist when I offer the suggestion that you purchase something larger, shove it in one of them, and share it via NFS, SAMBA, and any other protocols your computers speak.
- A.P.
"Remember when the U.S. had a drug problem, and then we declared a War On Drugs, and now you can't buy drugs anymore?"
The first thing to do is expose all of your drives in the "same format". On Windows machines, share the extra disks as normal. In linux, use NBD (network block device) or iSCSI to expose the disks as raw partitions accessable over ethernet to the other linux boxen.
Now, on a special linux machine (the sucka, or Serialization and Uniformity Cache Kludge Administrator) mount all these exposed drives via "mount.smbfs" for the windows boxen. Use loopback filesystems the size of each Windows disk to create virtual devices accessable on said remote winboxen. Use md or LVM to stitch the exported linux box disks and loopbackfs-over-smbfs together into a software RAID disk.
Finally, format this UBER meta disk via your favorite filesystem. Expose it to windows via samba, and linux via NFS.
Of course, this whole setup serializations all the operations through one machine. Everything takes one round trip over the network. And unless you use RAID striping, if a machine goes down, so does the whole disk!
Other method, more complex:
Check out the Parallel Virtual Filesystem. What you do is for each spare linux box that has a disk, you run both the IO server and a client. One machine also has to pick up the slack of the metadata manager (no big deal...) Of course, for each linux machine, you have to pick and mount certain the Windows disks (via mount.smb) and run IO server procs for each mounted volume. Finally, you have to run samba on at least one of the linux machines running PVFS to expose those files back to the Windows machines. If you can tweak the samba source to use larger than normal block transfers, do so, because PVFS suffers when you transfer data between nodes that are too small.
Or you can use OpenAFS. Someone else mentioned it here. But it's not as much fun, and it is a big deal to set up if you haven't done it before.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
2.4 GB hyah, 4.6 GB nyah,
here a gig, there a gig
everywhere a gig-gig
Old McAdmin had a farm, 01 01 000000....
"Can of worms? The can is open... the worms are everywhere."
Check out Hivecache. Their software supposedly uses all your free bits of disk space for online backups. Not exactly what you're after, but a pretty cool idea none-the-less.
that by clumping the drives together in a NFS, all PCs must be up and running for the whole to work. If you have more than ten PCs, having them all up at the same time without having one stuck at a BSD may be a problem...