Making Use of Terabytes of Unused Storage

kernspaltung writes "I manage a network of roughly a hundred Windows boxes, all of them with hard drives of at least 40GB — many have 80GB drives and larger. Other than what's used by the OS, a few applications, and a smattering of small documents, this space is idle. What would be a productive use for these terabytes of wasted space? Does any software exist that would enable pooling this extra space into one or more large virtual networked drives? Something that could offer the fault-tolerance and ease-of-use of ZFS across a network of PCs would be great for small-to-medium organizations."

Comments

Not without heavy utilization of other resources

[Mostly a lurker]

If you have a very robust local network with plenty of spare capacity, and can accept a performance hit on the client computers, I am sure some kind of linked filesystem would be possible. In most practical situations, I think this idea would be a non-starter.

Do you really have control of the boxes?

[Marc Rochkind]

If they're in a computer room, then such a scheme might work. But, if they're on user's desks, you don't really have control. They're subject to filling up, being shut off, being knocked about, crashing, etc. I don't think in this case you would really get the reliability that the diversity and independence would suggest.

--Marc

Space is not that important any longer

[eebra82]

It's a very interesting question, but from my point of view, hard drive space is so ridiculously cheap nowadays that it is utterly pointless to look for a useful application that will fill it up.

Let's assume that the average computer has 80 GB of storage. Multiply that by 100 and you get 8 TB of space. That's what you can get into one or two computers nowadays without plunging out too much cash.

What's more interesting is how much processing power you have as well as how fast the internet connection is.

Re:Space is not that important any longer

[jaxom]

I disagree with this and face this question all the time in work. Disks are cheap, storage systems aren't. If this is for a business that requires reasonable uptime, then the only solution would be to implement a SAN using Fibre Channel or iSCSI and then take out the drives. With the right array, all of a sudden those drives become superfluous (you decide if boot from SAN is right for you), management is easier and you'll be able to get a lot of reuse out of the drives.

Now a lot of people will start to question the cost of doing all of this and it isn't cheap, however you have to analyze the data correctly. We migrated 200 servers from DAS to a SAN and had our money back within 12 months. Add on top of that the implementation of VMs, all of a sudden those 200 went to 20. That's a big difference in cost of ownership.

Re:Space is not that important any longer

[LWATCDR]

Yep a better question is Why do all these PCs have harddrives?
If they are really only using it for the OS, a few applications, and a few docs why not use diskless workstations?
Less power, heat, and fewer things to break.
In other words don't use all those drives, get ride of all of them.

Looking at the problem another way...

[pedantic bore]

You might want to ask yourself why, after more than a decade of research and countless papers and prototypes that address this problem, your PCs storage are still underutilized...

It's harder than it looks to get something reliable. Your PCs have extra capacity because it's cheap, but mining that capacity is not cheap. As other posters have pointed out, putting together (or just purchasing) a server with a few TB of storage is simpler and cheaper, less prone to getting wiped out by a virus, easier to manage and backup.

Re:Not without heavy *use* of other resources

[DarrenBaker]

Hrmm... Funny, he didn't come across that way to me at all. You, however, come across as a pompous linguistic Nazi, much like Orwell. If you compose sentences for people who don't have command of the language, then you are really quite delusional.

As is my understanding, resources are utilised, while tools are used. He was correct in its usage.

Re:It's been done by Microsoft: DFS NameSpaces

[darrenkw]

Been there also and I disagree on the "just works" part. We're using it successfully but we've run into issues with losing files from some of the computers. Let's say that the admin changes permissions on somebodies directory so that they can write to it also. DFS will think that the file with the changed permissions is the newer one and blow the other one away. I hesitate to call that "just working".