Slashdot Mirror
Making Use of Terabytes of Unused Storage
kernspaltung writes "I manage a network of roughly a hundred Windows boxes, all of them with hard drives of at least 40GB — many have 80GB drives and larger. Other than what's used by the OS, a few applications, and a smattering of small documents, this space is idle. What would be a productive use for these terabytes of wasted space? Does any software exist that would enable pooling this extra space into one or more large virtual networked drives? Something that could offer the fault-tolerance and ease-of-use of ZFS across a network of PCs would be great for small-to-medium organizations."
If you have a very robust local network with plenty of spare capacity, and can accept a performance hit on the client computers, I am sure some kind of linked filesystem would be possible. In most practical situations, I think this idea would be a non-starter.
If they're in a computer room, then such a scheme might work. But, if they're on user's desks, you don't really have control. They're subject to filling up, being shut off, being knocked about, crashing, etc. I don't think in this case you would really get the reliability that the diversity and independence would suggest.
--Marc
You could try to use something like "Localhost Azureus" for distributed data storage. The only problem will be that it will cost you in terms of processor and network hogging.
Is it cost effective to reclaim that (small) space? Probably not. My suggestion is to realize that no-one tries to save clock cycles any more and maybe this is the way disk storage is probably heading that way.
It's a very interesting question, but from my point of view, hard drive space is so ridiculously cheap nowadays that it is utterly pointless to look for a useful application that will fill it up.
Let's assume that the average computer has 80 GB of storage. Multiply that by 100 and you get 8 TB of space. That's what you can get into one or two computers nowadays without plunging out too much cash.
What's more interesting is how much processing power you have as well as how fast the internet connection is.
Full Tilt
This is the dumbest /. question I've seen.
Decentralized network storage pooled together with no means of practical management? Sign me up! Oh yeah, let's rely on the ditzy end users to help make sure it doesn't crash. I'm sure everyone will leave their computers on 100% of the time so you can make use of it.
Don't tell anyone at work of your idea, they might not ever stop laughing.
Please stop typing words like "utilization" when you mean "use". You sound like a PHB trying to sound smarter than he really is and you make it a pain for people to read what you write, especially non-Anglophones. Read George Orwell's essay on this topic.
You might want to ask yourself why, after more than a decade of research and countless papers and prototypes that address this problem, your PCs storage are still underutilized...
It's harder than it looks to get something reliable. Your PCs have extra capacity because it's cheap, but mining that capacity is not cheap. As other posters have pointed out, putting together (or just purchasing) a server with a few TB of storage is simpler and cheaper, less prone to getting wiped out by a virus, easier to manage and backup.
Am I part of the core demographic for Swedish Fish?
Hrmm... Funny, he didn't come across that way to me at all. You, however, come across as a pompous linguistic Nazi, much like Orwell. If you compose sentences for people who don't have command of the language, then you are really quite delusional.
As is my understanding, resources are utilised, while tools are used. He was correct in its usage.
Great, let's all dumb down to the lowest common denominator. English is a rich language and all the better for it. If you're too lazy to learn it, your choice. I'm a non-native speaker but prefer a vibrant, expressive language to some "for-dummies" international pidgin.
I'm sorry if I haven't offended anyone
The difference is that I wasn't nasty about it, I explained a problem and gave him a link to an essay about it. You, on the other hand, called Orwell and me names, attacked a straw-man, and said something incorrect about the words that is trivially debunked by glancing at a dictionary.
AFS would be applicable if you were interested in turning each end user workstation into a centrally managed AFS server and dedicate storage for holding replicated readonly volumes. I wouldn't store single instance read-write volumes on a machine that at the mercy of an end user to turn on or off. I would also be resistant to deploying centrally managed storage on end user controlled machines in any case due to the access control issues. Anything that is stored on a machine that the end user has physical control over can be accessed by the end user.
As others have pointed out, storage is so inexpensive these days. 8TB can be obtained for a few thousand dollars and managed in a much more reliable manner.
Detailed knowledge of any technological artifact will make you better at using it, maintaining it, knowing when to use it, whether it's an automobile or an AK-47. Yes, some people find guns interesting to a greater degree than others (I don't, personally, nor do I own one) but whatever floats one's boat. Let me ask: do you find someone that has an advanced knowledge of computers creepy? Probably not, if you're on Slashdot ... but there are many that do, until they need him.
When the time comes that I need a brain to pick, it's those "creepy" nerd types that I seek out. They're the ones most like to be able to help. Maybe you're anti-gun, and the fact that some people are not is offensive to you, I don't know. Regardless, you should look at people who know much more than you about a given subject as a potentially valuable resource, not an object of scorn.
The higher the technology, the sharper that two-edged sword.
Having tried this in college, I can tell you a couple things.
1. You will noticeably reduce the lifespan of the discs. (Which can anger cost conscious supervisors)
2. Doing ongoing hardware maintenance, because of this reduced lifespan, on closed, used by others, boxes is a *serious* pain.
Storage setups make hot swapping discs easy, trying to do this with full blown systems just gets tiresome. The solution I eventually came up with was the following.
Implement a two tiered hardware replacement cycle where you reduce the time a user is allowed to keep any hard drive in their box before replacement. Then using the still reasonably good drives, create a centralized storage solution in which the drives can live out the rest of their useful spans. Data security, user happiness, and redundancy are all good selling points of this system. You still have to deal with monkeying around in user boxes but if it's on a schedule and it nets you more drives, it's not so bad.
-Ian
You make it sound like it's a bad policy keeping all business data somewhere properly managed. It won't mitigate any damage done to your company or your career because you told them to be careful. People will store data in the most convenient location, thats not stupidity - just human nature.
Been there also and I disagree on the "just works" part. We're using it successfully but we've run into issues with losing files from some of the computers. Let's say that the admin changes permissions on somebodies directory so that they can write to it also. DFS will think that the file with the changed permissions is the newer one and blow the other one away. I hesitate to call that "just working".