Slashdot Mirror
What To Do With a Hundred Hard Drives?
Makoto916 writes "In five years with my current employer as the IT administrator, I've amassed a sizable cabinet of discarded hard drives; just shy of 100, in fact. All of the drives range in size from 20GB up to 300GB. They've all been stored in anti-stat bags, and spot checks of even the oldest ones show that most of them still work. Individually, they're mostly useless for our line of work, which is digital video production. However, the collective storage potential is quite significant. They are of varying size and speed, but the one commonality is they're all IDE. What is the best way to approach connecting all of these devices and realizing their storage potential? On a budget, of course. Now, I'd never use such an array for critical data storage, but it certainly would be useful as a massive backup array to our existing SAN that does store critical data. I have several spare and functioning PCs, but not nearly enough to utilize their internal IDE controllers; even with multiple add-in controllers, it still wouldn't be enough. Not to mention the nightmare of managing a bunch of independent PCs. I've looked into ATA Over Ethernet and there's a lot of potential there, but current 15 to 20 bay AoE cabinets are expensive, and single device enclosures are so rare that they're also expensive. Are there any hardware hackers out there who have crafted their own home-brew AoE systems? Could they scale to 100 drives? Is there a better way?"
With almost a hundred hard drives, the gold leaf discs inside them must really add up in weight. What's gold trading at now? $850 or something per ounce.
Admit it. You post strawman arguments as AC so you get modded Insightful for refuting them, rather than Troll
I got a $20 enclosure with 17 drive bays in it, and a 300W power supply. I've got a dozen SATA drives, each drawing under 10W, and 5 EIDE, drawing under 20W each.
At first I just got a dozen SATA/EIDE USB slaves for $10 each, and plugged them all into a USB hub, with just the single USB cable stretching out of the case over to another full PC's USB socket. But that is so slow, especially when copying big music or video files between drives (and through the single USB cable to the CPU and back). Playing multiple media files to different terminals in my house is too much bandwidth for the single USB, too. Running 4 USB from the big enclosure to the 4 sockets in the server PC isn't much better, because it all goes through the same CPU and PCI bus.
So I got 3 Sabrent SBT-SRD4 4xSATA controller PCI cards, because they were $25 each. But when I tried to boot them in a few different motherboards (pre-HP Compaq P3/1.2GHz, IBM P4/3.2GHz), none of them got past the POST to even start booting the OS. I want to use them with Linux, but with the failure to even boot I'm not hopeful about driver support, either.
I bought them from CompUSA (still alive, online only), which hasn't replied to (email only - no phone available) tech support requests. Nor has Sabrent itself. I'm not hopeful that they'll refund my money, since everything else about this transaction has sucked.
So what I want to know is what cheap motherboard (no need for graphics or anything else other than at least 3 PCI slots and 100Mb-1Gb ethernet) will work with these SATA cards? If they're really duds, what is the cheapest way to get 12 SATA drives controlled, even if they're not that fast, over to 100Mb/Gb ethernet? Either SATA cards + motherboard, or even a fat mobo with a dozen SATA ports. I'd even settle for just 4-8 SATA ports to get started. I'm talking under $200 if possible.
Ideas? If it works, then 8-9 of them should support the 100 HDs the original question was asking about.
--
make install -not war
I'm going to have to second this, but probably more towards the charity side.
/, 67% /home used so far) - without working drives it's next to worthless and unusable.
It's quite easy for computer recycling charities to get working computers, but because of data security policies at a lot of companies they are not allowed to recycle hard-drives. This means that a disproportionate number of computers to hard-drives float around until they're finally scrapped (which overall costs the charity more time, effort and money).
For example, I have a 9gb and a 26gb drive in my main development machine - with a few 40gb and 125gb drives waiting for me to upgrade to (80%
I would call your local data recovery service as they sometimes are interested in buying old drives of no particular size to use the controller cards on them.
Apparently, a lot of failed hard drives are not bad because of their physical platters, but because of the drive logic. These places need old drives for replacement controllers that you probably can't buy from the manufacturer.
ft
You've obviously never priced an enterprise level SAN. The GP quite obviously works for a very, very, very large company and got a huge price break, or purchased a lower end enterprise/mid-sized SAN. Joe blow mid-size company would've have spent $30K or more for the same or less for an enterprise level SAN from Dell or the like.
Just keep in mind these are *STRONG* magnets. When you take it apart the magnets may smash into each other. This could send particles flying away in a direction that, according to Murphy, is where your eyes are. I know this by experience, lucky for me I wear glasses. And if some of your flesh is between the magnets, it's painful.
Generally speaking, time alone will not reduce the strength of a permanent magnet. Heat, vibration, magnetic flux, and other forms of energy exposure can weaken permanent magnets. But in a box in your cabinet they are unlikely to encounter any sufficiently strong energy source to have a significant impact.
Rebate them with the manufacturer. That way, they're out of circulation (in case of privacy concerns).
Using Your Hard Drive magnets to make a wind Generator
http://www.fieldlines.com/story/2006/2/9/13128/15117
http://www.fieldlines.com/story/2006/10/8/112046/572
http://www.fieldlines.com/story/2005/9/24/152446/359
How to remove Hard Drive magnets from their mounting plate
http://www.fieldlines.com/story/2006/10/4/181345/402
Recycling parts from Hard drives
http://www.fieldlines.com/story/2006/11/9/01948/0162
We are Dead Stars looking back Up at the Sky
Typical Rare Earth magnets have a maximum temperature of 150F. Other than abuse, heat or water getting inside the outer shell . . . magnets have a very long life.
A lot of our donated computers don't come with hard drives, so we're always in need of hard drives more than just about anything else.
We wipe all drives to DoD standards before ever putting them in anything, too. (Well, anything other than the machines we use to wipe 'em.)
If you don't want to ship them all the way to Eugene, there's lots of other charities that do the same kind of thing, and probably have the same disproportionate computer to hard drive donation ratio.
If the masses can keep you down, you're not the Ubermensch.
Most SAN's aren't built with IDE/SATA drives, they are generally built with fast Fiber Channel or dual port SAS drives so that they don't have a single point of failure even in the backplane. The applications that are hosted on most SAN's care about I/O's per second as much as they do storage space. If you don't care about performance then there are cheaper solutions like a Sun Thumper or an HP DL320s that get you pretty good TB/$ while still being more reliable then most DIY whitebox storage projects. I have plenty of storage on SAN, medium speed direct attached storage and on a couple DL320s's, you use what's most appropriate for the job at hand if you're doing your job right and your management allows.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
The drives alone will consume close to 1000W. It's probably another 1000W for the equipment to run them, plus whatever the hardware costs are. When you add in A/C costs, thats going to come to around $8-10/day, and depending on the average drive size, you're going to end up with less than 10TB of redundant data.
Now the alternative is 12x1TB RAID6. It will consume around 250W, and cost around $4000. That's around two years before before the power budget catches up assuming you already have all the necessary hardware. Since you have to buy all the hardware, you'll catch up in under a year.
This isn't at all considering the limited lifespan of the already used equipment.
Sorry but if you are serious your steps should be.
1. Call a recycler and dump the drives. smaller than 200GB (keep the largest ones to give out to other employees for their home systems)
2. Buy 2 or 3 1TB HDD's
3. Install them in a box.
4. Done.
Start with the shear cost the additional equipment, then add in the cost of the electricity to run the drives and their controller. then add in the cost of HVAC to keep the room they are in cool. Will by far exceed the cost of 2 or 3 1TB drives. Not to mention the cost of your time to build, deploy and maintain.
In short. Nothing you can do with these drives will save your employer money. However proper recycling might bring in a buck or two. Not to mention the good will when you hand the largest drives to fellow employees to use at home.
I'm sorry, I'm to tired to be witty at the moment so this message will have to do.
Here you go.
http://www-03.ibm.com/systems/storage/tape/virtualization/index.html
Average HDD power consumption varies but 5W at idle and 8W under load is probably a reasonable average.
500-800W to run 100 HDDs. Some PCs use that much alone. Even these days, it's still worth using older HDDs, because the cost of replacing them with bigger and more energy efficient ones is still not low enough to cover the cost of running an older drive for a few years. Especially if your NAS supports power saving.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Umm, we don't even know if a magnetic monopole exists. Currently, theory is the only place you can find one.
yep... DBAN (Darik's Boot and Nuke) FTW!!! I just DOD-wiped about 75 old 10-20-40GB drives that we're gonna be excessing. It took me a bit over two weeks.. You bet your bippy I didnt sit there and watch them wipe...
THANK YOU, Edward Snowden!! Americans owe you a debt of gratitude (whether they know it or not..)
You can get a bunch of firewire to ide bridge boards, and run scsi over firewire.
Keep in mind this will be noticeably slower than native ide once you get more than a certain number of drives on a single bus, but for some applications, fast disk access isn't as important.
Technically speaking, you can use USB for this too, however there are many more downsides.
Many times slower than firewire, due to the method usb uses to communicate bidirectionally.
Its not that much cheaper, and also you cant use nearly as many drives per bus.
As an example, try http://www.fwdepot.com/
Their prices are a bit high i admit, but you can build a shopping list there and look around for best price.
4 BUS firewire cards. Note that a 4 -port- card is not at all the same. That will be one bus, with a 4 port hub built in. The less drives on each bus, and the more buses you have, the more bandwidth is available to each disk, and the speed up is exponential.
One bridge board per hard drive, a few hubs and some cabling, and spread them out over your few spare pcs.
Then run something like http://evms.sf.net/ to cluster the machines together and create one giant pool of storage space out of all the drives over all the machines.
It's probably as cheap as possible for getting use out of them storage wise. Any other 'better' solution will cost a lot more too.
Of course, useful for storage and just plain useful are two different metrics.
A lot of others already mentioned donating them.
Just remember to hook 4 up at a time to a spare pc and run a good HD wipe app like http://dban.sf.net/
But there are many options to get rid of them to others with.
Charity donations for a tax write off, local community projects in need of hardware, friends, family, stocking stuffer for the staff, make a craigslist post and offer them for free (or next to), buyer comes to get it or pays shipping, do the ebay dance, etc etc
A "one sided magnet" would be a monopole. Draw your magnet, draw the field lines: these lines must end up being closed loops, neccessitating one sout pole and one north pole for every magnet.
We're all born with nothing.
If you die in debt, you're ahead.
A decent IT shop will have a policy of wiping drives if they're operational when they're taken out of commission and tagging them as wiped. I work in government and this is routine procedure for us.
If the drive doesn't work, it's shredded.
But I have to agree with the previous posters about the power required. If you have a bunch of 300Gb it might be worth it,but less than 100Gb you'll end up wasting more than you gain. What I usually like to do with them is if I have an extra slot on the HD IDE I put the smaller drive as a dedicated swap. Takes some of the wear and tear off the main drive and gives you a nice little speed boost as well. But that is my 02c,YMMV
ACs don't waste your time replying, your posts are never seen by me.
That link on removing the magnets gives a lot of difficult solutions. Here's what I do:
* Bend open plate with the magnets and clamp it into a vise. (Usually the plate is a "U" shape, with the magnets inside the U.
* Heat up the plate with a heat gun: the magnets are glued on there, and this will melt the glue.
* Pull off the magnet with some pliers.
Easy, and it works every time.
Disk based backup solutions are worth the effort, so I can see why you're leaning this way. Unfortunately, trying to utilize ~100 PATA drives for this is going to give you nightmares for ages. Find a way to reclimate them for cash, either directly or indirectly. Hell, you can donate them to charity for a tax writeoff if you like (just make sure you DoD wipe the disks first). Take the reclimated capital and buy yourself a new data-deduplicated VTL, or a NearStor, or similar. Backup solutions need to have some level of trustworthiness to be useful, and I doubt you'll find that in a pile of aged PATA disk.
The standards for data sanitization is more stringent now. Anything that is more sensitive than Classified, and leaves the control of the organization disposing of the drives, needs to be either put through a degauser, chopped up into tiny pieces, or turned into slag. If the media is simply going to be re-used with-in the organization then wiping is okay.
Err, monopoles, no. They are one-sided flux devices though.
We experimented with that at the shop. Your typical degaussing ring doesn't generally have the field strength to wipe 'em. Heck...in our test, after zero-writing 'em, and checking 'em after 5, 10, 30, and 60 seconds of D-ring exposure we didn't appear to lose a bit.
Note: dedicated hard drive degaussers can get really expensive, too... It's MUCH cheaper to stick with software methodology. Have a look here for details on both methods...
Don't tell me to get a life. I'm a gamer; I have LOTS of lives!
dd if=/dev/zero of=/dev/hdb bs=1M &
dd if=/dev/zero of=/dev/hdc bs=1M &
etc., substituting
DRM 'manages access' in the same way that a prison 'manages freedom'
(geek)
There are no magnetic monopoles in theory, either. Maxwell's four equations that define all of Electromagnetism, includes Gauss's Law of Magnetism. This law states that magnetic fields don't in net diverge.
Its usually written in differential form as: del * B = 0 (del dot B = 0). Note that Physics students from bush-league universities might write the equation in integral form, but that's either a product of their deficient education or maybe some kind of genetic defect.
More here (wikipedia):
http://en.wikipedia.org/wiki/Gauss's_law_for_magnetism and here:
http://en.wikipedia.org/wiki/Maxwell's_equations
Yeah, I suppose magnetic monopoles might exist and then we'd re-write the laws, but there's no reason to assume so. There is a natural temptation to look at magnetism the same as electricity (individual charges, like electrons and protons, being analogous to "North" and "South" monopoles), but probably the most useful way to think of magnetism is as a relativistic effect of electrostatics... once you do that, there's no reason to assume any kind of magnetic monopole at all.
(/geek)
My favorite quote doesn't fit into 120 characters. Now no one will like me.
SAN drives are also generally smaller in capacity, around the 250 GB range, and a 2.5" form factor. The reason for this is to maximize the spindle count. For example, having five 250GB drives with RAID 5, each running at 10,000 RPM can handle a lot more I/O (especially random seeks) than one 1TB 3.5" SATA drive. SAN drives also have a lot more cache. The reason that a most SAN companies have moved to 2.5" for their drives is because even though the platters have less capacity, the data is faster to get to on a physical basis, and case engineers have more room to engineer around the drives, especially in 1U rack enclosures than 3.5" drives.
Enterprise drives are definitely more expensive, but in this case, one gets what they pay for -- a lot more speed (especially with large, random seeks), and decent redundancy. The drives themselves are in the million to 1.4 million hour MTBF range, while consumer level drives, either don't have a rating, or the MTBF is hard to find, so the best guess is 250,000 to 500,000 hours, although some drives do have a million hour MTBF.
The key is to figure out the task at hand, and one's budget, and decide that way. Some tasks, just hooking up drives to the motherboard and using software RAID is more than workable. Other tasks are so time dependent that one has to have full hardware RAID with as many low-capacity spindles as possible to distribute the I/O far and wide. This is why Flash drives are making a good dent in the enterprise RAID market -- they are not perfect, but there is zero time wasted waiting for the head to move, and the right sector to float by.
What I usually like to do with them is if I have an extra slot on the HD IDE I put the smaller drive as a dedicated swap. Takes some of the wear and tear off the main drive and gives you a nice little speed boost as well. But that is my 02c,YMMV
This is always a good idea. Move the swap and the Windows temp to this drive and keep it formatted FAT32 (or lower). If you can, partition the disk up and give it two 2 gig partitions. Each partition should be formatted FAT16 (aka: FAT, no 32). FAT32 and FAT16 need to read/write to the disk less for each transaction than NTFS and is much faster for it. Since it's just swap and temp files, you don't need NTFS.
If you do this, you leave the rest of the drive open for users' personal files or whatever. The two partition setup should be one for swapfiles, the other for temp files. You can get more creative and create another for a web browser cache, but as you create partitions the drive head has to move farther to span the space and slows down the operation. A large FAT32 partition works well if you dig deep and move a lot off onto this second drive. Another thing to keep in mind is what people are going to be putting into their temp directory. Video work might create files bigger than the partition. In this case, create a 2 gig swap partition at FAT16 and leave the rest FAT32 for normal files.
I always get a second drive now for this reason. Helps in both Windows and Linux. For even better results, keep the drives on different IDE channels. Just think, the overall strategy is to keep one disk working on program data and the other working on the memory swap data - a major bottleneck, especially at IDE speeds.
Get your Unix fortune now!
Plutonium, not uranium.
Nothing to see here; Move along.
A caveat with FAT. It does not support Unicode, so if you use software that tries to save files with Unicode (i.e. different to your OS locale) names it will fail.
NTFS is not significantly slower than FAT, and in fact can be faster due to improved caching and resistance to fragmentation. Sure, sometimes more data need to be written to the disk than with FAT, but in practice it just gets cached until the disk is free to write it without interrupting anything else.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC