Slashdot Mirror
Terabyte Storage Solutions?
DeMechman asks: "As many on Slashdot may know, storage is one thing which you can never have enough of. Given the current situation with CD/DVD rot (Personally I can attest to a 10% attrition rate) hard drives in a RAID configuration seem to be a better and more economical solution. If you own more than fifty CD/DVDs, it can be a daunting task to find a file. I am wondering if anyone has found a hardware solution that can inexpensively be set up to handle 10 or more 250GB HDDs in a RAID configuration. Primarily, has any case manufacturer tackled this niche market yet?"
I'd say that $2.82/GB, for a well-built, well-designed 14-drive 3U RAID (0, 1, 3, 5, 0+1, 10, 30, 50) hardware cabinet with dual-2Gb/s fibre channel connectivity, dual-100mbit ethernet and serial for monitoring and management, excellent Java setup, management, and montoring software, redundant hot-swappable power supplies and fans, and that works and is qualified for use with Windows, Linux, and Mac OS X, qualifies as "inexpensively". But that's just me.
http://www.apple.com/xserve/raid/
Academic prices for:
1.00TB - $5399
1.75TB - $6749
3.50TB - $9899
It's not RAID, but you could buy a 1-terabyte drive from LaCie.
In soviet russia, You ask not what country do for you, but what you do for country!
Oh wait...
Why buy a specialized solution when the easiest solution is usually in your basement (or under your desk, or stacked up against a wall somewhere)? Grab a few PII/PIII boxes and load them up with drives.
Apple is one of the cheapest, at 6000$ (with drives)
See page here.
http://www.intel.com/support/motherboards/server/c hassis/sc5200/index.htm
Just bought one myself.
You can get em at:
http://www.bellcomputer.com
Let em know G Force Hosting sent ya!
I have a TB here, and rather than raid, I decided to do a nightly "rsync" mirror to a "yesterday" partition.
The two advantages of the nightly rsync over RAID are
- It protects against user-error too. If I make a bad edit, I can always 'diff' against
/yesterday/home/me/...'
- It makes upgrades of both hardware and software easy. Since my live backups are excactly that (live, and tested every day), one machine can be fully upgraded while the other acts as the primary one for a while.
Important data also gets backed up to another large HD in my car and DVDs in a safe occasionally, to protect against a fire or burglars.you can "cheaply" buy 3U rack mount cases that hold 15 drives in hotswappable SATA or SCSI cages up front. Combined with a 3ware 9500-12, and leave 3 cages empty(or spare drives just not cabled up), this will give you 2.75 TB in each unit of raid5 storage. If you were really hard up for space, you could use a pair of 9500-8's and this would give you 3.25 TB per unit. Some 4U units hold 16 drives, which gives you the full 3.5TB in 2 x raid5 arrays.
I have 8 x 160GB Maxtor drives in a RAID5 array. It's fast, relatively inexpensive [Fry's Electronics recently was selling the 160s for $69/ea]
/dev/md2 1.0T 521G 522G 50% /ext
The 160GB drives used to come with a Maxtor [Promise] ATA-133 card. Two of those will support eight drives. Not the most optimal arrangement because of the bus having two drives on each channel, but it doesn't seem to affect performance too much since it is striping the data across all of the drives. I'm assuming it stripes in order, so you'd want to stagger the drives such that 1 & 2, 3 & 4 are not on the same controller.
Output of df -h:
The cost to assemble something like this?
~ $600.00
8 x $70 for the 160GB drives
2 x $20 ATA-133 controllers
The biggest issue is that there is no easy way to back up the array. You could use RAID 6 and have two drives worth of parity info, but it still leaves you vulnerable to a catastrophic hardware (or building) failure.
Anyone have any ideas on how to back up 1TB in a home environment? i.e., not $3000 tape drives & $200 tapes
Good IDE hardware RAID controllers with Open Source drivers. Appears as a single SCSI drive to Linux. We swear by them.
My God! It's full of Voids!
www.raidweb.com Bought one of these at my previous employer and we really liked it.
I use a Hard Drive Enclosure for backing up files. With IDE HDD's getting less and less expensive, picking one of these versatile enclosures up for less than $50 is a good value. I own a DVD burner but rarely use it for data storage since the enclosure is way more convenient. Now as far as 10 250GB drives in a Raid configuration, how redundant redundant do you need you data to be? Or is it that you're just overly cautious after having your backup DVD's fail? Just curious.
Yes, I've used an HP/Compaq DLT auto-changer that will do the job.. Don't remember the price offhand, but I remember it was in the over-$100k range.
I can answer your question, as I've just built one as a giant backup solution for our hosting company.
:)
I went with Serial ATA for a couple reasons:
1) It's cheaper and has more capacity than SCSI;
2) Cabling is not a mess as it is with regular IDE (if you've never seen serial ATA cables, the first thing you will notice is that they are small!);
3) It can hotswap, unlike regular IDE;
4) It's not that much more expensive than regular IDE.
I custom-built a 3U server from InterProMicro. They are a small (local if you are in the Bay Area) SuperMicro reseller that does great work. (If you need something, call and ask for Andy. Tell him Erica from Simpli sent you!)
The machine I specced out was as follows:
* 3U case with 8 hot-swap SATA drive bays;
* 8-port 3Ware 8506-8 SATA RAID controller;
* 5x250GB SATA drives in a RAID-5 array;
* Dual Xeon processors.
The 5 drives give you 1TB of storage, and expanding up to 8 gives you 1.75TB. I would also recommend a separate mirrored SATA 10KRPM array for the OS if you want really fast speeds.
This whole solution (Xeons; 5 drives; 3U case) cost just over $3000... which is pretty reasonable for 1TB of network-accessible storage. Interpro has solutions that go up to 24 SATA drives, which at 250GB each gives you an ungodly amount of space (5.75TB, if my calculations are correct.)
My suggestion is to go with a niche server builder like InterproMicro over Dell or Compaq or any of those guys. You can get the same high quality from a custom manufacturer without paying the steep brand name price from a larger manufacturer. As for the drives, any time the goal is "as much space as possible", SATA should be your first choice.
Good luck!
Simpli - Your source for San Jose dedicated servers and colocation!
-What RAID level you want (5 usually requires better hardware)
-Whether you want hardware RAID (I strongly recommend this) or soft RAID
-How much redundancy you need (Battery backup cache? Redundant controllers? Hardware environmental controls?)
If you are looking for good pci cards, I would strongly suggest a card from 3ware, and a card from a place such a Seagate. Getting a super-duper cheap card when terabytes of data are on the line is just fundamentally stupid. You can save some bucks now, but be ready with your next Ask Slashdot: "How do I recover data from my dead RAID?" Seagate now has a nice 5 year warranty, which match well with good quality and reasonably cheap drives. Look at some of the SATA drives like the Barracuda. However, any decent quality drive maker can work. If you have even more money, you can look at some of the things offered by places like StorCase. A larger initial investment can become cheaper as you scale up the cheap harddrive count, and it can be a good thing in the long run. Obviously, the more time you are willing to invest doing things yourself, the cheaper you can get to some extent vs premade items. However, no support as well.
Do read up on some of the fundamentals of RAID: Everything you need to know (and lots you don't) is probably at least mentioned in the PC Guide on RAID. Look through that. Things like hot swap and hot spares are important to understand. Finally, you should remember to check compatability. Unfortunately, I for instance have not been able to find much of anything in the way of controller cards that is compatable with OS X (except the obvious, the XServe RAID). So I have something set up on a BSD box in my server closet that I then link to, more like a storage appliance. Happily, the 3ware cards and many others are now compatable with a wide variety of *nix and BSD flavors along Windows, but do check to make sure.
Last but not least, remember this!: RAID is *not* a backup solution, but an highly redundant onsite storage system. Have another form of backups, even if it is just a RAID 1 off site, or DVD-Rs, or something. If a disaster happens (thieves, fire, nuclear destruction, John Ashcroft) on site storage won't save you.
Promise has a nice off-the-shelf solution and you can get it for arround $3600.
If I were going to do it I'd build it my own by combining a nice case and a 12 port 3Ware controller with whatever server configuration and SATA drives I wanted to get.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
- Find any tall beige-box case. ($150)
- Find 9 good 250g Serial ATA drives. ($100 each = $900)
- Get an 8-port serial ATA hardware RAID controller like these ($300)
- Get a good 400-500W power supply ($200)
- Any motherboard and CPU will do ($200)
- Spend a few extra bucks on gigabit ethernet ($50)
Put 8 of the hard drives into a RAID-5 array. (1 for your O.S/system use). That makes about 1.4 TB for only $1800 total. The 3Ware IDE raid thing works great with FreeBSD, which is what we use for everything.Rip all your CDs as FLAC so that (1) you never have to rip them again (it's lossless), but (2) it's half the size of saving WAV files
At least that's what we've done with our 68,000 CDs we have here.
I never knew this, and apparently many others didn't either, but if you use hardware RAID the disks are tied to that card.
m l#28
More info here, plus the ever-acidic jwz calling people dumbasses, dipshits, and more fun!
http://jwz.livejournal.com/368307.html
http://www.dnalounge.com/backstage/log/2004/07.ht
John Kerry is a Joke!
We (The Binghamton University Computer Science Department) employ 2 debian raid servers. They make use of a 3ware ATA 12-port card and their (3ware's) hot-swap enclosures (whoever said hot-swapping with ATA is not possible is incorrect, we do it).
It uses a 9 external 5.25 bay case (enlight) with an Antec 550W power supply to handle the 12 drives (plus a seagate system drive in the internal 3.5" bay). This has worked very well.
We use Maxtor 300GB drives in one machine (RAID55) and have lost 5 of 20 drives we purchased in 6 months. The other uses Western Digital 200GB (RAID5), and we've lost 1 of 12 in a year. Manufacturer DOES matter. WD replaced our drive in days, Maxtor makes you jump through hoops and tries to deny the problem for a while, just to finally decide to replace the drive, then take 5-7 mroe days to get it to you.
All in all, these machines cost us under 7K each and perform very well. However, if I bought one today, I'd get 3ware's SATA card and Seagate's new 400GB SATA drives instead. Whoever said ATA cables are a pain was NOT wrong, and these drives would give much better performance.
And i've installed quite a bit of these:
* SuperMicro motherboard (any of the newer ones, depend on your choice of architecture). Be sure to get one with PCI 133/64 and gigabit onboard.
* 3Ware RAID board(s).
* Chembro rackmount cases (they have a very nice one with 16 SATA hotplug slots with backplane and all)
* Don't go cheap on the power supply. You'll need at least 600W. I always go for redundant ones.
* 16 SATA disks of your choice (250, 120 or 80GB)
* Linux!!! (Be careful with fedora core2, it doesnt support nativelly the 3Ware cards - you'll need to compile your own)
Of course you could save about $1000 by using a cheap motherboard, chassis and PS. But it really pays off using the good brands on those.
By the way, you should always get an extra hard drive (or two). They will fail (sooner or later) and you don't want to be left hanging.
My server (with a smaller by far RAID) used to be a dual athlon too. I got tired of paying for the electricity, so I switched it to a Athlon-M 2500+ and setup all the powersaving stuff. (It took ages to find a desktop board with a PowerNow capable BIOS and voltage regulator...) Kernel compiles are a little slower, but 90% of the time (even streaming data at 100mbit) the processor stays in it's low power mode. What once took 350watts now takes 70. Highly recommended.
LVM and RAID are not mutually exclusive. They compliment each other nicely. I'm not sure how (Linux's) LVM could be easier to set up than RAID (hardware), though. Most hardware RAID has some form of a "use these disks as a RAID[0,1,5,whatever]" interface. My only experience with Linux's LVM was my last Gentoo install. However, I have quite a bit of experience with Veritas Volume Mangler^WManager.
If you want to spend the extra money and have a warranty and fancier case, look at Nexsan , or EMC's AX100. Scary that EMC is selling something cheaper than the competition, but they are. Sorta disturbs the natural order of the universe. Still, either will set you back several thousand. The AX100 looks pretty impressive on paper. Options for dual controllers, and up to 3 TB in a 2U space. Haven't tried one myself yet.
Disclaimer: I work for a storage integrator, both are brands we sell.
Ignorance is the root of all evil.
Remember that some manufacturers offer better warranties than ithers (ie Seagate's is now 5 years).
Also, remember that some drives nowdays don't allow for a 24x7 duty cycle. Given that the SMART diagnistics in the drives can tell quite a bit to the person examining your warranty return, don't try to 'cheap' your way through and then claim on warranty.
Um... ever consider the mind-bogglingly simple solution of:
ls -R> ~/dvd.index/<disc_label> for each dvd
grep "<whatever_youre_looking_for>" ~/dvd.index/*
Instead of looking at a semi-commodity 1TB solution - which is a PITA for needing an industrial strength case, power supply, drive controller card and HVAC, you need to look at the other end:
.8TB.
Two or three fairly normal PCs with STANDARD drive controllers, PSUs and HVAC.
Look, we're talking file servers here. 128MB RAM is gobs if you aren't running any other service on 'em. Pick and OS, any OS: 2000 gets you dfs, *nix gives you NFS. Both give you a homogenous networked file system.
So...
Standard case/PSU/cheapo CPU (Athlon mobile or Via or P3, for lower power consumption)/RAM - That's $250, maybe. Add another $20 for a gigabit NIC or two per machine.
4x 200GB drives @ $110 apiece (pricewatch shows $96 as the low price, but I'll go $110 for a little wiggle room)
So... something around $700 gets you
Buy three machines. $2100 gets you tons of storage and scads of redundancy no matter how you look at it.
This is the philosophy I use in setting up my file servers (now serving 6.5TB!). Over time I've added 3ware cards, upgraded PSUs and added gobs of RAM, but my basic starting point is a very modestly-appointed system.
-- I wanna decide who lives and who dies - Crow T. Robot, MST3K