Slashdot Mirror
Home Network Data Storage Device
It happened again- a machine on my home network died. Taking with it tons of data. It's mostly backed up. No huge loss. But I finally think it's time to get some sort of network raid disk. A unified place to safely store data accessible to the numerous machines on my home lan. So now I pose to Slashdot readers- what are your recommendations? I'm looking for something with RAID and SMB sharing. At least a quarter TB, probably a half, but with some room to grow. What have you used? What works? What fails?
As CmdrTaco, I'm sure you have money coming out of your ears that you've harvested from the pseudo-religion that is Slashdot.
... I know, I know, I'm going to catch hell for using such a crappy generic product. And I know many people who will tell you that VIA is crap when it comes to RAID controllers. Maybe you're one of them. If you are, I hear that the brand Promise provides excellent RAID controllers, you'll just pay a whole lot more for them. A couple of these babies in RAID 1 and you're set.
But for those of you with fewer fiscal resources, I will tell you the stories of my friend and me, a.k.a. The Master Rebaters.
My story is a simple one. I love music. I have over 1,000 CDs and have spent a lot of time meticulously ripping them with my friend CDex. So, I have some 350-400GB of data that I would like to archive. There are a multitude of possibilities but, since I'm short on cash, I opted for a simple $13 RAID 1 controller
My friend, however, opted for a huge and expensive RAID 6 array controller made by Promise. Then he waited and waited until there was a 250 GB Maxtor rebate at CompUSA or Outpost and went in and bought five with cash. Then he filled out the rebates for relatives and played the waiting game. Huge initial investment but he received a lot of money back slowly. Result, a 1.1 ~ 1.2 TB RAID array. He got a lot more storage and more efficient use of the disks since a RAID 6 with striping allows for drives to be rebuilt in the array.
What he wasn't planning on was the logistics of what he would have to do to his Antec case as a result of all these drives. Fans. Airflow. Heat. These all became huge issues for him--especially in the summer. I'm not sure what your situation is with a case but I made no alterations to my case.
Now, there's a lot of things I skipped over that you can take into consideration, like SATA or ATA? 7,200 RPM or 10,000 RPM? 8MB or 16MB buffer? Striping size? etc. Honestly, those issues aren't worth my time to mess with. Sure sure, I'm losing precious ms seek/read time on my disks but I'm not that motivated.
In the end, if you're only looking for half a TB, do what I did. Those 500 GB drives will only get cheaper and if one blows, just pop another in. And if you really need that room to grow, grab the nice RAID controller that supports RAID 0-6 and just use two 500GBs leaving the other three slots open for the future when you might buy them and RAID 6 it.
What fails? The old IBM Deathstars. Beware!
My work here is dung.
First I'd recomend using a size formating in your question that better fits your situation like "At least 250GB, probably 500GB, but with some room to grow".
On to solutions. Buy yourself a big case (you can do rackmount or regular "large" ATX cases) and stick a decent computer in there. Add Gigabit NIC. Add an 8 port 3ware SATA Raid controller (configured to RAID5). Add 4 120GB 7200RPM SATA Drives (or what ever you can find cheap, even 200GB drives are relativly cheap these days). Install Linux, share your harddrive using Samba. Done.
You have 4 extra ports to expand your RAID if you need too, or you could get bigger harddrives. I think 3Ware cards can support up to 2TB of HD space - so that gives you some expandability. Plus you have a RAID5 which has fault tollerence built in.
snowulf.com
Tera Station
Everything you need probably. I saw a 1TB version for $700 at Fry's the other day.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
A linux box with a 3ware RAID card.
i reccomend you eat myy ass
There are dozens of products out there to do this. Linksys alone makes several. You obviously didn't search slashdot, google, etc. The fact this article got accepted... Words fail me.
-Foxxz
Raid 0.
If you have the funds, this looks promising.
:)
LaCie Biggest F800 1.6TB RAID Storage 300943U (USB 2.0, Firewire 800).
It's only CAD $2,599.99, so that's like US $100
I haven't tried it out yet... but I'm looking at the NetGear Storage Center SC101.
Who wrote this? Why aren't there any of those lovely italics?
;-)
You know, we wanted you to add nofollow, not take credit away altogether...
To take a wholistic view, first, tell me more all-encompassing details about your previous cheap-n-shitty hardware purchases...
you can download all of that porn you lost from the usenet. think of those binary newsgroups as an always on, multiply redundant network backup of all of that porn you lost
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
How can you dupe so many stories yet fail to dupe your own data? :)
You might as well start big right away. Digital media is sure to explode in the near future, even more so than it already has. Soon enough you'll find that .5 TB is nowhere near enough capacity. You may be requiring 500 to 600 TB in even just two or three years.
While you'll want to leave yourself room to grow, of course, don't underestimate your future needs. You will need more disc space, that is almost guaranteed. Set your target very high.
Cyric Zndovzny at your service.
At one point, 250GB SATA drives could be had from newegg for about $120 each. That's pretty cheap.
I've been looking on-line trying to find this sort of possibility and the only prefab system I've found that has configurable RAID in a consumer NAS is the Buffalo Terrastation. I've seen lots of NAS devices but basically they are all just a single hard drive with a network connection.
I have not used one of these and do not know if it's any good, but like I said, I haven't seen any other options for a prefab system. I've priced out what it would cost to roll my own system like this and it ends up being only a tad more expensive to get a prefab device. Actually, I think the price dropped on the terrastation so I'm not sure that's true anymore.
Also, if you get something like this, you should seriously consider upgrading to gigabit Ethernet if you haven't already. I have a network mounted share for most of my files and it works pretty well, but when I try to do things like synchronize my ipod against it, it totally crawls. Having a networked file server works better if it doesn't feel like your files are on a network.
This sig has been temporarily disconnected or is no longer in service
My condolences on your recent loss.
Couple questions:
1. SMB only? NFS is faster and plain better, but only for mac/linux.
2. Noise/size/power constraints.
3. Price.
SMB only, moderately cheap, quiet and small, go for a teraserver from buffalo networks. Easy to setup, runs decently, 4x250 drives that can be raid-5'd into a 750 array. Costs about $800.
A good midlevel solution is an nforce4 motherboard, with 4 250 sata drives, total cost around $600 w/ cpu mem, etc. You need a decent case though, and it will be noisier and louder. Plus side is better performance, full customization, and ability to use it as a router or such. You will have to configure it yourself, and likely throw windows on it because the nforce raid support is tricky on linux for a novice.
I use a heavier 2tb solution myself with a HW raid card, but for most purposes a sw raid is better, and the performance difference is almost never noticable. Personally I recommend the buffalo if you don't need nfs, just for the size, quietness, and convenience.
The first rule of USENET is you do not talk about USENET.
Recipe for xbox raid network storage:
Add 1 Xbox ($130)
Add 1 mod chip ($30)
Add 2 250 GB HDDs ($250) -- you can either disconnect the CD-ROM or follow instructions on adding a second hard drive, but disconnecting allows everything to be internal
Add 1 Linux for Xbox ($0)
Stir in Raid 5 set-up and samba
And you've got yourself a headless quarter terabyte raid 5 network server for a low-budget of $410.
Figure out what your data's work, come up with a budget for protecting it, and go from there. Without a reasonable budget, nobody can intelligently recommend specific solutions.
For myself, I'm using a VIA Epia motherboard (quiet, extremely low power consumption) with an Areca 4-channel SATA RAID controller w/ four Seagate 7200.9 250MB Drives in an hot-swap enclosure with extra cooling - configured in a RAID-5 array (all of my data), and two WD 160 GB drives on the IDE channels in a RAID-1 configuration (OS, programs). It's running Fedora Core 4 and SAMBA (and a bunch of other stuff). The performace is reasonable - I can saturate a 100Mbit LAN connection, which is all I really care about.
For off-line backup, I use Mitsui Archive Gold DVD-R disks - supposedly they're good for 300 years. I'll believe this 300 years from now if my discs still work, but it's the best available right now. I'd love to hear some feedback from people with more knowledge than myself in this area.
The most important thing - discipline. Store important stuff on the array, use rsync or whatever to synchronize with your PCs and laptops, and backup on schedule!
Help save the critically endangered Blue Iguana
I record a lot of concerts at 24/96 and also have a large collection of music in FLAC format. Current archive is 3.5TB and rapidly growing. It mostly consists of 320 and 250GB S/ATA WD drives.
I have good enclosures and run all my drives cool, 25-29C. Two 120mm case fans, one front, one rear.
I am guessing there isn't anything that can compete with the price-performance of just building another Linux box with 7 or so drives. Is there?
... buy an LTO2 or LTO3 tape drive. LTO2 can be had for ~US$1.5K and can hold 200GB per tape before compression. We have a 14 tape LTO2 library here and it makes my backup work a cinch.
Trolling is a art,
"...real men just upload their important stuff on ftp, and let the rest of the world mirror it " :)
My MythTV HowTo
Never trust your data to any one box.
As for the solution, the cheap and easy option nowadays is to simply use stock motherboards - most will accomodate 4 SATA drives and up to 4 PATA drives with no extra work - and run Linux with software RAID on them. It's still a problem to boot from a RAID disk, so one can be set aside for that purpose. Motherboards have GigE nowadays, so speed is not limited by the network link. 300 GB drives are cheap, making a 1.5 TB server affordable if you acquire it piecewise over the course of a year or two.
Now duplicate this setup into 2 boxes and you're good to go.
[an error occurred while processing this directive]
It's quite easy to find used, tiny 700-1000 MHz Celeron PCs for a fraction of $130, and you do not need to bother with a $30 mod chip.
Cyric Zndovzny at your service.
looking for good SATA RAID controllers with external ESATA ports, and good external hard drive cases to match. Hard drives bought with rebates will fill the cases cheaply.
RAID with parity (at least 5) is mandatory. RAID 1 is just daft these days. Cheap, fast but very stupid.
If you don't try to cram all the drives into one case, you avoid many of the heat and power problems....
RAID could help with downtime, but is not a substitute for backup, really. Tape backup is still very expensive (high inital cost), and DVD's are limited in both quality and storage capacity. Well, I use both, but then my storage needs are slight since I burn my most important data to a DVD-RAM disc every night.
What OpenBSD thinks about RAID:
I have the same situation. At least one of my hard drives dies every year. They just get worn out. So what I've done is build a server out of an old Pentium-233 I had laying around. It's not a speed demon, and only has UDMA-33. But my network is only 100mbps anyway. But it does the job perfectly fine for things such a subversion repository/samba server. It runs Ubuntu 5.10 without an X environmeny, and has a single 300GB hard drive that I fully intend to beef up with an identical drive in RAID-1 at a later point in time (I have a RAID controller, but the motherboard is too old to support it, and I lack funds anyways).
My laptop and my desktop both use the server for things such as music and video, and I rarely have an issue with speed. Samba is stupidly easy to set up via SSH, and it honestly does everything I need it to do at this point in time. At a later date, I might beef up the server with a new motherboard/CPU, and turn it into a media server, with a LOT more storage, and hook it up to my TV. But that's a task for another time, when I have more money.
But for what it does, my server functions absolutely beautifully. I've even built a similar box for my girlfriend, and she uses it also for sharing things with her friends via FTP.
The requirements need to be a bit clear-er. Do you want something that sounds like a small mouse? A rider lawn mower? How about a two-story jet engine turbine fan? Are you willing to spend 500$? 1000$? Important things to consider.
My SAN box has some particulars: 1.1GHz CPU (Intel celeron, I think...), 1x MegaRAID controller card (SATA) [RAID 5 is a requirement], 4x 200GB SATA drives, 1x 30GB IDE hard disk (Operating System), 512MB RAM, 1x Gigabit PCI Ethernet NIC, and a nice beefy PSU, something that can handle a lot of strain. Since this is a home network devouice, it doesn't necessarily have to be snappy, unless you're particularly anal like that (3 or 4 sec / mp3 isn't so bad, in my opinion).
I would reccomend Seagate disks, and definately the MegaRAID controller. I would also reccomend some of those hard disk cooler fans. They take up a bit more space, but I'm convinced that they're the reason my SAN box has continued to run on rather warm days. You'll also want to make absolutely sure that you use SATA and not IDE, incase you ever have to rebuild a disk on the fly. IDE takes _forever_, especially if your controller card doesn't have a lot of buffer memory (PS: Get a controller card with an onboard memory module).
Informatus Technologicus
The only non-obvious thing (i.e. a lot of people are telling you to do the wrong thing) is that you should use software RAID instead of hardware RAID. The cheapest CPU that you can buy, will still be 99% idle.
A less non-obvious thing (but some people still forget it) is that you want a well-cooled machine, because heat is what kills hard disks. Get a nice case; pretend you're building a machine that you wanna overclock like an 31337 h4xx0r, but then of course, don't really overclock it.
Oh yeah, and keep an eye on /proc/mdstat -- when your first disk dies, you want to know it happened, instead of finding out a year later when your second disk dies. (I use a lil' python script that displays the array status on a VFD using lcdproc. But there are lots of other ways to deal with it. Just make sure you deal with it somehow.)
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
I use a 3ware 7810 and 4 250GB IDE disks in a RAID 5 configuration. The controller can be had for $200 or so on Ebay and works quite well (though you may want to use a 7850, somewhat better RAID 5 performance).
Ubuntu does a great job as a server.
Mark
Buffalo makes a product called "Terrastation" (or something similar in naming).
It can do RAID5, and can expand using USB2.0.
The only problem is that it uses Western Digital hard drives, which are in my experience proned to sudden death syndrome, more suddenly than newborns. (I've had about 9 out of 17 WD drives die, mainly when the partition table mysteriously disappears....of course, IBM deathstars have the all-time record....7 out of 9 and not just partition table death....but entire drive death)
I dunno if Buffalo sells it empty so you can use your own drives like the 500GB Seagate (the ones in it are 250GB WD's).
copy C:\*.* D: Done!
This message printed on 100% post-consumer recycled electrons.
I have used a Promise and 3Ware controllers on server setups (and they worked great), but now that software RAID has matured in Linux, I plan to save some money for my home setup and use software RAID.
I found an external case for 4 drives, without hot swap (which I'm told doesn't work that well with Linux software RAID anyways) for like $150. A 4 channel SATA controller is like $60, and the Multichannel bracket the external case's output back into 4 single SATAs is like $80 or so.
Once I confirm that I am happy with the RAID performence and reliability, I am going to pull the trigger and get some 7200.9 Seagates in the 300GB range. With RAID 5, I'll get 900GB storage and no more drive crashs causing data loss.
I will not ever use a Maxtor again. The only drives that I have had crash that weren't Maxtors were IBM Deskstars. Seagate has 5 year warranty on their drives.
Sharing to Windows is easy with Samba, and I have also configured rdiff-backup on cygwin to backup the local drives to UNIX.
I used a similar method, though not raid and not gigabit nic. I decided I needed a centralized storage place for my media. I took an old PIII Celeron I had laying around with on board video and lan. I then bought a 4 port Promise SATA card and two 250 GB Seagate SATA hard drives. I installed slackware, stripped it down while making sure it had NFS support and LVM support and finally configured it. I got it down small enough to run off of a 64 MB flash card, that was just a fun side project. I went with LVM since it would be easier to manage, plus I wanted my data to all be on what seemed to be one drive. I didn't want to have to have a bunch of directories that I would have to drill down in. I don't have the money currently to setup a RAID setup with integrity support. LVM appealed to me because I can add and remove drives very easily. I can also swap drives out very easily. I don't have all that much experience or knowledge with RAID, but it just didn't seem like it would be as easy. Now this computer is on a 100 Mbit network. I have been able to stream movie, DVD quality, to my HTPC and Roku PhotoBridge with out any problems. Obviously, music and pictures are fine since video works. I am currently sitting at 1.5 TB, with only 11 GB left. It is a Jerry ridged case, al though I plan on custom building one (to save money) to house all the HDDs and fans to cool them. LVM only goes up to 2 TB from what I have read, but that shouldn't be a problem as 8 drives take up a lot of space and require a descent power supply. I will just start the project all over again after I add drives 7 and 8. Though, I am unsure how I am going to migrate all the data to look as one big directory. It has been up and running strong for almost half a year now. Obviously minus the times it had to be shutdown to physically install the drives. I got a large portion of my DVD collection archived to it along with CDs and my photos. Has worked out very well for what I needed it for. I just need a house now with a place to have a rack-mount to hide all this crap. The fiancée isn't a fan of computers laying all over the place and being in the room, although, surprisingly, it isn't that loud.
It is basically a network appliance (no TM) box that allows you to put your own IDA drives in to make your own storage device. Seemed like a good idea for the newbies out there.
Recently I was also shopping around for a storage solution. At the store, I saw a promising looking device called the Netgear SC101. You pop any two IDE drives into it, plug an Ethernet and power cable in the back, and you have yourself a NAS. Because you can pick out your own drives, you can even do a terabyte in a cheaper and much smaller unit than 4 x 250 GB units like the Buffalo Terastation.
Unfortunately, where this device failed for me was that it doesn't just share the stuff as a SMB share like a real NAS box does. It uses some weird proprietary protocol, and only machines with the right drivers installed can talk to it at all. Such drivers aren't available for Linux, or Mac, or BSD... even versions of Windows that are old (98, ME, etc.) or 64-bit won't work. It has to be a 32 bit version of Win 2k3, XP, or 2k with the right service pack level for the drivers or no data for you.
No self-respecting geek would want a device with such limited compatibility. If a piece of network equipment only lists Windows in its compatibility, that normally means the manufacturer only officially supports Windows, or maybe you need Windows to set up and administer the thing. When even many versions of Windows can't access the device, it's a junker. I took it back the next day, and will start researching hardware purchases more carefully in the future.
In short, Netgear's short-sighted decision to use some strange proprietary protocol instead of SMB turns this unit from something I would have strongly recommended into that gets a definite thumbs down.
After the recent debacle involving even high-end CD-Rs only lasting five or so years (even though they were claimed to last 100 years), you should be careful about trusting those DVD-Rs. Do you take care to store them in cool, dark places?
Cyric Zndovzny at your service.
I own the X6 and love it.
- It's GBE is very fast.
- It supports raid-5 with up to 4 drives. (mirroring on 2 drives)
- You can just keep adding bigger drives. so it'll be highly expandable down the road.
- Supports SMB, NFS, FTP, etc.
It's $600 for the unit with no drives.
Check out the toms networking review, it's linked from Infrants site.
I discovered something the other day which sounds perfect, you can expand it almost infinately to hold as much data as you want, although on the downside the upload takes a while... Still, it never crashes - not even in a thousand years.
the sales guy said it was called "paper"... strange thing
*''I can't believe it's not a hyperlink.''
What exactly are you worrying about - and will RAID protect it all? I think maybe not. Some things RAID will *not* help with:
s /
:-) At least Seagate give a 5-year warranty, which suggests you might reasonably get a 5 year MTBF.
1)Theft of the machine
2)PSU failure in the machine (this happened to me, and fried every single drive with 240V on the 12V rail!)
3)Lightening (could kill every machine in your house)
4)Fire.
5)HDD failure.
6)Catastrophic OS failure (filesystem corruption, conveniently mirrored), or a worm/trojan/virus.
RAID does give you convenience, slightly better performance, and ease of repairing the most common fault. But it sounds to me as though reliability and backup safety matter more to you than a few hours of downtime. My suggestion:
1)Don't use RAID; use separate machines. [Maybe mini-itx ?]
2)If possible, put one of them somewhere else.
3)Rsync + SSH - see here: http://www.mikerubel.org/computers/rsync_snapshot
4)Offline Backup (CD ROM, or external HDD) in safe place (eg bank) for really important documents.
5)If you have remote backups, make sure you encrypt them. If you dispose of old CD-Rs, destroy them first. Likewise old HDDs.
P.S. I've had a lot of HDD failures over the last 5 years (Mainly IBM Deskstars). 2 years ago, I switched to the Seagate Barracudas en masse. So far, so good
This is what you should put those drives in
5 SATA bays, hot swap, fits in 3 5.25" bays, dedicated fan that will pull 100% of it's air past your drives. Oh, and it comes in black, too.
And use a RAID controller that allows automatic rebuilds using a hot spare as well as online capacity expansion. I've had good luck with Raidcore/Broadcom controllers. For 500GB, use RAID-5 with three drives, hot spare in the 4th bay, and room to add another in the future.
As far as cost - auto racers have a saying: "$50 helmet for a $50 head" - what's your data worth?
In contrast my brother-in-law just built a file server with four 250gb SATA drives configured in RAID-5. He bought all new equipment, motherboard, processor, gig of ram, drives, power supply case, etc... He spent considerably more then my $230 (over $1100) but he now has a terabyte at home. It was important for him to have not only the file storage but a location for server type applications to run.
I could have purchased a terabyte NAS for $870 or so but I didn't see the need. The only downside I have found with a NAS is with my laptops. At home they can connect up to the NAS and get access to my photos, music, documents, etc. On the road however, they are not very useful. I've now had to start classifying what information I need with me at all times, and what things I can live without. If you have a VPN solution that will allow you to access your home network at a decent speed this may not be an issue for you.
Having data on a central server also has implications with the software you use. For example you will need to decide where iTunes stores downloads, local or on the NAS? If it is on the NAS then you can only download new music when you're connected to your home network. My best recommendation to you is to look at how many people access the same data in your household and use this to identify who would be effected if they didn't have their data with them all the time.
One more option is using your NAS primarily for backup. This reduces most of the concerns above.
I found that a four hard drives in a RAID 5 worked well in Fedora. If you get a board that has four SATA connectors on it, you can use them for your RAID 5 and have n/(n+1)=3/4 of the hard drive space available. You can then also use PATA for the OS drive and keep the RAIDed storage physically seperate. I used the built-in software RAID in Fedora.
You can use less drives for storage, but then your RAID 5 becomes less cost effecient. With two drives, you only get 1/2 the space. With 3 drives, you get 2/3 of the space. I found 3 or four drives is a nice compromise.
Power was one issue. You need to have a good power supply to be able to spin up all those drives at bootup.
The other issue was heat. I like the Antec Super LANBOY cases for airflow. They have 120mm fans for the front and back, and the air travels right over the hard drives. The hard drives also get mounted with the the longest edge parallel to the front of the case, which makes it easy to get at the connectors.
I combined this with samba (also acting as a primary domain controller) and it serves our small office.
My file server recently kicked the bucket. The good news, I had a mirrored array. The bad news, it was the raid card that died. (Thank you promise!) So My ext2 partitions got thouroughly gnarfled. Needless to say, I didn't get 100% recovery, but I did get most of the important stuff back.
That having been said, a decent RAID card and a motherboard with RAID onboard are about the same price. Now a chip and other hardware for that board will cost more, but you can still get a really good RAID-5 system for ~ $500, excluding hard drives (my preference is the A8N-SLI Premium). Add to that a few hard drives (any size you like, I'd go with 300Gigs or bigger) and maybe some swap kits for easy maintainence, and you're good. The swap kits usually have the benefit of fans, but make sure you don't go cheap, otherwise the fans are basically for show (not to mention LOUD!)
--Not to be worried, Pitr fix.
It's around $70-$80. Uses external usb2 drives. Doesn't do raid but you can get an extenal raid disk if you want that.
Easy. Just get yourself a basic box, connect it to the internet on a 100Mbps pipe, and get yourself a free 7TB of storage, with no risk of data loss due to your drives failing!
Reality has a conservative bias: it conserves mass, energy, momentum...
Parts:
- some usb enclosure (Belkin makes awsome ones, with fan and built-in powersupply)
Install the OS.Attach your drives
mdadm and lvm2 get familiar with them, you will use the man pages and google.
The only quirky thing will be the scsi emulation for usb under linux will send a REPORT_LUN command, that causes some enclosures to puke. Google REPORT_LUN usb resets or check this link for the basics to disable lun checks scsi-scanning
While you're out slapping drives into beige boxes to stuff in a closet, think hard about checking out ZFS running on OpenSolaris.
You're a smart guy that knows how to use Google to look up the fuzzy bits, but if someone dropped 250-500G in my lap, ZFS lets me manage it in a much more dynamic way, and more simply, than most other solutions.
I use external USB2 drives for backup and archiving. You can easily get them up to 400G these days. SATA drives and bus-powered USB2 drives are other options.
I think network attached storage just isn't worth the hassle.
Hello,
I'm looking to make a full diskless desktop client. That shouldn't be much of a trouble for linux. But how should i do it for windows?
I plan to do it over a Gbit network, but no windows terminal server, it 'must' be some virtual network C: drive, so i could benefit of all video acceleration.
I'm thinking to some kind of linux disk server, i've read about iSCSI & iBoot, but i'd like to have a free & software (linux?) based solution.
Any recommendation?
Dear CmdrTaco, I am sorry to have to announce this to you, but honestly I just don't care.
Love, this great guy.
Between home and work, I've done three setups like this. One has the optional battery backup module, which I think is worth it if you have the extra $100/US. All have been solid as rocks. I use Lian Li drive drawers and Silverstone black aluminum cases for eye candy. The drawers are a nice touch. Thus far I've used Windows 2000 as the OS, AMD processors, Seagate drives (5-year warranty!)and Abit NF7-S mainboards. Lots of bang for the buck, and nice to look at as well. At some point, I want to see how the 3ware cards do under Linux - - I hear that they are fully supported, just haven't gotten around to trying it yet. On the 8-port cards, I use a mirror for the OS and the remaining six ports for a RAID5 setup. The fans should be of good quality, as the heat is substantial. You can use multiple 3Ware/AMCC Escalade cards in one installation for really big storage.
It's only funny until someone gets hurt. Then, it's hilarious.
From Inventgeek. Seems like it would do the trick.
You may also want to consider offsite storage of this array. It's one thing to loose a disk but what are you going to do in the case of (fire | flood | other natural disaster).
Maybe you could set up a cooperative type situation with another geek friend and rsync your arrays to each other's house. That would be solving two data storage issues at the same time.
I'll create an amusing sig when I have something meaningful to post.
ReadyNas X6 is very nice. It has support for upto 4 SATA drives and can grow the raid array if you want to only start with 2 drives. I would recommend this with the SATA 400GB western digital raid drives.
I went with a 3ware 7506-12 PCI-64 card.
It supports up to 12 parallel IDE drives.
For drives, I did the rebate thing, kept buying Seagate 300GBs over a few months
whenever Frys or some other shop had them for under $100. (they can be had for that now without rebate)
I have a Dual A2200 Motherboard , Tyan Thunder K7 S2462 with 64bit PCI slots.
Right now I've got a 6 drive array that dupes all my individual drives, I'm testing
the system for reliability and stability before I move live data to it.
I will have to move this to a bigger case when I add the 2nd set of 6 drives.
The controller can only handle 2TB per array, so I'll have 2 RAID-5 arrays on it.
I'm going to get a Coolermaster Stacker, it will fit 12 IDE drives using 3 of their 4 drive modules. Each module has a 120mm fan for cooling.
For power I'll upgrade to a Dual 12V Ultra 550 supply. It has 2 12V supplies of
17A each, enough to spin up 12 IDE 7200 drives and run the CPUs. If that supply
gets too hot, I can put in a 2nd ATX supply in the Stacker. I havent seen any sort of failover supply option for ATX from facter, that would be nice.
For OS.. well, right now it's XP, mainly because that's what the system was running before I put in the RAID card. But it does have a few things in it's favor.
It will network well with my other Win32 machines.
It works the the OSX machine as well.
I figure now that I have the hardware faults mostly covered with the RAID5, my
biggest risk is filesystem corruption. The tools for XP recovery are well developed
and several vendors offer file recovery tools. For Linux EXT3? what happens if
something blows up the filesystem? I dont want to be picking around in a 2TB drive array, trying to pull files out one node at a time.
The XP will of course be severely neutered, no Media player, no IExplorer.
This system wont have direct access to any outside network.
I'm thinking of setting up a 2nd network of Gig-E links on a different IP range
just for mass storage. Most systems/users will only have read access since this
is archival and media storage.
Starman97@Gmail.com (bring it on spammers)
Although I've only got a pair of 80 GB spindles, its supported out-of-the-box by debian and redhat... And last I checked you could put 4 x 200 GB spindles in a single standard case... Sounds like 400 or 600 GB to me (raid 1 vs raid 5)...
Maybe you'd like to try that with Ubuntu?
We just got a couple Buffalo TeraStation units at work. The software that comes on the CD is a peice of junk, but the unit itself seems good. The major drawback I've heard about it is that it's really slow in RAID5 mode. Not too big a deal for us, as it's a cache sitting in front of tape, so it's still a faster backup medium. It's obviously running Samba in the background, but it doesn't support NFS mounts. I don't know if that's a big deal for you or not.
The other company I've heard about is Infrant. Similar setup to Buffalo, only instead of being mistaken for a Bose subwoofer, it looks like a small radio circa 1920. It claims an impressive set of awards, but I don't know if it's any faster in the RAID5 department than Buffalo.
But, for home backups that are occurring overnight, and if you're not pushing 100+GB at a time, you're probably good with either. They're both, depending on capacity, between $800-$1,500.
I used to be a live alone geek, back when I first started reading /. I loved to mess around with Linux boxes and play with my NeXT. But, eventually, even geeks get lonely, and I married a wonderful woman who just happened to be a Mac fan. On top of this, I don't play well with Wine, but I love to play computer games, which makes me run Windows too (Yeah, yeah, I know.)
We take way too many pictures with a digital camera of our boy, and needed a central place to store them, I like to edit them with gimp, while my wife likes to edit them with Photoshop on her MAC, we needed a central place to store them, and music, and e-mail and backups... and... and... and...
So what did I do? I bought a vintage Dell with a top-of-the-line Pentium 2 233. I slapped in a pair of 80g hard drives and ran Samba as well as netatalk on my vintage 7.2 SuSE. No muss no fuss and everyone's happy, I even have the 4 year old storing pictures he makes in his own folder.
A happy family story, brought to you by Samba, SuSE, and netatalk.
You can hook up several hard drives (or other usb toys) via a usb hub. Performance is not great, but totally fine for storage of music and movies if you only have a few users on your network. It supports samba, ftp, nfs, http, probably any other way you'd like to access the files. You could do software raid or some other type of mirroring/backup if you'd like.
The main reasons I really like this thing for an at home server:
I was amazed at how quiet my office became after replacing my PC file server with this guy and PC firewall with a wrt54g. I could actually hear the gf talking again, which is the only downside so far.
-Lod
Lightweights.. Try 3.5+ TB
You know that just about have to lure out the guy with that 200TB array I remember reading about there on slashdot. It's like mentioning low uids, and suddenly all CmdrTaco's boyhood friends show up.
Live today, because you never know what tomorrow brings
1. Buy a large tower case. Or use an old one. Whatever. Make sure there are lots of 3.5" drive bays.
2. Put in some kind of crappy, low-heat motherboard and CPU. Use the Celeron 300A you bought back in 1998. Whatever. Pop in 128MB RAM or so.
3. Buy a large, name brand PSU. Enermax, Seasonic, PCP&P, something like that.
4. Put in some kind of crappy boot drive. The 10GB drive that probably went with the Celeron 300A will be fine. Load Linux or Windows Server. Whatever makes you happy (yes, Windows Server will run on 128MB, especially if it's not doing anything but serving files).
5. Install a multiport IDE or SATA controller. Sil, Promise, Via, whatever. They're all OK. You want to be able to handle at least four drives. I prefer SATA at this point, 'cause I like big drives.
6. Speaking of big drives, 250GB disks are dirt cheap. Buy four of those. I prefer Samsung and Hitachi drives. We're using spanned 250GB drives 'cause 500GB drives by themselves cost four times as much.
7. Configure some a nice spanned, mirrored volume (RAID10 or the like). Two copies of a 500GB volume will be just fine. I prefer to use software RAID, in case I have to move the disks to another machine that doesn't have the same controller, but if you have a hardware option for RAID10, more power to you. Remember that RAID mirroring doesn't protect you from your own stupidity and cheapo PCI disk controllers never do RAID volume management.
8. Or don't mirror, and just use the second volume as a backup destination for the first.
9. Stick the resulting PC in a closet someplace. Administer with VNC or SWAT or RDP or whatever makes you happy.
Total cost for this project is probably $500 or $600, almost all due to the hard disks.
Alternatively, you could use an NSLU2 + a 500GB drive in a USB enclosure. That would also be a $500 setup, and there's no redundancy there.
-- I wanna decide who lives and who dies - Crow T. Robot, MST3K
I bought a $79 NAS device - a Hawking HNAS1, which runs on embedded Linux and works with both Windows and Linux systems (and I would guess Macs). It goes on my network, and gets mounted locally via NFS (on Linux) or as a Share (on Windows) (Linux can also access it via Samba, in addition to NFS, but NFS is faster). The Hawking comes without a drive, so you can put in any IDE drive. You then set up cron jobs to run scripts that back up important files and databases, rotate the archives, and save them on the NAS device. Also, for really critical stuff, another script breaks up the archive into 6 meg chunks and sends it to gmail (where each backup job comes out as one thread). I use a 120G drive in the NAS device. Stuff that doesn't change (music, photos, old docs) periodically get backed up to DVDs. Stuff that actively changes gets backed up onto the NAS device and off-site email. No point in repeatedly backing up stuff that never changes. The automated email part also works well, but obviously has very limited capacity, and constantly has to be manually cleaned out - but for really critical stuff, it is great.
The Hawking is somewhat primitive and quite slow, but it gets the job done for me. I don't pay any attention to it, yet it has worked witout a hitch, completely in the background, for about a year. There are much faster NAS solutions out there, but most of them require Windows (or Samba).
Of course, this is all for home/SOHO tinkering. At the enterprise level, there are truly sophisticated systems involving mirroring, snapshotting and all kinds of stuff like that.
I built a Raid 5 box with 900GB of usable space using a 3Ware card and 300GB HD's. Aside for patching, I haven't had to take it down in 2 years. 3Ware makes a very stable (and inexpensive) hardware controller.
100 gmail accounts, GmailFS installed on your PCs, and highspeed innuh-net! ;-)
http://forums.phoenix.craigslist.org/?ID=37426515 From my Craigslist post # 2 and # 3 just got 750GB drive upgrades, with my Media Center, I am now sitting at over 2TB of storage. In the end, it has cost me about $7500 dollars for everything and about 3 years to put it all together.
"Hey Gary, why are we wearing bras on our heads?"
PC: Get yourself one of those crazy cases with room for 6+ harddrives and a decent power supply. Decent CPU, but lots of ram (for caching)
OS: Linux of some sort.
Raid Hardware: 3ware or some other equivalent
Network: 1-3 GigE network cards (I'd go with intel) something that support 802.3ad (provied your switch supports it as well)
Drives: biggest SATA drives you can get your hands on.
Yes Francis, the world has gone crazy.
My method is not the geekiest way but it works great, is reliable, is cheap, and not bound to any specific hardware.
/home is just tarred and compressed. I even have enough space left over on the "backup" drives to allow my Windows and Linux workstations to backup themselves to a computer specific shared directory setup via Samba (share is path = /mnt/hdh1/pcbackup/%m). Depending on the space, you can even rotate the resulting backups just like the system logs.
Back up data one HD to another on a schedule that fits your need. You can use rsync or if the data can compress well use tar with a compression option.
I have been using this method for years on my home network and it has saved me at least 3 times. My Samba server has 2 200GB drives ($90 each now) that get synced up in one way shape or form with other drives in the system (2 other 200GB drives and a 1GB drive that contains the entire OS) via some basic scripts run via cron. An example being my mp3 collection uses rsync and my
I know, not sexy, not chic, but for a reference point of under $200, I have 400gb of backup that is very flexible and not tied to any specific hardware or software raid setup and config. I guess I could expand it with another elcheapo PCI ATA card and more drives until I hit a limit with the PS. The machine is only a headless P3 so I have some power and space to spare.
Bad boys rape our young girls but Violet gives willingly.
Problem for the home user as much as the data center is MTBF and energy consumption. A constantly spinning disk will wear out quicker and consume plenty of electricity in a year. By the time you have 5 or 6 drives all spinning away in your RAID system you have a hot, noisy, hungry thing. Not what most people want at home. Luckily I was involved in a project last year on building Petabyte scale storage. One of the things I learned about was MAID (Mostly Idle Arrays..). You switch on the disk when you want to read or write from it. Unfortunately not many standard consumer drives support true full power down, especially over USB2 interfaces which just piggyback onto a partial implementation of IDE. At the BBC they have an old fashioned but elegant solution, they use 10 quid plastic caddies and pull the disks to a shelf in the basement. I decided to go one better for my home backup system and implement a real MAID array. It only has 3 disks so far but the extra cost is a 12v relay per disk and a parallel port controller in the host. Requests to LVM blocks in a certain range get intercepted and power up the disk, mount it etc... You have to tweak some other stuff (including a few lines of source and recompiles) to get the timeouts to all work properly, and you have to disable ext3 auto checking fsck so it always powers up in a guaranteed time. The next step is to make it all boot and run from a flash device
on the main controller because at present I need at least one disk running. Anyway it's a good idea to separate your bootable rootfs from the actual data storage.
Not RAID, but on my network I have a Linkys NSLU2 with two identical external USB harddrives. It has built in SMB support and can be configured to do a full drive backup daily. Its not the most scalable solution but it is low cost and easy to maintain.
The NSLU2 runs Linux and can be hacked so that you can setup more complex cron jobs to do your backups, if desired.
I have the Thecus N4100. 4x 400GB SATA Drives, RAID5, 1.2TB Storage space. and now that seagate have the 500GB Drives, I may have to upgrade to 1.5TB RAID5! hehe.. only issues with it, and I'm not the only one who thinks so, is that the data tranfser is slow. doesn't even utilise a 100Mb NIC let alone the Gb NIC it has in it. And that it doesn't have removable & redundant PSU's
----- Concentrate on promoting more than demoting.
I went with a RAID 5 on 8 x WD 80GB ATA-100 Drives on Fedora Core 3. It was fast and about the size you are shooting for, although little room for expansion. My issue was that my power supply didn't provie enough power plugs. So being a cheap skate I bought Y-spliters to adapt. This caused a few of the drives to crash as a result of shorting. My issue was that that RAID mounted on boot, when the RAID crashed I have no flipping clue how to get Linux to activate the RAID 5 functionality to restore the drive(s) from the strippings. This might be something to consider, also find out how to use RAID 5 under Linux/UNIX and practice a fail over before putting critical data on the RAID. Nothing like feeling totally safe until the inevitable happens and being up $4!t creek.
I've recently build myself a "Poor Man's RAID Array" out of 8 x 300GB drives in software RAID5. After reading about Sun's ZFS, I'm wondering if there's a way to do something similar to their "disk scrubbing" with Linux's md raid.
Basically run through the array and check each stripe against it's parity, and re-write (to attempt to fix) the stripe if it doesn't match.
At least it's something to think about, since two disks could get a small error, each on seperate parts of the disks, and then the RAID5 rebuild could fail when you try to rebuild it after a disk fails, even if there is enough data to reconstruct the missing parts.
I checked the list before posting, but yes, the Infrant boxes are the best. Go for the X6 - it adds external disks via the previously existing 2 USB ports. In addition it supports rsync, and will backup itself if requested.
1 box plus 4x320gb NCQ WD drives is ~$1,200.
For a home network a 500 Gig or so drive in a USB-2 or Fire-wire enclosure might make a good backup system. Just copy your data over every night with a cron job and your all set.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
http://lime-technology.com/unraid.htm
They're just starting out and they're targeting the HTPC crowd. There's a HUGE thread on the AVS HTPC forums about it. So far it seems almost everyone is happy with theirs. The cool part is that you can add hard drives on really easily, so it grows as you need more storage. Pretty affordable considering the other options.
From their website:
Lime Technology believes that there is a market for a new wrinkle on the old RAID technology. Our approach uses a storage organization we call unRAID(TM). unRAID(TM) is similar to RAID-4 in that for every n hard drives, there are n-1 data drives, and a single fixed parity drive. Unlike other RAID organizations, however, files are not striped across the data drives. Instead, each data drive is formatted normally with its own file system.
You have enemies? Good. That means you've stood up for something, sometime in your life. --Winston Churchill
Yeah, I like 3ware's stuff. I've got a 3ware 9500 series controller pushing four Maxtor 300GB units around ala SATA on FreeBSD 6.0 with a stripe set (parity) and it's been great. The wife and I have a common music directory, a repository of iso images for every piece of software we own, and home directories along with a partition for storing ghost images of our machines over the LAN and it's been awesome.
Watching some auctions such as Dove bid you might find something like this http://www.dovebid.com/assets/display.asp?ItemID=c pt49558 to keep you warm in the winter. Nothing like 57 36 gig drives and some fiber for your diet.
Well I have one dedicated Linux server where I have
200$ cool looking server box (2 power units), cheap motherboard with SATA controller cards, 8 x 250GB Maxtor HDs (the more expensive ones with 60 month warranty), software RAID 5 (MDADM) + LVM + SAMBA + NFS. + I have other system with 2x250GB (RAID 1) which do other tasks + provides secondary backup of the most critical data.
This way I feel quite protected against software/hardware/user stupidy failures.
You can still lose all your data:
- thieves
- house burns down etc.
Data encryption prevents thieves to get your data (crypto devices), to protect against data loss it would be wise to setup duplicate backup server at some other place with (fast enough) network access.
PCI SATA controllers+software RAID+LVM+encryption combination is quite slow (you can notice this) so for faster access HW RAID can handle everything else except LVM+encryption (maybe some support encryption also).
I had terrible trouble getting a SATA RAID 5 array set up on my home server, a dual-processor box running a slightly old Linux distro (SuSE 9.0, 2.4.21). I went through several controllers. First, a HighPoint RocketRaid; it kinda worked, but the driver was buggy on 2.4 SMP kernels. Then an Adaptec, but that was a non-starter as the driver is not open source and they did not have one built for this kernel. Then I just got a Promise SATA controller and ran software RAID, and that worked for a couple of months, but I ultimately saw filesystem corruption, which I attribute to the driver, again, not being fully debugged (maybe the SMP was again a problem).
Finally I gave up trying to use a low-cost controller and picked up a 3ware 9500S-8. This seems to be working fine. (The reason those controllers are so much cheaper than the 3ware is that they are basically software RAID anyway. The 3ware, and other high-end controllers like the Areca, do much more work (like all the XORing) on the card.)
Also, on a different machine (this one 64-bit) I just wanted to set up a SATA software RAID 1 pair, and I had trouble with that also, until I installed a very recent kernel (2.6.14).
My conclusions:
BTW the 250GB drives have the lowest price per byte, or did when I last looked.
Good luck!
Your god may be dead, but mine aren't!
1TB Raw, 750GB Raid-5, GigE, easy, cheapter than what Dell sells even without rebates. Works fine out of the box for "home purposes", but it's linux based and you can always go in through the back door and fine tune their samba config. They have the original image in flash if you screw up (I've never tried it personally). It's pretty 0 maintenance which is exactly what I want at home - get it working, set up a structure, and don't think about it.
You can always back it up via another terastation (yeah, right), or portions via a USB hard drive.
Redundant Array of Inconvienient Duplicate (stories)
:)
I spent quite a while trying to find a perfect RAID NAS device for home use. Everything I came across was either ridiculously overpriced or grossly underpowered.
:)
My basic requirements are:
RAID 5 support
Extensive HTTP based admininstration
Samba, NFS, rsync share support (browsing files over HTTP - a plus)
Gigabit Eithernet Interface
20MB+ Read/write speed
Support from 1 to at least 4 SATA disks
Disks should be easily swappable
A few month ago I finally came across Infrant's ReadyNAS X6 box. Specs read like just what doctor ordered - everything I wanted seemed to be there. I got it and after 3 months of use I am not disappointed. I purchased 4 300GB Maxtor MaxLine drives and got about 850GB of NAS disk space. I use it as a primary storage for MythTV, backing up two laptops [rsync], and (obviously) the rest of my data which is now much safer on RAID. The box runs Infrant's custom Linux distro and (I think) Motorolla 350Mhz CPU. It has a dedicated XOR chip. Array upgrades are seamless - you can start with just a single disk, then to RAID 1 (add another disk), then RAID 5 (3 and 4 disks).
The only thing that I was hoping would be better was write speed - I get about 15MB sequential write and 25MB seq. read speed. After some digging, I get a feeling this is actually a problem with network card not being able to keep up with packets. If that's the case, I might be able to pop another network card in one avaliable PCI slot.
As far as price goes, Infrant's box and 4 300GB drives cost me under $1K USD which seems quite reasonable. I highly recommend taking a look at this unit if you are considering purchasing NAS.
BTW, I am in no way affiliated with Infrant, just a satisfied customer
"You mortals are so obtuse." -Q
Perhaps you should look into an Online backup solution, the costs are very resonable for such critical data. I would recommend http://www.backup-live.com/
Huh !!!
/. you may ask from Jobs to send one of these beauties...
After that much Apple hype on
http://www.apple.com/xserve/raid/
[My english is better than most other people's Turkish, so please point out mistakes politely. Thank you.]
As has been pointed out, you have more money then the average slashdot reader, so I'm going to post a ghetto solution for the rest of us.
Linux, SATA, software raid5.
Promise SATAII150 TX4 PCI SATA
Motherboard with onboard SATA, onboard video, onboard 10/100
AMD Semperon 64
512m DDR RAM:
6x 320GB drives
case/powersupply
$1144.39 with shipping at newegg, 10 minutes of shopping.
Total storage: 1.5 gig (1490 * 2^20)
Software RAID5, LVM2, NFS + SMB/CIFS + WebDAV (apache2) exports.
For home use, it's perfect. Sell one extra google stock option and
add on gig-e, DL-DVDRW +/-, and a gig switch for your network.
I'm running a similar setup, except it's a full AMD64, more ram, bigger
video card and it doubles as my always-on linux workstation Smaller drives,
though, I bought it last year before the 320s came down to $140 a piece.
So now you know how much a 1.5TB home storage array costs. Don't
lose data again!
I just built my own 1TB server, so here are my thoughts on the process.
You could probably find an out-of-the box solution that will work really well for $800-$1000. However, it will likely not be as configurable as you would like.
My server is 4 320GB Western Digital 7200 ATA drives, a Duron 2800+ processor, 1GB RAM, and a Promise IDE controller. There is also a gigabit ethernet card in the PCI slot. All four disks are set as master in their IDE channel. I have an extra fan on the hard drives to keep the whole thing cool, and it runs at about 95 degrees F. I built the whole thing for about $750 from NewEgg, including case and power supply.
The system is running Gentoo Linux and software RAID (1 and 5), sharing over NFS to all my other boxes. Setting up SMB would not be a big deal.
SATA or SCSI drives would be faster, but they will also be more expensive, and honestly, there is not much I can't do in realtime with ATA.
Processor and memory are not a big deal if the majority of data transfer is from read operations on the disks. It's only when writing data (or reading data from a RAID system with a broken disk) that the processor/RAM is utilized heavily, due to all the XOR operations that have to be performed.
Putting more than one disk on a single IDE channel (master and slave) will cause a HUGE performance hit. Also make sure that DMA is enabled on all the drives.
I RAIDed all the system and data partitions (except swap, of course -- that would be a speed hit), so that the loss of any one drive would not be enough to take down the system.
Data throughput is all a matter of priorities: will you be copying large files back and forth all the time, or will you be mostly streaming data by playing/recording music and video? If you're just streaming, a plain ol' ATA100 over PCI and gigEthernet is plenty wide for sending 12Mbps MPEG-2 video realtime, with bandwidth left over for playing music. If that's not fast enough, start upgrading at the bottleneck (PCI->PCI-X, ATA->SCSI, Ethernet->GigEthernet, etc.)
Good luck to you.
For security, the MD5 hash of this message and sig is 09f911029d74e35bd84156c5635688c0.
1. Many solutions did not support and RAID 2. Closed-source meant no ability to hack/upgrade hardware 3. In some cases, inablility to upgrade disk size.
My solution was to build a PC with 2 SATA disks and install SME http://contribs.org/. I wanted the additional benefits of a NT domain controller with roaming profiles and a VPN endpoint, but these were just added benefits to the file service. Under the hood, its just a suite a packages sitting on CentOS. The 2 drives are mirrored using LVM (which was sufficient for my needs). You still retain 'root' access to the host but can manage most tasks from a webpage. I find it to be a near-zero maintance appliance after the fairly quick & painless install.
Another solution that I look at was OpenFiler http://www.openfiler.com/ which also appeared to have good file service (including snapshot capabilities). I decided on SME purely because of the 'extras'.
LinkSys NSLU2. Plugs into your home network. (10/100) Then you get yourself 2 IDE drives and 2 USB 2.0 enclosures then plug them in. Then you can set it to periodically back-up one drive to the other. Sure, it's not as bullet-proof as RAID5. But it's dead simple, cheap, and it just works. Failure recovery is dead simple. Also, the system is has some of the same flexibility as the Buffalo Teraserver. (Plug in your friend's USB 2.0 drive when he comes over.)
Also, with this scheme, you can delete a file and change your mind. (Recover from the back-up before the weekly copy job.)
And, if this is too simple for your geek quotient, it's Linux-based and hackable!
I'd agree that the 'best' solution is to set up two file servers and use their on-board disk controllers and mirror them to each other. But I'd also recommend using DVD as storage. Less heat issues.
Two servers; Have one be a primary file server, and allow the other system to spin its data storage disks down until they're needed to back up the primary server. The secondary server can be available for other network server needs. Maybe your desktop system would be the 'Primary' server, and a system in a ventilated closet would be the backup server, but with three or four drives, noise becomes an issue in desktop/deskside systems.
I've worked in a disk storage solutions division for 15 years and have seen a wide variety of 'solutions'. I think mirrored file servers is likely the best idea. Pay attention to power supply quality, and to case ventilation. Mount the drives flat, not on edge. I prefer Seagate disk drives. My newer IBM drive crapped out recently and I lost data. The rather old Seagate drive is still doing fine.
Best regards.
I use a couple Windows Server 2003 computers running Distributed File System which replicate data between themselves whenever there are changes. I know there are some drawbacks when using this setup, but it has worked out great for me. Honestly, I have yet to see a failure on the hard drives that run the DFS, so I am not sure of some of the negative consequences. But one thing that I like with it is if one computer goes down for some reason, such as reboot (which we all know can happen atleast once a month due to patches), or a hardware failure, the other system will take over. It is not like a seamless failover, like what you would see with a cluster, but it works for me. I know it is not like having a single-system RAID solution, which requires less hardware, but for what I do it is nice because I need both systems anyways for other services I run.
I did try to use OpenAFS on my servers at one point, but was having some trouble getting it to do what I needed. Plus I read had major performance issues on Windows at the time, but that was over 2 years ago. I know it is probably alot more reliable and efficient on Linux, but I didn't have the chance to test it out with my setup.
Can anyone suggest another distributed file system solution that works on Windows and/or Linux?
I just recently upgraded the case on my Linux Software RAID setup (see pictures). At some point in the future, if I have the money, I can get six 500GB drives for a 2.5TB RAID 5. As it is, I'm content with 80GB RAID 1 for now, but will be adding a third controller card and 80GB drive for a 160GB RAID 5 in about six months.
Here is my setup for a 1.2TB JBOD array.
4x Maxtor MaXLine III 300GB Hard Drive
4*$132.70=$530.80
Asus A8N-SLi Premium Motherboard
$177.59
AMD Athlon 64 3200+ Processor
$168.80
GIGABYTE Geforce 6200TC GVNX62TC256DS Video Card
$58.63
Dvico FusionHDTV 5 Gold Plus TV Card
$168.60
Total: $1,104.42 (including tax and shipping)
It duals as my PVR and fileserver. I could have made it a 900GB RAID5 array but then the disk wouldn't show up as 1.12TB; which makes my epenis huge.
The human race is artificial intelligence created using object orientated programming.
However I did make the mistake of not using LVM to set up the drives so adding drives is going to be complicated. (I also did a poor job of partitioning which isn't helping my cause.)
Since I built this there are now multiple cheaper solutions w/ at least a TB of storage. Little appliances you just plug into your network. I think they are the cheapest, quickest, and easiest way to do what you want. The second easiest given your low storage needs is to use the on-board raid chip on most new mobos and set up a raid 1 array with 2 400GB or 500GB drives.
But if you do go the full system approach, (which certainly lets you be flexible with how you share the data), Think about how you will expand the array when you need to, (both physically and partition wise). Think about the partitioning, (I basically have 1 for the OS and 1 HUGE partition with home media and temp, (basically everything I download), partitions that are that are bind mounted to where I want them. This means one partition won't fill up while the other has space. Think about how you will replace drives, (seagate drives have a 5 year warranty. Not a bad idea when you are making a system w/ 7 or 8 of them.) Think about power consumption of large numbers of drives. (I have mine on battery backup because the box is also my server. The poor APC XS 800 (800VA, 540Watts) can bearly handle the server and the cable modem.
And think about how you want to make the data available. I personally like being able to get to my data anywhere, (includes my email which is on the server). I make smb available inside the network and ssh inside and out. I tunnel X over ssh and run evolution. I however haven't found a good way for my friends/family to access my photos/music to play/download. I also use webmin to interact with it for the most part.
Also if you go the computer route, it has to be one of those computers you don't fiddle with. Install debian, set it to install security updates, and forget about it. The more you mess with it, the more likely you are to fubar it and the array.
Anyway, thats my experience. I have to say that I am VERY happy storing everything that isn't an OS or a program on a storage computer. I don't have to worry about my data dieing w/ the drives. I don't have to worry about it being inaccessable when I'm on the road. I don't have to worry about trying to find something in the "pictures" directory on 6 different computers, (some a "pictures" directory on multiple hard drives within them). And I don't have to worry about about 100 different physical drives running out of space!
I do security
Call me crazy but I need my data to be accessible locally most all of the time (usually from a laptop that can be connected to any nasty network). A big fat disk on my home LAN just isn't gonna cut it for me unless I use it to backup my workstations and servers.
/. and other web browsing.
The solution that I've found that works very well for me is using IBM's Tivoli Storage Manager (TSM) on a Linux server at home. Each night (my computers are on all the time) a TSM scheduler agent backs up every node on my network and keeps it on a big(ish) disk (just a plain ole ATA disk, no RAID, nothing fancy). Once setup it's pretty much hands-off except once every week or so I copy the "storage pool" that sits on the disk to DLT carts on my old slow stand-alone SCSI DLT IV tape drive ($100 on ebay). Then I burn a backup copy of TSM's database onto a CDRW (about to be either 2 CDRWs or a single DVD+RW) and store these at the office.
The way I figure it I can lose a disk on one of my workstations and recover from TSM (bare metal even on Windows though untested with the iBook). If I lose the disk on the backup server, I replace it and restore from the tape and database backup CD I made a week or so back and the next night my nodes will refresh the data that changed since that stale image. If my house burns to the ground (a possibility given that I live in a dense and dry forest in the Colorado mountains [well, foothills]) I still have a copy of all my stuff (albeit a week or so old) at the office. This solution is the only thing I've found that allows me to be mostly hands-off and just have it work. I invest an hour or so making my tape copies while I read
I know it's not exactly what you had in mind with the question but there really is no reason not to consider a real recovery solution that scales very well (I've worked in enviornments that backup whole datacenters with TSM), is fairly painless to manage (once it's setup) and is pretty reasonably priced (especially considering what it costs to lose something critical). I'm sure you can find an old server and can scrape a few bucks together for a decent disk (or array if you feel it necessary). It's well worth the piece of mind it offers.
So I know what 0, 1, 5, and 0+1 are, but not 6, 10 and 50. Care to enlighten those of us who are too lazy to Google?
+++ATH0
Data initially gets stored on my Linux server, a 60 gig IDE RAID-1 that I set up a few years ago using a platform-agnostic controller (I forget who makes it).
For a while, I would just periodically cut backup DVDs of my important bits. I'd leave a copy and home and take a copy to work, figuring if anything ever happened to destroy both copies, getting my data back would be the least of my worries.
Burning DVDs, however, turns out to be a pain in the ass. So, I went out and bought a 100 gig Seagate pocket drive. Backing up to this is far less of a bit deal -- I periodically drag the folders before I go to bed and let it copy over my network while I sleep. Then I take that drive to work with me. Combined with the occassional DVD backup, I figure I'm golden.
Every year during my review, I just pray the words "slashdot.org" aren't mentioned.
I'm running a Gigabyte Nforce 4 Ultra motherboard for my AMD X2 cpu. I have a RAID 5 setup on it with 3 drives. This motherboard doesn't have active cooling on the chipset, and the temp reading on it reguarly hits 70C. That bothers me, mainly because I'm thinking that if the motherboard dies, I'm SOL with my data right? If I understand right, I should probably get myself a dedicated SATA RAID card and dump the motherboard solution in case the board dies, correct? I have this board:
u cts_GA-K8N%20Ultra-9.htm
http://tw.giga-byte.com/MotherBoard/Products/Prod
This summer, I finally tired of fighting a constant maintenance, recover, and backup battle with seven different PC's in the family (my notebook, a print server, a LINUX test box, plus wife and kid's machines). It's worse than keeping a small corporate network up and running (where there is no gaming or downloaded "free" software). And we have some real $ tied up in the wifes audio books and the IPOD downloads. So I overhauled our network and backup process.
For backup, I installed a SNAP 2200 800 MB network attached storage device ($1,600, yes expensive but my time is worth it) configured as RAID 1. I'm partitioning each PC into operating system and data drives, then ghosting the operating partition. The data on each machine is backed up incrimentally and automatically to the SNAP server with the SNAP supplied software.
System works OK, Snap appliance is a little noisy (fans) but I put it in a large fireproof box in the crawl space so I don't hear it. File transfer rates are nothing to write home about. But I've got the peace of mind that the data is much more secure against theft, hard drive crashes, most house fires, etc. AND it's not just one more PC to administer, as it is a self contained appliance running a stripped down LINUX kernal. The capacity should hold us for a few years, then I'll probably expand with an additional NAS appliance.
10 and 50 are just the result of combining RAID 0 with RAID 1 or RAID 5. I don't know for certain what RAID 6 is, but IIRC it's someone's proprietary attempt at getting around some of the limitations in RAID 5.
"They redundantly repeated themselves over and over again incessantly without end ad infinitum" -- ibid.
Do you store your sensitive DATA in RAM?! That's bizarre. Or do I not get it.
"When the solution is simple, God is answering." -- Albert Einstein
It was originally Redundant Array of Inexpensive Disks. The term "independent" came in later when RAID came be be used more for reliability than cost savings and RAID vendors wanted to charge premiums for "RAID optimized" hardware. The term independent really adds nothing to the descriptions, they should just have changed it to RAD. Cause.. that would be totally rad.
-Ryan C.
Yeah still new, but it works great, is EXTREMELY easy to use, and you can setup samba.
My only "problem" is configuring 2 wireless cards at once, the ath driver seems to have problems right now. So I have to go through my router instead of having a dedicated wireless driver.
But I now have a wireless server (created from my old game machine that I don't use anymore) that will have a terabyte of storage and more all setup with raidz (raid 5 sortof).
Once I get these 500G sata drives I will be a happy camper. Once samba is setup my mac can see the shares easily, I don't have a windows box to test that area out with anymore, since the server was the only windows box I had left.
But you might not like Solaris, but zfs is awesome. I can't wait until it is in Solaris 10 general release.
How do you recommend planning for the eventual drive failure that happens in 2 years? Don't these raid controllers want the drives to be the exact same model? One of my drives is "failing" (going offline periodically but not dead yet). I bought it a couple years ago and don't think I can find it for sale anymore (at least new).
Seems like a home raid solution would want as a feature the ability to stick in incrementally bigger drives. Either that, or you buy 1 or 2 extra drives and figure that'll last you for 5-10 years and by then you'll want to swap the whole thing out with something that's 10x faster and 3x cheaper.
Yellow Machine
I read the review. I bought it for my office. Works really well.
The only problem I am having is with the windows backup software that was supposed to come with it. I only have 2 user licenses where it was promised 5. But with SMB, NFS and (i think) FTP there are many other solutions you could use besides the bundled software.
I just wanted to use it to make my life easier on the windows machines... sigh.
My only wish was that it spoke FireWire, as if it did that, I could in principle hook up some of those 1TB little FireWire arrays.
It takes a little bit of hackery to get the NSlug useful for Linux; you need to install some alternate binaries so that you can put it in as an NFS mount. It's pretty slick from there...
If you're not part of the solution, you're part of the precipitate.
At the moment, I'm looking at a NAS box from D-Link.1 5
http://www.dlink.com.au/Default.aspx?ArticleID=17
Throw in any 3.5" drive you want, set up whatever folders you want for whichever users, it has 2 USB ports so you can use one for hanging a USB hard drive off it for regular backups and another port for a printer, and (hopefully) the Sonos system will see it, saving me having to leave a Mac powered up for housewide music.
When they came for the communists, I said "He's next door. Take him away. Goddam commies."
The trouble with using expensive hardware like dedicated RAID controllers is that if/when they break, you'll have to replace them (ie. buy a new one of the exact same brand/type/firmware rev.) to get at your data. Not something to recommend for a home-setup. You're better off with commodity hardware and software RAID.
Also, it's probably not a good idea to set up a big NFS server with your data and have all your other boxes mount their filesystems there, since that leaves you with a nice big single-point-of-failure. Power-supplies will blow, motherboards will fry; stuff like that is inevitable. Not being able to get at your data will become a major nuisance then.
I'm currently using unison to keep my homedirectories on both my boxes synchronised. That works quite nicely. I'm not sure if it scales to multi-machine setups though.
Error: password can't contain reverse spelling of ancient Chinese emperor
This thing looks cool.
http://www.yellowmachine.com/products/index.htm
Half a TB of porn? I'd watch out dude, your gonna go blind
"The preceeding, informative post brought to you by Seagate! ...Seagate, available everywhere Maxtor is sold."
So I have an NSLU2 at home. Had it for about a year. The length of time the thing has been actually useful is maybe two days. Let me give you the counterpoint...
- Silent operation, no fans in the nslu2 and you can get fanless enclosures for the HDs
Make sure it's an aluminum case at least. And be prepared to try several different ones until you find one that works well.
- Takes very little space away from your home office
No, other than the six thousand cords you've got hanging off the back of it to plug in these external drives.
Oh, and don't accidentally disconnect a cord. The NSLU2 doesn't support anything approaching to Plug and Play. You'll likely damage data on the drive, but the most annoying thing is you gotta shutdown and restart the whole thing.
- Very small power draw
True.
- Easy to add/remove drives without any reboots
Not in my experience.
- Can power off drives that aren't used frequently, then turn them on when needed
Again, not in my experience. This is most likely going to lock up the whole thing so it stops responding.
The other problems with the NSLU2 besides the speed(might as well hook it up to a 10baseT hub, cause it can't fully utilize 100baseT), is that if you do try to transfer a large amount of data(say 15 gigs of MP3s) more likely than not the whole thing will lock up on you.
In short... The NSLU2 is unreliable, for a variety of reasons mostly having to do with software, but also having to do with the external drives and the lack of support for hot plugging USB devices. The NSLU2 is slow. The NSLU2 is a pain to manage on the table because of all the cords hanging out of the thing. The NSLU2 is not well supported by Linksys, they periodically release firmware updates but 9 times out of 10 they don't help. The NSLU2 is particular about what type of USB enclosure you use, as well as even what drive, so it's hit or miss whether it will work.
To be fair, I did look at buying a Netgear SC101, and everything I have read indicates that it's even worse.
I ended up just taking my drives and sticking them on my computer and leaving them there. I thought it would have been nice to have this running all by it's lonesome in another room with some batch scripts periodically replicating data over to it. But it's simply not reliable enough.
I've been meaning to try to sell my NSLU2 on ebay. Maybe someone who wants to install their own copy of the nslu2 Linux on it can have some fun. But it's not a good device for a SOHO server, that's for certain.
Format the filesystem that you intend to serve with Netatalk thusly
/dev/hdX (the "O" is an uppercase "O" as in "Oscar")
mkfs.jfs -O
This formats the filesystem with "OS/2 Compatibility" which means that it is case-preserving like HFS and NTFS. In other words "file" and "File" and "FILE" would all refer to the same file and only one of those variations could be used as a filename in a directory on this filesystem. Samba will enforce case-preserving semantics on case-sensitive underlying filesystems by default. If a Linux server will only be serving Windows clients then samba+Linux default case-sensitive filesystems is perfectly fine.
Netatalk will not enforce case-preservation and this can break Mac applications if their data is stored on case-sensitive underlying filesystems. Unless "case-folding" is used which forces filenames to be all upper or lower-cased, Netatalk simply lets the underlying filesystem namespace behaivor show through. From the Finder, you can indeed create "file", "FILE", and "File" in same shared directory if is something like reiserfs or ext3.
I've been experimenting with this for awhile and I've found that only JFS is suitable for the purpose. Firstly, it is actively maintained and robust when put into operation. Linux native utilities can create and fsck the filesystems. Performance is at least acceptable even if other choices like ext2 can be a little faster. Finally, the filesystem can be created to explictly support only case-preserving behaivor. HFS, HFS+, NTFS, FAT, and HPFS are other routes to attempt this with. Either the filesystem is buggy for read/write as in the case of HFS+ or support is incomplete like NTFS or bizzare things happen in the case of FAT and HFS(severe filename limitations here). HPFS works well but I have never succeeded in finding the necessary accompanying mkfs and fsck utilities for Linux.
The downside is Linux native scripts and apps that rely on case-sensitivity may break but those can be run on other filesystems if all else fails. The server needs to give the client what it expects. I have to support Mac apps that use data stored in a server share while still allowing human users of the shares to name files the way they are accustomed to naming files. This method supports both those goals well.
To my understanding RAID 6 is like RAID 5 with an extra parity block, so 2 disks can fail at once instead of 1. More info at http://en.wikipedia.org/wiki/Redundant_array_of_in dependent_disks#RAID_6
I use NASLite from server elements. I gotta say this is the simplest cool way to do NAS Small, sweet and fast. The free version runs on a floppy, and the commercial version can boot from USB (leaving 4 drives available for shared data) This thing is prety cool, and even if you don't decide to go for it, you owe it to yourself to check it out. small, simple, fast and doesn't require too much computing power. In fact, as a geek (like me) you probably have machines that are overkill for it sitting in your basement. www.serverelements.com
What doesn't work:
Dan
Add storage to a separate machine, and keep a backup copy on it.
My last hard drive failure was caused by a power supply failure. It roasted both the main hard drive and the backup hard drive in the system. Even if they had been RAID I would have been out of luck.
A company called Server Elements offers a port of Linux OS for a small fee that makes any old computer into a NAS. Unfortunatly at this time their are not offering raid support. With that said, this may not be for you. However I find it useful for backing up my data incase of a large drive blowout. I am hoping in the future they do come out with a version that supports RAID. Here's a link to them
There are also services like http://www.dataprotection.com/ that start at $125 per month (what's your data worth!) You don't have to worry about hardware or your house being wiped out by the apocalypse, it will still be available. They take care of everything including the electric that a 4-hard-drive-RAID-Pentium 4 machine would drain. You don't have to worry about upgrades or where to store the stupid machine so its constant hum doesn't keep you up all hours of the afternoon.
Thanks, because I don't know what I'm talking about and never claimed I did...
With your relatively small storage requirements, I suggest a 250-500GB drive with a cron job running rsync every night. I do this with my significantly smaller storage requirements. I have a drive in my system to which I copy via rsync /home/user1 and /home/user2 (basically, the entire contents of /home). The great advantage to this is if /home dies (it is a separate drive), I can simply edit /etc/fstab and mount the backup drive as /home. All my files are there. Since this is for home use, I cannot imagine that you have frequently changed files. If this is the case, nightly (weekly? monthly?) backups would suffice. This method provides you with a drop-in replacement without the need to actually "drop-in" anything.
Life is short; think quickly.
The problem I have with NAS boxen is that I feel like I'm buying another computer that I don't need. I figure the best value has to be an existing computer I already have stuffed with SATA drives doing some kind of software RAID. The problem generally ends up being "How many drives can I fit in the case?"
Which leads me to the next problem/question -- are there many E-SATA enclosures and controller cards that support E-SATA interconnects? It seems to me that this might be the next best thing -- external drives in E-SATA enclosures and a controller card that supports E-SATA. It should give you some portability if your server goes meltdown, and with software RAID you should get some data redundancy.
I also kind of wish that there was a USB2 "hub" that was actually a hardware RAID controller; plug in external USB2 drives to the hub and instant, portable RAID. The computer would see just the LUNs created by the controller, not the individual drives.
Matched drives give you better performance but they are not technicaly required. Some raid cards might have checked for this but none that I have worked with. 3ware specificaly does not use a chunk of a drive so that different drive sized can be accomodated I have a 4 drive 2 maxtor 2 WD raid 5 on a 3ware 95xx and it works fine. Cheep windows mirror and stripe software "raid" controlers probably have this issue but it should work fine putting a larger drive in to replace the failling unit, as there logic is a simple write every block twice and say your as big as the first drive or write ever other block to each drive and say yours 2x as big. If your using a "raid" card that cheap you might as well use software raid and get a better feature set (expansion, raid level migration, raid 5 support and sub drive arrays for starters)
No sir I dont like it.
Hi, folks :D
/dev/sda /dev/sda:
I'd set up an Adaptec 2810SA Raid controller with 8*160 Seagate SATA disks just a year ago. The budget, well 83 euro each disk, and 420 euro the raid controller. I've set up 2 LUNs in the controller, with a 64Kb stripe size . After that, LVM2 and XFS file systems. The system has 1GB or RAM, and 2xe1000 NICs. It has a nice througput for video streaming
The hdparm:
mini-me:~# hdparm -Tt
Timing cached reads: 2012 MB in 2.00 seconds = 1004.65 MB/sec
Timing buffered disk reads: 770 MB in 3.01 seconds = 256.19 MB/sec
mini-me:~#
I've been working with some peers (true friends, not just network peers) in spreading my personal hard drive across what I would consider a Wide Area RAID (WAR).
I have a huge pipe right now (soon to be about a 12Mbps pipe in beta). Some of my friends have Comcast's 6Mbps (download) pipes. We're all XPMCE and MythTV heads, so we have literally terabyes of audio and video files. My wife has 2 60GB iPods maxed @ 96kbps.
Why isn't there a better way to use torrent or somet other protocol and set up a WAR? I don't need to back up 99% of my data -- the stuff that I do need to back up I just drop onto a few other hard drives I have access to.
I believe a company Google will be the company to provide a service like this. I'd rather see the ability for millions of users to offer a % of their hard drive to share with the world -- anonymously of course. There has to be a reasonable way to send your files out to millinos of users (in packets) and still have the ability to recreate the files if needed. On my hard drives at one of my offices, we have about 60 drives of 60 gigs a piece, and I bet we're only using about 1.2TB of the total. The rest is wasted. I'd love to offer this space freely (for myself and for others) to be used.
Will Google do it? They're sort of there now with gmail. If they can find a way to tap into the mass storage that is out there (safely), and offer it to be shared and mirrored and copied in a redundantly safe manner, we could get rid of backups entirely.
I don't have the solution, but the idea seems solid.
I'm just using a cheap Linux box using software RAID with two internal 120G drives.
On http://rootprompt.org/ (one of my other home tabs) was this article posted on Saturday (1/14/06) which showed how to build a high performance, 0.5 TB, SCSI RAID array for less than $300. Here's the link, so go take a look, get out your tools, & have at it. http://www.inventgeek.com/Projects/PoorMansRaid/Po orMansRaid.aspx
I've never used it though. I wish it was part of md, so I could use it on a standard distro.
http://lime-technology.com/
There's a long thread about this over on AVSForum.com. LB
If Chaos Theory has taught us anything, it's that we must kill all the butterflies.
I would highly recommend against a RAID-5 configuration. You're not going to save much money over RAID 1+0 and it's a ticking time bomb. See here for a cost analysis, basically you save about 33.3% by going RAID-5, but the write performance is worse than a RAID 1+0 array.
Do the math for a multi-disk failure in a RAID-5 versus a RAID 1+0 system (let's say four disks). Both the RAID-5 and RAID 1+0 system will handle a single drive failure without any problems. Now, see what the odds are of the RAID system going down when a second drive fails; in the RAID-5 case it's 100%, in the RAID 0+1 case, it's only 33.3%. More information about this is available here. I don't know about you, but having to possibly restore a terabyte array from either tape or optical media is not my idea of a good time; spend the extra cash for RAID 1+0.
Kind of along the same line - any sort of good *free* (as in beer) hsm solutions for Linux? Shore would be nice just to have the files migrate to off line storage all on their own...
Just do what I did: buy a used server. I got a used Compaq Proliant with RAID 5+x, with room for a total of 12 drives. It was $450. Easy, cheap, and it works. No dicking around with software and shit necessary (unless, of course, your idea of a good way to spend a weekend is configuring software). And of course, the hardware is designed to work perfectly. I've got one in my business, and one at home.
I don't respond to AC's.
I used RAID1 exactly as you suggested for four or five years, then one day after doing something stupid, I realized that the main danger to my data was me, not my hardware, and that by throwing hardware at the problem, I was making it _more_ likely that I'd have a hardware failure, not less, and that every hardware failure was another opportunity for me to make a really bad mistake. Additionally, I was getting tired of every year or two having to scramble to either find a piece of matching hardware to replace a dead drive, or rebuild the array onto a matched pair of bigger drives.
I immediately scrounged enough hardware to build a backup server with a big-enough hard drive, and started running rsnapshot to back things up. This box does ONLY backup work. No servers, no file server (except for backup access), no nothing, so it's reasonably safe from me asking it to nuke itself. Rather than do RAID1 on this box, I got 2 external drives which I use to snapshot the backup once a month. Those drives rotate between home and work, so at all times I have the original data, a very fresh onsite online backup, a less fresh onsite offline backup, and a somewhat stale offsite offline backup.
This was no more expensive than RAID to setup, but I've found it to be a LOT more useful in the real world. Upgrading machines is much more enjoyable now, because of the netapp-like snapshotting that rsnapshot does. I used to do an intricate dance of replacing a mirror so that I'd have a trustworthy backup, then do the upgrade, then be afraid to reconnect the disconnected drive to check how something changed. Having tested rebuilds a couple times with the new system, I just build the OS and copy over stuff from the backup server as needed. If I need to check on a configuration mistake I made in July, I can just compare the pre-July copy to the post-July copy.
This all happened about a year-and-a-half ago. It quickly became apparent that this was _the_ way to go. Now I've got it reduced to a SFF case that sits off in the corner doing its job, and I don't touch it.
The ratings and reviews on their homepage http://www.infrant.com/ say it all. This thing blows a Terastation away in terms of ease of use, supported protocols, and goodies. Buy an empty ReadyNAS X6 from http://www.eaegis.com/ for $579 (no tax, free shipping). Fill it with two of whatever drive is dirt cheap this week (cough-newegg-cough). Here's the kicker...ReadyNAS will expand the drive array automatically each time you add a drive. So buy a couple 300GB's for $100 each and you'll have 300GB of mirrored storage. A few months from now, you run out of room, you just drop in another 300GB drive and now you've got 600GB of redundant storage. Add another drive and you'll have 900GB with redundancy. Still need more room? Replace those 300GB drives one at a time with higher capacity drives and watch it automatically resize the set to use the extra space. Without ever having to rebuld the array! Trying to backup a TB of data so you can move your NAS from 300GB drives to something higher really sucks the big one.
Of course it does CIFS(SMB). But it is one of the only NAS products to support Apple File Protocol, which is a must for networks with Mac/OS X users that insist on using filenames with colons, slashes and question marks and other things that make CIFS/SMB explode. It also supports NFS and rsync for the UNIX/Linux crowd and both FTP and HTTP for the web browser crowd (hi, grandma). It also streams in both flavors of home media server protocols (UPnP and the HMS) so you can buy a $100 Linksys media extender and watch anything you have stored on your RAID. It also has a SlimServer plugin for streaming music to those SlimServer devices that you can hook up to your stereo or a cheap pair of speakers.
It's also supports Gigabit with Jumbo Packets (write only currently) so you can copy 200GB of HD camera footage to the NAS in a couple hours instead of a couple days. The RevB case is cable-less with just thumbscrews between you and swapping a drive. It also holds the drives vertically because who is the idiot who thinks stacking heat factories horizontally on top of each other is a good idea. Also, I can't tell you how many RAID products only lets you specify an alert SMTP server name but no authentication information, which means e-mail alerts don't get delivered (boo Promise, boo 3Ware). ReadyNAS has its own MTA so the mail gets through without a problem, and it can also let you set login/password to authenticate to your ISP's SMTP server. It looks nice, clean, and it certainly not the noisiest thing I've had in my room, although I will be happy when future firmware lets you put the drives to sleep so the case fan can be completely turned off when you aren't using it.
I spend three weeks shopping for a NAS for my network, and I'm glad I looked past everyone telling me Terastation. I've had this ReadyNAS X6 for a few weeks now and I love it. I'm already shopping for a second so I can recycle the old drives from all my other rag-tag household systems into one nice neat package.
-JoeShmoe
-- I wonder which will go down in history as the bigger failure: the War on Drugs or the War on Filesharing
I scrounged some WD 20GB PATA drives for $5 each, tossed in an ATA card for extra channels, and run RAID 5 using Linux software RAID. I'm not doing realtinme video editing or anything, so it's plenty fast for my home fileserver. About the biggest disk bandwidth hog is the occasional, gigantic GIMP file being read or saved. I rsync it to another system for backups. Someday I'll get a tape drive or something for archiving, but for now this works fine. I did buy some cheap, fan-cooled drive bays so I could stick these in the 5-1/4" slots. They're a bit noisy. But they should extend the drive life. Eventually I'll buy some new, bigger disks. I'll stick with WD, thanks; we use WD exclusively at work, and have *very* few drive failures across 200 systems with about 250 drives.
You want 500GB storage, you need 500GB of backup, the more storage, the more backup. Always think of that. Very important.
Especially if you say, "I'll just make a selective backup". Especially then, when you loose the disk, the data you want right now, is not in the selective backup.
Right now I have a 250GB + 40GB and I have 250GB + 200GB just as backup. I have just no interest in lossing a single bit of data. if one of the disk dies. I change and just copy 1:1 back from the backup disks (or the other way around).
"Freiheit ist immer auch die Freiheit des Andersdenkenden" - Rosa Luxemburg, 1871 - 1919
Every item is stored on at least two hard drives, not on the same machine.
Periodically, I add any new stuff to a large portable USB drive.
When I have enough to fill a DVD, I burn one and verify it. They're tucked away in a different location. (I used to use cdrs, rather a lot of them, but switched to dvds as soon as burners were affordable.)
I don't need on-line RAID. Redundant, off-site storage and very little discipline is adequate for my needs.
first think i'd like to ask is, what systems are running on your home network and what kind of array you would like to run on it. I have tools that can bring some pretty far gone drives come back to near pristine when I thought they were dead depending on what they were formatted as. Are we talking about *nix Drives or Win* drives? or a mix of both running over a mixed network. and regarding the network is it a one to many server/worstation network or an everything connects to the router dhcp network etc.
.... then you lose that lovely extra drive!!!
I have a weird codependant relationship with Dead Tech, so I like to tweak with it on occasion and get it to do things it shouldn't be able to do. so I play around with Older machines and operating systems squeezing every last bit of use out of them. Doing this makes me lose the occasional bit of data yes, but it also allows me to learn a lot about not letting go of it that easily. Raid arrays are fun but add an extra drive into the mix so now you have 2 drives to worry about. striping them means that if one goes down you lose your data. backup raids are good but
ah the humanity. talk to me rob.
what are we talking about here.. NT, Win* 8nix or what.. maybe i can give you some answers.
) mag, that geeky girl on the left side of normal.
-Magdalene --"there are 10 types of people in the world, those who read binary, and those who don't"
Total overkill. Here is how to get a 250 Gb Raid 10 storange free of charge 1: Get 125 Gmail accounts 2: Get Google filestem 3: Job done
After all, nothing like 48 SATA connecters plugged into a dual opteron board to give you the storage solution that you need. I mean that is like 24 TB with 500 GB drives, or 16 with RAID.
I have mod points and I am not afraid to use them
Check out Adaptec's Snap Servers. They used to be made by Quantum until Adaptec took them over. I have an older one I got off of ebay, added 4x 120Gb in a RAID5, and can get to the same shares via SMB, NFS, and allows from a host of other protocols. User-friendly, plug and play, and fast.
Lime Technology has a pretty cool product for this. As I understand it, it is a version of RAID 4. You don't get the performance benefit of striping, but performance is still more than adequate for the typical home media server.
A couple of nice points about it are that the array can be as small as two disks (1 parity and 1 data), it's easily expandable and drives can be different sizes (though not larger than the parity drive).
Their prebuilt box is a little pricy, but you can also just buy their software and build your own box.
-pischke
(no affiliation, just a happy customer)
Beware the slug!
I've had mine blow away the superblock on drives twice! along with that it asses up the inode bitmap so have fun getting your data back. Do not trust the device with data, always back up if you're using the linksys!
-nB
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
A raid drive is only half the solution. You still need a backup system of some kind.
Raid arrays are only partial protection from physical hardware failures. It doesn't guard against executibles being infected with viruses, from the bad updates or trojans being installed, from data being deleted either by a user or deleted by a process gone bad.
I am going to build a pair of RAID drives, with one in a different physical location and then do an rsync from the main to the backup machine 4 times a day.
I am going to use an incremental back up that will let me do a daily backup and allow me to checkpoint systems before I install software so I can easily roll back system updates.
On my systems I am going to create partitions so that the software is seperate from the data which is seperate from it's configuration. This will let me update just the webserver, while leaving the configuration and data files in place. I can update a website while leaving everything else alone.
It happened again. Another instance of short sentences. Awful. Periods everywhere. Attempts at being dramatic. Failed. What a loss.
People, lost the dramatic writing style and don't worry if it sounds dry! Sheesh.
Isn't the simplest, for the problem as stated, just external USB or Firewire enclosures, one per machine, with automated backups, big disks, and the enclosures the kind with temperature gauges and fans? Just buy them a bigger hard drive than you think they will ever need for the computer, and another one to go in the external to back up to.
Its only when they start to need more storage than will fit on one drive that this doesn't work, so over around 400G today. But you are only thinking about less than this for the whole house. Its by far the simplest solution. And if they do ever need more, odds are disks will be bigger and cheaper by then.
If they get real nervous, you can even buy some extras and keep them off site, rotate them every couple of weeks or months. That way you can almost never lose more than two weeks/months data from one machine even if the house burns down. You do have to make the trips to the off site location though.
The great thing is, everyone can relate to my backup, which is labelled with my name, that I plug into my machine. Hard to go wrong.
Since it's only purpose was to be a fileserver, I opted for an older AMD Sempron (2800+ 754 socket). I purchased 4 Seagate 300GB PATA drives (this was a slight mistake).
Going slightly off topic here, but since I had 6 PATA devices, I needed a ATA133 PCI card to handle 2 of the 300GB drives. Everything went fine during the installation (Ubuntu), although it was a tad slow. I found out why once I could access the system. It seems that the IO transfer for the PCI based drives always reverted to 16bit. Since 2 drives were on 32bit, and the other on 16bit, it created a nasty little bottleneck. hdparm fixed it, but one drive seems to drop the settings after a while which recreated the bottleneck (Over 10 hours to sync 30% with bottleneck, 2 hours to do over 40% without).
In hindsight I should of bought two PATA and two SATA to keep everything on the motherboard. I've got a couple of IDE to SATA converters sitting here ready for tonight, I plan on moving the CDROM and OS HDD onto the SATA system, and then placing the 300GB directly into the motherboard to see if it fixes the problem.
So now I have a nice 900GB SW RAID5 to play with that utilises Samba so the Windows machines can play with it too. Its fast and the CPU doesn't even break a sweat. The bottleneck is a pain, but it's temporary, and only noticeable while copying a lot of files.
Don't these raid controllers want the drives to be the exact same model?
My 3ware couldn't care about model. It's the *size* that matters. The replacement drive has to be >= the size of the drive it's replacing.
http://slashdot.org/~tf23/journal
I flirt with paranoia occasionally and have recently been keeping my backups physically unconnected to anything electrical. You might keep a regular backup on a separate drive in your machine, but if there is a major electrical disturbance, it might take out all the drives in the machine. I keep daily magnetic backups physically separate from my machine in a steel filing cabinet (though it'd be even better to use magneto-optical or even CD-R), and make periodic CD-R backups and leave them with a trusted family member off-site.
Of course so far my main losses have been due to error on my part, and a few times software corruption of the drive data, so an in-machine backup on a second drive would have been sufficient.
haha, pretty slick, but what happens to your data when google makes some subtle changes that no longer makes using a "gmail filesystem" possible (or at least very hard)?
I am assuming that you have a rather good internet connection. The problem with all these raid systems is they are on-site data systems. For true data security you need to have an off-site back up of the data. There are some companies that will do this for you over the internet. But no matter what sort of RAID system you have it wont withstand a fire.
30% Troll, 50% Underrated, 10% Interesting
Score:5, Troll
Once you understand that RAID is reliability strategy and are prepared to have appropriate backup measures in place, then RAID 5 becomes an attractive option for the home network. I've recently looked at several options.
In my case, a three disk RAID 1 solution proved more appropriate than RAID 5. I value high reliability on the home system and wanted to use a rotating third disk as a backup in the event of catastrophic data loss (e.g. house burns to ground). FWIW, I also use a DAT for differentiatial backups. For many users this may be overkill -- sacrificing three disks plus fixed hardware costs to greatly reduce potential data losses -- but for priceless coding projects and digital pictures, this might be good for you as well.
For some users working with video or having large audio collections, much larger disk systems may be desired. First make sure that you have an appropriate mechanism for backing up a terabyte or three. Then, the Vanguard V5 may be an excellent solution if the $2-3k price is acceptible.
Given one hour to live, the student replied: "I'd spend it with professor FP who can make an hour seem like a lifetime."
I am looking to set up a very similar system (running Fedora) at home, but with the exception that I want everyone’s home directories on the network share. Is there any way to pull this off with both Linux and Mac OS X clients? Part of the challenge here are the extended attributes on HFS+ (which would not be implemented over CIFS and certainly not the native file server of the box). Am I all out of luck or do I need a Mac for hosting this if I want that kind of functionaliy?
Join Tor today!
Maxtor OneTouch III is available upto 1TB, doesn't require special drivers or software to interface, and is cheaper, easier, and arguably more reliable than the build-your-own cheap raid PC ideas already mentioned. Only disappointment is that it uses USB1/2, Firewire 400/800, but has no network interface. If this is required, you could use something like a Linksys NSLU2 (under $100).
Maxtor sells external RAID1 units now that come with Firewire 800, 400, and USB2 interfaces. They are a bit more money than putting something together yourself, but they are very simple. Get an Adaptec (or other) FW800 card with a couple ports and you could add two 500GB RAID1 Maxtor units to your system. If you ever need to take your data with you -- emergency or not -- just unplug and go. Everyone has USB2, if not FW400. Make sure your system and your external drives are plugged into a good UPS.
Messing with software RAID (which most posters have recommended) is simply not worth it. Too many bugs, too many issues. It works for some, but becomes a nightmare quickly for others. If you go for RAID inside your PC, get a 3ware card, say the 9550SX-4LPK, for $325 or so, and then add two Western Digitial 400GB RAID Edition drives and run RAID1. This gives you hardware RAID1 vs. cheaper software RAID1. You make the call depending on how much your data is worth.
Of course, back up your data in two places. I had a fancy RAID server fall off of a moving truck. Not anticipated and it cost me 2TB of data lost. This is one reason that I suggest the smaller Maxtor external RAID1 units. They are not as fast or as fancy as a RAID server or PC RAID. But they can be easily replicated and put into big cases with lots of foam padding.
Anyway you go, you are moving forward. Good luck.
First, buy one or more Generic SATA adapters. You can do the same thing with IDE. Note: AVOID PROMISE at all costs. They put something in their BIOS to prevent too many cards in one machine - so you have to buy their RAID crap.
The get as many hard drives as you have ports. I like Seagates - cheap, fast, low temp, 5 year warranty. Good enough. You can get nice 250's for about $100 each.
Shove this into a PC with a couple of extra fans. Go with a low-end Athlon solution, install cpudyn. Heat won't be a problem now, and you might even save on electricity. You can even have cpudyn spin your drives down for you if you don't mind an initial delay.
Make sure you have enough power for all the drives that you're buying (so they can spin up together - only more expensive controller support staggered spinup).
That's it. Install Linux, use md to make your RAID, add NFS and Samba, enjoy. It'll run for years without attention once you're done. The only downside? You're limited by the PCI bus for throughput, but that doesn't sound like an issue for this application.
jh
is that it will be relied on, and then it will break and you will lose something important (eg your entire life in photos, all of your music and your wedding video). Then you will want to kill yourself, and if you don't then someone else (eg your wife) surely will.
RAID is not backup. In fact in my experience it adds moving parts which (when not done properly) can actually impair stability/security. You need remote, incremental, automated and regular backups. If only duplicity were ready - then you could just backup to some untrusted third party like a neighbour over wireless or a friend over DSL (by untrusted I mean someone you would rather not have trawling through your stuff).
At work we bought a Buffalo Teraserver. I'm not impressed. OOB it won't support files > 2gb, and >4gb (It actually specifies both of those restrictions in the documentation). It was also fairly slow regardless of the access method. The thing couldn't saturate our 100Mbit switched link, making the gbit enet redundant to say the least. The afp interface places unconscionable limitations on the filenames usable.
:) ) but we didn't want to give up the space.
So we hacked the box a la instructions found on the net, installed ssh, and began exploring. As nearly as I can tell, the onboard processor is not fast enough to support high speed writes in raid 5 ( pegs the processor quickly ). It might have been faster in RAID1 or RAID0 (misnamed, that is - there's no redundant in RAID0, so it should be AID0.
Solution? We popped the front of the teraserver off, got a couple of fairly long IDE cables, put an inexpensive IDE RAID controller in an old 833mhz DELL PIII. Ran the ide cables out the slot next to the controller, up through the faceplate, and viola, the Teraserver becomes a nice external IDE enclosure that happened to come with four nice 250GB drives. It is very quiet, however
Result? It can now source or sink 11.5-12 MB/sec (just shy of the theoretical limit of 100mbit enet, 12.5mBYTEs/sec) all day long, quiet, and the little fans in there keep the drives cool. But save some money and buy your own external enclosure, is my advice.
Thinking outside my Head
A pair of NSLU2s with big USB drives was my first thought, too. The NSLU2 is small and silent, and you can pop them open and remove the intentional underclocking by clipping a resistor. (http://www.nslu2-linux.org/ has details.)
Because that's when I was spec'ing out the goodies that are making their way to me in the big, brown trucks as we speak. I'm satisfied with my purchases, but it would have been nice to hear what others came up with before blowing my year-end bonus.
4 x Maxtor 7L300S0 300GB SATA drives for $120 each from Newegg. (D'oh! They're $5 cheaper than last week!)
4-drive capable SATA external enclosure with hot-swap bays for $250 from Cooldrives.com.
Highpoint RocketRAID 1820A to turn it all into a RAID-5 with ~900GB capacity. $200, from Newegg.
This will all be driven by a G4/2x450MHz running Tiger Server.
get a 1 or 2 u rack mount case, a couple samsung 250gb drives, mini-itx mobo, openBSD, and roll your own. i started out in '98 using linux for my personal home server(redhat>suse>debian>redhat>fedora>openbsd), and without a doubt openbsd has been the most stable and the least problematic... i've been using it for the last 18 months, and the only reason for rebooting was when i experienced an extended power outage, when i moved, and when i added a new hard drive (because of a noisy case fan). http://www.doink.org/geeklog/public_html/article.p hp?story=20051212224355152
BIG TIP!!! get a frackin' UPS! i'm currently using an ancient APC smart 2200, but i've had fewer flakey problemsthe last three years i've been running with a UPS, and i think alot of it is just having clean power... of course my sysadmin chops might have gotten better as well, but i'm pretty sure clean power goes a loooonnnng way.
finally, as far as file sharing is concerned, i prefer netatalk cause i'm a long time mac user(as is my wife) and i've been a sysadmin in the graphic arts for a long time. netatal 2.0.x works very nicely on openbsd. but you should run whatever file sharing (netatalk, smb, nfs) is most conducive to your client OS.
i can't tell you which backup/archive is gonna be the best for you... if i could run legato networker on bsd cheaply, i would. i'm leaning towards bru for the time being, but i'd like to explore amanda some more.
good luck.
three can keep a secret, if two are dead - benjamin franklin
I don't know RAID 6 and 50 well enough to explain them, so the link will do the job. While I'm at it, they can do 10 (compare it to 0+1 to understand better)
RAID 6
RAID 10
RAID 50
-Turkey
Since it sounds like home data, and it is not disasterous to have it go offiline for a while, I'd suggest you do what I do.
I have an old P3 something-or-other box running debian/samba, with a single 100gb drive in it (capacaty can be adjusted to your needs, obviously.) This drive has a share on it for my music, and serves as a target for rsync backups from various boxen about the house.
Sitting on top of it are 2 200mb IDE drives in USB 2.0 enclosures. I have a cron job that runs rsync to copy everything from the internal drive to whichever drive happens to be plugged in at the time.
Whenever I think of it, I unplug one usb drive and plug in the other one.
(For those not familiar with rsync, its a cool little program that copies only data that has changed...which makes it practical to do huge backups daily.)
The benefits of this system are:
At any time I have 3 separate copies of most of my data.
Every time the cron job runs, I have 2 separate copies of all my data.
If I should do something blindingly stupid with rm, or a massive power surge should fry my box and all within, I still have a drive offline to restore my files from.
If you have an elaborate installation on the server that you want to make an image of, you can boot with a live cd and dd the relevant partitions onto your USB disks.
Not sure what pricing is these days on this stuff, but you basically dig a junker out of the pile, install drives to capacity, and you're on. I would imagine you could do it for under 500 bucks.
You get all that, plus the fun of dweebing with rsync for a few hours. What could be better?
Why choose a card (and the requisite set of drivers and/or other software) instead of a box that manages the RAID for you and presents a single drive to the host (like Raidweb boxes)? I don't work for Raidweb, but I know some of their customers and the people I know are satisfied with the devices.
If a home media jukebox drive fails, who will be at home to replace a drive with a cold spare? Do people normally build their card-based systems with fallback power supplies and a hot spare?
Digital Citizen
Here's something that works for me.
Following a power supply 'incident' that took out a motherboard and 2 drives in one hit, I thought about it a bit and decided that:
1) Speed was not what mattered most.
2) Immunity to hardware failure matters a lot more.
So I bought a USB2 card, 4 IDE drives and 4 USB2 to IDE adaptors which came with brick style power supplies for the drives. Linux. Software RAID5.
Performance is not stellar, but is good enough for serving mp3/xvid and I can be fairly happy that the setup is not only likely to survive a HD failure, it is also likely to survive a PSU failure without loss of data. One quirk is that md autorun doesn't seem to like USB drives, but I can live with this.
If I need to expand I can add more drives using a hub or (probably better performance) another USB2 card.
As I said, not stellar performance, but it works for me, didn't cost a lot, and it would be very very hard for a failure to take out more than one drive.
I require 1TB of fully backed up network storage and I've been researching my own solution. My thoughts:
1.) None of the "home" NAS appliances came close to meeting my requirements and the pro setups are too expensive. Building my own boxes is the way to go.
2.) I looked at SATA raid 5 cards, but then decided to forget the redundant aspect of RAID. Raid 1/3/5 are for uptime, they enable a properly configured server to hot-swap failed disks without rebooting, not important in a home setup. I realised how useless it is for backup if, say, the PSU were to overvolt and fry all the disks simultaneously. I'm still considering RAID 0 for performance though.
3.) Following on from above, anything in the same case is vulnerable to a catastrophic failure. To properly back things up, I need two boxes sited as far away from each other as possible.
3.) I want decent performance and I've got a lot of data to move across the network. Gig ethernet is obviously the way to go. Gig ethernet NICs struggle over PCI, so I want either a PCI-E NIC, or preferably an intergrated motherboard solution that bypasses the PCI bus.
4.) To maximise future expansion potential, I want motherboards with lots of SATA ports and big disks so I don't take up too many of them.
The ethernet throughput and SATA port requirements led me straight to NForce 3/4. I'm going for Nforce 4 as the price difference is not too great and PCI-E with socket 939 gives better future upgrade potential. (I've already decided to use the boxes for more than just storage, who knows what I'll want to do with them in a year's time.) If you're certain you'll only use them as pure network storage, then NForce 3/socket 754 and the cheapest Sempron should easily suffice.
I'm going for two Seagate 500GB disks per box. The cost per gig is marginally higher than mid-capacity disks (optimum seems about 300GB at the moment), but the benefits of future expandabilty outweighed that for me. If I want to expand further in future, I can easily take it to 2TB by moving all four into one box and purchasing 2TB more in whatever disk capacity is available/economical at that time. The 5 year guarantee won me over to Seagate.
To ensure backup is constantly maintained with the minimum amount of effort and network congestion, I will be running rsync between the two boxes. I'm also looking at the possibily of a dedicated backup network between them by using dual gig-ethernet motherboards. As I will have full data redundancy between the The two boxes, I figure I have nothing to lose striping the data between the two disks in each box to maximise throuput performance.
The setup I'm going for should, theoretically at least, come close to maxing out gig ethernet throughput and be as fast (in data tranfer terms, don't know about latency) as a typical locally connected hard drive. However, none of this is implemented yet, so if anyone can see a glaring error, please tell me before I waste money!
I've been putting together the specs for such a beast. I decided to go with SATA for cheap drives and "SATA-II" (or whatever you want to call it, since there isn't a standard name for NCQ and 3.0Gbps support) for future-proofing.
1) The natural first choice was 3ware. 12 port SATA-II controller (9550SX-12), for about $800. 3ware products are very well supported on Linux. The only downside is that it's a PCI-X device (this is NOT "PCI Express"!), and PCI-X busses are generally only found on very high end motherboards for servers and workstations. Any athlon motherboard or single-processor opteron board claiming to have PCI-X is lying, they really mean PCI express (AMD chipsets did not support PCI-X at all until around the time dual opteron motherboards were being created)
So since I didn't want to spend $500 on a motherboard that had built in scsi raid, support for 16GB of ram and dual opteron processors just to use that $800 card, I looked around some more...
2) And found a serious contender, the 12 port Areca 8x PCIe ARC-1230 (also about $800). While most low end motherboards don't provide an 8x PCI Express slot, they DO provide a 16x slot which will work just fine for this card (after all, this will be the fileserver, so a motherboard with crappy built in video will do, we're not playing Doom 3 here). Linux drivers are provided as source, even including a kernel tree patch which will build the driver into the kernel rather than as a module, making booting directly from the RAID controller easy.
Slap the Areca into Tom's Hardware's 37 watt computer (motherboard has built in GigE, but pentium-Ms are 32 bit processors, making giant files/filesystems a pain. An Athlon 64+cheap mini-ATX can be had cheaper, but uses more power), add in a stack of 10 watt 400GB WD Caviar Raid Edition 2 drives, and you're set for a very low power fileserver with a lot of storage.
Now, my turn to "ask slashdot":
Where do I get a 250-300 watt powersupply with 12 SATA power connectors?
Alternatively, do the SATA drive cages (like 3ware's RDC-400-SATA (pdf) have their own SATA power connectors built in and use standard molex connectors on the outside? Do I need special cages to support 3Gbps drives (ok, not a serious problem for now, but futureproofing)? 3ware's website says it'll work, their product PDF doesn't.
If I have been able to see further than others, it is because I bought a pair of binoculars.
I didn't see anyone else mention it.
? id=10007
I use various Lacie products, both Network and firewire/usb drives.
the network drives are either windowsXP embeded or linux.
Theses are both perfect for a home network.
here are some cheaper versions of their products:
http://www.lacie.com/products/clearance/products/
here are their top of the line network devices:
http://www.lacie.com/products/range.htm?id=10007
for backup servers i dont use any raid, except maybe software mirroring.
For backups its about reliability not speed, and less devices to monitor ie:no raid 3,5,... also seams more stable for a home backup.
These devices are quiet, rackable if you have that at home.
_JS
It is not about cheap you are talking about. You are talking about your data security (in context of backuping it). First requirement is the data to be safe. Next is to be cheap. So if you wan't real sure solution it is probably big iron. :) A *nix (Samba or whatever access you wish - it will probably have it) server with decent disk (like 2x 300GB) configured as file server. Additionaly (for backup purposes) some form of tape backup (automated library would be the best) and frequent backups/working backups for your data.
It will not be cheap. It will be normal price I guess.
And also do not forget basic backup security measures like:
1) Having *two* (more is better actually) copies in independent (geographically) locations.
2) Testing disaster recovery often - to make sure your backup is worth anything.
It will not be home-cheap. It will be such-solution-normal-price.
First, it was S L O W. My (Windows-only) NetDisk was over 2x as fast, but it was Windows only, needed drivers, etc... so I gave the NSLU2 a shot.
I could live with SLOW, since it was just for backups, but after the 2nd time it barfed the file system, I took it out of service... That reminds me, I need to put it up on eBay.
I don't know what happened the first time, but the 2nd time I stongly suspect it got trashed when a I was using it from Windows, doing a massive delete operation, then decided it was taking to long and aborted the operation...
I can not trust it anymore.
This issue is a bit more complicated than you think.
However, even a low-end system would be pretty pricey for home use, and you'll probably balk because it's many times more expensive than a cheap and easy linux box with a samba, some cheap disk drives, and a raid controller. However, the features and performance that a Netapp server provides more than make it for it.
Still, you might be better off trying to find an used, older Netapp model that's still supported and buying it second-hand. You'll still have to license the software, but you might be able to find a system that's more in-line with your needs.
Bruce
On to solutions. Buy yourself a big case
If you want a monster (sized) machine, a full tower will readily give you 2+8+10 bays into which you can properly mount a 3.5" HDD, and probably provide adequate cooling capacity as well.
Not to sound too much like an advertisement, but if you want elegant and realistic (18 drives???), the Lian Li PC3077 provides 7 external 5.25" bays, which will give you one CD/DVD drive, plus up to 8 3.5" HDDs with the Lian Li EX-34A 3x5.25 to 4x3.5" mounting kit (personally I prefer the Thermaltake A2309 iCage, which only gives you 1:1 (so 3 HDDs in 3 bays), but has simply wonderful heat dissipation). About as close as you'll get to a dedicated external RAID enclosure, but with decent cooling (many RAID enclosures suck for heat, and eat drives like candy) and a full PC inside the box.
Add 4 120GB 7200RPM SATA Drives (or what ever you can find cheap, even 200GB drives are relativly cheap these days).
The current dollars-per-gig crossover has reached the 400GB level (some posts actually suggested going with 80GB drives... while cheap per drive, that would cost almost a third more than going with 400GBs on a per-GB basis, not to mention only a fifth the total capacity in a given enclosure); and if you plan to go with a hardware RAID, don't worry too much about per-drive performance either... Go with the biggest cheapest drives you can get. Any modern drive will perform admirably in a RAID, and if you needed higher performance, you'd already know you need a dedicated NAS that eats low-capacity high-price SCSI drives.
Install Linux, share your harddrive using Samba. Done.
Agree completely. I'd suggest sharing most of the drive RO, however, with only a small portion RW. That way, any malware on connected 'doze machines can at worst wipe out the RW portion, with your "real" archive safely unmodifiable.
This is a very timely topic, and a segment of the market that has repeatedly been identified as a 'hot' by storage manufacturers.
There is a new line of Maxtor RAID products called One-Touch III, which is just about to come out, and very much address the kind of needs you are having. If you look at the hassle of setting up a chassis, enclosure, configuring and installing your raid card and drives, the cooling and ventilation issues that may crop up as someone else already pointed out, as well as the associated electricity costs, these Maxtor devices are really a pretty fantastic value in a really small form factor.
Just to be clear, this is not Network-attached storage, so it would still need something like a Mac Mini or equivalent to get it onto the network, or until they come up with something that is Network Attached Storage as they did with their previous line (The One-Touch II line has some Attached Storage devices, but they are not RAID).
I have had decent luck with their other One-Touch drives, must admit that after several years of pretty intensive use, never had a real failure yet. So although YMMV you may want to look into these newer drives, wish I could tell you more myself, but they are not out until the end of the month.
Z.
(Disclaimer: I do not own stock in Maxtor - now Seagate -, and definitely do not work for either of them; just honest personal experience.)
RAID is not a backup.. RAID is not a backup.
RAID was created to reduce downtime. it is not, nor was it ever, intended to be a backup system.
Get yourself a big tape drive.. Sleep soundly. Be happy
---- Booth was a patriot ----
You rang Milady?
...
o rage_platform/
...
Some of us don't get out of bed for less than a Petabyte
http://www.hds.com/products_services/universal_st
I have six. You have to love 2Gb fibre channel. I like using RAIDSilly (Redundant Array of Independent Datacentres)
I have written two articles on this:
- Tapes are still the most efficient and cost effective form of backup.
- How to backup a Linux home network.
Granted, disk capacity is growing faster than tape capacity can keep up, but RAID cannot be an offsite bacup solution.
Perhaps two disks in a USB portable housing is a better solution. One on site and off site, rotated weekly.
2bits.com, Inc: Drupal, WordPress, and LAMP performance tuning.
..I'm getting an old Compaq rackmount server with a boatload of disk for nothing. Apparently, it's not longer useful because it isn't a multi-gigahertz platform. LOL!
My plan is to slap Solaris 10/x86 on it, fire up SVM and do a RAID 10 disk set with two hot spares. Hopefully, that will last me long enough that Sun T3s will come into affordability for homeusers.
Why SVM? Well, simple -- I use it all the time at work, and it will require minimal effort to make work. Assuming, of course, that SVM on Solaris 10 x86 is works the same as it does on Solaris 9/SPARC. Last time I ran Solaris x86 (version 7), I don't think it had the option to run DiskSuite (now called SVM).
Do daemons dream of electric sleep()?
Thecus 2100S (at newegg.com)
What about this thing? It sure is a hell of a lot cheaper than a Buffalo Terraserver (at $355 vs ~$800)
and seems to do what you want. It supports up to 1TB, has gigabit ethernet, uses SATA drives
and runs on an Xscale processor.
Nope, I don't work the company either. I'm just thinking about buying one for someone. (Disclaimer: I've
never used one of these, it just fits the specs...)
I have mod points, but I feel it's more important to just correct you. He already has everything backed up and the LVM idea doesn't do anything to help his situation.
He does care about downtime. Downtime = time spent restoring. With a RAID level > 0, all he has to do is replace a drive and tell the raid to rebuild. He's done in 5 minutes. It would take that long just to queue up a restore job for the tape.
I've been running raid arrays for my music collection for about +7 years now. I've carried the data across multiple new machines several times. I've used onboard controllers (no good, because you're stuck if the MB fails. I've used the promise cards, which are ok unless you use more than one. They conflict, and locked up my PC. Promise wouldn't investigate the issue. Similar issues with the other cheap drivers.
I even got a mesh 3U rack blank, mounted a 3-bay unit on it that would hold 5 SATA drives, and got an E-bay power supply to run it. I ran cables from that down to the PC.
Currently I'm using the Kingwin SATA drive trays. I bought 3 bays and 10 extra trays. That way I could put 2 in one PC to transfer files between drives, and have on in another machine just in case. The advantages over a raid or SATA controller is that I'm only powering 1-2 disks at a time. While with raid I have all the disks going, and I'm not using them. This should help them last a lot longer as well. When it comes to backup, you could just manually do raid 1 with
insync or something similar.
It works ok, I run XP which picks up the drives automatically, and sometimes I have to reactivate them. I always have to re-share them. Other than that it's working pretty well for me.
I still have my music on a mirrored drive, and a backup on one of the inter-changeable drives.
Hardware VS Software Raid
l Hardware-c.htmln ual/custom-guide/s1-raid-approaches.html. html4 349/2/
The $13 card you purchased is software Raid. Promise cards are mostly hardware RAID. I recently purchased a Promise FastTrack S150 SX4-M for less than $100 hardware RAID5 card compared to the $30-50 software RAID5 cards. I'm pretty satisified with the purchase but unfortunately there isn't room for much upgrade. I currently have 4x160GB in a RAID5 configuration giving me 480GB of space and 1 disc of redundnacy.
Some useful links to tell you the difference between software raid and hardware raid are:
http://www.pcguide.com/ref/hdd/perf/raid/conf/ctr
http://www.redhat.com/docs/manuals/linux/RHL-9-Ma
http://techrepublic.com.com/5100-10880_11-5715216
http://www.linuxplanet.com/linuxplanet/tutorials/
HD Trailers
If a hard drive craps out, I don't necessarily want my system to go with it--would rather have my raid monitoring system inform me of a bad disk so it can either be hot swapped, or if not swappable, replaced after hours.
If you're using the RAID storage as your primary storage, you'll want a pair of arrays instead of a single array; better, you'll want a duplicate system as your real backup box. RAID is not on its own a backup as the system itself can still fail or data become corrupted. So the second part of the recommendation is: use rdiff-backup. It's standard on Fedora Core now, and it's a breeze to use. It won't take up space with anything that hasn't changed, either.
Let us live so that when we come to die, even the undertaker will be sorry -- Mark Twain
I would assume most of the users on here are quite linux savy (if not, ask a friend, they may be able to help, but prepared to supply beer and pizza).
.6tb of storage: /storage
/store/picturelibrary/ directory, and share my pictures with family .6tb volume.
:) ). Mine is in my basement, also has a cdrw on it (for quick doc backups), a 2gb orb drive (for quicker backups) and a shared printer.
l
:). The linux md device also supports a hot spare, which i recommend you consider if the data is important enough to you.
Rather than a dedicated cheap nas device, i decided to go with a DIY linux software raid array. The current linux software raid is pretty reliable. if your doing mission critical data, i recommend hardware raid instead however. To estimate this, try to attempt to guage the cost associated with re-gathering all your data, and spend at least 1/4 that much for your storage.
My solution was 4 200gb ata seagate baracuda drives in sw raid 5. the cost was about:
- 4 drives @ $125 cdn
- case & powersupply @ $100 cdn
- board, cpu, 512mb ram @ $200 cdn
total cost - 900$ cdn
i used the onboard ide controller for a 80gb os disk, and a separate 2channel pci ide controller for the 4 disks, in raid 5, giving about
achilles:/storage 559G 474G 86G 85%
i've been using this volume for about 1.5 years now with no problems *knock wood*. I've also rebuilt a sw raid 5 array at work, so i know that part of it works (for the most part).
A few benefits i find using linux rather than a hardware device:
- i can ssh/winscp in and get any of my files, anytime, from anywhere
- i can run apache, mounting my
- nfs or smb mount the volumes to any other linux/windows machines
- the geek satisfaction of having my own
my next step from this is to purchase 4-8 SATA drives, a 8channel sata controller, and go with that. One thing to consider, is the location of your system. With this many drives, it can generate substantial heat (and noise), so you probably dont' want it sitting in a warm location in your home, where you have to listen to it droning away (4 cudas make some noise
hope this helps! a good linux sw raid howto is at:
http://www.tldp.org/HOWTO/Software-RAID-HOWTO.htm
One other thing. you can also use the mdadm tools to monitor the volume for any issues, and if/when they arrise, you can have it email you a message. This way you can pick up a spare 200gb drive on your way home from work to replace the failed one
dwight s.
You don't need 1000 CDs.
You don't need 500 DVDs.
You don't need hours and hours of shaking, badly focused home videos.
You don't need 5000 bad pictures.
(if you really do you know I am not referring to you).
Nobody is going to watch all that crap, and unless you have not got a life, you are included on that select group.
Prune your digital trash.
You will find that a moderate amount of disk space is more than enough to hold all your files.
If you want to make a datacentre of your home, go ahead and enjoy, but the lamest excuse is to house all those GBytes of data that are never going to be seen again.
IANAL but write like a drunk one.
For a home application where you just want to backup data why not just use hard drives to backup? Most of what I have doesn't even need to be backed up on a regular basis. Currently, I have about 4 drives sitting in bags with most of my stuff backed up. (not an elegant solution)
But I have been thinking about a slightly better solution for a long time. Of course the needs may be different than mine.
What I intend to do is get a couple of 500 gig firewire drives to store stuff on. Then I will run software to backup one 500 gig drive to the other on a regular basis. Also the backup drive can remain turned off most of the time.
My concern with RAID:
1. It is not really an archiving solution as I understand it.
2. Huge use of power all the time.
3. Noise.
RAID 6 is pretty much the same as RAID 5, except that the parity is stored on two disks not just one. This is so that the array can cope with the failure of more than one disk.
:)
--
This executive summary was bought to you by cheese
I've taken to taking a portable drive and sync software to replicate from my main drives to a portable that lives most of the time at work. Need to do something though about my data getting corrupted and replicated in corrupt form. 95% plus is static, so I can probably lock those files down and only replicate active stuff.
Get one of these: http://www.topmicrousa.com/combo-205.html It's a level 5 hardware based raid. Plugs into any computer via usb 2.0hs or firewire 400/800, appearing as a single monster drive. (1tb if you stuff it with five 250's) The bays are hot swap and online rebuild, so you never have to go down to recover.
Since it's a hardware solution that's external, even if your server buys it you can swap in another PC and be off and running again. This also makes the storage portable for when you need to take your data somewhere else. Heck of a lot easier grabbing that by the handle with a laptop under your arm than lugging a behemoth PC tower around.
That design is actually available in several places under several different names, but that enclosure I linked is the most common, and one of the cheapest. It also comes with a good warranty, and good customer support. Nice idiot lights on all bays, as well as LCD display with (lockable) menu. You can admin it via telnet with serial cable or through the front panel.
I suppose you could opt for NAS, but then you have the server tied to the drives and have less flexibility.
I work for the Department of Redundancy Department.
This is what I do:
- I have a linux box at home, on a DSL line.
- I have a unix system at work.
- I have exchanged the ssh keys to my user account between the two sytems.
- the work box has a cron job that runs two rsync commands at 3am
1- rsync my work home directory to my home box
2- rsync the home box home directory to my work box
More and more people have two systems at home, so you could set this up at home also. But this has the advantage of giving me an off-site backup.
If your work situation doesn't allow this, then perhaps you have a close friend with whom you could arrange something similar.
But yes, this is only good for, oh I guess a few GB. after that you are going to run into bandwidth issues. But leave out your audio and video data and I would wager that this would work for many people.
I have a file server at home.... I is pretty basic I suppose...
...cause I'm sure if fully need that .php hello world test page I made 3yrs ago and stuff).
3ware Escalade 7000TX (with 2 Hitachi 120GB HD's) in raid 1
I use it to backup my mp3's and documents... so far space is fine (about 30GB's or MP3's and 10GB's of documents... I actually need to review the documents at some point
anyways I am very happy with my escalade I suppose if i had it to do over again it would be nice to have a RAID 5 card but oh well.... not really a big deal.
basically I use m$'s SyncToy to update my backups on the Server once a week.
actually I am happy to see you, however that is in fact a banana in my pocket.
Hardware RAID solutions require exact hardware in case of RAID controller failure. You'll need the card replaced before you can get back at your data.
Use Linux md() instead. The Linux Software RAID, combined with EVMS is a fantastic multiplatform solution. Software RAID is negligably slower, and if you get some old cheap hardware, it doesn't matter. I bought a huge old IBM xSeries 4-way server and it's got 2 procs- slap IRQ affinity for the cards to one processor, and you have yourself a dedicated RAID processor.
Plus, you can recover the RAID on any linux system. Hardware agnostic.
Best part of all? It's cheap. I use regular off-the-shelf SATA cards strapped to the drives.
remove the intentional underclocking
The underclocking is to keep it from overheating.
If you remove the underclock you better have a little fan around to keep it from dying...
I have 435GB spread over a laptop, a desktop, 2 external hds and an iPod. Today after loading a file in Azureus on my desktop, I realized that 2 directories with over 30GB of data weren't visible on the hd. They dissappeared. Switched to Windows to realize that of the 190GB partition, almost 150GB were either in weird characters or the files themselves were unrecognizable by the program that should be able to read it. Lost tons of media, including over 12GB of MP3s and tons of videos.
The point is that we depend on technology way too much. A couple years ago I destroyed a partition (accidentally) and all I lost was about 100 MP3s, some game saves and some images. Today I lost a lot more. Makes you wonder what will happen when someone loses a HD in 5 years with no backups.
/usr/bin/rsync -avbS --delete-during --exclude=.snapshot/* \ --backup-dir=.snapshot/`/bin/date +%Y%m%d%H%M%S` /source/. \ /destination/. >/var/log/backup.log
Realistically you should use "--delete-after" but Fedora Core 4 ships with a broken version of rsync (and to the best of my knowledge they haven't fixed it, despite it being THE ONLY BUG with that version) which will actually NOT backup changes if you use "--delete-after". Meh.
In other words, "I want one really good basket to keep all of my eggs in." What... did they stop teaching problem-analysis in the CS dept after I graduated from Hope? {smile}
Might I humbly suggest that you buy/build/salvage a pair of inexpensive computers, each with a fair amount of RAM, a hard drive (or RAID 0) of your desired capacity*, and the fastest NIC your switch can handle. (Forget the fancy RAID controllers, and of course anything better than a PCI VGA card is wasted.) Install the OSOS of your choice on both. Turn on Samba on one of them: that one's your file server. On the other one, set up a nightly cron job to synch (without deletion) the shared directory on the first machine to its local copy of that data: that's your redundancy.
This solution effectively protects you from fried electronics, accidental deletions, and even small fires if the boxes are in different parts of the house (and if the whole house is going up, you get your choice of which box to run back in and rescue), scenarios in which the really-good-basket approach will still scramble your eggs.
*Consider getting different brands to reduce the likelihood of near-simultanous failure. I've had multiple drives from the same lot start failing within months of each other, and you don't want to have a second drive failure while you're still browsing for a replacement for the first.
http://alternatives.rzero.com/
Like the original poster, I am a home user. I'm considering getting a NAS or dual-drive enclosure (hooked up via Firewire... I'm a Mac user).
ATA drives are still being made. SATA enclosures are just starting to show up. I'm not into buying SATA because the drives are going to be 'faster'. If I'm running on 100 Mbit ethernet, I'm sure my limit is the network and not the drive. I really only care for the long-term replaceability of the drives in the enclosure. How much longer will ATA drives be manufactured? Is SATA 1.0 and 2.0 compatible on the chipsets in most of the enclosures?
I'd like an opion on the choice. Judging by the newer PCs and the fact that new Macs will be made by Intel, is there a chance that Firewire will be eliminated? That being the case, would it not be wiser to use a NAS, since ethernet will be around much longer?
I'd like to be able to get a case that will last 5 years or so. If a drive dies, fine, its replaceable. I just don't want to buy other cases.
www.permastor.net
Many Mobo's now have onboard gigabit NIC and SATA RAID5. Many also include PATA RAID 1, 0, 0+1. So I would suggest 4 SATA in RAID5 for Network file storage and 2 PATA in RAID1 for OS boot. Something like the MSI K8N Neo4 Platinum would be ideal. The Gigabyte GA-K8N51GMF-9 would also work but provide less expansion. I recently bought a Seagate PATA 120Gb drive for $30. You can find hard drives cheap now and for the boot drive would not need to be large.
Just a Tuna in the Sea of Life
I have two redundant servers on the home LAN... one with 960GB and the other with 840GB of space - all raid-5. rsyncs every 12 hours keep all of the data together. Each server is on a separate circuit in the house, with UPSes, just in case. I also make good use of redundant power supplies... Every 6 months or so, I make an incremental tape backup to keep in my safe deposit box.
Since I still have every mp3, movie, video, soundfile, game, email, document, paper, and everything else, dating all the way back to 1990, I certainly don't want to risk losing any of it.
I've seen 160 GB Seagate ATA drives for $10 after rebates.
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
MegaRAID SATA 300-8X
LSI Logic is one of the few companies that actually supports and helps out the free software community, and therefore their cards are supported more places than binary drivers would allow. Also, they do hardware-assisted RAID so the CPU can do its own thing.
Support these guys!
8 x SATA2 250GB HDs. Might as well make sure they have NCQ.
Some say NCQ has a benefit. Some say it doesn't. Might as well get it, it doesn't cost any more than the SATA2 drives that don't have it.
Cheap server motherboard w/ PCI-X (NOT PCI-Express.)
The MegaRAID above uses PCI-X. All 8-port RAID cards that I'm aware of need PCI-X. There are a few that support PCIe, but they're not proven and they're not (yet) quite supported everywhere the way the 300-8X is.
2GB RAM
Might as well load up on OS-accessible cache.
One of the *BSDs, since they're easier to use.
Arrange them in a RAID10 configuration and use ~64KB stripe size, or less.
The only reason you might consider larger stripe sizes would be for large media files (like for a MythTV box for example.) Otherwise, it's not worth the overhead. It should be RAID10 because RAID5 is slow and wasteful, and RAID10 with a 2x4 configuration (2 sets comprising 4 striped drives each) is capable of losing more drives simultaneously than other configurations are.
Put it all together and you have 1TB of redundant storage made from open source components which you can pretty much tug along with you wherever you go. Plus, the MegaRAID cards are blindingly fast. I mean come on! 8 drives!
Now then, if you have more money, grab one of the MegaRAID SCSI U320 solutions and get SERIOUS speed for your money. You'll end up with a smaller, overall, drive array, but the speed will be ridiculous.. Just buy two of them and slap them in there, and put them on a ccd/vinum/lvm partition. Droolicious.
D-Link already has a product out that will solve your problems. It also appears from the description that it may even be better than requested. You can find the D-Link DNS-G120 802.11G NW STORAGE ADPTER for under 120 and it will connect to your wireless network or cable(cat V not bnc) no problemo. It connects to usb hdd's. For those who need larger storage solutions IDE to USB2.0 Adapters are under 10 bucks. I hope this is usefull info personally this seems to be a fairly simple solution to a home network storage problem. Marcus
I use cygwin and rsync to backup everything to two Linux boxes in the house.
All my PC's can rsync to the master repository.
Every machine in my house can use this, using rsync on cygwin on the PC's, the
Mac runs unix already...
Every single 3Ware that I've ever encountered has unfortunately been .. how shall I put it .. less than stellar when it comes to rebuilding a damaged array. From the odd drive trays of some models, to the lackluster rebuilding logic that silently corrupts on-disk data too commonly to be trusted, I just really don't like that hardware.
Stay away from it. Support LSI, they're more friendly to the free software community!
I had just this same issue, roughly 300GB of porn, 100GB of digital audio, 100GB of music, and probably 50GB of miscellaneous stuff. I scrounged up all the drives I had, bought two computers from the University Surplus (a PII and PIII, 450 and 400 MHz, respectively), put 128MB of RAM in each and split the drive space evenly between them. Then, I set up LVM on each box to create one giant disk, set up RSYNC on each box to create automatic backups, and I was done.
The drawback to this setup is speed, the drives are not SATA drives, there is not RAID to speedup your throughput, they use 10/100MBit NICs. But, to be honest, I've not noticed any issues with my Home Media server (Freevo-based), nor backing up my Digital Audio work, nor going through an automatic backup while watching videos on 3 different machines on the house.
Now, is this pluggable like a RAID array? Not exactly. However, adding disks to an LVM is ridiculously easy, and if you're out of IDE channels, trash your LVM (remember your redundant backup on the backup machine), swap the smallest and largest disks, restore the data from the backup machine, and go. Will copying ~1TB of data take all day over 100MBit? Yes. In fact, if all goes well and you don't use your network at all, it'll take you 22 hours (12.5MB/sec). So gigabit is clearly the way to go, but this whole thing cost me... $150. That's cables, power strips, everything. Of course, I had the disks too. You could probably do all of this for:
2 Surplus computers: 160
2 Gigabit NICs: 50
Gigabit Router/Switch: 100
Cat 5e cables: 40
Linux: Free
Total: 350
At that point, all you need's disks... which is honestly going to be the bulk of your expense. But hey, you wanted storage, right?
The one issue with this setup (I considered it myself) is that S.M.A.R.T. will not work on USB enclosed drives. Hence those nice little smartctl jobs that you should be running every night will not be able to warn you of impending drive failure.
$120 for a 250GB SATA? How about $123CAD for 250GB of SATA2 w/ NCQ? NCX Rules That's like... what.. $106USD! Cheep.
I had a similar experience with the YellowMachine. It was advertised as 1TB RAID, but the fine print reads "on some models" and the ones I could actually find only gave 650G in RAID5 configuration. No big deal. But it's slow. SLOW. It's got an ARM processor that runs at 100 bogomips and it has 64M of RAM.Mounting is slow. ls is slow. I got it to store media files but I find I can't play mp3s from it unless I tell my player to cache the whole song (and forget about crossfade). It came with telnetd running and no ssh, but fortunately it was based on Debian Woody so fixes are easy. And boy has it required a number of fixes.
I'm thinking of doing something like you did--copying critical configuration info off of it and reusing its md in a faster x86 box.
I want my Cowboyneal
Pair the NSLU2 (The Slug) with one or more Netgear MP101s (available on eBay for under $50) and Twonkyvision's Media Server, and you've got a cheap media center. Add a wireless router and beam MP3s all over your house. I like Acomdata's drive enclosures because they're fanless...dead quiet drives, dead quiet slug.
I've found the Linux support for this hardware to be severely lacking. I wouldn't recommend this product to any "real" slashdotter.
You may not like this file system ; but it works. Set up some shares with Samba on your Linux boxen with RAID; and use one Windows 2003 server for the replication. A logical file system with redundancy. You may all mod me down now....
Every mans' island needs an ocean; choose your ocean carefully.
Hrm, I don't dispute your experience however the NSLU2 has been an absolute godsend to my network. I regularly push 25-30Mbps through the thing (SMB) and while I agree that's slow it is plenty fast enough for my needs. I can play DivX and XviD files directly off the device on my Xbox. I use it for the central MP3/OGG store for iTunes. It is my backup device (rsync on my other machines). It takes up an otherwise unused cupboard shelf and the slug is absolutely *silent* because the USB enclosures are fanless. Perhaps there's some disk chatter when it's pushing bits but I honestly wouldn't know because I'ver never heard it. Mine has been perfectly reliable for the past 12 months; no moving parts means I'm confident it will give me years of reliable service.
For the price, and the fact that you can upgrade to UNSLUNG firmware and use it for a general purpose always-on Linux server, I can't recommend the NSLU2 strongly enough.
Most desktop macs (except iMacs/eMacs) can hold two drives, and anything that runs OS X will do RAID 0/RAID 1 in the OS. So if you're not too-too concerned about screaming performance, go find any old G4 desktop, plop 2 drives into it, grab a copy of Mac OS X.2 or better (but I strongly recommend X.4 if you're doing SMB). Format the drives using Disk Utility, then in Disk Utility, drag the two drives into the RAID tab as RAID Mirrored. Hit "apply" and wait a while as it builds the RAID set.
Then you can configure the RAID set for sharing (System Preferences, Sharing, don't forget to check 'Windows File Sharing'), and when you do, it'll tell you how to access the RAID set as an SMB URL.
Note you can also mount the same RAID set using NFS and AFP.
Frankly, this is not as cheap as the Linux-based alternatives, but I'm adding it here for people who are interested in RAID in a cross-platform environment. The next step, if you want to do cross-platform, is to look at Mac OS X Server, which has really great cross-platform capabilities wrapped up in a -relatively- easy-to-use package. (I bought a copy of OS X Server to support networked home directories. It'll do networked directories for both Macs and PCs off the same file system, which is kinda cool. After I get the Mac side all worked out, I'll give that a try on our token PC.) I'm running a 10-license copy of Tiger Server that I got off eBay for $300, running on a Mac Mini, but with the home files mounted from a RAID array in my G5/2DP. The Mini will host Open Directory for both Windows and Macs. Another neat feature of Tiger Server is 'roaming users', where the system mirrors the directory information and the home directory when the machine is plugged into the home network, but the machine runs just fine standalone when on-the-road.
dave
Stuff to watch out for is overheating, make sure to monitor harddisk temperature and install enough fans (I use a Antec P180 case which has 12 cm fans, and all 8 harddisks run at or below 35C at all times and it's all still pretty quiet).
Also set up smartmontools to mail you when a harddisk failure is imminent or when RAID5 has a disk failing. I've had 2 harddisks fail in the past 3 years (Maxtor) but both were in warranty and they simply send me a replacement.
Also make sure you have enough power. I use a 300 watt PSU, which only barely is enough to "start" the machine (it sometimes refuses to start because of the harddisks spinning up taking too much power, but simply starting it again while the harddisks are spinned up halfway is usually sufficient to get it started all the way -- quite funny when I discovered this for the first time).
The first version of this setup was a standard ATX case which I stripped for as much 3,5" disk room as possible (I mounted 4 in the 3 top 5.25" bays, and another 4 below the disk drive by hanging the harddisks from plastics strips with mounting holes in them. The antec case I'm using actually has enough bays as standard to mount all HD's normally.
The crypto I added is just some wierd thing I wanted to try sometime. At the time I set it up, it was impossible to change the password of your encrypted devices once created. I worked around this by creating a small encrypted partition, which has the ridiculously complex password for all the others. Whenever the password needs changing, I just change the password on the small encrypted partition which has the script that mounts all the others. Performance impact of the crypto is harsh though, average performance of the RAID5 is about 15 MB/sec (reading/writing), however this is still enough to feed a 100 MBit network, and far more than you'd need for video streaming (if that's your intention).
My humble home setup consists of 8 300 GB drives, 1 extra promise IDE controller, a gigabit NIC and linux to do software raid 5, plenty of cooling, plus some stuff to mail me when things go wrong.
One of these babies, maxtor shared storage, the 500gb version, offcourse. It all runs Linux ;) OpenMSS and got nice, high transfeer speeds (in difference to an NSLU or ASUS wl500gx wich uses USB and dont reach more than 1-1.5mbytes/sec).
I would then plug in two decent USB disks, set up LVM and then have the device to store the most used data and all data currently being uploaded at the local drive, and all old and not to frequently accessed data would be stored on the slower USB drives.
Nice and convenient... And, considering the storage capacity, I dont think you get off cheaper if you where to buy a regular fileserver (ordinary computer whit a lot off disks), and it drains less power aswell.
On the other and, if you want to get away really cheap, I would go with an xbox and two ordinare IDE drives at about 250gb each, and a little chip...
I like the idea of NOT going with identical drives. Use RAID10 and use two sets of drives from different manufacturers. That way if there is a common failure component like many lines of drives have had over the years you aren't out all your data when the drives decide to give up the ghost at the same time due to an identical fault.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
A networked, mirrored RAID system is nice and somewhat safer than not mirrored and all but that is still not anywhere near anything like a "backup".
:)
If you get a 1 TB mirrored RAID array going say, you will then need a third 1TB of REMOVEABLE storage to "backup" that system if you actually care about the data on it. A messy power outage, lightning strike, fire or other mishap can take out data even on the safest of RAID configurations. Your data will be MUCH safer sitting somewhere not connected to electricity or being accessed when disaster strikes.
Make backups on removeable media and then keep the media away, in a fireproof container or off-site.
In fact, if money is tight, i.e. can't afford THREE times the drive space as you really need for mirroring+backup, it is far safer to buy two sets of drives but do not RAID them, instead of mirroring use the second set of drives for your removeable backup!
SATA now allows you to use hard drives as live removeable media (as long as your SATA controller and driver actually supports it).
Then of course, remember to sync your on-line and off-line storage regularly
Contrary to popular belief, coding is not all free blow-jobs and beer. Those things cost MONEY!
RAID is an overly-complex solution to this problem. Get a fileserver with two big disks. I've done Suse with both Samba and NFS with little or no knowledge of either. rsync nightly. Every computer in the house has shortcuts in "My Documents" or "home/myhome" that point to directories on the share. Life is good.
If it's really that important that you are worried about your house burning down, then a RAID machine wouldn't solve the problem either. You need to print stuff out and keep it in a firebox.
-KB
Quantum is synonomous with failure. In six raid5 arrays all of the Quantums failed over three years. This was not a fluke in purchasing or a one time production problem. There were two Quantums in each array and the others are IBM. All of the Quantums failed...the last one in December. If you use Quantum watch the log files closely.
Stay away from it. Support LSI, they're more friendly to the free software community!
Not sure why you'd say that. The drivers are in the kernel tree and 3ware includes Linux drivers on the CD.
Assuming you've already discarded the idea of RAISS (Redundant Array of Inexpensive Samba Servers...which is the solution I use) or are just looking for something different...
If you can find a drive case (like a regular case but the supply does not have mainboard headers and there's no MB plate), you can build a whopping great USB backup unit. Get two of those 4 port hubs for a 3.5" bay and shave them down so they'll stack in a 5.25" slot. Then get a bunch of IDE drives and a USB adaptor for each one. Assemble with the cable ends labelled and sticking out of the gap beside the hub(s). Voila...1.8TB of plug-and-play storage for ~$1000.
every one needs a backup solution, and we'll be witnessing 1TB+ needs in the home. maxtor made the one touch button backup(retrospect) drive solutions popular, but the one touch button solutions will eventually have a diminishing point of return as drive sizes get larger. if you can roll up your sleeves and roll your own, i think a 2 raid system is the way to go...
2 raids on a single system: primary raid built for performance, secondary for backup and redundancy.
primary raid = raid 0, 2x250gb sata
secondary raid = raid 5 3x250gb pata
most modern mobo's come with both ata/ide and sata, or if necessary add the extra host cards to run the additional drives.
hardware raids are preferred over software, for performance and allowing i/o functions to be offloadede to a dedicated controller.
software for back up to the secondary raid? how about good ol' tar or even tarpipe?
tape drives are great if you can afford it. unfortunately i can't afford an LTO2/3 drive(and please no references to refurbished drives! refurbished tape drives are as dependable as retreads). the tape drives which are affordable are either too slow or not capable of handling the capacity needed(or requiring spanned tapes)
three can keep a secret, if two are dead - benjamin franklin
This may not be as appropriate for large amounts of data, but it is what I use at college.
I use rsnapshot to incremently backup data every four hours to my server (which then allows access with samba or ssh). rsnapshot creates hardlinks for every new backup it makes, and then uses rsync to only create files which have changed. This allow backup snapshots to be at intervals (from hours to months) with relatively small disk space.
Additionally, I use unison (basically a two-way rsync) to sync documents, photos, music, and files for projects, so I have the more important files backup'd in two places.
Now, I don't know about you, but I could certainly live with even a month or so of lost info on my *home* network... 1-2 nights certainly isn't going to kill me.
If RAID 5 isn't quick enough for you, go for RAID50 or whatever.
Just don't expect that to be cheap...
smash.
I run: Windows, OS X, Linux, FreeBSD. Just because you have a hammer, doesn't mean everything is a nail.
Lots of posts. Here's my 2 cents.
:)
Find a cheep machine, used. The MoBo, PwrSup, CPU, Case is cheapest that way. Load it with about a GIG of RAM. That'll set you back about $30 and save wear & tear on your disks.
Get 3 4-port SATA cards (non-RAID). There is a reason why MoBo's come with only two IDE ports (for 4 devices). IDE (PATA) cable suck. SATA is so nice. If you make more than $10/hr it's worth your time to get SATA. And the SATA drives are just as cheap as PATA/IDE now.
Get 12 large HDs. Get the best GB/$$ ratio. Don't try to predict the future. Prices will drop, availability will always be there. Just get the best price. Oh yeah, and get them from a small shop, you can get a good deal that way. I got 250GB WD's for my file server.
Get the latest Redhat. The install time is the shortest, thus cheapest. And your fileserver will not need the best compiled binaries from GenToo. Your network will be your bottleneck, even if you're using block device encryption. CPUs are fast, and so is SATA.
Setup raid using mdadm. It's simple and easy. Thus, cheapest. Oh yeah, use the largest chunk size you can. 1024k is what I used. There are reasons for this which I will not go in to. Point is: music/movies are large. Let your machine eat up your RAID data in the largest pieces possible.
Format it using ext3. It's supported by all rescue disks. This will save you time when things go bad (don't they always?) Also, ext3 has stepping optimization for RAID. Use the largest block size you can and google for "ext3 stepping".
Load SMB, ftp daemon, and rsync (in daemon mode). Then get a modded xbox to play all this in pure candy.
Get friends who love:
- music
- movies
- taking photos of their lives
Make a pact to give them all your music and movies (which are not copyrighted of course) and make them agree to rsync your personal photos in exchange. The Key: Automated Offside Backup of Vital Data Using Rsync and Your Friends (or "AOBVDURYF" for those who have no life).
Then you're off to the races.
Personally, I've got an IBM Netfinity 5000 (Dual P2-400) with an external EXP300 drawer I got used for almost nothin'.
:)
I've got (5) 18GB drives and (13) 36GB drives for a good chunk of storage, driven by two ServeRAID 3L cards. Got a DLT drive hooked up to it to get extra backups of the really important stuff.
The electric bill isn't very nice, but it's reliable.
Forget RAID, get tape. It's cheaper, so you can have, you know, more than one tape with similar data on it.
Hey, at work we use a NAS server from Dell. The system works very well. Network Attached Storage is an easy to use system.
It's sort of ironic, I'm actually in the process of doing just this, but for work. The data is not mission critical data, but it's important enough that we need some redundancy.
Here's the basic solution:
1. Slackware linux
2. Multiple pairs of harddrives set as mirrors (raid1)
3. Combine these mirrors into a single volume with lvm
4. Share the volume using samba &
5. For easy managment, use webmin
I'm using standard IDE drives because their cheap, and we've got a half dozen laying about. One nice thing about raid1 on linux is that you do NOT need the two harddrives to match in size. The size of the mirror will be the size of the smaller of the two drives. LVM allows me to take a lot of the smallish mirrors (60gb-160gb) and combine them into one large volume (total is 580gb). Because of the mirror'ing, there isn't much concern about a single dead drive taking out LVM.
I'm running this setup with a Pentium III-733mhz w/128mb memory, and have found the only bottle neck to be the speed of the drives themselves. A suggestion about that: Make sure that the two drives of each mirror reside on different ide channels; this improves things noticeably.
peace,
nathan o'brien
A local scrap dealer near me acquired 15 raid drive array boxes. Most of them (10 I think) are HP Netstore 6000 series with 23Gb drives in 10 bays. Most of the boxes had some superficial damage but appear to be in working order. I plan on attaching several of these devices to my home network when I get some time to work on them. The best thing is I was able to pick up the better boxes (12 or 13 of them) for very little money as electronic scrap. Check your local scrap/recycling centers!
I have been considering using my mac mini, and an external dual drive firewire case using software raid 1.
I would rather have gb ethernet, but since I already have all the hardware its very cheap.
Any comments?
While it's certainly not low budget, I still saved a bundle by not going with an enterprise class solution. I have a little money to burn so here's what I got:
10x 200GB Seagate SATA 7200.7 Barracudas (RAID 50)
2x SuperMicro 5-Drive Hot Swap SATA Backplanes
2x 80GB Seagate SATA 7200.7 Barracudas (RAID 1 for OS)
Adaptec 21610SA 16-Port RAID Controller
I use FC4 Samba and NFS for shares to clients.
Just over one year ago I had the hard drive on my primary box die a sudden death (no chance to recover anything). That became sort of the last straw for me (fortunately, I did have the most important data backed up). I began to search for a good backup solution. After doing a bit of research, I settled on using BackupPC ( http://backuppc.sourceforge.net/ ). I have an old 500Mhz pentium Linux box which is dedicated to running the backup server. I only have one 160GB drive in that box which is used for the backup pool. Currently it is at 81% capacity (but the usage grows very slowly. I am backing up about 5 other computers throughout the house (all in the middle of the night, when I am usually not awake). The machines being backed up have probably an average of 100GB of storage each (it is a mix of Linux/FreeBSD/Win2k boxes). Each backed-up machine runs an rsync daemon which BackupPC uses to grab the data off the machine. I have been very pleased with the results. I have one weeks worth of backups (1 or 2 full, and the rest incremental) of each machine. I have a second 160GB hard drive in the backup server which I can use to make a 2nd full backup of the entire backup pool (I just have to shut down the backup server, plug in the 2nd hard drive, boot from Knoppix CD and run 'dd' to copy the original drive to the 2nd drive). Some users of BackupPC use external USB drives to copy the backup pool. I don't have any USB drives, and the second 160GB drive was originally purchased for growing the size of the pool (which has not been necessary in a years time).
BackupPC makes extensive use of hard links for identical files, which is why copying the backup pool is a little more challenging--it is the reason many folks do a full disk copy to make a backup of the backup pool itself. However, I don't feel the need to backup the pool very often (I don't need many backups of backups), and even if I did it's not that difficult of a job.
All in all, I am very happy with the solution I ended up with. It does the job well, and restoring files is pretty easy (using the web interface). BackupPC also emails me automatically if one of the machines it is supposed to back up has not been backed up recently (like, if it was shut off or unplugged from the network for a few days for some reason). Very nice feature. I would highly recommend BackupPC to folks looking for a reasonably cheap and reliable backup solution for their home network. It is highly configurable, and not too difficult to set up (and hey, it's free [dollars-wise] and open source, written in Perl, which makes it easy for me to customize).
Sorry. I have no suggestions when it comes to hardware. But, I will tell you this: For anything (Other than windows clients... so perhaps run samba for them), save yourself the trouble and use NFS if at all possible. I worked and I worked, trying to get SMB set up. After all, it was supposed to be easiest, especially if connecting to a windows machine. Wrong. Wrong. Wrong. SMB is a very fragile protocol IMO. So many things I dislike about it. NFS is a much much better solution for data storage. Jeeze, if you're using a *NIX on your client you can even NFS mount your home directory. (Consolidating your data making it even worse if a hard drive crashes. Just kidding). Ok, yes, windows can't (Natively at least) make use of NFS mounts, but I see no reason why you can't run nfsd and samba beside eachother. I even got NFS working on my windows machine. (Serving... never tried client)
He, he, he. Did you ever get the user mappings to work? :)
For those who have had the ***s to use it, they will know what I am talking about.
There are some great posts on this topic in a past Slashdot discussion (Taco should've done his Googling ffs, it was only 2-3 months ago that the discussion in question was on Ask /.)
3 37226
The discussion in question http://ask.slashdot.org/article.pl?sid=05/11/26/0
The basic idea:
Split drives into small partitions, say 20-25 GB each. Since most drives available now are a multiple of 50GB, I suggest going with 25GB or 50GB per partition. Make software RAID devices out of sets of these partitions, one on each drive. e.g. md5 = sda5 + sdb5 + sdc5 + sdd5. Take all of those smaller RAID drives, and then LVM them together.
I just set up such a system on my dad's fileserver back at home, and will be doing the same with a machine I'm building within the next week or two. So far my opinion is that this approach ROCKS.
There are more details on neat tricks you can do with such a RAID + LVM setup in the discussion I posted the link to. Among other things, if you have a 150GB drive and three 250GB drives, you can have four-drive RAID for the first 150GB of the drive set, then 3-drive RAID with the remaining 100GB.
retrorocket.o not found, launch anyway?
Can you use two power supplies and connect the grounds as a reference? Or is it going to explode?
Thanks.
This is what I did: http://secure.newegg.com/NewVersion/WishList/WishS hareShow.asp?ID=1764600
/boot partition across all 8 disks. (Linux raid 1 will do more than just 2 disk mirroring) The main data partition is a 2.2 TB raid5 but I seriously considered raid6 and I'm keeping the option open. I like the idea of being still protected in the case of a disk failure since I won't be keeping a spare on site (unless I do.. hmm.)
It's a 3U rack mount 8 drive server with a nice dual-core cpu to keep the software raid moving along. I directly attached all 8 drives to the motherboard (it has 8 SATA ports). Grub has some trouble figuring out which is bootable but since I wanted all disks to have grub on them (in case of a failure) that's just required a bit of fiddling. The big thing I needed to do was plug all the fans in the box into 5v instead of 12 since they are insanely loud otherwise. This required a bit of fiddling to rewire the power supply fan but nothing hard. Since not too many people have enclosed racks at home a quieter desktop case would make that a non-issue.
I did a raid1
This is a decent box but its a lot more work than should be involved in getting a simple home server. I will note that I get about 120 MB/s write speed and about 150MB/s read speed (sequential) which seems quite good to me.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
Companies such as streamload (www.streamload.com) will allow you to store up to a 1000GB for a fairly compelling price (compared to buying a Buffalo Terrastation or similar).
Pyrmaid scheme o
Check out the Buffalo Terastation. Great for small businesses. I have installed a few for my friends. No longer have to go scouting with them for used Micro$oft servers. http://www.buffalotech.com/products/product-detail .php?productid=97&categoryid=19
...if you want it to be.
...also, with s/w RAID, you can use partitions as raid devices, and so you can use devices of different sizes - it's the partitions that must be the same size. You can even make an MD/LVM device from multiple other devices until they get close in capacity to a bigger one, then RAID them both together. You can do pretty much what you need to do with s/w RAID. Of course, you'll loose space from any mismatch in size.
Just make a RAID1 over two disk, wait for it to complete the mirror, then fail and remove the second drive - store it somewhere safe like a normal backup.
Get an extra drive, and you can cycle the disks.
Think of it like a your backup process (tar/whatever) is running continuously, and you choose when you want to make a snapshot.
100baseT is too slow when you're dealing with doing a complete backup of 250 GB is data. I did a backup using their software of 100 GB and it took 24 hours.
Any type of computer-to-computer transfers of large amounts data requires gigabit ethernet, made better with jumbo frames (at the expense of regular tasks). Don't forget to defragment your drives as well because this has a huge impact on performance.
That's the funniest post I've read in a while. Let me get this straight:
-You're using cheap old drives that are most likely going to fail soon (20GB drives aren't exactly this year's model)
-You're using the worst brand of HDs - the only ones we have failure problems with. (we have over 2000 workstations just at this site), and they're the ONLY HDs that ever go bad (no issues with maxtor/seagate/samsung and others). Same thing for friends, relatives and family members. I see TONS of WDs failing all the time - more of them than all the other brands combined. They're so bad that I ensure every computer we buy doesn't use WD HDs. They're the ONLY brand that's total crap. My data is worth too much to put on such unreliable junkers.
-You're using such small drives that to get any kind of size one is looking for (say, 1TB at least), you'd need like 50 drives or more. That's like the stupidest thing ever (well, you're using WD drives too). So you have to have tons of them around, maximizing risks of having one fail, maximizing noise, maximizing heat (which kills drives and costs more AC in the summer) hence also ventilation (more noise), maximizing the amount of space used by the drives, maximizing the amount of large clunky 80pin cables in the case, maximizing the amount of controllers [ports] one needs to run the thing (I can imaging already see all the PCI slots filled with IDE controllers), maximizing the amount of power required to have any amount of storage - that alone probably makes it more expensive to run than running off bigger yet more expensive drives (15 of those VS a single 300GB drive...) - not counting the extra cost of the dozens of coolers, which also makes more stress on the PSU (especially since you don't get staggered spinup), etc. There's also no expansion possible here, short of adding another PC full of PCI IDE controllers and clunky drives (14 of those drives - 3x 4port-IDE controllers + 2 on the motherboard - will still give you less space than a single 300GB drive but the case will be crammed full of drives, messy wiring and noisy fans)
Or perhaps you're making a joke, and this should have been modded +5 Funny!
That's the absolute utlimate most suckiest setup one could ever make. Way more expensive to run and more troublesome to run and maintain in the long run - I'd basically take ANYTHING else over that.
Keep it Simple...
Primary data storage should always be on your pc(s). That is the most convienent and easy place to use it...
Setup a linux box with four matched SATA drives (e.g. 300GB) setup as a large striped drive. Most modern motherboards will handle this without problems.
Use backuppc to backup your pc(s) (linux, windows, mac, anything that runs SMB or rsync) to the large striped drive.
You don't need RAID as the large stiped drive is just a backup copy. And (hopefully) won't crash with hardware problems at the same time as the drive in your pc(s).
For extra redundancy place it offsite somewhere and backup over your cable connection. Swap machines with a friend. Host his backup at your place, he has your's.. Then in case of disaster all of your data is still safe.
Cheap, simple and robust. Just setup, configure and ignore.
I don't know about all you wackos with the 600$ CPU's and 600$ RAID controllers at home, but I have better things to do with my money. Like invest it rather than spend it on useless trinkets, theoretical seek-time figures and unused gigaflops.
/proc/mdstat is wrong.
Here's my brew:
1. Old PC. Any one would do, probbably even a good'ol P1. 128MB RAM is more than enough. I consider this FREE. I run a dual-PIII-450MHz that I have lying around.
2. 4x[BIG-SATA-DRIVE]. How big? When I built mine, highest bang-for-buck was 250GB. So I went with 4 of those.
3. 1x PCI SATA controller.
4. 1x PCI GbE NIC.
[3] and [4] are peanuts. [2] is worth, what, 500$?
The entire rig will easily give you ~10-25MB/sec, which is, for any home use I can consider including pumping 10GB files over the network, plain enough.
Plug any crap old 2GB or greater IDE harddrive in for sport (or two and do yourself a RAID1 configuration).
Install Linux.
Install SAMBA.
Configure RAID.
Set up healthchecks that email you if something in
[OPTIONAL]
1. Grab several old IDE drives. Not neccesarily same sizes.
2. Stick them in some other box (I did it on my windoze box cuz that's where I had case space).
3. Configure a RAID0, or better yet, a spanned volume. Use windoze dynamic disks, use LVM, whatever makes your boat float. Set up a compressed filesystem if you think that would help any. Usually, with the kind of things people store on huge arrays at home, it won't.
4. Do a daily dump of everything from your RAID to your backup array.
DONE. Forget about it and go do something better with your time.
-
I've wanted to build a fileserver at home for a while, and move everything over there. Between all our computers, it'd probably have 3-400 GB.
/*.... Oh shit!" problem.)
I realized, though, that I'd be creating a single point of failure, and that routinely backing up 400 GB of data would be a royal pain. I know RAID is supposed to reduce the problem of disk failures, but in my experience, outside of big, professional setups, RAID is often poorly setup and is just asking for problems. Back when I was in high school, we lost our entire fileserver because one drive in the RAID array died. (Few people saw the irony?) RAID actually caused more problems. (But nightly backups saved the day.)
So my project of building a home fileserver has morphed a bit. I don't want to build a fileserver anymore. I want to build two, and keep them mirrored. (Or possibly, have a primary one that's a beefy system, and the second one a lower-end machine that I do a nightly rsync to. This solves the "rm -rf
________________________________________________
suwain_2
A guy set up a very cheap .5TB raid 5 system. go check it out. www.digg.com
Nope, but it's SO far beyond easy to implement that it almost wraps around to painful and difficult again. If you aren't trying to save a few bucks by using some old machine, just go to the local Fry's and buy a Mac Mini and two or three FireWire drives. Take it all home, find someplace with 1/2 square foot of empty space, plug it all in, boot up, run Disk Utility to make RAID 0, RAID 1, or concatenation as desired. Then finally, turn on "Windows Sharing" under Sharing in the preferences.
If formatting disks didn't take time, once the machine was booted up, you'd be done with the whole process in about 60 seconds.
After using 3ware pci p-ata controllers (2/4 Ports) for years and enjoying their great linux support, i needed more space (huge audio cd collection ripped with cdex, regular tv show recordings, digicam).
I didn't want to buy an expensive server mobo for the pci-x slots needed by newer 3ware cards so i started with a nforce4 939 from asus and its onboard sata + one medium prized promise. 8 250gb seagate 7200 drives equals a solid 1.5TB using mdadm (raid5, 1 spare) and lvm2 on top of it. Never again, evil driver situation with nforce and promise and after a massive data corruption (endless fschks) I went on to an areca 1220. Afaik the only available 8-port (12/16 avail.) for pci-express. after an upgrade of the mobo's bios as mentioned in arecas FAQ, i now have a decent raid6 with 1.5TB. This would allow 2 simultaneous drive failures against 1 with the former setup. Easy driver installation on rhel4 (centos), great management tools and webinterface. I'd never go back....
I've just built a 2TB RAID server as a hot backup server for work, there's plenty of pre-built stuff on the market but I found it's all way overpriced if you're looking for RAID-5 (or 6). I've got 2TB of usable space on a RAID-5 server for around £1,700, and it's upgradable to 3.5TB. Standard solutions would have cost me anywhere from £5,000 to £12,000.
:D. asp?segment=RAID%205%20HBAs&product_id=148
? chs=743.
:)
If you're after RAID-5, don't forget that if a drive fails you want to be able to swop it as quickly and easily as possible. That means buying a spare drive at the outset and ideally having a case with hot-swop capability.
My solution was a Promise RAID controller, a whole bunch of cheap SATA drives and a case with 8 hot swop SATA bays. I'd always recommend Promise for IDE or SATA RAID, I've had too many problems with Adaptec cards in the past, arrays lost, cards fried, etc... Promise cards have always recovered from everything I've thrown at them, even when I've expected to loose data.
I used the SX8300 which, thanks to Slashdot, I've just found is upgradable to RAID-6, sweet
http://www.promise.com/product/product_detail_eng
I also found a superb case from Supermicro, the SC743. It's a dream to work in and has more than enough cooling for all those drives. It is a little noisy though:
http://www.supermicro.com/products/chassis/tower/
Unfortunately Promise don't guarantee hot swop ability with this case - they'll only guarantee it if you're using their own enclosures, but personally I'm happy enough to take that gamble. My feeling is that the worst case scenario is that it doesn't recognise a new drive automatically and I have to reboot, I can't see me frying a drive or loosing the array.
Good luck with your project, let us know how you get on
Ross
Had this about 2 years now, without fail...
2 * 9 gig Raid 1 SCSI OS drive
5 * 250 Maxtor Maxline II, in raid 5
Running Gentoo Linux, no hardware controller, all handled by the kernel. The rest of the machine is an AMD 500, 256megs ram. No X, nothing un-needed installed.
CPU usage never gets maxed out, and I can copy too, or read from at the max a 10/100 network will allow.
I'm very happy, as its got an uptime of about 200 days.
I've tried Naslite for NFS backup disk, but was not really happy with the 4 IDE drive limitation.
Ideally, I'd like to have a linux live CD on SCSI CDrom, 4 IDE drives over two onboard channels, and another 2 drives over an add-on PCI drive - All of it booted from CDrom, configured with LVM, shared with either NFS or Samba, and a config saved to USB hard drive. Although Naslite has somewhat the right idea, it only allows 4 drives, and doesn't support IDE PCI cards.
Can anyone recomment a Live distribtion that contains drivers for IDE controllers, and in addition, will save a config on a USB key?
Naslite comes close, except for its limit upon the # of IDE drives, and lack of IDE card support.
Seeking any recommendations.
As mentioned elsewhere in this endless commentary, I can confirm both that the Buffalo TeraStations are quite slow, even using a 1000base-T network, and that Buffalo's tech support is horrid. I would not recommend a TeraStation to anyone, even those looking to save money. Fortunately for me I was able to convince Outpost.com to accept it back for a full refund, even after a couple frustrating months of ownership; I doubt Buffalo would have given me the time of day if I'd been forced to ask the same of them directly.
I wound up building my own solution for less cost, but one which has much better performance:
LSI Logic MegaRAID SATA 150-4 controller
4 x Hitachi 250GB SATA II drives
Addonics 4-in-3 drive enclosure
I got the Hitachi drives for about $100 apiece, though a guy in Fry's was buying what might have been the same drive yesterday for $50 with hefty rebates. I chose the Hitachis because of good internal performance and overall specs and because they demonstrated the best effectiveness-to-price ratio that I could find at the time. The fact that they happened to also be SATA II wasn't even an issue, since there's not a single current 7200 RPM drive with a platter-to-buffer data rate that can even match the 150MBps bandwidth provided by SATA I. Basically SATA II is, for the moment, nothing but a marketing gimmick.
The LSI Logic controller was inexpensive and somewhat "obsolete", but received good reviews and demonstrated average or better performance. I had also decided that I wanted to avoid partial-software RAID controllers, so that excluded a number of competing products, and since I didn't have PCI-X or PCI-Express slots, only PCI 2.2, there were several newer products I couldn't even consider without a new or major system upgrade. Of those choices that remained that didn't cost a small fortune, I concluded the MegaRAID was my best choice.
Finally, the Addonics enclosure allowed me to cram the four drives into the space of three 5.25" bays. It also offered flexibility, since Addonics also sells an external drive chassis into which the enclosure fits; that would allow me to make the RAID array external if I should ever need to do so.
I've had the result functioning for almost two months now without a hitch, with much better performance than the TeraStation, and for less than what I paid for the TeraStation. Since I already have a 1000base-T network (in part thanks to the Buffalo misadventure), I'm also sharing the RAID array on the network with good results. I also don't have to deal with Buffalo's uncooperative and unresponsive excuse for tech support; the unfulfilled promise of support is far more stressful than having no support at all. My couple conversations with LSI Logic's support staff have been notably more productive.
The old cliche is true: if it sounds too good to be true, it probably is. That cliche applies to Buffalo TeraStations.
Mark
A combination of:
- A failed hda system disk
- Which took out the superblocks (yes, all of them)
- Which meant my md0 4x40Gb IDE software raid array wouldn't recover (no superblock)
- And the backup on hda went too
I can only say - Raid is good for h/w failures, but ecstasy is a backup volume on an independent device! (preferably located in a different reality where they don't have lightning, flood, fire and children with fizzy drinks)--
But I'm Conroy's plant!
--
And remember that some 40Gb drives are not the same size as other 40Gb drives. And the drive sizes aren't fixed in my example.
We recently had a requirement to replace a drive in a RAID 5 set but the 160Gb drive we bought to replace it from the same manufacturer as the original was a different generation and actually smaller.
We ended up having to purchase a larger drive to put in there.
Matt Thompson - Actuality - Insert product here.
The easiest solution is to get a Linksys NSLU2 and two external (USB) drives of any capacity you like. That'll get you the SMB shares.
Then configure the NSLU2 to back up from one drive to another. While technically this isn't RAID, it's accomplishing the same ends.
That should do the job. Then you can proceed to hack it, if you are so inclined :)
I'm in a Unix state of mind.
How about daisy chaining a few external firewire cases loaded with HDD's of your choice? I'm starting my little pseudo-raid like that.
Cheap... costs me $150 AU for 200GB HDD and $50AU for the case.
Portable... we had bushfires recently and had to evacuate the house. I realise thats not everyones possible dilemna, but hey.
Easy... I wrote some simple batch files for backup and syncing. Works perfectly. If thats doesn't work for ya, get some software.
Just my three cents for anyone who's interested.
In my home I have a cobbled together system that cost me almost nothing and made up of: x1 old salvaged mini atx case £0.00 (taken from the dumpster outside work). x1 200W atx salvaged power supply £0.00 (having been stripped out of an old system years back and kept because it is totally silent when powered up). x1 Cyrix K 300 MHz processor and motherboard £0.00 (stripped from a system found in a dumpster -an old Packard Bell I think- but kept because it has on board video connector which I thought might be usefull someday and set to share 16 Mb of system Ram). x3 64Mb PC100 Ram chips £0.00 (leftover from previous upgrades). x1 Heatsink from a damaged/ unrepairable Athlon 1 GHz machine £0.00 (thrown out at work and reshaped/ cut to fit processor above so as to replace CPU heatsink and noisy fan on above mentioned mobo with a passive replacement). x2 10/100 Mbit network cards £0.00 (again salvaged from machines found in dumpsters) x1 CompactFlash to IDE adapter £8.00 -including delivery (ebay -make sure to bid on items ending midweek between 1 and 4am to get the best chance of getting a good deal on ebay). x1 512 Mb Compact Flash Card £0.00 (kept from an old digital camera I no longer use and used as the boot/ os drive for the RAID 5 system -technically £28.63 including delivery if bought from from www.aria.co.uk -E51 512MB Kingston Elite Pro Card). x2 IDE Ribbon Cables £0.00 (salvaged from various machines machines found in dumpsters). x3 200GB Maxtor D/Max+10 133/7200 rpm 8Mbit cache £198.30 (the most expensive item at -£66.10 ea. including delivery from www.aria.co.uk- but could have been done for nothing with salvaged drives if I wanted less capacity). x1 OS £1.00 (download FreeNas BSD based OS which supports Raid 0,1,5 from www.freenas.org/download.html and burn to CD hence £1.00 -cost of Cd media). * temporary use of Floopy drive (to install firmware on Harddisks and Motherboard to allow the proper recognition of the drives). * temporary use of CD drive (to install OS onto CompactFlash Card). Total cost of system: £207.30 (or £366.63 if you include the cost of the Compact Flash Card)- overall not bad when you consider that a commercial Nas system with similar capacity and redundancy will cost around £1200 as my system would still be considerably cheaper even if you added the cost of a raid 6 card and another redundant drive). *Array capacity at RAID 5 -400 Gb * Note that many will criticise a software based RAID 5 array due to the processing overheads -but this isn't really a problem with a NAS system as the primary bottleneck to performance is the bandwith of the network connection (100 Mbit connection is the limiting factor here). * On the pluss side -this system is relatively easy for anyone to set up -even a FreeBSD newbie such as myself- and It has run smoothly for a good while now. *I have now moved it to the hallway cupboard and rerouted an old hub in there to connect it to my household system and it appears very happy indeed).
My setup which works well is simply a Windows 2003 server with 2 x 250GB secondary drives, 2 x 30 GB drives for the OS, using Windows to manage them as mirrored sets.
Then, on my XP desktop, I map the "My Documents" folder to a shared drive on the server.
Then, I set the "My documents" folder to have "Offline Access".
Now if the desktop dies, the server has the files. If a drive dies, I can swap it out. If the server dies, the desktop has the files. And they are all synchronized automatically when my PC is idle.
I've lost a few HDD's over the years, but have never had to rebuild anything. Just shut down, pull the dead drive and replace it, reboot, and let the server rebuild itself. And even while the server was down, I can still work on files on my desktop.
http://translate.google.com/translate?u=http%3A%2F %2Fsupertank.iodata.jp%2Fproducts%2Fsotohdlgw%2F&l angpair=ja%7Cen&hl=en&ie=UTF-8&oe=UTF-8&prev=%2Fla nguage_tools
supports bonjour, itunes streaming and webdav.
I bought a P400T at Fry's for $700 which gives me ~700 GB of RAID 5. Performance w/ NFS is ~ 4 MB/s write & ~ 6 MB/s read. I've seen complaints about slow SMB/CIFS performance, but I don't do that so I can't say if it's actually slow or just unrealistic expectations.
h tm
System is very quiet. Power consumption is ~75 watts (i.e. ~5 times one disk drive) It's an ARM processor runnning Debian.
http://www.anthologysolutions.com/products/index.
rhb
Best solution overall I Have see is the Buffalo Terastation.
They arent cheap, but they wont break the bank either.
The only drawback to the standard Terastation is its lack of support for large (>2gb) files. I just noticed they now have a "pro" version, which I have to assume supports large files but have not confirmed this.
I have a client using two standard terastations and they works beautifully.
What kernel are you running, and are you using any non-mainline patches? I've noticed fairly massive differences in NFS performance lately when comparing 2.6.14/2.6.15 & the -ck patchset. In summary, 2.6.14 had crappy NFS performance for me in every incarnation. 2.6.15 mainline is much better, although some small decrease occurred beteen -rc5 and the release version. The -ck patched kernels have _atrocious_ NFS performance. With 2.6.15-ck1, I can't get more than 60Mbit over my gigabit network, whereas 2.6.15-rc5 (which had the highest NFS performance of the kernels I've tested recently) gave me over 500Mbit.
The other replies to your post, with tuning tips, are more likely to fix your problem, though, but I figured this might nonetheless be interesting as a data point, especially if the standard Ubuntu kernel uses any of Con Kolivas's interactivity patches from the -ck kernel patchset.
Floppys...it's funny how more replies are useless than useful here. Everyone thinks they are a comedian (and most of you, don't quit your day job). And those same people mod up each other with score 5: funny. I don't know why anyone bothers to ask a question. Feel free to mod me a troll despite the fact I'm giving an actual viable answer to the person's question.
Because I'm a developer and my source code is my life, I use a combination of
- mirrors on my dev system
- nightly offsite backup via Quick Online
- hourly backup via GHOST to a second drive on my dev system
- daily WINRAR of all my source to date based file name
- And of course source control
My MP3 drives:
- Mirrors
- nightly copy to a second (third actually) via a really cool program called Second Copy (www.secondcopy.com) It's a smart copy that only gets what has changed and will basically do a drive sync (in this case, I'm not doing a sync - the mirrors are the master copy)
IMHO, the best of both worlds and cheapeast - Mirrors for where you work (play) and third drive to copy the data to periodically (offsite backup with MP3s isn't practical except manually)
http://www.openfiler.org/ This is a NAS distro that has a lot of features that can be seen on commercial NAS boxes such as NetApp solutions. One of the features that could be very beneficial are customizeable snapshoting capabilities which allow for instant data retrieval in the event you have some sort of data corruption. I have tested this software several times and found it very easy to set up and configure. There are plenty of other bells and whistles that I wont go into detail about but it is one solution I would definetly look into.
Assuming it works as designed, this Freecom FSG-3 storage gateway looks like an answer to most home user's needs. It does webserving, SMB, email, network printing, firewalling and more out of the box. It even runs Linux, which assuming on how open they make it might even allow you to make it run a Subversion server or other esoteric things. It's not RAID though it does have USB ports so perhaps you could attach one.
Hence it's best practice to configure your RAID (be it hardware or software) to use slightly less than the full capacity of a drive, in case you can't get an exact match if/when you need to get a replacement.
Just hope you don't get fired and walked out the door. It may take weeks or months to convice the powers that be, that the HD is yours and does not have any company secrets on it. You may not personally fact this situation, but others may not have a good working relationship with their boss.
Never trust a man wearing a coat and tie!
This doesn't work right?
** For GPL license information and source code availability, please refer to the CD-ROM included with your Infrant Product.
Found on this page http://www.infrant.com/learn_features.htm
You should check out NasLite from www.serverelements.com if you are short on cash. It doesn't support RAID but here's what I did: I rescued an old Celeron 366 eMachine from the trash and removed the CDROM and 4GB HD. Then I found 300GB Hard Drives for just under $100 each on Newegg.com and bought 2 of them. I put a $19 D-Link Gigabit NIC in it, and booted up to the NasLite floppy. You can administer it from telnet, so it can run headless. Now I have a .6TB file server that only cost $220.00. I basically use one drive for data, and I use an XCopy script each night to back it up to the second drive.
Netgear SC101
a Network drive enclosure support raid, using SMB and FTP to transfer.
so just get 2 300G HDD and then u got a 1/3.3333333 TB raid file storage with SMB and FTP access
AND it only cost about 100-150 for the enclosure...
Assuming no rebates or other sales:
PATA
200GB $85 - $0.42/GB
250GB $100 - $0.40/GB
300GB $125 - $0.42/GB
400GB $260 - $0.65/GB (or $220, $0.55/GB)
500GB $380 - $0.75/GB
As you can see, the price per GB stays flat up until you hit 300GB drives. The 400GB drives are slightly more expensive, but still reasonable if you can find the ones for $220.
SATA
200GB $90 - $0.45/GB
250GB $100 - $0.40/GB
300GB $120 - $0.40/GB
400GB $205 - $0.52/GB
500GB $340 - $0.68/GB
Same deal, prices are near identical per GB all the way up to 300GB, with a slight rise at the 400GB level.
Now, here's how I setup my current linux file server with software RAID.
2x300GB - RAID1 (boot, swap, root, LVM2 partitions)
2x300GB - RAID1 (LVM2 partitions for more data)
4x300GB - RAID5 (used for near-line backups of data)
For my MP3/media server, I did something simpler with software RAID:
2x300GB drives, split as follows:
- 32GB RAID1 (boot, swap, root)
- 275GB partition for media on the first drive
- 275GB backup partition for media on the second drive
Since the media folder rarely changes, the backup partition stays unmounted for most of the time. If the primary disk crashes, I replace it and copy the files from the secondary drive. If the secondary disk crashes, I replace it and then re-run the backups to refresh the backup partition.
Software RAID makes it easy to do these hybrid systems (rather then using all of the disk on a RAID1 array).
Wolde you bothe eate your cake, and have your cake?
Slashdot | Which RAID for a Personal Fileserver? http://ask.slashdot.org/article.pl?sid=04/06/16/16 58250
Slashdot | Experiences w/ Software RAID 5 Under Linux?
http://ask.slashdot.org/article.pl?sid=04/10/30/18 4256
"A gun is a tool, Marian. No better, no worse than any other tool. An axe, a shovel, or anything." Shane (1953)
I think that misses the point:
Step 1) Buy device that works out of the box.
Step 2) Configure it for your requirements using web based interface, in under 5 minutes.
Your then free to spend your life doing something less boring and tedious than setting up (and supporting, when it eventually expires / and or goes wrong) a naff POS-hardware server.
"Total cost for this project is probably $500 or $600, almost all due to the hard disks"
Personally, I'd rather spend the 500-600 USD on something that meets my needs and just works out of the box like a Buffalo TeraStation than re-invent the wheel.
Data backup has always been a big concern for me, but I don't have a few thousand dollars it takes for a real backup system. I believe in having data stored in three places:
:P
A) Where you access your data
B) Backup location A - onsite
C) Backup location B - offsite
The cheapest way I found to do this was in fact to SCRAP my RAID design. The problem I found by having data in RAID is you were still suseptable to hardware failure taking out two locations for your data by having a bad power supply or a failure on the PCI card / motherboard. Instead I have a computer that my main server turns on once a week via WOL and does an rsync of all the data (about 400 gigs right now) Then about once a month, I get 2 external hard drives that sit in a safety deposit box, plug them in, and rsync changes. This covers me from fire, theft, and the ability to sustain 2 failures without loosing more then a months worth of data (which for me is acceptable, but could be adjusted to fit your needs) As well, if I run out of room, I don't have to rebuild the array or anything like that, I just throw more hard drives at it. Right now I'm using a mix mash of 60s, 120s, 200s and a 250 to spread it around as evenly as possible. It's the cheapest method (since i had most of the equipment already) and I find it to be extremely reliable. No wondering if RAID card A will work with Linux reliably or not
hi, on the subject of raid, i was thinking of creating a software raid array using debian. I was wondering whether there were any flaws/bottlenecks in my plan. a raid 6 array using 8 drives (7200rpm 8mB cache), connected to the sata2 connectors on my motherboard, (these are grouped into 2 groups of 4 connectors connected to 2 controller chips). as they are sata2, the cache speed of the array should be 300MB/s*6=1.8GB/s, with dual channel ram at 400mhz*8=6.4GB/s, i should have a minimum of 4.6GB/s ram bandwidth for the rest of the system. i do not do any graphics intensive work at the same time as i use the harddrives intensively. can anyone tell me the sort of processor load the harddrives would use at maximum, on a 2ghz amd64 processor? are there any bottlenecks i have missed? how would the amount of memory affect performance, i was thinking of 2x512mb sticks, would i see any difference with 2x1gb or 4x512mb, or 4x1gb, assuming timings are the same? thanks. :)
http://openfiler.org/ ? I'm about to give this a shot
Wow, the IRS should upgrade to these!
100X2gb=? How much of it is big files(you know the kind)? And how much is documents, photos etc... I can pleh.... says professor baffles....
Sig Hansen?
On the other hand, some of us have been known to get *into* bed for much, much less than a petabyte...
Install a smoke alarm in the understairs cupboard and make sure there is ventilation in there.
How many 3ware controllers have you used? 1? 2? Less than 10? More than 10? At Fermilab we use A LOT of 3ware controllers, where "A LOT" is in the thousands. We've got ~1PB of disk attached to them (yes, that's 1PB - 1000TBs). I brought them in here back in early 2001, starting with the 6800 cards. In RAID 5 mode these card *would* corrupt data when a disk went bad. In RAID 1/10 mode, they worked just fine. 3ware replaced all our 6800 cards with 7810 cards, which worked just fine in R5, albeit, somewhat slowly in block writes (~18MB/s) but block reads were just fine (>90MB/s). But, I digress.
Yes, we have experienced problems with our 3ware ATA RAID systems, but more often than not, it was due to the disk drives failing in spectacularly subtle ways. In one instance, the "quiet bit" was enabled on some WD drives that would force the drives offline, even when they were fine; some Maxtor drives failed in such subtle and unexpected ways that the only solution was to replace all of the drives in a system because you just can't tell which drive is bad.
Some 3ware cards have failed, but face it hardware fails, and these were rare. There was a problem with the 8506 cards; the first SATA RAID cards. These were missing a capacitor that made them very susceptible to electronic noise in the chassis, caused by the system fans. Again, 3ware replaced these cards.
Overall, I've been quite satisfied with the service and support that I, personally, I have recieved from 3ware's support staff.
That said, there are some other interesting SATA RAID products on the market that has piqued my interest, specifically the Tekram Areca controllers. I have yet to personally lay hands on one of these cards, but they sound very interesting.
I'll vouch for the Linksys NSLU2 solution as well. It's simple, elegant, and does exactly what you're asking. Additionally, it runs Linux and can be tweaked - mine runs a DAAP music server, for example - quite extensively.
As far as backup goes, you could either simply backup from one disc to the other, or maybe (haven't tried this) install cron and tar/zip/ftp files to some other server.
Too late I fear, but mod parent up.
Parent is absolutely correct. Netgear SC101 does not claim to be a NAS.
Why you might want such a small SAN would be different matter.
Also I think it might be mistaken for a toaster and either end up full of breadcrumbs, or you would ruin your HDs by trying to put install them in the two convenient slots in your real toaster.
The problem with Laserdiscs was they used silver for the reflective layer. Some early CD plants were converted Laserdisc plants, and did the same. The result was a bunch of Philips-pressed CDs that started to tarnish. Philips replaced them all. 99% of CD pressing plants were unaffected.
I've never seen a documented case of aluminium CDs dying. Apart from anything else, if a regular CD is badly sealed, the aluminium will simply oxidize to a thin layer of alumina, which is still reflective, and effectively seal the rest of the disc; it won't tarnish progressively, like Laserdiscs did.
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
It was never intended to be seen (or more importantly: critiqued ;-) and I'm sure everyone will make fun of my code. But you can get it here. The raid.py and lcdproc.py modules are the main thing of interest; the actual program that is running all the time, is systat.py.
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
More to the point, the reward for a good cleanup job of the sort you described is. . . finding that you desperately need something that you dumped (usually for something having to do with making money), and that you can NOT get another copy.
Tech Public Policy stuff
I'd say that because they're one of the very few RAID companies actually working with OpenBSD. What, you never heard of OpenBSD's complaint re: hardware RAID?
OpenBSD Doesn't Like 3Ware
Notice the quote: "3Ware has lied to us and our users so many times they make politicians look saintly."
Or, a more detailed account is here:
ONLamp Interview w/ OpenBSD Devs
"Is there any vendor that chose to contribute with hardware or specifications?
Marco Peereboom: LSI has been very nice in providing hardware, certain pieces of documentation, and engineering help. In the end, to make all this happen, there was quite a bit of reverse engineering done as well."
More than 10, and especially during the time when I was system admin of a 60,000+ customer ISP, I had more problems with 3Ware's hardware than with any other RAID card I've ever used.
The fact that some drivers sit in the Linux kernel is meaningless--all that shows is that someone had the wherewithal to actually spend the time to write one.
Binary drivers mean nothing. Support for *modern* hardware is the crucial factor in running these kinds of devices under *modern* open source operating systems, and 3Ware is falling down hard in terms of their support of the free OSes.
Why support them when an alternative exists that has a good reputation, supplies documentation and hardware to the free OSes, and doesn't seem to come with all the negative baggage that you yourself just described!