Slashdot Mirror
RAID Trust Issues — Windows Or a Cheap Controller?
NicApicella writes "My new system has two sparklin' SATA drives which I would like to mirror. After having been burned by a not-so-cheap, dedicated RAID controller, I have been pointed to software RAID solutions. I now stand in front of two choices for setting up my RAID: a Windows 7 RC software RAID or a hardware RAID done by the cheap integrated RAID controller of my motherboard. Based on past experiences, I have decided that only my data is worth saving — that's why the RAID should mirror two disks (FAT32) that are not the boot disk (i.e. do not contain an OS or any fancy stuff). Of course, such a setup should secure my data; should a drive crash, I want the system up and running in no time. Even more importantly, I want any drive and its data to be as safe and portable as possible (that's the reason for choosing FAT32), even if the OS or the controller screw up big time. So, which should I choose? Who should I trust more, Microsoft's Windows 7 or possibly the cheapest RAID controller on the market? Are there other cheap solutions?"
You lose your data twice as fast!
Do you really want to trust Windows with your data?
"I want any drive and its data to be as safe and portable as possible [...] even if the OS or the controller screw up big time."
If the controller screws up and writes crap to your raid, your data is dead. Not sure if your expectations are realistic.
Try a drobo instead.
Comment removed based on user account deletion
Repeat after me: RAID is NOT backup
You data is most important and you plan to use FAT? Good luck with that!
Seriously, though. No RAID solution that is not totally S/W is portable. But do you really need RAID? It sounds like what you need is a good backup solution with frequent backups. Does you data change so much that losing one day's worth of data would be a problem?
The real "Libtards" are the Libertarians!
RAID is not a backup. Get a backup solution or you'll realize you can be even more frustrated.
... and I'll say it again: unRAID
If you want data integrity, use NTFS. Using Fat32 is like saying you want a reliable car, so you're buying a Edsel because they've been around a long time-- it doesn't make sense. Every other OS on earth can read NTFS (if not write it), so it won't affect your portability requirement.
Secondly, before you make any decision regarding Windows 7 RAID, make sure the edition of Windows 7 you want to buy ships with software RAID support before you put all your eggs in that basket-- early betas and RCs of Vista had software RAID enabled, only to have it disabled before release. I've seen no guarantees about Windows 7 software RAID support, and which editions will have it enabled. (If any.)
If you're planning to move to a server OS after Windows 7 expires, I can practically guarantee software RAID will be enabled, but that still doesn't mean you can necessarily upgrade your Windows 7 software RAID array to a Windows Server software RAID array. Do your homework.
Comment of the year
You sound like someone that need to be reminded that RAID IS NOT BACKUP! Google for that sentence. All you talk about is saving your data, and RAID will not do that for you. You'd be better off just using the second drive as a backup. RAID will not save you from accidental overwriting of data, corrupt filesystems, broken chipsets, etc. The only thing RAID will save you from is downtime. If you lose that much money on the downtime it takes to recover from a backup, then by all means, use RAID, but don't treat it as a backup solution that will protect your data. That's not what it's made for.
c++;
FAT32? Are you fucking insane?
I've been using on board RAID controllers for years and have never had a problem with data loss or crap being written to my drives. A mirror solution is nothing major for a RAID controller to handle. I would never trust windows to software raid anything important.
Please back up your data. RAID will not save it when your PSU dies and takes both your drives with it.
RAID is not for data security. It only brings speed and/or less downtime in case of a single drive failure, depending on the flavor. One would think everybody would have learned it by now.
Hello,
the only advise I can give you is that performance wise both RAIDS are the same.
In essence the work is done entirely by the CPU by both solutions.
I would not recommend RAID as a backup solution.
Cheers
These motherboard "raids" are called fakeraids.
All that it is is that it writes the metadata on the disk in specific format so that you can see the raid volumes via BIOS. Note: Only "see" their status - in case you replace one drive, the resync is still done by software and you must boot to operating system. One clue is the fact that in Linux the dmraid package uses exactly same driver for accessing fakeraid-mirrored drives and Linux's own software-raids - device mapper just does a bit of magic at init.
However, if faced with choice of Windows-only or motherboard-raid, I'd go with the motherboard-version, because that's at least supported both by Windows and Linux so in case something goes wrong with your Windows installation you can always pop in Knoppix or some other Linux CD for recovery.
For one thing I wouldn't trust FAT32, use a journaled file system for christ's sake if you're concerned about integrity. For Windows stuff NTFS is pretty damn good, Linux has been able to read it just fine for a while so getting to your data if an issue with Windows arises is a non-issue.
Cheap RAID controllers suck - at least you can trust Windows to be consistent between installations if need be. External (preferably offsite) backup is also a must! As I'm sure you'll be reminded 1000 times in this thread, RAID is not backup.
NEVER trust Microsoft with any data that is;
Valuable
Important to you
Cannot be easily reloaded
You're welcome
I've already lost a software RAID setup when Linux wouldn't let me put it back together again. A friend of mine also lost his RAID setup on Windows a couple years ago. Also, a mirrored RAID doesn't prevent you from losing data when you accidentally delete that folder of your favorite photos.
I now keep the drives separate and setup a simple nightly batch file to robocopy the files from one drive to the backup directory on the other. It's even better if you can put the second drive in a different PC. This allows me to decide which files are important to me and only do backups of them. That leaves all the rest of the free space on the two drives for more volatile content (stuff I don't care about if I lose). It also gives me time to get a file or directory back if I accidentally remove it. I've done that numerous times when Vista first came out and it didn't move the focus to the right hand side of the explorer window.
RAID1 serves only one function. Increased uptime. If avoiding having to spend 2 hours restoring from a backup is your primary goal, then RAID1 might make sense for you. Do you have an office full of workers that will all lose productivity if you have a system crash? If so, then RAID may make sense. Any other use of RAID1 is fool's gold. It will not protect your data from a system-level problem. It will not protect your data from corruption (especially not on a FAT32 file system, which was never intended for any partition size above 32GB in the first place). It will not even always protect you from a single drive failure, since the rebuild process in a RAID1 setup often kills the second drive while trying to recover data. As many have said already on the thread, RAID is not backup. Backup needs to be a completely independent device. Unless you have serious uptime considerations, RAID1 should not be part of your backup strategy.
RAID is no substitute for backups. RAID is very good at propagating errors and problems very quickly, be they software glitches or human errors.
For consumer class storage, weekly / daily backups might be more efficient than investing a lot of effort into live RAID. Since I'm a Mac guy, I see the best answer to this question as Time Machine to a network / USB attached drive -- hourly (configurable for more or less often) differential backups, almost transparent to the user. To my knowledge, Windows has no similar set of software to allow reinstallation to the last hourly backup -- my wife had the misfortune of having to restore a blank drive from her last backup and it was a flawless process that truly left her where she left off less than an hour before the hardware failure. The reinstall wizard just had to ask where the backup was. Casting aside MacOSX advocacy, there is truly no substitute for a good automated backup solution that is regularly tested. I think the best method would use the fewest common components, like a NAS, followed by an external drive with its own power supply. My least favored option would be an internal drive with every single component shared.
If you were doing this with Linux, I would suggest just using md. It works in most cases. For raid1 it mirrors to the point where you can take one drive out and mount it as a non-raid single disk somewhere else. Handy for troubleshooting when ubuntu decides to change uuid drive mappings and thus refuses to rebuild the now broken raid set.....
-Tm
Support TBI Research: http://www.raisinhope.org
The only way to keep your data secure in any reasonable fashion is to make a copy of it and store it offline, off site. Ideally "off site" would be in another building or city, but it at least has to be on something not attached or accessible to your computer.
Without regard to if you use software or hardware RAID or the quality of the RAID system, RAID only protects you from a physical disk failure. If you as a user screw up (delete or change something you didn't want to) or if some software bug screws up for you, or if you have a non-disk related hardware failure (causing a data corrupting machine crash) then you have lost your data -- RAID doesn't help.
Even if you are only trying to protect against disk errors, if the RAID system fails (even expensive quality ones can), or if you don't know and follow the recovery procedures EXACTLY, you can lose all your data.
The only reliable solution is making a copy or a "backup". Backup does not mean making a copy of the data on the same machine. (Whatever took out your RAID might also take out the other non-RAID disk or directory that you put your copy on.) If you are paranoid (or just prudent) your backup should not be a mapped or mounted drive on another machine. (Viruses can write to the network as well.)
And finally... Backups only count if you have tested your restore process.
I won't try and improve on the comments above, almost all of which I agree with, but I will make one observation. The reason for mirroring is to protect against drive failure. The one time I had a drive failure, mirroring saved the day's data. The best way to protect against drive failure is to buy server grade SATA drives, which are designed for 24/7/365*5 operation, and not cheap PC drives which are designed for 10 hours per day for 3-4 years. Buy server grade SATA drives, mirror them using a hardware controller, back up daily, sleep at night.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
... do it with software. This CPU overhead is minimal for that.
Things like RAID 5 are where you'd be better off with a dedicated controller.
Consumer editions of Windows only ever supported Software RAID1. I've made a few experiences with SW RAID1 on WS03, and it's pretty much crap. Linux SW RAID on the other hand worked fairly well.
RAID is not a backup. This is the most important observation. RAID is a high availibility feature. If you lose your RAID array, you shouldn't lose any data. If you do, your backup strategy sucks.
Generally, skip RAID in a consumer setup. RAID is complicated, it's a PITA and especially the low end stuff can do more harm than good. Even expensive stuff can fuck your shit up (I'm looking at you, ServeRAID 8k). Better in invest in a proper backup - to a local harddrive and maybe offsite. Online backups make sense in a home office. For servers, i recommend LTO tapedrives.
RAID is only marginally valuable. In my experience, for all but the most carefully controlled environments, RAID simply adds complexity, the number of things to go wrong increases, along with the likelyhood of lost data. Do it only if you want the *experience* of running RAID, but don't count on RAID to "save your data".
I've worked as a system administrator for more than a decade, in medium-large scale deployments with good success, (think: servicing thousands of users, hundreds of domain names, tens of thousands of email addresses, etc) so I think I have some useful experience you can benefit from.
IMHO, you most likely to lose data from the following things (in order)
1) Aw sh1tz. "I didn't mean to delete that folder"... or "Whoops! I formatted the wrong drive", "I saved the wrong version of the file!", whatever. Although I *myself* don't have this happen often, it does happen. And even in my case I've lost about as much useful information this way as by drives dying. Users delete stuff all the time, and it's usually my job to bring it back, which is why I perform redundant, historical backups EVERY SINGLE DAY.
2) Malware. Don't minimize this - it's real, and it's why I reply to Parent. You are more likely to lose information from a virus/worm/malware and/or b0rked install of something that hoses your filesystem than by a hard disk crash given stable hardware.
3) Bugs. Filesystems have bugs. So do applications, utilities, anything with software. Strange, unexpected conditions, often caused by bugs in applications can cause data to "disappear", files to get corrupted, filesystems to get corrupted, folders to be incompletely written, etc. This is about as likely to cause lost data as:
4) Hardware failure. This is one of the lowest orders of lost data, although when it happens, it can be one of the most extreme.
Let me say this: RAID 1/5 only PARTIALLY protects you from the last one. Actual, bona-fide backups protect you from all of these. If you care about the data, get backups. If you care about uptimes at great expense, RAID *may* be worth it.
My advice is something most people don't want to hear: for personal use, get backups online for $5/month. Mozy/Carbonite/etc. There are zillion vendors, just Google it. In two years, it will cost you about as much as that 2nd hard drive. It protects you far better than that 2nd hard drive, and it's so automatic that you'll hardly notice it until the moment it actually matters: when you just have discovered that your data is gone.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
The only thing RAID will save you from is a dead drive.
There are infinitely many ways to lose your data, and a dead drive is only one of them.
If you are going to use RAID, you might as well use RAID 0 unless you can't afford the downtime.
If you start talking about data loss, then you just lose, because you should have backups.
The article smacks of false dichotomy. There are a number of solutions, not just Windows 7 or a hardware RAID controller.
To begin with, every NT-lineage Windows version ever produced supports software RAID out of the box. Add that to the fact that any major Linux distro today supports software RAID. And so do the *BSDs. And Mac OS X. And Solaris. And probably a bunch of other platforms I can't think of right now.
Hell, you could buy one of these one of these and throw the drives in it, connect it to your network switch, and presto -- instant RAID+NAS.
I think we would all like to know why you think Windows 7 is your only option, because if that's what you think, you don't know how mistaken you are.
My blog
With RAID mirroring, if you overwrite or delete an important file, it's copy on the mirror is immediately overwritten/deleted too, and the file is lost. Wouldn't you rather need a good regular backup?
And as someone pointed out already, FAT is really not a reliable file system. If you are on Windows, use NTFS. It is still portable, having read/write drivers for both Linux and Mac (see this guide).
Since the files you want to keep safe appear to be regular files, not system files, any simple file copy mechanism could do. For an easy and simple system, you can use the Windows robocopy.exe tool in a batch file. For a more sophisticated system which can keep older file versions, and can easily be adapted for use over the network, you could try a Windows version of rsync like cwrsync. There are also a few rsync GUI frontends for Windows.
If you decide you really want RAID mirroring and go with the hardware solution, my understanding is that you need a replacement controller in case yours breaks. Since your controller seems to be embedded in the motherboard, you would need a replacement motherboard.
With the Windows software RAID, you are dependent on that software, and have portability only between machines with this Windows 7 software RAID (possibly even only this particular version).
You can buy a real RAID controller for $400-$500 nowadays. If your data is not worth that much...
Using FAT32 means you do not care about your data. Period.
Use NTFS if you're using Windows. Period.
That said - you should be okay with Windows SW RAID. I used it for years in a simple RAID1 two-disk configuration. I've sustained loss of a single disk with no ill effect. I would not, however, recommend doing an upgrade between versions of Windows while doing this. It might work; it should work; but if you go from, say, XP to Vista (Or Windows 7 to Windows Whatevercheesynameisnext) - yeah, that's scare the hell out of me.
Also, as plenty of others have suggested, RAID is not a backup solution. Honestly, it's almost always 'good enough' - but if you're dealing with a simple setup - say, two disks that you've purchased right this second off of NewEgg... You've got to understand, in that sort of situation, if you're one of the unlucky to have purchased two disks from a bad run, RAID won't help you when both disks die a horrible death.
Plan for this. Plan for this even if you take the time to buy disks from different batches. Because at the end of the day, no matter how infintesimal the chances of total failure are, the chance remains. You really need to ask yourself just how important your data really is. Let's face it - it's honestly probably not important enough to go nuts with hourly backups to off-site locations and such. Is it important enough for you to set up, say, an NFS/CIFS/whatever box and sync once a week? Or even slap on an external USB drive and manually copy over your data once a month? Probably! That's still not 'ideal', but it may be 'good enough' in terms of prevention of complete data loss in the event of catastrophic RAID failure.
Oh, and test your backups occasionally. Untested backups, aren't backups.
I would recommend an eSATA RAID enclosure, similar to this. Run the newly purchased SATA disks mirrored (RAID-1) in the enclosure. Power up, run backup, power down. Rinse/repeat.
...then you probably don't need RAID. Use NTFS and set up some kind of scheduled backup to the second drive. Or, build a Linux NAS device and run BackupPC (backuppc.sourceforge.net). BackupPC works great for this sort of thing, it can do incremental and full backups of all your data, on the schedule you choose.
The best way to protect against drive failure is to buy server grade SATA drives, which are designed for 24/7/365*5 operation, and not cheap PC drives which are designed for 10 hours per day for 3-4 years. Buy server grade SATA drives
This is just pure marketing baloney. Do you have any real-world tests that actually back this claim up? I've never used "server-grade" drives, and never will. I've seen "server grade" drives fail in large quantities and "desktop" grade drives last for years running 24/7/365.
The only thing I've seen vary greatly in quality are power supplies. Cheap ones are designed to last a couple years and fail. The more expensive ones higher grade ones tend to be better.
AccountKiller
Screw RAID. While it will protect current data in the event of a drive failure it is not excuse to not have good backups. RAID does not protect against overwrites, deletions, filesystem corruption, etc. A good backup, preferably offsite and in multiple snapshots over time, is much much more important.
As has been said many Times RAID != Backup and IMHO RAID 0 SUCKS ROYALLY,
there is a place for other Raid variants. These is a case to be made for using a NAS device for backups.
I use one that is configured to use Raid 1 But there are some devices out there in the market that should be avoided. The NAS I use runs Linux and formats the disks as EXT3 volumes. It fully supports NTFS so windows access is not an issue.
If your Raid Controller imposes a propritary format on the disks rather than using other widely used ones is just plain madness. Why?
If your hardware gives up the ghost you could also end up with major data loss.
If the hardware uses standard disk formats then you can take the drives and recover the data.
For example, the Netgear SC101 is a NAS device that uses a proprietary on disk format. The protocol between the PC and the SC101 is also non standard and IMHO should be avoided at all costs.
Being an old school H/W engineer (I started designing H/W using 74 series Logic) I am suspicious of RAID 5. For m Raid 1+0 is a more recoverable solution.
Whichever RAID solution is chosen, I'd check how the underlying disks are formatted.
mirroring with RAID 0 does not take the place of regular backups of data. RAID 0 is for rapid recovery to minimize downtime is MOST instances. Data should still be backed up separately in case of a catastrophic failure.
... and in the DRM, bind them.
They've been doing RAID in hardware for quite some time now - the hardware may fail, but I don't think it'll mess up your data. Think of it this way: The hardware controller only has to do one thing, which is to serve RAID. The OS, on the other hand, has to do a bunch of things, any one of which could go bad and kill your RAID.
I've done both hardware and software RAID-1 in the past, with Windows, Solaris, OS X, and Linux. For Windows, go for the hardware RAID.
Software RAID 1 is only supported on server versions of Windows, and I haven't read anything on Windows 7 that mentioned any changes in that regard.
I don't really trust cheap motherboard RAID controllers, but I'd use them for RAID 1, since each drive has a complete copy of the data. For any sort of redundant striping (RAID 5/6/10/50), I would use software RAID or a real hardware RAID controller.
"Safe" FAT32, cheap RAID, RAID implied as backup, Microsoft.
Nice job, you successfully trolled the /. frontpage.
Leben Sie jetzt die Fragen.
You'd be better off putting the second drive in an enclosure and using it for off site backups. That way when the main drive fails at least you have a backup to recover from. RAID 1 will just fail on you when you come to re-build the array and you are making work for yourself before then. RAID is good for uptime and throughput but nothing else.
The article smacks of false dichotomy. There are a number of solutions, not just Windows 7 or a hardware RAID controller.
Agreed.
As I see it, if you want guaranteed repairability then you basically have two options: enterprise-class hardware with a support contract (and price tag to match), or an Open Source software solution.
Put another way, either you pay someone to take responsibility for fixing it, or you take responsibility yourself. A Microsoft solution doesn't give you enough control to take full responsibility, because you can't be certain that it will be legally or technically possible to recreate your current setup in five years time.
I don't normally comment but WTF is this doing on front-page of slashdot? As for the question, go with a dedicated hardware based raid controller. Just cause you gut burned once doesn't mean that the dedicated controllers are bad. I can guarantee the software RAID via windows is worse because of Dynamic Disk partitioning is slow and its not swappable between platforms for recovery. Also FAT32 should never be used outside a USB flash drive or similar device; its terrible for handling large files over 2GB and it inflates the size of small files. If your data really is important go with NTFS for windows or EXT3 (other?) for Linux; it should not be an un-common file system as rare file systems dont plug easily into other computers for data recovery. I've tried several different motherboard integrated raid controllers (intel, nvidea, amd) and all were either unstable which caused disk synchronization issues, slow, or had very little bios-level controls. The little Promise and Highpoint raid controllers aren't what they used to be in quality and performance. Get yourself a 3ware PCIe 14x controller for SATA/SAS controller which costs $$ but you'll thank youself for the investment. Its fast, stable, handles 4 x 2TB SATAII hard drives and is bootable for Windows / Linux. The only time I've ever trusted a software based RAID was with Linux EXT3 based software raid. The actual hard disks are standard EXT3 (not some funky format) which means I can remove any 1 disk and plug it directly into another Linux system without having to do anything special to read the data. That the linux software raid is still very fast, easy to administer, and is most importantly stable. For the super cheapo just don't use raid and get yourself an external eSATA drive and do regular backups. Regular backups aren't the same as RAID but its better than Windows Dynamic Disk any day and still protects your data; oh and its fast and as reliable as you do backups.
There are some important differences between server SATA drives and consumer SATA drives - for example the number of retries until an error is reported to the controller. The price difference of course is not really justified by that.
If you're just want a convenient backup of your music collection, porn collection, musical pr0n collection, or your pr0n musical collection then RAID is not a horrible thing. However, if you're backing up important files, like the only existing scans of the now-burned dossiers William Mark Felt left you, then you should not stop at RAID. Statistically speaking, if something happens to one HD in your machine, like a massive power surge or being confiscated by tight-lipped men in black suits and black sunglasses, it has a pretty high probability of happening to the other HD. Offsite backups are, therefore, prudent. Leaving a HD in a box at the bank and giving the key to your lawyer is one of the safer things you can do, but not terribly convenient. There are a variety of online backup services available that are decent. I'll leave it to others to speculate on which ones are least likely to be fronts for the NSA. If you feel that your data might actually be interesting to more than one human being on Earth, don't forget to encrypt it. (Be honest with yourself. You are posting to /. after all.) I'm rather fond of emailing moderate risk files to my gmail account. (Stupid, I know, but very low effort and they're available anywhere you feel safe enough to check your email.)
As for Motherboard RAID chipsets... Keep in mind that your motherboard has a non-zero probability of frying, having it's caps go bad, being peed on by irate government agents, etc.. I once had a RAID 0 array that was hooked up to one of those things. After the Mobo died I had to do without letters K through P of my Japanese horror-comedy-porno-game-show collection until I was able to find a used computer with the same RAID chipset. (I don't know if it's changed, but at the time each different RAID chipset made RAID 0 arrays that were not compatible with anything else on this lump of rock.) If data portability rather than performance is a priority for you, my advice would be to avoid hardware RAID entirely.
Do you work for one of these online backup places?
I would sooner trust a WD drive with my valuable data.
you shouldn't be using release candidate software as your production box. everything else become academic after that. when Windows 7 is released to the general public in a production version and you upgrade, you're going to need to wipe your RC version. so just don't do it.
Actually RAID1 is quite good for reading data: it minimizes seek time . Of course, it works fine as long as there are not many writes. For example think analytic databases, cubes, etc. Those are not written to in real time (like the more common transactional databases)
http://revj.sourceforge.net
There are a number of reasons why not to use FAT: 1. Unreliable 2. Doesn't support large files 3. Doesn't support advanced permissions Since you are running Windows, use NTFS, an external USB drive for backup (also NTFS) and the free Microsoft SyncToy to make periodic backups to the external drive.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
You can buy a PC for less than that, and install OpenFiler, or any of a number of free/Free soft-RAID solutions that support Samba.
If you buy a RAID controller, you move the SPOF to the controller rather than the disks (though admittedly it's not got moving parts and should last longer). If you do use a RAID controller card, though, and you want safety, you need a spare RAID controller of the same sort in a drawer somewhere if you expect to get your data back, unless you're sure that the RAID doesn't use a funny on-disk format to store the options it's using.
If the machine is doing RAID and not much else, what are you saving by buying expensive hardware to offload the RAID processing when you already have a mostly idle box sitting there already?
Linux software RAID is about as portable as it gets. It is totally independent of software vendors and/or hardware vendors. It also gives you great flexibility with file systems and growth. You can even make a RAID 5 array and just slap another drive in later to extend your space, something which other solutions don't let you do.
RAID is not a backup solution.
You should be using mirroring in addition to frequent backups.
RAID does not protect against (accidental) deletion, file system corruption, or user stupidity.
The best way to protect against drive failure is to buy server grade SATA drives, which are designed for 24/7/365*5 operation, and not cheap PC drives which are designed for 10 hours per day for 3-4 years. Buy server grade SATA drives, mirror them using a hardware controller, back up daily, sleep at night.
Err, thats been proven questionable. More specifically, here is an article from eweek and here is the google talk about a large study of drive lifetime characteristics. "Server" drives are just as good as "consumer" drives when it comes to lifetime. The only benefit you get with the more expensive drives is slightly better performance (NCQ, higher rpm, larger buffers, etc). I have several machines at home that run 24x7x365 on the "cheap PC drives" in raid1 pairs (linux md) and a non-raided windoze box, and have had to replace 2 drives out of 12+ over the past 8 years due to failure: I tend to need to upgrade to larger ones before they fail. Even at work we have used the cheaper drives in clusters, next to others running enterprise level drives and found no benefit to the extra cost in most situations.
-Tm
Support TBI Research: http://www.raisinhope.org
No where did he suggest he required Windows 7 for RAID, just that thats what he planned to use for this machine. Personally I would just put the second drive in my server and backup my data to that rather than use any of the two options he lists.
Normal people worry me!
Personally, I haven't yet encountered anyone who really got benefit from those personal Internet backup services like Mozy. In regular use, it always seems like the person exceeds their storage allotment or Internet connectivity issues prevent them from recovering what they need, when they need it.
I tend to recommend people buy an inexpensive external USB or firewire drive, leave it attached and assigned as a backup device, and have some software package run a daily backup of all the relevant folders and files they might need to save.
It's great that your data is stored offline and off-site ... but I'm just not sold on most of the implementations for "home use" being as great a solution as they first appear to be. Many of the providers have come and gone over the years, too. What happens when your offline backup company goes under?
You're in a special situation using Windows 7 RC. So despite the below, you're especially at risk to the possibility of a new (but not yet discovered) bug in Windows release-candidate software, don't use Windows or other RAID capabilities for your "data protection", use backups. Use RAID mirroring for improved performance or to reduce the probability of downtime occuring, and allow for rapid recovery from common drive failure scenarios.
RAID controllers (Software or Hardware), all suffer from various scenarios where the entire array can fail, and data recovery becomes so hard that it may as well be lost. 90% of the bits may be there redundantly, but you can't get to them for one reason or another. Also, RAID will not protect against system compromise, software data corruption, accidental deletion, or any type of volume corruption that occurs as a result of software running on the system.
Don't pick FAT over NTFS on the basis of portability between systems, if reliability is more important, the NTFS filesystem uses a technique called journalling which makes data corruption less likely after a system crash, eg power failure. NTFS _can_ be read by common solutions, if you need to recover data. Recent Knoppix CDs and various rescue disks can read NTFS, and the filesystem checking tools available for NTFS filesystems are better. FAT is more susceptible to certain failures, including excessive fragmentation leading to poor performance.
Research what type of RAID solution your integrated hardware really is. If it is hostraid, or fakeraid, that requires Windows drivers to implement RAID, then don't use that, avoid like the plague as it's SOFTWARE RAID, even though the software is running inside a driver provided by the controller vendor and A FEW functions may be offloaded to hardware, the main RAID code is still running in software, which is bad, mmmkay?
You can often detect this in that there will be Windows only drivers, or the product will be labelled a hostraid solution, but each of the major drive controller/RAID chip manufacturers has a different name for their ultra low-end solution that isn't really hardware RAID, but has hardware offload of just some functions (checksumming, mainly).
(Fakeraid/Hostraid adapters that require special drivers in the guest OS to implement RAID, also generally suffer from the RAID5 write hole if you utilize RAID5. And RAID code may be more susceptible to certain problems, when it isn't running on card firmware.)
I would actually favor implementing RAID in Windows over that. However, there is hardly any point of doing this, except if you are mirroring your boot drive, or you need RAID for improved performance (e.g. You could use RAID1 for all drives to improve read speeds, RAID1+0 to improve both read and write speeds, or RAID5 for redundancy and scalability at the cost of slower write speeds and a read speed penalty).
I mean that: since you aren't mirroring your boot drive, there is little point of utilizing RAID in your case. One of the most performance-effecting files on your disks is the page file on the boot drive. If you were utilizing RAID for improved performance, you should definitely want to maximize read and write speeds to the boot drive.
If your non-redundant boot drive crashes, your system will be down and need to be re-installed on a new system drive. You may as well just pre-image a backup drive with your system, keep a continuous backup on another machine, and in the event of a failure, pop in the backup HD, and start restoring continuous data from backup, to bring your 'spare' up-to-date.
This will probably even be faster, as an OS reinstall and re-up of Windows is not required
Second, your RAID controller can fail, make sure you have a plan. That would mean either two identical controllers with the exact same firmware version, or you use a very common controller that you are CERTAIN you can easily buy anoth
FAT32 has a 4 GB max file size.
This can be somewhat inconveniant if you have say a linux dvd ISO thats larger than that, or if you record HD video, etc.
I would suggest going with NTFS, there are ways now to use the actual microsoft NTFS driver in a wrapper with linux, then you get full native compatability if thats what you are thinking (it used to be that NTFS compatability was an issue with linux).
The cheapest solution for backup would probably be an external USB drive. And you may also want to have 2 of them that you swap out for off site storage, or something like that, depending on how important your data is to you.
controller for your money. I've had bad luck with the cheap ones in Windows, always a blue screen of death that leads to data corruption and other problems.
You want a RAID controller that is properly reviewed, one that has been proven to be stable. One that has good driver support and a good warranty. You want the same with your hard drives that data will be stored on.
I am not sure why you want FAT32 drives, is it because you are running an old version of Windows that does not support NTFS? Most moden Linux systems have NTFS support built in if not an option to load, just read the FAQs for your version of Linux and look for how to add in NTFS support. Unless you have software that breaks on NTFS and needs FAT32 (like old video games and software applications with anti-piracy technology that only works with FAT32 and not NTFS), you should really use NTFS for your hard drive format.
You need more than just RAID drives, you need a backup solution. Either use DVD-RW disks, or buy a Tape drive, or have an older system you can set up a network share and copy data files over to it, or a USB hard drive or thumb drive.
Remember, Slashdot does not have a -1 disagree moderation, and no, troll, flamebait, and overrated are not substitutes.
seems to be one of those buzzwords that are taken for granted, but under duress are not really desirable attributes. The easiest (only?) way to achieve consistency in times of trouble is to destroy all information constent that is not consistent. That means destroying all files (and directories) which are not all there.
You are stranded on a desert island desperate for news. You have a choice between a month-old newspaper which is intact and yesterdays newspaper which has been mouthed over by hungry sharks.
Your network server with all your orders, shipments, invoices etc just went up in smoke. You have a choice between 97.5% recovery of all the current data or a00% recovery of month-old data.
Seems like the people that promote these things are never in the position of actually having to use them. (Sounds like politicians. Lots of plans sound great, as long as you do not actually have to implement them).
There is a basic rule of medicine which seems to not apply to systems analysis. First, do not damage (or something to that effect).
Heat kills drives so avoiding it is a big plus. I've had friends buy neat looking cases that didn't have proper drive cooling and the hard disk died. At three separate employers we had weekend air conditioning failures and each time hard drives died not just over the weekend but the following weeks as well.
My machines are in 4U server cases with a dust filter on the front, a 120mm fan in front blowing on the drive cage, and two 80mm fans in back pulling air through. Temps in the case stay low.
The other key is quality power supplies. A poor p/s can kill your drives or the motherboard with that on-board "RAID" functionality.
Make backups and take steps to avoid hard drive abuse through heat or unstable power. That should be good enough
Run and catch, run and catch, the lamb is caught in the blackberry patch.
Western Digital "RAID" drives feature Time-Limited Error Recovery, and it is a user adjustable setting on many WD consumer drives via a DOS utility.
You are neglecting:
- the rogue app that overwrites your data
- the drive that reads back incorrectly, right into your mirror drive
- the drive that croaks as you attempt to re-mirror from it
I have personally encountered all three of these.
If you don't have disconnected backup, you are just kidding yourself. Any connected media can be trashed by bad software/firmware/whatever.
Your risk factor is never broken hardware. You don't care about hardware. Your risk factor is always lost data.
To begin with, every NT-lineage Windows version ever produced supports software RAID out of the box.
I'm sitting in front of a copy of Vista Ultimate right here that doesn't. Do you have a citation or anything? Or are you using some alternate use of the word "supports" that doesn't mean "supported by Microsoft?"
Comment of the year
I worked for Microsoft for over 6 years, and I swore, when I left, that I would find a better way. I ran into Ubuntu Linux after going from Red Hat, to Fedora, and Suse... I tested it out for some time, setting up a server to handle all those functions I was paying hundreds of dollars for. Then I converted all the machines over that I own, and I have been more than happy. Every machine I set up has a RAID, My "user" machines all get RAID 1, my server and my play machines get either RAID 5 or RAID 6 I have had drives go bad, operating system troubles, hardware troubles, and everything imaginable. My data is still safe. Stuff that I have collected for 10 or more years, files, mails, etc... I don't worry about any of it. I tried raid in Windows, and without purchasing expensive hardware, forget it. It doesn't work, or at least, it doesn't work well. While I worked at Microsoft, I went in for a "blue badge" interview. While I waited, a currier brought in a box for a group in building 53. This box was a copy of Red Hat Enterprise. I thought that was funny, but when the guy came to pick it up, he explained that they had sensitive data on their servers, and they couldn't trust that data to Windows Server Addition, (back in the Win 2K days). If Microsoft cannot trust their own software, why should I?
--E--
Just use some good incremental backup software. Send the backups to the second drive.
FAT32 is not a good choice, it can have file-system corruption more often than hard drive or controller failure occur. If you're running Windows I highly recommend you go with NTFS. Linux can read it pretty reliably if you're looking for portable data, you probably don't have immediate need for portable write support to the drive.
Windows software raid is slow, but it is phenomenally more practical than cheap hardware RAID found for "free" on motherboards. The worse thing about hardware RAID is that it is not portable to different controllers. If your motherboard breaks you can be stuck being unable to use your RAID volumes anymore.
“Common sense is not so common.” — Voltaire
We're talking about an environment that has more then TWO MILLION VIRUSES in the wild, and if your choice of Window-hardening software doesn't stop every one of them, you're screwed anyway, headed for a flush and fill. (And potenitally paying another $100). Welcome to Windows.
Sure, it's been a LONG time since Stacker, the 'wonderful new idea to double drive space', but since it was in Windows, it would last about a week before some memory-hogging virus or poorly-written program would stomp on it.
Hardware RAID can be a disaster; I got in a place and time where corporate data was on a set of drives, there was a failure, and the OS maker decided NOT to make a driver for the replacement RAID card we had. (at great cost and by FedEx!) I had to downgrade the OS, load the data to another device, upgrade it and throw away the raid entirely. What a bitch!
It's even part of the reason I delayed using software RAID on Linux- I was gunshy.
But I tried it. You'd expect the extra overhead to cost access-time, instead it speeds up reads! And the writes (at least in Linux) happen in the background so you don't notice any lag. I've run software RAID in Linux for YEARS, replacing drives and adding spares, etc. It's solid. Not only solid, but (for small applications) the best thing out there.
(If you're gonna approach Amazon.com, hardware RAID, all the way!)
Across the distributions, Redhat's got the lead in RAID-at-install-time, but every Linux out there has the ability. It's worth a shot!
--- For a good time mail uce@ftc.gov
None exactly of that. You have to find cheap RAID controller, with documented or known format, recognized by Linux softraid. There is some. In this case, even whole computer busted (including controller, it can be EOL, when something will happen, and you cannot find new), or system partially corrupted, you can still recover your data with help of Linux (yes it can read Software RAID created by cheap softraid controllers without those controllers. Check dmraid documentation.
Huh? Vista Ultimate supports RAID.
http://www.neowin.net/forum/index.php?showtopic=612319
Anyone can "stand up for what they believe", but it takes a very brave individual to change what they believe. - Loundry
Years ago, when I was a system administrator, we spent a lot of money on a RAID box for our Sun machine.
It came in a really pretty cabinet, and looked VERY impressive.
Some time later, the controller failed, resulting in complete, unrecoverable data loss.
Yes, we had backup, but the array was toast.
I've had several foul run-ins with RAID, JBOD, and lack of true backup solutions. The failure rates of HDDs are unacceptable, and the manufacturers should be accountable for the safety of your data (Insert car analogy here). Because this is not the case, we now have cheap RAID.
Cheap raid isn't worth a copy of Vista, and no matter if you use hard or soft raid, there is still a single point of failure, and no accountability for the array just going poof (I've had it happen, and it's the ugliest thing to ever have to try to explain to a boss/client).
I think it is unforgiveable for manufacturers of drives to get away with some of the poor quality I have seen, but it's twice as bad for makers of RAID controllers to make the garbage they do, and market it as helping to ensure the safety of your data. And I have seen RAID controllers do more harm than if they had just been left as a bunch of disks, even in RAID 1. If they had to foot the bill for the data recovery, they would be a little less cavalier about releasing shoddy hardware.
Again, as has been pointed out, RAID is primarily to ensure high availability, and is NOT a backup solution. If you're not running a web server or other service, while RAID may be useful when one of your hard drives dies randomly for no reason, it leaves a single point of failure, and that single point was probably designed poorly.
--Not to be worried, Pitr fix.
RAID != backup but it's still quite useful.
I'd much rather swap a disk when it dies then go through a long and painful restore process from tape or a network. Disks fail.... a lot. Especially cheaper SATA and IDE drives.
Anyone that says "HDD's are reliable, RAID is useless, just use a backup device" needs their head checked. You need RAID AND A DECENT BACKUP.
RAID won't save you from yourself or OS bugs but it is far from useless. I would not trust a single HDD with my data. I also would not trust myself to keep an impeccable backup schedule where I back things up several times a day so without RAID if a drive dies I WILL lose data. Not a heck of a lot but I'm going to lose SOMETHING.
On another note, maybe I should take a look at Time Machine in OSX.
Solaris/BSD ZFS w/snapshots and rsync to an external backup location.
Wrong wrong
Who says that one needs to recover data from the remaining drives? What if I also have the data stored somewhere else on different disks on a different controller?
If you lose data when your RAID controller dies, then your real problem is that you are not backing up your data.
SSDs minimize seek time. They are an order of magnitude below spinning discs, more often two.
Software RAID may look good in benchmarks, but a real RAID controller is where the real performance is.
I used software RAID 0 on linux for many years, and I thought I was getting good performance.
Then I bought a hardware RAID card, and was totally blown away. I had no idea disk access could be like this. No more waiting, no more bogging.
I'm still questioning my new computer: "No, you didn't really do that THAT quickly did you? There must be some kind of mistake!"
RAID 1 hopefully prevents that a server will go down and it makes it possible to easily replace a bad disk.
At the fairly large ISP I work at, all servers run Linux, and wherever redundancy is not easily implemented otherwise, we use software RAID so that if a disk fails, the system can keep going, and the hot-swappable disk replaced without any outage. We use only software RAID — RAID 0 for syslog servers where write speed is an issue, RAID 1 for disk redundancy, RAID 5 for redundancy and larger volumes, and RAID 6 for some larger systems. Even where we use 3ware RAID cards, we still combine the individual disks (exposed as JBOD) with software RAID.
This RAID is totally distinct from backup, which we do to a number of large filers, but we only backup data. Configuration management is the critically important tool that makes it possible to restore the OS and software with its configuration after a failure. LVS is the other important tool for redundancy.
everything I see says only stripping drives is available in XP/2000. Only Windows 2000 Server/Adv. Server/Datacenter, and Windows 2003 Server and up feature redundancy through software.
So it sounds like the originator was onto the cheapest for home, windows solutions.
"You're posting to Slashdot asking whether you should trust Microsoft?"
Nope, he is asking whether he should trust a software RAID for data protection...
...and no, he should not. RAID (both software and hardwar, all RAID levels except 0) are about minimizing downtime and not about protecting data.
"the only way you'll be able to access that data"
Gee that's funny, because I can read the data from my backups just fine.
edit/
Windows supported solutions, you can hack XP into installing server add-ons. Doesn't sound like a good solution to recommend.
"I want any drive and its data to be as safe and portable as possible (that's the reason for choosing FAT32)"
.
Hardware RAID often uses a header to store meta data about the RAID configuration. Software RAID for windows does too (8MB), but that is portable across it's lineup.
In other words if you want this to be portable you had better check. It may not be possible to just pull a hardware RAID disk and slap it into another system due to proprietary meta data headers.
Simple answer: buy more drives. Put the extras in the closet. By the time you run out of spares, it will be time to move on to new drives.
Fast facts:
I prefer pure software RAIDs, for a simple reason: They do not depend on available hardware. If one controller dies, switch to another one: Other brand, other type, other drivers, and the RAID still works. If you insist, you can even mix an IDE drive, a USB drive, a SATA drive and a SCSI drive into a single RAID. Try that with a hardware or host RAID. Some people even built RAIDs of floppy disks or USB sticks (not for pemanent use, of course).
My faithful old Linux home server runs two RAIDs, both in software: a RAID-1 for the OS (remember: the BIOS does not know about the RAID), and a RAID-5 for the data. The RAID-1 used to run on old SCA drives, but recently, I switched to two small IDE drives due to unrecoverable SCA cabling problems. The RAID-5 is composed of four IDE drives, connected to two IDE controllers, each disk on a single IDE cable. An external USB disk is used to back up my data, rotating through 10 days. All filesystems are ext3, all disks are monitored using SMART, all RAIDs are monitored. If anything wents wrong, I will get an e-mail from the monitoring software.
Until recently, one of the controllers was an el-cheapo non-RAID controller, and the other one was a donated, expensive, well-known brand, RAID-capable controller running in non-RAID mode. The latter one decided to randomly take some free time on the job, and either disconnected from the PCI bus or disturbed it, causing panics in the OS above. Only pure luck protected me from data loss. I ripped it out of the machine, kicked it into the trash bin, rewired the RAID to use two disks per IDE cable, and verified and reconstructed my data. Some days later, another el-cheapo non-RAID IDE controller arrived, the same brand, model and type that already sat in the next PCI slot. So I rewired the RAID again to work with one disk per cable, everything was fine again.
For a new small business or home server, I would use nearly the same setup again: Two software RAIDs, one for the OS, and one for the data. Upgrading the OS is just fun when you can
Denken hilft.
Vista/XP, etc supports stripping, but not a mirror (or raid5) which was what the original article requested. Granted stripping is the lowest form of RAID, so technically true.
Should have read the image in your link. doesn't protect your data, only gives you a performance advantage. to the contrary it makes any single disk failure capable of destroying data on all drives.
Definitely not raid, definitely not what was requested.
If you are dealing with enterprise hardware and a proper contract, then Sun will come in and fix it right up for you.
If you are not, then you are just wasting time with very expensive hardware.
There are a number of solutions, not just Windows 7 or a hardware RAID controller.
To begin with, every NT-lineage Windows version ever produced supports software RAID out of the box. Add that to the fact that any major Linux distro today supports software RAID. And so do the *BSDs. And Mac OS X. And Solaris. And probably a bunch of other platforms I can't think of right now.
Perhaps he has already decided to run Windows 7 for other reasons... having done so, none of these options are avaiable to him. Except the NT-lineage Windows software RAID, which is what he's asking about.
Hell, you could buy one of these one of these and throw the drives in it, connect it to your network switch, and presto -- instant RAID+NAS.
My experience is that NAS is a _lot_ slower than local storage for many applications. If he's a developer, for instance, he really wants local storage, otherwise he's going to be suffering substantially slower compile times. Video encoding and DVD authoring likewise is noticeably slowed by access over 100Mbit networks. And he may not want to update to 1Gbit just so he can use a fancy NAS box.
I think we would all like to know why you think Windows 7 is your only option, because if that's what you think, you don't know how mistaken you are.
His options, basically are:
* A hardware RAID controller. Either use the one on his motherboard, or shell out cash for a better one. Summary makes it sound like he's decided he doesn't want to spend any more cash, and I can't say I blame him. It isn't worth it.
* Software RAID as provided by whatever OS he decides to use. Generally it isn't worth switching OS just to get a better software RAID implementation, at least not for a desktop machine, so for him this comes down to Windows 7.
* NAS; this is an expensive solution that provides suboptimal performance. If he has only a single desktop machine there are no real advantages to the approach, either.
* SAN; this is an even more expensive solution, although admittedly it would provide the best performance. Not really a realistic option for a home desktop system, though.
So I don't really see any viable options for him other than the two he asked about.
RAID1 "might" protect you from a failed drive.
Sorry, but RAID is so early 2000. RAID is a technology based on, the REAL meaning of the RAID name, Redundant Array of INEXPENSIVE Disks.
Raid is not backup. Raid is NOT faster (on average). Raid is NOT reliable.
RAID does not backup your data because RAID arrays do fail.
RAID is not faster because every type, in one way of another, requires some amount of disk-write overhead.
RAID is NOT reliable because you need to maintain it to keep it reliable.
Backup is backup.
Disk space is disk space.
If you need disk space, use raid0+1 it is the most reliable of the bunch.
If you need cheap disk space, use raid 5
If you noed fairly reliable disk space, use raid0.
In the end, I recommend a device like a dlink NAS box with a could sat drives. It does LINUX software RAID and provides good speed shared to windows, linux, and mac systems. I bought one a few weeks ago, and I'm not turning back.
If you need REAL high speed for something like database, that's one thing. A NAS on gbit ethernet is fine for most everything else.
All three major OSes, Windows, Mac and Linux, support R/W access to NTFS. With FAT32 you're limited to 4GB file sizes and you get NO journaling - which means you actually have a greater chance of data loss when you lose power. RAID won't help you there.
Think about your needs for a bit. Do you want to be able to access your terabyte plus of data from other computers or other OSes? Depending on your needs you might also be able to just use a setup like that of my place: networked Linux based storage over a Gigabit LAN. It won't work if you're grinding away 4 VMs at a time on VMware or editing video, but it's fine for storing downloads and music. Going with a Linux based storage server means you can also access the drive over whatever protocol you want, Samba/NFS for local traffic and even SFTP or HTTP for accessing it away from home. You also get a lot of flexibility with your RAID options. Try picking up a third drive (the same size) or even two more and putting them all in a RAID 5 configuration.
Whether you use local or network-attached storage, go for software RAID. It takes almost no CPU time on modern computers and you can move the array to a different computer. That's an invaluable disaster recovery path to have. And use NTFS, for previously stated reasons.
Nope, he is asking whether he should trust a software RAID for data protection... ...and no, he should not. RAID (both software and hardwar, all RAID levels except 0) are about minimizing downtime and not about protecting data.
Err.. he actually says "should a drive crash, I want the system up and running in no time", i.e. his goal is minimizing downtime. I don't know where you got the idea that this was a replacement for a backup strategy.
I'm gonna just count on the RAID controller taking out all my drives and all my data when it dies.
This way I will not be disappointed.
I think the bigger issue is... he's asking about data integrity, yet he's using a Release Candidate OS, not a final version. (Yes I know Final Versions can and do have flaws, but a RC is going to have more than a final, a Beta more than an RC, an Alpha more than a Beta..)
I laugh when I see people say,
"If the RAID controller dies, you have to replace it with the same model. blah blah blah"
If my RAID controller dies, I am not going to trust any of the data on any of the drives that were attached to it when it died, so it doesn't matter what brand it is, because there is no way that I am even going to bother trying to recover the data.
You really need to make your backup strategies based on "worst case" scenarios, not just the "what happens when my drive dies" ones.
Just askin'.
Cheap hardware (mobo) RAID controllers are actually SoftRAID controllers that implement the RAID functions through the bios and the driver. You don't want those, because if the controller fails you might lose your data, because you have to get the same controller, because the data is stored in a proprietary format of that card or mobo chip.
So it is much safer and saner to use the os for software RAID.
Why you want to go with FAT32 is beyond me, though. Your data is much safer with NTFS. Linux has good NTFS drivers now anyways.
1) Don't use any RAID and schedule a daily Full backup to the other drive (using rsync, unison, even xcopy called from a .bat file)
2) Use Solaris and ZFS. Linux with software mirroring (you could always use wine or vmware to run windows apps you must have)
3) Use Windows Software RAID. But I wouldn't use Windows 7 RC. Use a proper supported O/S.
I don't think that hardware RAID on a windows PC is not worth it because if the controller goes it's hard to find a replacement.
MOD PARENT UP. Good explanation. Statistics often assume randomness, and often events are not actually random.
Speaking from experience: The software RAID in Windows XP (and to a somewhat lesser extent, Server 2003) is *VERY* temperamental. Unless your drives and controller channels are identical and are both configured exactly the same, the RAID would need a re-synch after every single boot. Also, to get it working requires that you use Microsoft's "dymanic disk" format, which probably makes the partition table nonstandard (and on one occasion I had a dynamic disk created on one computer not readable on another computer running the same version of Windows). Linux software RAID on the other hand is very reliable and operates very simply: the RAID superblocks are placed so that you can just ignore them and mount the drive normally; the filesystem is exactly where it would be if there were no RAID. It doesn't screw with the partition table, so all the disks are just normal disks and you can read them in any computer, in any OS that reads whatever filesystems you used. The hardware can be very different -- I have a few arrays where part of it is on SATA and the other part is on PATA. For fun, I once tested an array where half was on PATA and the other half was on external USB. Worked fine, even across reboots (as long as you remembered to turn on the USB drive first :P).
You can even (if you're sadistic, or rsync isn't realtime enough for you), have half of the array on a network mount. If you set it with the write-mostly flag, your read performance won't even be that much worse (but large writes will probably suck).
Add that to the fact that any major Linux distro today supports software RAID. And so do the *BSDs. And Mac OS X. And Solaris. ... Hell, you could buy one of these one of these and throw the drives in it, connect it to your network switch, and presto -- instant RAID+NAS.
So basically your "other options" are "use a different OS than what you want to" and "buy more stuff"?
You need RAID AND A DECENT BACKUP.
This is, of course, idea.
That said, if you're going to have only one... go with the backup.
Dude.
You sound like your constipated and you just gotta get out that windows sucks....ughh ugghh uggh ahhh thats better...
He didnt say that windows was his only option, thats just what he was using..... Yup...Cheap low end raid controllers, they're the only ones on the market too, dont see ya listing all his other options there...
I do not believe you can use Fat32 formatted volumes to make a software raid mirror in any version of Windows, ever. If I'm wrong, tell me, but I have been working with the 'Disk Management' in windows for 15 years and have never been able to do this. Not that I would. 'For portability' is not a good reason to use a file system that was left behind by Microsoft years ago. You can mount an NTFS partition in several O/S's these days effortlessly. I agree with every one else here. Build hardware RAID 0 into your system, use all the space on those disks that you paid for, get some performance out of them. And most of all, PRACTICE RECOVERING YOUR DATA. Do mock-crashes, do some research, READ what other are doing about their problems, make some notes, get acquainted with your hardware, and your backup software, and whatever tools you end up using for data recovery. This is essential for real data loss prevention. Am I preaching? Sorry!
Microsoft dropped software RAID (except RAID 0, which is not redundant) from "non-server" versions of Windows starting with Windows XP.
--
"Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next.
I often use inexpensive SATA RAID controllers from Promise (I do not work for promise). They don't cost very much and they have been absolutely reliable for me (for many years now!). I often stick with RAID 1. I've built several RAID5 arrays and I don't find a lot of value in them for low capacities. Mirroring (RAID 1) is straight forward, and if you ever have a problem you can always read one of the RAID 1 drives using a SATA to USB interface, or if you ever need to clone a hard disk it's easy. Promise seems to use the LAST 64k of the hard drive for it's mirror info, not the FIRST 64k! this makes any of the two drives in the RAID array easy to use out of the array when/if you're in a jam (for whatever the reasons). As far as RAIDing your data only, in my opinion RAID is designed to avoid lengthy recovery procedures - don't put yourself in a disadvantaged position - all hard drives fail eventually - RAID the OS, your data, everything! If your server is a very busy server - start looking at higher end RAID solutions.
I'm IT for a small consulting company, not unlike your home.
a) RAID is not a backup; a backup is more important than RAID
b) which file system to use is a critical decision
c) Don't mix OS, Applications or Data on the same file system
Plan A: Don't use RAID at all. Use a single disk with 3 partitions (OS, Apps, Data) and mirror them automatically, nightly, weekly, whenever you fill like it. Weekly is a good trade off for a home system and lets you have a complete recovery should the system get infected by a virus, but you don't notice it for a few days.
Plan B: Setup a NAS device with 2 disks, mirrored. No RAID in the PC. Keep your data on the NAS. Whenever something important changes, backup the PC to the NAS.
If you weren't stuck with Windows, you'd have many, many other options, like
1) setting up an OpenSolaris ZFS as a file server and performing snapshots of your RAID-Z (or mirrored, or RAID1+0) protected data. ZFS is very impressive replacement for a LVM and FS, but not worth using unless it is inside the kernel (Linux doesn't like the license, so it won't be anytime soon).
2) setting up a Linux server with mdadm as a file server and running RAID-whatever-you-like and a good, solid, proven file system like JFS or XFS.
As for backup software, there are many choices. I like rdiff-backup if I'm not on a ZFS backup server receiving remote snapshots from each of the production servers. Lots of good ideas for home backup here: http://lifehacker.com/search/backup/
I agree with you that hardware RAID is a hassle for home users. Software RAID leaves control of your data with you, not some card maker. I've been burned too.
"should a drive crash, I want the system up and running in no time"
That's a fair reason for RAID and indeed specifically for mirroring. But the most common TSHTF scenario is stuff like power supply or mobo
failing so your goal should be to increase hardware redundancy at every level. Get yourself a good ESATA enclosure for each drive with
independent power supplies and use software RAID.
"I want any drive and its data to be as safe and portable as possible (that's the reason for choosing FAT32)"
I assume FAT32 is for portability (everything reads it) and not for safety (uh, you'd be nuts). This is fair but a better choice is NTFS. It is also
readable by everything (writing used to be a problem but not anymore) and much better for safety.
"even if the OS or the controller screw up big time"
If the OS or controller (or more likely the user) screw up big time, you'll need backups.
This is the best setup for "saving your data." Why? It's cheap, convenient and guaranteed to save your ass. Probably $100 for a USB HD /w pretty external casing and backup software, it's a set and forget thing (for the most part), and the HD is on a seperate power supply (so your PC becomming toast won't effect the backup). What a nice solution. I don't even have a need for a backup but I'm thinking of getting this setup for myself because, well, why the hell not? It's too good.
I tend to recommend people buy an inexpensive external USB or firewire drive, leave it attached and assigned as a backup device, and have some software package run a daily backup of all the relevant folders and files they might need to save.
Buy two drives.
Every Monday take one to work (make sure your name is on it) and bring back the one you had at work and hook it up to your system. This way if there's a power surge or fire at least one set is offsite.
A backup isn't really a backup unless it's offline (and preferably there's also an offsite).
Hope for the best, but be prepared for the worst.
The one downside of a Drobo is vendor lock-in, if your controller dies. However, this seems to be the case with every RAID controller.
I use a mirror drive in a mobile rack as a backup and an rsync script on a modified liveCD to allow me to point and click to the rsync script to tell it to run the backup. I plug in the mobile rack drive and boot from the LiveCD. When the mobile rack is not in active use, i.e. either being backed up to or restored from, it is stored well away from the computer.
This has served me well for years, including after a hard drive crash. I was up and running in 15 minutes after the crash.
Though since the Knoppix 5 LiveCD it was based on does not recognize SATA running in ACHI mode, I have to rebuild around another LiveCD. The big advantage of this is that a drive in a mobile rack that is not plugged in is completely immune to anything awful happening to the main system, including the events which are most likely to blow up a hard drive.
Tech Public Policy stuff
As others have mentioned, I also agree that RAID0 isn't raid. Even if you do consider it RAID, it doesn't address the submitter's request.
Comment of the year
Well, Windows 7 Ultimate does support mirroring. Sadly RAID 5 support remains for server versions only. Hopefully they'll finally wise up and go all the way with whatever comes after 7.
ZFS period end of story. No other production stable filesystem equals it for level of paranoia about EVERY bit that comes off the disk. Using ZFS in either mirror or RAIDZ2 with many disks is the best possible solution IMO.
Robocopy your data from one drive on the 1st computer to a 2nd drive on a second computer, nightly while you sleep. If you need more assurance, robocopy your data to a 3rd drive on a third computer.
#3 - I had a lovely experience with an application error causing massive data loss. STEAM. I was an early adaptor, but it took me a long time t be convinced to go back to it after this event. I was in the process of rearranging data -- I was going to make the drive STEAM was installed on a data-only archiving drive, instead of a mixed data/application drive -- and ran the STEAM uninstaller. Well, it uninstalled alright. It also wiped everything else off the hard drive. Not too happy about that one.
raid will make your data highly available, not secure.
if you want security, you need backups, and backups are:
- off line (viruses, power surge, sabotage...)
- off site (fires, theft...)
- tested (i've got horrors stories of people that THOUGHT they had backups...)
- multiple (... and of backups that turn bad at the worst possible moment)
Raid is none of that. I know plenty of people who thought their data was safe because they had raid. It isn't, it wasn't, it ain't ever gonna be.
The Cloud - because you don't care if your apps and data are up in the air.
I use Mozy. It's cheap and easy. They are also owned by EMC so I'm not too worried about them going under. I only use a laptop so I don't have to worry about having a usb disk permanently dangling from it, or having to remember to plug it in periodically. I get unlimited storage so no worrying about storage allotment. My broadband connection is fast enough that it will backup any large changes overnight at the longest.
It also gives you the added advantage of being off-site, which all reliable backup systems should have as a component. So no worrying about house fires etc. This brings us to the interesting use case they are currently milking where a guy who was on the Hudson river crash used Mozy and his data was perfectly safe, whereas another guy lost both his laptop and USB backup to the bottom of the river.
(http://blogs.usatoday.com/technologylive/2009/03/when-your-lapto.html)
Just use the integrated raid (yeah, softraid or fakeraid) to do it, make sure its the chipset (intel or nvidia) that is doing the work.
Sure the performance isn't there in general (and if you raid0 more than 3 drives on the NV chipsets, not there at all) but it is OS agnostic and is relatively cheap to get the data back if your board dies (cheap mobo + celeron CPU) since all chipsets from each of those two makers support each others raid.
As others said though, what you are doing is not protecting the data, but just increasing your up-time if something bad happens, backups are your only protection.
I would be one of the first to suggest using an areca raid card (or similar quality) should you be looking for performance, but your not, so I won't :)
...
My data is VERY important to me. Therefore I want the cheapest, lamest solution possible. Please recommend the "best" way to protect my data...
>everything I see says only stripping drives
*sigh*
computer porn is everywhere, now :(
hawk
Build a cheap headless NAS that uses ZFS, such as FreeNAS 0.7RC1. Best way to go hands down for a high-availability home file server.
I have a hardware RAID 0 setup for the data that I need immediate access to.
Yes I have drive failures and yes they are catastrophic. But I am prepared for them and they don't slow me down for long.
I have the whole thing mirrored on a networked file server. It's software RAID because it doesn't need to be fast, it just needs to be there.
I do the backups from the remote system. I backup onto hard drives, then I unplug them and put them in a safe place. I rotate several drives in this fashion. I check them regularly for hardware issues. I have to trust these drives more than normal, so I give them a really good working over when they are new. If I were rich, I would buy server drives for these. I am not, so I condition them myself.
I only bother to back up my home directory. Why backup data that can easily be restored with an OS install? It only takes me about 15 minutes to get a newly-installed machine set up on my network.
When I do an OS upgrade, I wipe everything and restore from my backups. It's good practice for when I will actually need the backups. I like nice clean systems, not ones with years of accumulated cruft.
If my RAID controller fails, I can mount the backup mirror and continue working, albeit more slowly.
I haven't lost data in years.
raid is for high availability, not backing up your data. Put another way, it's not *data* redundancy, it's *hardware* redundancy: it allows you to store one copy of your data on multiple devices. But it's still one single filesystem spread across those drives, and if you screw it up you're still fscked (pardon the pun).
JWZ (Netscape, DNA lounge) has a very simple guide on using rsync. rsync is really about all you need. Apple's Time Machine is based on it. Now, this guide is not eïxactly windows friendly, but 1) you stated that you only need to back up your data, not your apps, and 2) you sound like you're pretty technically adept so you're probably running cygwin on your windows boxes. Rsync should be able to handle this just fine on win32.
For data serenity, two separate single drives beats raid every time.
"What happens when your offline backup company goes under?"
You start using a different one? Personally I have a colocated server where I keep encrypted backups along with a hard drive in an external enclosure attached via USB. I keep backups of all my important files in two locations. Now, if the online backup service went out of business and my house burnt down before I could find a new one, I'd be out of luck I suppose. I think I'm a little more likely to win the lottery though, so I'll take my chances. This also assumes that the backup service shuts down and cuts you off without any notice whatsoever.
Why is it that Open Source WEENIES have this irrational fear of the raid controller? OP says he got burnt, but its probably due to his own stupidity.
Over the world, masses of Linux users insist that hardware raid controllers are the devil, and will eat your data, and that we should all use cheap pata/sata drives and software raid. WHY ?
ARE THESE PEOPLE ALL COMPLETELY FUCKED IN THE HEAD?
Hardware Raid Controllers are the answer, but only if you know the right question. Most weenies who use windows or ZOMFG LUNIXES!!! have no idea what the right question is, and therefore, continue to have their data eaten. Mind you, these are the same idiots who would blame their computer for loosing data after they typed format D: into their ms-dos machine.
Use Raid for keeping uptime - only protects against a drive failure - corruption and the above described crap does you no good.
Dropbox - easy to use, cross platform Mac, Linux, OS X - even has built in versioning, web access, iphone etc.. great service!
Another strike against Vista I didn't know about unless it's just deeply hidden somewhere - however the related Server2008 should have it. I still have an old NT4 box to run legacy software that is using software mirroring of it's drives.
Better yet ... get two inexpensive USB disks, and swap them every day or two, maybe even have the inactive one in a separate location...
"My experience is that NAS is a _lot_ slower than local storage for many applications. "
It's really about storage requirements. If my Internet connection is only 20mb/s and I just want to stream some movies at 50mb/s (1080p) then does it really matter if the network is "only" 100mb/s?
OpenSolaris ZFS
Free and the best.
see http://en.wikipedia.org/wiki/ZFS
Amazing, all these posts and hardly anyone actulay answered his question. Frankly these people who say raid isnt a replacement for backups should just shut the hell up already. The reason people use raid as in mirroring is just for redundacy, and for that purpose it works well. Regardless of what people have been saying about how useless it is I cant say I ever saw a mirrored array fail in a way to lose data when the whole system was fried, talking about file system errors and mailware causing you to lose data is like not driving because you might get carjacked.
Also what is with the people recomending using 2 identical drives? I have seen failues on arrays like that where one will die and then a few hours/days later the identical disk will die as well. Its better to have at least 2 different models so that if you bought a bad run when you got your drives, they wont all die close together.
But to ACTULAY ANSWER HIS QUESTION...
I have never had an issue with a onboard controler. Unless your planning on having a large array instead of a mirror it should be just fine.
A false sense of integrity is worse than useless.
A structured backup system is the only sane answer, where the oldest technology (lowest density) is closest to your daily work, and the newest technology is your backup. Your backup will always have about 2x your online, which is about right. Careful snap-shots, with off-line validation [ ie. ensure that you actually backed up something useful ] is step 1. A well structure pipeline will stage the snapshots for integration into the whole [ ie. yesterdays filesystem ], and archive the accumulated snapshots.
As your storage demands outstrip your storage, discard your oldest [ ie. online ] storage, bringing forward the second oldest, and backfill the chain with what is new today.
In a critical event, you will lose the delta since your last snapshot [ you had 2x, why aren't you snapping more often? ], but it will be online immediately.
If you are paranoid, mirror your backup chain at some point - but don't be too aggressive. A backup is only as good as its validation, so the more redundancy you introduce, the more validation you must perform.
RAID is an accountants answer to integrity - unless storage cost represents something substantial to you, it should be avoided.
I had my but saved by mozy when two drives on my raid 5 array failed at the same time.
Yup RAID is not perfect, but it saved me a couple of times when just a single drive failed. Yes, a 4 drive array had two single drive failures, and a 2 disk at once failures. Suffice it to say I will never buy a western digital drive again EVER!
I've been using Slicehost for a couple of months, backed by Rackspace Cloud Files instead of Amazon. As far as I'm concerned it the best thing since the Original Nintendo.
I have two PC's and a laptop at home and a work computer backing up to it, the resync occurs and night so I don't see any performance issues and its completely encrypted on the backed so I don't have to worry about someone browsing through my collection of unpublished poetry about water fowl.
If you haven't actually used one I suggest you check it out.
I would go with software RAID. Hardware RAID will be faster, but software RAID gives you simplicity.
I have found hardware RAID controllers temperamental. In the situation where you can keep buying the exact same drive forever (or replace all at once), hardware RAID is nice and stable, and should be faster than software RAID. But they tend to be inflexible -- incompatible with some drives, you can't mix drives at all (if your initial drive is 500GB you can't put in a 500.1GB spare..), etc. My friend right now is dealing with finger pointing trying to get *any* 1.5TB drives to work in an array. Also, think down the road that in case of card failure you will have to find the exact same card on the market if you don't want to lose your array -- even later models of the same card may not use the same on-disk format.
I have no comment on the Windows 7 RAID, I'm assuming it's functional. This is what I recommend because you just install your OS and go, you don't have to worry about having a physical card (you can plug the drives into anything with SATA). Disadvantage, you are dealing with sending the same data to two different disks, going over the system bus and all, versus hardware RAID where the system deals with the I/O once and the card takes care of the duplication and such.
Second, DON'T USE FAT32. It has only 2 copies of the FAT (file allocation table) so if they are out of sync it's hard to tell what's what; and only 1 copy of other important data structures. It's very space inefficient, and has serious size limitations as well. As you are using Windows, feel free to use NTFS on the disk; Linux distros handle it, and OSX has "NTFS-3G" ported to it as well. As a Linux user I'd put ext3 on an external hard disk by default (I have a small FAT partition with a ext2/3 access software for windows on it too to appease the windows boxes.)
Also note, some of cheap RAID cards are software RAID -- they present SATA ports over the bus, and the card drivers implement RAID. Under Linux, these use Linux'es software RAID infrastructure, but under Windows it is in the vendor drivers and seperate from Window's normal RAID support, so performance may vary.
What's the real fear here... that you won't be able to recover data?
Use NTFS if it's windows.
Use a released version of windows - not a beta.
Software raid-1 is fine - perhaps even better in some cases.
Raid is not a replacement for backups. Please repeat that - many times. Raid is not a replacement for backups.
You real solution is probably a proper daily backup solution and whatever raid-1 is easiest to set up.
Scrolling through the first several posts, I see a lot of posts basically saying RAID is unreliable. I couldn't disagree more. I set up a Linux software RAID 5 file server in my house about 8 years ago. It started as 4 Seagate 120GB drives. Over the years, I've had 3 of those drives fail. I've never lost any data.
I can't afford to go out and pick up the latest and greatest in storage every time a drive fails. I replaced the failed drives with refurbished drives of the same size from random manufactures. I've never had a problem. The RAID is just as fast to access and write to as when the drives were matched. RAID is an excellent strategy if your concern is drive failure and uptime.
As others have pointed out, it won't protect you from accidental deletions and overwrites. Regular backups help in this area. However, if you accidentally delete/overwrite your backup, or the backup fails, you are in the same boat.
For data that just can not be lost, I have a copy on the RAID, and a backup on DVD that gets refreshed every couple of years. Everything else just sits on the RAID.
I've also run a file server at work for about 5 years now. It uses the same setup. I've only had one drive fail there. Again, I've never lost data. I have a redundant server set up that contains a mirror of the primary server. However, I've never needed to restore data from it. It is nice to have when I'm doing updates. Everyone uses the primary server while I test the updates on the redundant server.
Request a Linux Shockwave player here: http://www.macromedia.com/support/email/wishform/
I'd trust a true hardware RAID controller from a reputable manufacturer. I would never trust a software RAID via Windows, OS X, ATTO, etc. More evidence, less anecdotes from techies that probably don't have real world experience. imo.
"Time is nothing; timing is everything."
Perhaps that's the OS that is running on the computer that he wants to use the RAID in?
I tend to recommend people buy an inexpensive external USB or firewire drive, leave it attached and assigned as a backup device, and have some software package run a daily backup of all the relevant folders and files they might need to save.
1. I do this for my digital photography. (1 weekend can yield several GB of data).
2. I am better off manually backing up my files than trusting the computer to get it right. I haven't found backup software I trust. Yes I realize I can miss things too, but I periodically compare the list of backup directories with the ones on the main drive.
3. This whole regime is useless unless you regularly backup your 2nd device (external drive) to a 3rd one and copy it off site. (It gets packed off and sent to my mother's house). This mitigates against losing data if I have a house fire or if malware erases drives 1 and 2. I limit exposure to a few weeks.
4. Once you've backed something up to a 2nd or 3rd drive, you should NEVER overwrite it with another copy. Never copy over the top of one directory if a few files have changed in it. Otherwise if you've damaged/corrupted the files on your first copy, you're propagating that damage.
5. I wish I could get individual drives to show up as read only in Windows. I took it for granted that there'd be something like mount readonly. There isn't anything simple or standard. I think there are utils that do it but they're expensive, non-standard. There are free apps that will change registry settings to block all removable drives but that hardly allows copying from a backup to another USB drive. There are expensive hardware USB write blockers used in computer foresnics. If I were using Linux it'd be trivial to mount a filesystem readonly by default.
These posts express my own personal views, not those of my employer
The first thing to learn about RAID or any other technology that you're using for enhanced reliability: If you use it without first testing it in failure scenarios you might encounter and figuring out exactly what you need to do to fix it (e.g. How do you tell which piece of hardware has failed? How will you know to respond if the automatic failover doesn't work? etc.), you might as well just pretend that instead of spending your time and/or money on a redundant solution you just took a vacation and/or lit the money on fire because that's effectively what you will have done.
With that in mind, although I tend to think I know what I'm talking about, you can take the rest of my advice as help brainstorming those failure scenarios and hints about answering those how-will-I-fix-it questions, but it's no substitute for knowing and testing your own set-up and checking the answers yourself. Okay?
Good.
After having been burned by a not-so-cheap, dedicated RAID controller, I have been pointed to software RAID solutions
Gross generalization for the short attention span crowd: RAID 1 seems simple, but there are lots of options and lots of details to get right. And if you've run into the limits of your knowledge dealing with RAID 1 on a dedicated RAID controller, then going to a software RAID 1 solution (which often doesn't secure against all the same problems and gives you more things to troubleshoot directly when things go wrong) may not make your life simpler...
In particular, remember that although your RAID 1 array stores some metadata, it probably doesn't keep track of which disk has the most up-to-date copy of each block, so if you manage to get two working but out-of-sync disks, take care when you're deciding which disk to restore the set from. Also remember that as we approach 1 unrecoverable read error per disk territory, the same wisdom about replacing RAID 5 with RAID 6 also applies to 2-disk RAID 1.
Moving on... =)
In the world of always-busy transactional databases and server systems, RAID is seen as part of complete breakfast of 100% storage reliability, and people want it to guarantee that whenever a program's explicit request to sync a disk write goes through, the data is definitely going to make it to disk despite a power or equipment failure. This means:
You can test that syncs are working properly by doing plug-pull tests (e.g. see http://brad.livejournal.com/2116715.html)
If uptime is important, they'll also have:
At the other end of the spectrum, RAID 1 is sometimes used on desktop PCs as a real-time 'backup' solution for people who are too 'busy' (i.e. lazy) to make real backups.
In the world of single-user PCs, the conventional wisdom is that most applications operate on documents in RAM (where they are vulnerable to power failures anyway), and only write them to disk occasionally when the user explicitly asks for it, so it has become common practice to leave disk buffers on for better write performance at the risk of data loss if a file is being written during a power failure. Application-level recovery features and journaling file systems have allowed this approach to survive into the era of PCs with modern multitasking operating systems. If you're willing to take the performance hit, you
I sit here, writing this reply, on a motherboard fake raid that's down one HD. All of my data is still here. It's all still easily accessible and I can keep using this system while I wait for the HD that I ordered to show up so I can get back to full data duplication. It's a very handy feature considering that if I had had the HD fail and wasn't using RAID, I'd be SOL as far as this system is concerned. To those who say that RAID isn't backup, they're right. Making any copies of your data and keeping them in the same building as the original data is also not a backup, as a fire can easily wipe it all out. For general usage, keeping your systems operable during disk failures, I think that RAID is a wonderful thing to use and I just use the fake raid of the motherboard. On my linux system, I use a software raid because it's easier than getting the system to recognize the motherboard-based solutions. If that's changed by now, then that's great, but it wasn't an option when I built the system.
You don't have a hardware or integrated RAID controller.
What you have is a non-RAID SATA controller, plus software RAID support in BIOS + Windows driver.
This is easiest to see when booting Linux, whose policy it is to only export your hardware, without any fakery.
See Linux SATA RAID FAQ for a clue...
I bought eight of the mirrored drives from WD that have two 500GB drives that can be used in raid 0 or 1. These drives have been a major disappointment as they have all failed unexpectedly and the mirroring was of no apparent value because when the usb/fw attached unit stopped responding, it was useless whether there was a remaining platter with good data or not. These units weren't cheap and I paid about $3000 for the eight of them. I am really unhappy about this and will not be buying WD products any more. A have pile of these I am hesitant to throw away, but they are little more than doorstops now because I wouldn't trust them at all, even if I could get them going again. What a waste of money. (And my trust)
What happens when your offline backup company goes under?
Further more, what happens if that backup company's hard disk array (that contained your backup) fails at/around the same your hard disk fails?
Personally, I haven't yet encountered anyone who really got benefit from those personal Internet backup services like Mozy. In regular use, it always seems like the person exceeds their storage allotment or Internet connectivity issues prevent them from recovering what they need, when they need it.
I tend to recommend people buy an inexpensive external USB or firewire drive, leave it attached and assigned as a backup device, and have some software package run a daily backup of all the relevant folders and files they might need to save.
It's great that your data is stored offline and off-site ... but I'm just not sold on most of the implementations for "home use" being as great a solution as they first appear to be. Many of the providers have come and gone over the years, too. What happens when your offline backup company goes under?
Um, I recovered my data with Mozy. Yes, external HDD backups are good too, but off-site is a good idea. What if there is a fire, or water damage, etc. Your backup is useless if it is sitting next to the computer that is burned / waterlogged.
Both Windows RAID and most likely the controller on your board are doing software RAID. Only higher end add-on or on-board controllers can do hardware RAID. Last I checked Promise and the RocketRAID controllers were still doing software only. It may be an add-on board or option, but it is still relying on software for the RAID work. 3ware was the only vendor I ever knew of that did true hardware RAID on their SATA controllers, at least in the SOHO/consumer grade arena.
Good, you want to separate your data from your OS. That make a number of things possible:
- your OS upgrade shouldn't affect your data drives
- you can treat data and OS backup seperate
- you'll be classifying your data (at least somewhat)
Take the next step & put them on a NAS (I don't care which as I'm not supporting you)
- Now you can get your data on multiple machines, regardless of OS (if you do it right)
- Your OS won't affect your data
- Gigabit ethernet is not expensive and it usually beats USB 2.0 speeds.
- You can isolate your data physically
= away from foot traffic with liquid spills, etc
= locate in a cooler environment
= not put in dogy devices into the data device
= less noise & heat at your desktop
For a fixed database of static information, this may be true. The vast majority of database implementation, however, is one that is also being written to in realtime, such as a webserver database, or POS system where the overhead created by decreased write performance will more than negate any advantage in decreased seek times. Really, it all depends on the application you are running, and the specific circumstances of your network/system setup. RAID1 can be useful. I just rarely see it used correctly.
uhh... did you really not get that? Or are you just trying to make a point?
His options are HW raid - provided by a chipset on his MB, as he is unwilling to use a dedicated raid controller having experienced failures in the past, or SW raid - provided by the OS he is running.... which happens to be Windows 7.
Reading comprehension
what are the experiences using intel motherboards hardware raid 1 (for example P45 chipset)? I'm deciding my home network. I was thinking of a PC with Ubuntu Linux for server with two 1TB WD green power disks in raid 1 with the motherboard hardware controller. What happens if the mother dies? can I read the disks in another computer without raid? does software raid take a performance hit?
"I think this line is mostly filler"
Well... then I did not RTFA. Another flaw of some of us on /.
I was rather hoping that he already had plans to do backups by some other means. If not then I agree completely that backups are more important than RAID.
StoreBackup is an excellent incremental backup solution under Linux. It uses hard-links, so each incremental backup looks complete - none of the nonsense of figuring out which incremental backup has which file.
A bit of a tangent but what about viruses spreading to your backups on USB or eSATA drives? So if you leave your backup drive attached all the time (or even some of the time) how do you keep malware from being copied to your backups or restored to your fresh machine? What is the best practice for the family PCs?
The RC does. So did the Vista RC. You point?
You say you care about your data, and yet you are contemplating using software RAID with an RC OS, along with FAT32? This is like wearing a bicycle helmet to protect yourself during a round of Russian Roulette. (Motherboard RAID is actually software RAID done in the device driver... I would trust this even less than the OS's software RAID, given that the motherboard vendor's driver may not play nice with the RC OS.)
If you care about your data but must use the Windows 7 RC for some reason, what you really ought to be doing is putting the data on a separate box which is running a mature OS (WinXP, Debian Stable, Ubuntu LTS... take your pick). And for God's sake, don't use FAT32 -- use a journaling file system like NTFS or EXT3.
Regular Seagate drives are now Maxtors with different firmware. Seagate seems to have destroyed the quality of their brand. Fortunately WD's are now good drives, and even the consumer models work well in RAID.
There are already 355 posts to this story so maybe I'm late for asking, but...
WTF are you doing running pre-release software on a production system? The final build of Windows 7 is not that far away...
And spend a month trying to download it when your system fails and you need a full restore...
A house divided against itself cannot stand.
To begin with, every NT-lineage Windows version ever produced supports software RAID out of the box.
Except that it doesn't, really -- Even under vista Business x64 you can't create a software RAID device, you'll either need the Ultimate/Enterprise edition, or Windows Server.
Probably better to leave USB drives disconnected when not in use, to avoid wear, malware/virus infection and electrical surges.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
> I have decided that only my data is worth saving
Then forget about RAID. RAID is designed to protect the integrity of the underlying volume - NOT the data that's on it.
> Of course, such a setup should secure my data; should a drive crash,
Then forget about RAID. RAID will only secure your data under some very specific cases of hardware failure of the drive. It does absolutely nothing towards preventing data loss due to (say) a corrupt file allocation table, virus, accidental deletion, or corruption.
> Even more importantly, I want any drive and its data to be as safe and portable as possible
Then use proper backups - not RAID. Preferably off-site backup. I use Carbonite which backs up to the 'cloud' at minimal cost.
By all means use RAID to protect you from hard disk failure, but don't under any circumstances assume it stops you losing your data. For backups, I always use the rule that at any given point in time, assume that the next time you walk back into your house/office, that NOTHING in that building is still there. Do you have a copy of everything you care about somewhere else?
I'm still amazed by people that carry 12 months of work around on a single floppy disk/USB stick/laptop, then cry when they go to the helpdesk asking what "sector not found reading drive A:" means, or perhaps "A USB device attached to the system is not functioning".
Get your data in as many places as possible - preferably three. A drive which is mounted one inch above the main one is *NOT* a valid second place!
FAT32 is a huge mistake. First of all, there is no journaling. All operating systems, at least reads NTFS fine and with the new fashion NTFS-3G, you can write to them and with ''test disk'' or a Linux recovery boot CD, you can even repair them better than Windows ever can.
Don't use FAT. That is all I can say. Even if you buy a mainframe class IBM storage solution, it won't save you from evil FAT issues.
I would hope that their backup systems and failsafes are a hell of a lot more reliable than anything I could come up with.
In Windows 2003 at least Software RAID is not trustworthy. Especially under high loads, Windows will stop synchronizing the disks and then you have a secondary drive that all of a sudden boots into a 6 month old version of your system (if it boots at all).
If you have no other choice, I would go for the 'firmware RAID' - I have never had any issues except that it's not that fast and you might have issues with Linux bypassing the RAID controller altogether (but then again, software RAID on Linux is at least stable). You can get burnt with any RAID controller, that's why we have backups but any decent setup will usually work.
Custom electronics and digital signage for your business: www.evcircuits.com
Our company finally moved away from windows software RAID because of problems. One drive would fail, then while rebuilding we'd find errors on the second drive. Usually bad sectors that couldn't be read. Windows 2003 doesn't do any SMART monitoring so a drive can start failing and you'll never know until you try to read/write data to that part of the drive.
On my linux server I have smartmontools checking the drives nightly which has saved me once so far. I can't find an equivalent for windows so we've moved to hardware which can check for failing drives.
Outdoor digital photography, mostly in New Engl
My experience is that NAS is a _lot_ slower than local storage for many applications. If he's a developer, for instance, he really wants local storage, otherwise he's going to be suffering substantially slower compile times.
Agreed there, but he never said he was doing development and:
Video encoding and DVD authoring likewise is noticeably slowed by access over 100Mbit networks.
Actually, IME, no. Video encoding is mostly CPU intensive so unless you are using some sort of hardware video encoder, the primary bottleneck isn't disk, it's CPU.
NAS; this is an expensive solution that provides suboptimal performance. If he has only a single desktop machine there are no real advantages to the approach, either.
Actually, NAS is not as expensive as you might think. Several devices with 2 bays are available for under $100. And performance may not be important for his needs. Even if it is, the D-Link 323 NAS, for instance, supports RAID 1 and Gigabit Ethernet OOTB and is available in this price range. A GigE switch just ain't that expensive.
My blog
Well, the clue's in the name really RAID-0. Sounds to me like your understanding of RAID is at fault.
I had similar issues, which is why I went the full hog in trying to come up with a open storage system for my data -
I ended up with a nice small NAS using ZFS, if your interested see these articles -
http://www.adamretter.org.uk/blog/entries/diy-nas-requirements.xml
http://www.adamretter.org.uk/blog/entries/diy-nas-software_and_hardware.xml
http://www.adamretter.org.uk/blog/entries/diy-nas-build.xml
You've been "burnt" by an expensive RAID controller failure, so you plan on using:
- software RAID
- A beta (release candidate) of a major OS update
- FAT32
I have decided that only my data is worth saving
Are you sure, cause it sounds to me like you are trying everything possible to lose your data.
As mentioned before, RAID is not Backup. Use RAID if you need to keep something online, despite hardware failure. RAID *should* give you enough breathing space to rebuild without dropping service for a significant amount of time. But that's not to say your backup plan can be thrown out the window because you are using RAID.
Analyze your requirements, and find a solution there - don't shoe-horn in a RAID solution because you've got 2x disks & you think you know "I.T."
Requirement1: "I have decided that only my data is worth saving"
Requirement2: "I want any drive and its data to be as safe and portable as possible"
Requirement3: "I want the system up and running in no time"
Solution: RAID1 plus Offline Backup solution.
The more money you spend, the better a solution you can afford. Software Raid is junk. Cheap Raid Controllers are junk. FAT32 is junk. Beta's are not for production/live equipment. Expensive Raid Controllers can fail.
Disks will fail. Spend wisely.
When there is a serious error detected via SMART, Vista will bug the hell out of you trying to get you to back up.
I'd start by learning how mirroring works. It's obvious you don't have a clue 'cause if you did, you wouldn't be posting this question here. Slashdot is not the place to get this knowledge.
Start here:
http://fedoranews.org/mediawiki/index.php/How_to_setup_and_manage_your_disk_software_mirroring
and learn how software mirroring works first.
Go to the manufacturer's site of your motherboard for the hardware info.
If you need to do a different OS, search for "How do I set up disk mirroring in [OS]" in google.
Google is your friend. Once you know how the different systems of mirroring work, and what their advantages and disadvantages are, you can make your own informed decision. Seriously...
-AC
Sorry, I don't see this at all. Yes, I use Mozy, so I'm a little biased but if I had seen no use-case scenarios where it was helpful I probably wouldn't.
I pay for an unlimited account... there's currently about 90GB of data up there that I can think of... and yes, I've requested a restore on more than one occasion. Hell, there was a time I found I needed a particular file while I was out in Denver off my personal laptop which I didn't have with me (since it was a business trip). Yes, I can VPN into my home systems, but I also knew my laptop was closed and therefore asleep at home. I jumped onto Mozy and "restored" the file to my work laptop within 20 minutes at the hotel, modified it, used it, and copied it back to my personal laptop when I got home.
And bandwidth / internet limits? Uhm... OK... if you insist. Today is the 6th of July. Being a parent of two kids meant I took a lot of photographs on Saturday of my kids enjoying the day... and some video. Sync'ed to iPhoto on Saturday and then just left my laptop running while ~4GB of pictures got uploaded to Mozy. It's damned nice to know all that data is up there in the event I ever lost it. Yes, I also backup my critical files to a file server at home via rsync, but I find that less and less a necessity since Mozy has fulfilled my backup needs for some time. Oh yeah, and I periodically take an image backup of my laptop so that I can restore the entire system to a point in time... I just fire up Carbon Copy Cloner, plug in my Drobo and go out for the day :)
I am the exception to the rule I think; I keep LOTS of backups. But still, my point still stands; that Mozy and other online services are damned good value and don't suffer from the perceived problems that you mention... at least not in my experience.
If what you want is data security, you have a desktop system with just a handful of disks, and you're worried about an unreleased operating system trashing your files, what you want is not RAID, you want a backup strategy. RAID is for fault tolerance. It protects against hard drive failure and nothing else. It will not save your butt after an OOPS moment, or when your shiny new beta OS decides to write zeros all over your filesystem's critical structures.
If you really want a safe, reliable backup system, build a dedicated backup NAS, use RAID on *that*, and dump all your important files on it via a mapped drive or FTP. If you're really paranoid about your data, you can have the NAS run a nightly task to tar/zip and rsync the whole thing to a remote server or Amazon S3.
Not trusting RAID is the smart thing to do, because it is not about trust, it is about making a sysadmin's life easier by adding a little tolerance to the least-reliable hardware in a server. Trust/security is not a piece of software, it is a methodology.
-Billco, Fnarg.com
Your best bet is to set up an OpenSolaris server, configure a zpool and share the filesystem natively using CIFS.
You'll get incremental snapshots, so you can recover a file you accidentally delete. Snapshots are also good in cases of malware infestation.
You'll get end to end checksum integrity of your data, so you can avoid hardware RAID write errors.
You can replicate your data to an offsite server at the block level for backup purposes.
The only downside is you need another server, unless you had really nice hardware and you wanted to run Windows in a VM on the same host. But then again we've been suggesting a lot of solutions that require a seperate machine, when it sounds like the guy really wants a Windows centric solution.
RTFQ. This really sounds like a single-user system. In other words, all your experience (I mean your stats, not your general knowledge) does not apply.
The only reason some of it might apply, is the malware thing; this guy says he's running Windows. Other than that difference, though, this guy's usage is going to be like mine, where disk failure is the #1 threat; absolutely dwarfing "oops, didn't mean to delete that." RAID is fucking awesome.
And while yes, it does not replace backups, in this sort of situation it does replace backups most of the time. i.e. In the last 10 years, I've replaced 3 of the original 4 disks in my RAID5, but never had to restore a backup except for testing purposes.
RAID's job is to protect from ONE and ONLY ONE type of failure, DISK failure, NOT controller failure. Any controller, regardless of type, can fail. That's why you should STILL do regular backups. RAID does not protect from viruses, spyware, operator error, fire, theft, or any other type of failure other than a single disk. It relies on the statistical probability that one and only one disk in an array will fail, and that is all.
Backups are floss. You only have to floss the data you want to keep.
-- I am. Therefore, I think!
The poster seems to be stuck on raid, when he really wants a backup solution.
Apple has really hit it out of the ballpark with usability on their backup solution... but you need Apple hardware to use it.
I'm using a Airport Extreme with an external hard drive and couldn't be happier. It is very easy to go back and find old versions of documents, preview them, and restore them.
I personally wouldn't trust either of these environments. The vast bulk of my data is static, with usually only small or minor changes daily. Rsync CRC mirroring is how I handle it.
My setup:
1 TB main drive. All OS, config, and user data.
1 TB backup drive in same box. All OS, config, and user data mirrored from main drive nightly, using rsync with CRC checking.
2 - 1 TB backup drive in second box mirrored once a month. This machine is typically unplugged until in use. Each of the two backups are staggered two weeks apart, so that the oldest recent copy is no more than a month old.
Very important 'real time' data, on the order of 10 GB, are mirrored nightly off-site.
Note - CRC checking, in my anecdotal experience, is important. Modern drives actively look for 'questionable' sectors when reading; calculating CRC nightly ensures that the drive has to actually read every sector that's in use.
CRC checking mails you what has changed, which is easily filtered to make sure that you haven't lost something important.
Nightly cron job should dump any questionable SMART statistics from your drives. If any pre-fail counters increment, immediately consider that drive suspect and have a hot replacement ready.
How important is your information?
This is the level of data integrity risk that I am currently comfortable with. In a particularly nasty scenario involving my building being destroyed in an earthquake while I'm not in it, I have to fall back to my off-site data.
Alter Aeon Multiclass MUD - http://www.alteraeon.com
I do both. Mozy works great for backing up the files that need to backed up offsite, photos, financial documents, other important docs. You don't need to backup movies, music etc... those can be recreated but photos, financial documents, important letters can't. What I like about Mozy is that my files are encrypted before leaving my computer so no one but me can decrypt them.
In the scenario of your house burning your USB drive is gone along with it and all of your important files.
As you said, you're probably an "exception to the rule". The typical computer user I run across finds the entire concept of "backups" to be over their head.
Typically? They don't even understand how to burn a CDR with a selection of files and folders of their choice on it. (They probably only use their burner to make music or video slideshow type CDs using software that makes it essentially a "one click" process.)
They need assistance setting up a backup solution, and whenever someone makes that solution an Internet-based backup package like Mozy, it winds up only working well for a limited time. Eventually, the user copies too much data into one of the folders selected for backup, exceeds their storage limit, and successful backups cease. They aren't savvy enough to figure out anything's wrong for days, and usually can't resolve it on their own when they do figure out backups are failing.
And yes, bandwidth limitations are an issue for many people. Some of the people I've helped out with their computer problems in the past are still on 56K dial-up! One of them is a guy who makes a living selling goods on eBay, out of his basement! You'd think *he* would see the value in upgrading to broadband, but alas - no.
Years ago, I had this computer system without backup. Ordinarily, no big deal, but I had been working at a new startup company, and had our first six months of data on this PC... on a terribly expensive Seagate Barracuda 2.1GB SCSI Multimedia-capable 3.5" drive (this was the mid 90s).
So I set forth to find a backup solution... Travan was all the rage then, particularly on the floppy port, so I bough this TR-3 drive, hooked it up, proceeded to backup and !BANG! The BSOD. Again, with the same results... my PC didn't like the floppy-interface tape at all. So, on back to CompUSA, and I exchanged it for some new, proprietary tape backup, from Sony or someone.... slightly more cash, slighly higher data capacity. Take it home... same thing. Ouch.
So, not quite desparing, I return this unit and shell out the big bucks for a Travan TR-4 drive, for SCSI. Hook that baby up, run backup... and hours later, wow, I'm backup up.
A week later, the terribly expensive Seagate Barracuda 2.1 GB SCSI Multimedia-capable 3.5" drive dies, never to be heard from again. Since then, it may be tapes, it may be CD, DVDs, or BD, external drives, or online, but I have all critical stuff (and even some of the trash) backed up regularly. This also demonstrates that, when you think about doing a backup (if you're not automated), you should do it right then... the universe may well be sending you a message.
-Dave Haynie
...So, what the hell did you do wrong. I'm sure if someone else was watching, it would be one of those "you are doing it wrong" moments. I've never heard a travesty comparable to yours with WD drives.
RAID is great for systems that need to be running 24/7. If a hard drive dies, the system just keeps on chugging, someone swaps out the drive, and no one is the wiser. It's not a substitute for backup. RAID only protects you against a hard drive dying. There are lots of other things that can go wrong (floods, fires, lightning, malicious users, idiot users, virus', exploding power supplies, faulty RAID controllers...).
Backups are best encrypted and stored off site. Rsync over SSH to a remote server (or subscribe to a online backup service), or dump your files to an external HD (or two, rotating daily) and take off site.
If it's data integrity that is important, and not high availability, then RAID is not the best solution.
As many people point out RAID isn't backup which is something I've taken to heart. But what I haven't been able to find is a good solution to do cross platform backup.
I have three machines which I'd like to utilize in a backup scheme. One machine is a linux box with a large SW RAID5 setup I'd like to back up to. The other two machines are my clients one is running WIndows, the other Mac OSX.
Are there any cross platform backup solutions which will allow me to back up files to the Linux machine from my Windows or OS X machines? I've considered using rsync with the directories which I want to backup, but that will hose files if something gets corrupted on the client box. I'm pretty sure rsync will happily sync my backups to the corrupted file. Plus I don't really have any history as rsync will keep the two directories up to date, but won't generate snapshots. (Well not directly).
I could just occasionally tar/bzip up all my files, but I don't really want to have to manage gigabytes of tar files. Especially as it would be nice to be able to easily recover a particular version of a file without hunting through tons of archive files. Plus tar.bz2'ed files don't really support accessing a single document in the file.
I've also considered just putting all my files in source control with svn, git, or perforce (free for 2 users) then mirror the repo but that doesn't seem to be the best solution either. I'm not sure what the overhead of the various source control systems but at least a few of them support binary diff, and they have versioning built in. But I would like to avoid having to check out, and check in files on the client. This solution also doesn't lend itself to making backups very automatic.
So is anyone aware of a cross platform backup solution (Windows/Linux/OS X)? Which would allow me to mirror/store the backup files on any of the systems that doesn't have the overhead of making a TAR file per day?
Optimally I'd like a solution where I can configure specific directories to get backed up, and have backups occur as some sort of timed cron job. I also would like a way to browse/recover past files without having to mess with manually extracting files from an archive. An open source solution would be nice so that I don't have to worry about losing access to my data in the future, but I have no problem paying for such a backup program.
I don't think the OP believes only Windows 7 is the only OS that supports soft RAID, but rather he just built a new system and is contemplating on the best solution to add RAID to his system... his question is not "what operating system should I use" but "what raid solution should I use on my Windows 7 system".
The Binary Anti-Pattern [http://beyondboolean.blogspot.com/]
I attached the drives to my Macs with a firewire cable configured in mirrored mode. I was using TimeMachine. Things work for a while, then the drive stops responding. The drives were powered with an APC power system and were not allowed to get hot. They were not in continuous use but simply used for backup. When the product showed up on the shelves at my local store, I was very excited about the FW800 and dual drive mirrored functionality. I bought one for each of my computers as a "safe place" for my data. The data remained safe until the drives just quit within several months. I cannot imagine what I might have done wrong with them other than depend on them to do what they were advertised to do.
I would highly suggest doing some testing with Windows and raid before you even bother using it for speed (Hint: it does almost nothing in a desktop or workstation). All it does is create more complexity in the system and chance for failure. Windows has never been known for taking advantage of raid for speed unless it is for database use. How is mirroring going to improve speed anyway? Even if you use a mixed raid system, any gains from striping will be lost from mirroring unless you have enough disks and a nice card. Do some investigating you will see. There are a few reasons why I say this creates more chance of failure, beyond the obvious more parts. If one drive fails, yes you can just replace it, drives are pretty stable these days, but what if they are both from the same batch, odds are they will die within a very short time span of each other. Better datacenters will vouch for this. Then there is the quality of the raid itself, if it is a built in setup you may want to experiment with recovery BEFORE you have a failure, I have seen more than one refuse to recover. What if the raid system itself fails? Again this makes recovery even more difficult. One bone of contention I have here as well is not only the threat of badware out there, but where I live, lightning is a consideration. I don't care how good your raid is, a voltage spike or bad power supply and your raid becomes a big lump of melted metal. Yes there are backups claiming lightening protection. I have one, even the manufacturers will admit there is no guarantee, which is why they come with insurance in case of failure. Windows software raid makes some of these problems easier to deal with. The problem is, none of us want to really recommend that method. My advice, unless this is for a server, forget the raid entirely and create a good backup system that lets you recover everything fast.
While the op is asking about RAID it is clear that RAID is only one third of what people require. They require backup, disaster recover, and availability. RAID only addresses availability. If you want RAID done correctly add a multi-port controller (preferably one that is compatible with your motherboard RAID) and mirror two drives with a hot spare (3 minimum). That way the hot spare will auto sync to the working drive after a failure and be there if the second master drive fails shortly thereafter. That will provide excellent availability, but backup and disaster recovery are another matter.
Disclaimer: I'm the CTO of vitalEsafe. Having watched clients struggle with backups and disaster recovery for the last 27+ years. I finally decided (4 years ago) that nobody was providing the solution that people actually needed, so I designed one from the ground up.
Based on vitalEsafe's secure web services platform, shadowSafe address ALL the requirements of the vast majority of users. It is backup and disaster recovery as a service (nothing to purchase). We use StorageCraft's excellent ShadowProtect backup and disaster recovery software married with secure storage of your backups on our remote servers (which are backed up using Amazon S3). Data is fully encrypted and stored for you in the event of a catastrophic failure and a local copy is maintained for all those "other" times where you need to recover a file, volume, or an entire machine. No bandwidth limitations like some online backup systems out there. If you go out of business, you have a fully functioning backup and disaster recovery system on your system that is completely independent of vitalEsafe (without the catastrophic recovery option of course). Backups can even be imported into VMware virtual machine a booted in minutes.
Based on the list of potential failures shown above (and with one addition, catastrophic event), here is how shadowSafe address each problem:
1) Aw sh1tz. "I didn't mean to delete that folder"... or "Whoops! I formatted the wrong drive", "I saved the wrong version of the file!", whatever. Although I *myself* don't have this happen often, it does happen. And even in my case I've lost about as much useful information this way as by drives dying. Users delete stuff all the time, and it's usually my job to bring it back, which is why I perform redundant, historical backups EVERY SINGLE DAY.
Mount any point-in-time backup (yes it is like a "time machine") and restore your file. You can backup every hour if you like, but most people think 1, 2 or 4 times a day is adequate. This can be done because our software utilizes volume shadowing services (VSS) that is built into Windows (sorry Linux/Mac users, for now you are out of luck) that can make backups of Exchange, MS SQL, Oracle 11g, Pervasive 9/10 (and any other VSS-aware database or service) backups without shutting them down. Backups are made at the block level (not the file level) which means that intra-day backups are really small and very fast.
2) Malware. Don't minimize this - it's real, and it's why I reply to Parent. You are more likely to lose information from a virus/worm/malware and/or b0rked install of something that hoses your filesystem than by a hard disk crash given stable hardware.
Always do an incremental backup of your machine before installing software. Then if you 'hose" the machine, restore your entire machine back to any point-in-time backup in minutes (from your local backup). Works like a restore point for your entire system.
3) Bugs. Filesystems have bugs. So do applications, utilities, anything with software. Strange, unexpected conditions, often caused by bugs in applications can cause data to "disappear", files to get corrupted, filesystems to get corrupted, folders to be incompletely written, etc. This is about as likely to cause lost data as:
Restore your entire machine back to any point-in-time backup in minutes (from your local backup).
4) Hardware failu
I used to recommend this to my parents. However, it is extremely ineffective when people break into your house... steal your computer, and they steal your external backup device. There is definitely a plus to the online backups being offsite.
Having done RAID many times in different ways over the years I'd say that as long as the version of Windows 7 you plan on running supports it, I'd do it. But.. be warned! There may be issues.
For one, I ran into one copy of Comodo firewall that completely blew up the networking stack on XP if dynamic disks (required for Windows software RAID) were present. Also, any BSOD will be followed by a pointless RAID rebuild where it completely copies the contents of one of the drives to the other slowing disk IO for a long time. Also, the entire disks including partitions that aren't RAIDed will need to be configured as dynamic disks which can cause issues accessing them from DOS and Linux. Not usually an issue but it's worth pointing out. One of the things I liked to do was install an OS on each drive that was bootable and have the Stripe/Mirrored partitions accessible from both OSs. If you are going for this type of configuration there is something important in the order with which you create the partitions and convert the disks to dynamic disks. I think you have to convert each disk to dynamic from the OS that boots from that disk in order for it to remain bootable. If OS that boots from disk 1 converts disk 2 the OS on disk 2 will be rendered unbootable (if I remember correctly). If you are going for this type of config, install each OS and have it convert it's disk to dynamic, then create the mirror/stripe partition(s). Other than that, the ability to put the disks in any Windows machine and access the data makes software RAID the clear winner here.
In response to all the posts about how better to safeguard data by backing it up, he's not asking about that. He's asking about which way we'd recommend doing RAID. Your suggestions are off topic. Every time RAID is discussed the same arguments are made. Backups aren't RAID. RAID isn't a backup. Enough already.
I've done software RAID for a long time and I'm a firm believer that it's the right way to do RAID in a lot more cases than it's typically used. Hardware RAID has many proponents and is obviously a profitable industry so there is a lot of money being spent based on it's perceived advantages so I'm in the minority here but that doesn't mean I'm wrong. Expensive RAID array's advantages are getting harder to justify now that Serial ATA has brought dedicated per-drive bandwidth, fast cold (if not hot) swapabililty, and now that multi-core CPUs can handle the load of the software RAID work gracefully. The niche where software RAID makes sense is getting bigger every year. Many people argue that software RAID is too slow, that you need the hardware controller to offload the calculations. I'd like to throw out the fact that I use an 8 drive Linux based software RAID 6 array for my primary storage. Bonnie++ clocks it at 215.2 MB/s on block writes and 263.5 MB/s on block reads. CPU utilization is 45/31% on those respectively but with a quad core machine, using half of one core to do my IO processing when it's writing at that speed is perfectly acceptable. Remember, this is RAID 6 with 2 differently calculated parity chunks, not goofy simple RAID 5 and it's CPU is just a AMD Phenom 9600 Quad with 8 gigs of RAM, not something exotic. I run many virtual machines on top of this array simultaneously and they are nice and snappy. The configuration works quite well and I'd recommend it for a inexpensive, high speed virtual machine server configuration.
For my Windows desktop, I'm currently using my motherboard's Intel Matrix Raid capabilities and have configured half of my drive as a mirror and half as a stripe. The setup has worked seamlessly and I'd recommend it as a reasonable alternative to software RAID. It's slightly cleaner from Windows's point of view and the Intel Matrix controllers are fairly common and, from what I understand, I could put the disks into any machine with Intel matrix raid and access the data in a pinch. I honestly probably would have done software RAID had Vista supported mirroring.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
I would never trust a simple RAID1 solution to protect all of my data whether it be Windows 7's software RAID or the best hardware RAID solution.
Both my wife an I's computers have two drives. We run mostly everything off the main drive and daily backup is scheduled on the other.
Daily backup of all of computers is also scheduled onto a RAID5 FreeNAS solution.
Weekly backup from the FreeNAS is scheduled to a computer off-site (at my parent's home) via the Internet.
Each month, the most important documents such as personal and business financial records are also backed up to DVD media, which is deposited in my safe.
I have lost data many times in my youth and I feel this setup I have provides sufficient protection for my data today.
In no way is any type of RAID1 enough to protect your data.
Amazon's S3 has been around for some time now and will likely continue to hang around for a while. I'm using JungleDisk to provide encryption and a nice interface to it. The nice thing about using S3 is that you don't get an allotted amount of disk space or transfer, you just pay for what you use. A second backup of everything that I wouldn't want to lose (updated nightly) costs me about $5/mo. That's for ~40 GB storage and the incremental transfers.
It came in handy after a theft left me with no physical copy of some of my data. That month, after dumping about half of my stored data back to my home, my bill was a whopping $8.
Being that inexpensive, encrypted, and with an automated backup solution, I find that the WAF is really high. It was her computer that was stolen, too, so she's totally sold on the idea of backups now!
If you want a vision of the future, imagine a youtube comments section scrolling - forever.
If you got the money, do both. I wouldn't trust a 3rd party with my sole backup, but I don't know how much I would trust my backup to be safe in the same room/building as the original. Local problems like fire would take them both out.
Okay. Here's what i'd do if you're wanting this RAID system to help redundancy AND be a backup. Firstly, be sure to use a physical RAID card. Don't trust it to software, you're just asking for more problems than necessary (although the motherboard controlled RAID's all right for home use). Secondly you should buy another HD for backup purposes. At the end of an allocated amount of time (day, week, month) pull one of the drives in the array, and slip in the spare drive. It will mirror, and you'll have a snapshot of your system when you pulled the drive. If you screwed everything up, you can slip the 'Backup' drive back into the array and everything's restored back to that point right away. This would work for some users, and may for you. But it depends on what your exact backup needs are.
01001001 00100000 01101100 01101111 01110110 01100101 00100000 01001111 01101100 01101001 01110110 01100101 01110011
Keep running those systems on *NIX variants: I am glad you're stupid enough to do so. The entire UNIX movement showed us all just how smart your crowd really is, because when push came to shove, your UNIX vendors all started changing things so much that what ran on 1 UNIX wouldn't on another, & nobody agreed on anything. This fragmented that community so badly, that the sad fact is, we should all have been running some unified form of *NIX on everything from a mainframe, to a midrange, to departmental servers, and workstation (as well as in homes). The same things is being stopped by Linux, & Mr. Torvalds having the good sense to keep the core of that *NIX OS variant under HIS control, keeping things unified @ that level @ least... but, this measure took place far too late, & Windows took the world by storm. As nice as Linux has become, it is still a minority niche player that does not run as much hardware or softwares for purposes of any kind imagineable, as does Windows. The fact is, folks who are smarter about their futures stay on Windows. Why? Simply because it is the dominant player, overall, & on the most used hardware architecture there is in x86, from enterprise class/mission critical servers, down thru departmental LAN servers & workstation, & right into the home user/end user market. This provides anyone looking for employment in computers a far larger surface area of work on many levels, because of this. So keep being a "Pro-*NIX clown", you're only doing myself a favor, as well as anyone else who is a Windows fan. You're removing yourself as my competitor in the job market, and for that? I sincerely thank you. Signed SOMEONE WHO IS SICK OF THE *NIX USERS GANGING UP ON HIM who cannot think for themselves, spread FUD anytime they can about Windows, and try to gang up on anyone that shows a different viewpoint than they have, especially at /.
The article smacks of false dichotomy. There are a number of solutions, not just Windows 7 or a hardware RAID controller.
Seriously? You have no idea why Windows 7 is the OS he decided to go with so you have no idea if his reasons are valid. Is it too hard a mental exercise for you to perform to take that requirement as a given?
Also.. no desktop OS has supported mirroring for the last 10 years. Vista doesn't, XP doesn't, I'm pretty sure 2000 didn't. Apparently Windows 7 will.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
If your not trying to create a redundant bootable drive. Just replicate the data using something like "Karen's replicator" and schedule it to run as often as needed. The only case where it wouldn't be advantageous is if you have files open 24/7 which wouldn't be able to copy.
You guys are all missing the point!!! Raid is backup!!!!!!!!!!!!!!!!! If you are all stuck in the days of having a computer with one hdd for everything, wake up! I build all of my boxes with a primary hdd for OS and apps and then another internal drive for data storage. I also clone the primary immediately after installing the os and all apps and use it as a recovery drive. What we are talking about here is safe guarding against failure to the storage hdd. For that matter, guarding against same for any external drive. Remember, it's not "if" but "when" regarding drive failures. A rais 5+ one will protect you from data loss!!!!!!!
I posted as anonymous because i am not a member and don't feel like jumping through all the hopes!! Why should I. Just trying to help.
Simply put, an infallible design would be so 100 times the size of the actual logic the component would hold. Back when semiconductors weren't electrostatically protected, the product was resulting small and power efficient, and broke daily not from malfunction but from misuse. Now, a symptom of misuse is inferior components from competitors importing their products or components from Asian countries. You didn't have these problems in appliances back with RCA, General Electric, Atarii, or even Digital Equipment Corporation.
That just proves you how America and America nationals are being f*cked by foreigners, not matter how "inexpensive" someone says a product is it will become Expensive in the long run. I still have a RCA tube television, General Electric culinary equipment, Atarii cabinettes and machines, and this post was brought to you by a pre-Samsung all-American fabricated 12-year-old 64-bit CPU DEC Alpha computer that dual-boots Linux (2.6 kernel distribution) with Windows 2000.
Let me know when the eco-war against Americans comes to an end; no matter what Greenpeace and Communists say, the companies and countries they boast and support have always proved to be more dangerous than anthing that came out of a market in the Americas.
I'm interested in knowing what slashdotters actually think about the original question. I'm about to implement RAID on a htpc/file server and would like to know what would be better - linux software raid vs motherboard raid.
Everybody knows that RAID != backup, its been drummed into us enough. So can we get back on track and answer the goddamn original question?
"So, which should I choose? Who should I trust more, Microsoft's Windows 7 or possibly the cheapest RAID controller on the market? Are there other cheap solutions?"
He didn't say anything about RAID-0, he explicitly said "mirror" which is RAID-1. And the reliability of N drives becomes 1-(1-r)^N, or in layman's terms the probability of both drives failing at once. If I use your 95% figure, then reliability is 1-.05^2, or .9975. Since the probability of the software or the power supply failing is higher than that, not to worry.
The real point is that backups are to protect the data, RAID is to keep the system up if the hardware fails. In this case I think all the people who said get a backup instead are just right.
Okay. Here's what i'd do if you're wanting this RAID system to help redundancy AND be a backup. Firstly, be sure to use a physical RAID card. Don't trust it to software, you're just asking for more problems than necessary (although the motherboard controlled RAID's all right for home use). Secondly you should buy another HD for backup purposes. At the end of an allocated amount of time (day, week, month) pull one of the drives in the array, and slip in the spare drive. It will mirror, and you'll have a snapshot of your system when you pulled the drive. If you screwed everything up, you can slip the 'Backup' drive back into the array and everything's restored back to that point right away. This would work for some users, and may for you. But it depends on what your exact backup needs are.
01001001 00100000 01101100 01101111 01110110 01100101 00100000 01001111 01101100 01101001 01110110 01100101 01110011
My job is is a professional raid engineer. "Burned by a not-so-cheap, dedicated RAID controller" -please explain, usually people are burnt by software raid, because they think its true raid and it isn't. All motherboard raid and even cheap raid cards are software raid. Cheap software cards often only contain one chip, same as found on motherboard raid, so don't buy one if you already have it on your motherboard. Avoid.They are NO GOOD. Trust me. If a drive fails you'll be lucky to keep your data. You're better off using discreet drives and copying date between them than software raid. RAID !- backup as other posters have stated. Good hardware raid such as the 9650se series 3ware or lst 87** series cards are good and will protect data but expensive for the home user. Use single fast drive for os, and proper raid card for your data, if you can afford it. hope this helps.