Slashdot Mirror
RAID Trust Issues — Windows Or a Cheap Controller?
NicApicella writes "My new system has two sparklin' SATA drives which I would like to mirror. After having been burned by a not-so-cheap, dedicated RAID controller, I have been pointed to software RAID solutions. I now stand in front of two choices for setting up my RAID: a Windows 7 RC software RAID or a hardware RAID done by the cheap integrated RAID controller of my motherboard. Based on past experiences, I have decided that only my data is worth saving — that's why the RAID should mirror two disks (FAT32) that are not the boot disk (i.e. do not contain an OS or any fancy stuff). Of course, such a setup should secure my data; should a drive crash, I want the system up and running in no time. Even more importantly, I want any drive and its data to be as safe and portable as possible (that's the reason for choosing FAT32), even if the OS or the controller screw up big time. So, which should I choose? Who should I trust more, Microsoft's Windows 7 or possibly the cheapest RAID controller on the market? Are there other cheap solutions?"
Do you really want to trust Windows with your data?
"I want any drive and its data to be as safe and portable as possible [...] even if the OS or the controller screw up big time."
If the controller screws up and writes crap to your raid, your data is dead. Not sure if your expectations are realistic.
Try a drobo instead.
Comment removed based on user account deletion
You data is most important and you plan to use FAT? Good luck with that!
Seriously, though. No RAID solution that is not totally S/W is portable. But do you really need RAID? It sounds like what you need is a good backup solution with frequent backups. Does you data change so much that losing one day's worth of data would be a problem?
The real "Libtards" are the Libertarians!
RAID is not a backup. Get a backup solution or you'll realize you can be even more frustrated.
If you want data integrity, use NTFS. Using Fat32 is like saying you want a reliable car, so you're buying a Edsel because they've been around a long time-- it doesn't make sense. Every other OS on earth can read NTFS (if not write it), so it won't affect your portability requirement.
Secondly, before you make any decision regarding Windows 7 RAID, make sure the edition of Windows 7 you want to buy ships with software RAID support before you put all your eggs in that basket-- early betas and RCs of Vista had software RAID enabled, only to have it disabled before release. I've seen no guarantees about Windows 7 software RAID support, and which editions will have it enabled. (If any.)
If you're planning to move to a server OS after Windows 7 expires, I can practically guarantee software RAID will be enabled, but that still doesn't mean you can necessarily upgrade your Windows 7 software RAID array to a Windows Server software RAID array. Do your homework.
Comment of the year
You sound like someone that need to be reminded that RAID IS NOT BACKUP! Google for that sentence. All you talk about is saving your data, and RAID will not do that for you. You'd be better off just using the second drive as a backup. RAID will not save you from accidental overwriting of data, corrupt filesystems, broken chipsets, etc. The only thing RAID will save you from is downtime. If you lose that much money on the downtime it takes to recover from a backup, then by all means, use RAID, but don't treat it as a backup solution that will protect your data. That's not what it's made for.
c++;
These motherboard "raids" are called fakeraids.
All that it is is that it writes the metadata on the disk in specific format so that you can see the raid volumes via BIOS. Note: Only "see" their status - in case you replace one drive, the resync is still done by software and you must boot to operating system. One clue is the fact that in Linux the dmraid package uses exactly same driver for accessing fakeraid-mirrored drives and Linux's own software-raids - device mapper just does a bit of magic at init.
However, if faced with choice of Windows-only or motherboard-raid, I'd go with the motherboard-version, because that's at least supported both by Windows and Linux so in case something goes wrong with your Windows installation you can always pop in Knoppix or some other Linux CD for recovery.
Cheap RAID controllers suck - at least you can trust Windows to be consistent between installations if need be. External (preferably offsite) backup is also a must! As I'm sure you'll be reminded 1000 times in this thread, RAID is not backup.
I've already lost a software RAID setup when Linux wouldn't let me put it back together again. A friend of mine also lost his RAID setup on Windows a couple years ago. Also, a mirrored RAID doesn't prevent you from losing data when you accidentally delete that folder of your favorite photos.
I now keep the drives separate and setup a simple nightly batch file to robocopy the files from one drive to the backup directory on the other. It's even better if you can put the second drive in a different PC. This allows me to decide which files are important to me and only do backups of them. That leaves all the rest of the free space on the two drives for more volatile content (stuff I don't care about if I lose). It also gives me time to get a file or directory back if I accidentally remove it. I've done that numerous times when Vista first came out and it didn't move the focus to the right hand side of the explorer window.
RAID1 serves only one function. Increased uptime. If avoiding having to spend 2 hours restoring from a backup is your primary goal, then RAID1 might make sense for you. Do you have an office full of workers that will all lose productivity if you have a system crash? If so, then RAID may make sense. Any other use of RAID1 is fool's gold. It will not protect your data from a system-level problem. It will not protect your data from corruption (especially not on a FAT32 file system, which was never intended for any partition size above 32GB in the first place). It will not even always protect you from a single drive failure, since the rebuild process in a RAID1 setup often kills the second drive while trying to recover data. As many have said already on the thread, RAID is not backup. Backup needs to be a completely independent device. Unless you have serious uptime considerations, RAID1 should not be part of your backup strategy.
RAID is no substitute for backups. RAID is very good at propagating errors and problems very quickly, be they software glitches or human errors.
For consumer class storage, weekly / daily backups might be more efficient than investing a lot of effort into live RAID. Since I'm a Mac guy, I see the best answer to this question as Time Machine to a network / USB attached drive -- hourly (configurable for more or less often) differential backups, almost transparent to the user. To my knowledge, Windows has no similar set of software to allow reinstallation to the last hourly backup -- my wife had the misfortune of having to restore a blank drive from her last backup and it was a flawless process that truly left her where she left off less than an hour before the hardware failure. The reinstall wizard just had to ask where the backup was. Casting aside MacOSX advocacy, there is truly no substitute for a good automated backup solution that is regularly tested. I think the best method would use the fewest common components, like a NAS, followed by an external drive with its own power supply. My least favored option would be an internal drive with every single component shared.
If you were doing this with Linux, I would suggest just using md. It works in most cases. For raid1 it mirrors to the point where you can take one drive out and mount it as a non-raid single disk somewhere else. Handy for troubleshooting when ubuntu decides to change uuid drive mappings and thus refuses to rebuild the now broken raid set.....
-Tm
Support TBI Research: http://www.raisinhope.org
The only way to keep your data secure in any reasonable fashion is to make a copy of it and store it offline, off site. Ideally "off site" would be in another building or city, but it at least has to be on something not attached or accessible to your computer.
Without regard to if you use software or hardware RAID or the quality of the RAID system, RAID only protects you from a physical disk failure. If you as a user screw up (delete or change something you didn't want to) or if some software bug screws up for you, or if you have a non-disk related hardware failure (causing a data corrupting machine crash) then you have lost your data -- RAID doesn't help.
Even if you are only trying to protect against disk errors, if the RAID system fails (even expensive quality ones can), or if you don't know and follow the recovery procedures EXACTLY, you can lose all your data.
The only reliable solution is making a copy or a "backup". Backup does not mean making a copy of the data on the same machine. (Whatever took out your RAID might also take out the other non-RAID disk or directory that you put your copy on.) If you are paranoid (or just prudent) your backup should not be a mapped or mounted drive on another machine. (Viruses can write to the network as well.)
And finally... Backups only count if you have tested your restore process.
I won't try and improve on the comments above, almost all of which I agree with, but I will make one observation. The reason for mirroring is to protect against drive failure. The one time I had a drive failure, mirroring saved the day's data. The best way to protect against drive failure is to buy server grade SATA drives, which are designed for 24/7/365*5 operation, and not cheap PC drives which are designed for 10 hours per day for 3-4 years. Buy server grade SATA drives, mirror them using a hardware controller, back up daily, sleep at night.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
... do it with software. This CPU overhead is minimal for that.
Things like RAID 5 are where you'd be better off with a dedicated controller.
That's nice, but the submitter is asking about RAID 1.
Actually, it depends on the reliability. 95% reliability becomes 90.25% reliability. 50% reliability becomes 25% reliability. 1% reliability becomes 0.01% reliability.
So if your drives are very reliable, it's very slightly less than twice the failure rate. If your drives are not reliable, then it asymptotically approaches an infinitely greater risk of failure.
Statistically speaking. :)
Consumer editions of Windows only ever supported Software RAID1. I've made a few experiences with SW RAID1 on WS03, and it's pretty much crap. Linux SW RAID on the other hand worked fairly well.
RAID is not a backup. This is the most important observation. RAID is a high availibility feature. If you lose your RAID array, you shouldn't lose any data. If you do, your backup strategy sucks.
Generally, skip RAID in a consumer setup. RAID is complicated, it's a PITA and especially the low end stuff can do more harm than good. Even expensive stuff can fuck your shit up (I'm looking at you, ServeRAID 8k). Better in invest in a proper backup - to a local harddrive and maybe offsite. Online backups make sense in a home office. For servers, i recommend LTO tapedrives.
RAID is only marginally valuable. In my experience, for all but the most carefully controlled environments, RAID simply adds complexity, the number of things to go wrong increases, along with the likelyhood of lost data. Do it only if you want the *experience* of running RAID, but don't count on RAID to "save your data".
I've worked as a system administrator for more than a decade, in medium-large scale deployments with good success, (think: servicing thousands of users, hundreds of domain names, tens of thousands of email addresses, etc) so I think I have some useful experience you can benefit from.
IMHO, you most likely to lose data from the following things (in order)
1) Aw sh1tz. "I didn't mean to delete that folder"... or "Whoops! I formatted the wrong drive", "I saved the wrong version of the file!", whatever. Although I *myself* don't have this happen often, it does happen. And even in my case I've lost about as much useful information this way as by drives dying. Users delete stuff all the time, and it's usually my job to bring it back, which is why I perform redundant, historical backups EVERY SINGLE DAY.
2) Malware. Don't minimize this - it's real, and it's why I reply to Parent. You are more likely to lose information from a virus/worm/malware and/or b0rked install of something that hoses your filesystem than by a hard disk crash given stable hardware.
3) Bugs. Filesystems have bugs. So do applications, utilities, anything with software. Strange, unexpected conditions, often caused by bugs in applications can cause data to "disappear", files to get corrupted, filesystems to get corrupted, folders to be incompletely written, etc. This is about as likely to cause lost data as:
4) Hardware failure. This is one of the lowest orders of lost data, although when it happens, it can be one of the most extreme.
Let me say this: RAID 1/5 only PARTIALLY protects you from the last one. Actual, bona-fide backups protect you from all of these. If you care about the data, get backups. If you care about uptimes at great expense, RAID *may* be worth it.
My advice is something most people don't want to hear: for personal use, get backups online for $5/month. Mozy/Carbonite/etc. There are zillion vendors, just Google it. In two years, it will cost you about as much as that 2nd hard drive. It protects you far better than that 2nd hard drive, and it's so automatic that you'll hardly notice it until the moment it actually matters: when you just have discovered that your data is gone.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
The only thing RAID will save you from is a dead drive.
There are infinitely many ways to lose your data, and a dead drive is only one of them.
If you are going to use RAID, you might as well use RAID 0 unless you can't afford the downtime.
If you start talking about data loss, then you just lose, because you should have backups.
-Tm
Support TBI Research: http://www.raisinhope.org
The article smacks of false dichotomy. There are a number of solutions, not just Windows 7 or a hardware RAID controller.
To begin with, every NT-lineage Windows version ever produced supports software RAID out of the box. Add that to the fact that any major Linux distro today supports software RAID. And so do the *BSDs. And Mac OS X. And Solaris. And probably a bunch of other platforms I can't think of right now.
Hell, you could buy one of these one of these and throw the drives in it, connect it to your network switch, and presto -- instant RAID+NAS.
I think we would all like to know why you think Windows 7 is your only option, because if that's what you think, you don't know how mistaken you are.
My blog
With RAID mirroring, if you overwrite or delete an important file, it's copy on the mirror is immediately overwritten/deleted too, and the file is lost. Wouldn't you rather need a good regular backup?
And as someone pointed out already, FAT is really not a reliable file system. If you are on Windows, use NTFS. It is still portable, having read/write drivers for both Linux and Mac (see this guide).
Since the files you want to keep safe appear to be regular files, not system files, any simple file copy mechanism could do. For an easy and simple system, you can use the Windows robocopy.exe tool in a batch file. For a more sophisticated system which can keep older file versions, and can easily be adapted for use over the network, you could try a Windows version of rsync like cwrsync. There are also a few rsync GUI frontends for Windows.
If you decide you really want RAID mirroring and go with the hardware solution, my understanding is that you need a replacement controller in case yours breaks. Since your controller seems to be embedded in the motherboard, you would need a replacement motherboard.
With the Windows software RAID, you are dependent on that software, and have portability only between machines with this Windows 7 software RAID (possibly even only this particular version).
You can buy a real RAID controller for $400-$500 nowadays. If your data is not worth that much...
What RAID is good for:
Better to just throw a disk in an old machine and back up to it regularly.
I would recommend an eSATA RAID enclosure, similar to this. Run the newly purchased SATA disks mirrored (RAID-1) in the enclosure. Power up, run backup, power down. Rinse/repeat.
...then you probably don't need RAID. Use NTFS and set up some kind of scheduled backup to the second drive. Or, build a Linux NAS device and run BackupPC (backuppc.sourceforge.net). BackupPC works great for this sort of thing, it can do incremental and full backups of all your data, on the schedule you choose.
Fakeraid is software too.
Get a real hardware RAID controller, or don't use RAID. Windows SW RAID or a Fakeraid controller is just plain stupid.
FAT32? You are fucking lame!
There, fixed it for you.
The best way to protect against drive failure is to buy server grade SATA drives, which are designed for 24/7/365*5 operation, and not cheap PC drives which are designed for 10 hours per day for 3-4 years. Buy server grade SATA drives
This is just pure marketing baloney. Do you have any real-world tests that actually back this claim up? I've never used "server-grade" drives, and never will. I've seen "server grade" drives fail in large quantities and "desktop" grade drives last for years running 24/7/365.
The only thing I've seen vary greatly in quality are power supplies. Cheap ones are designed to last a couple years and fail. The more expensive ones higher grade ones tend to be better.
AccountKiller
mirroring with RAID 0 does not take the place of regular backups of data. RAID 0 is for rapid recovery to minimize downtime is MOST instances. Data should still be backed up separately in case of a catastrophic failure.
... and in the DRM, bind them.
They've been doing RAID in hardware for quite some time now - the hardware may fail, but I don't think it'll mess up your data. Think of it this way: The hardware controller only has to do one thing, which is to serve RAID. The OS, on the other hand, has to do a bunch of things, any one of which could go bad and kill your RAID.
I've done both hardware and software RAID-1 in the past, with Windows, Solaris, OS X, and Linux. For Windows, go for the hardware RAID.
"Safe" FAT32, cheap RAID, RAID implied as backup, Microsoft.
Nice job, you successfully trolled the /. frontpage.
Leben Sie jetzt die Fragen.
The article smacks of false dichotomy. There are a number of solutions, not just Windows 7 or a hardware RAID controller.
Agreed.
As I see it, if you want guaranteed repairability then you basically have two options: enterprise-class hardware with a support contract (and price tag to match), or an Open Source software solution.
Put another way, either you pay someone to take responsibility for fixing it, or you take responsibility yourself. A Microsoft solution doesn't give you enough control to take full responsibility, because you can't be certain that it will be legally or technically possible to recreate your current setup in five years time.
There are some important differences between server SATA drives and consumer SATA drives - for example the number of retries until an error is reported to the controller. The price difference of course is not really justified by that.
If you're just want a convenient backup of your music collection, porn collection, musical pr0n collection, or your pr0n musical collection then RAID is not a horrible thing. However, if you're backing up important files, like the only existing scans of the now-burned dossiers William Mark Felt left you, then you should not stop at RAID. Statistically speaking, if something happens to one HD in your machine, like a massive power surge or being confiscated by tight-lipped men in black suits and black sunglasses, it has a pretty high probability of happening to the other HD. Offsite backups are, therefore, prudent. Leaving a HD in a box at the bank and giving the key to your lawyer is one of the safer things you can do, but not terribly convenient. There are a variety of online backup services available that are decent. I'll leave it to others to speculate on which ones are least likely to be fronts for the NSA. If you feel that your data might actually be interesting to more than one human being on Earth, don't forget to encrypt it. (Be honest with yourself. You are posting to /. after all.) I'm rather fond of emailing moderate risk files to my gmail account. (Stupid, I know, but very low effort and they're available anywhere you feel safe enough to check your email.)
As for Motherboard RAID chipsets... Keep in mind that your motherboard has a non-zero probability of frying, having it's caps go bad, being peed on by irate government agents, etc.. I once had a RAID 0 array that was hooked up to one of those things. After the Mobo died I had to do without letters K through P of my Japanese horror-comedy-porno-game-show collection until I was able to find a used computer with the same RAID chipset. (I don't know if it's changed, but at the time each different RAID chipset made RAID 0 arrays that were not compatible with anything else on this lump of rock.) If data portability rather than performance is a priority for you, my advice would be to avoid hardware RAID entirely.
Do you work for one of these online backup places?
I would sooner trust a WD drive with my valuable data.
you shouldn't be using release candidate software as your production box. everything else become academic after that. when Windows 7 is released to the general public in a production version and you upgrade, you're going to need to wipe your RC version. so just don't do it.
Actually RAID1 is quite good for reading data: it minimizes seek time . Of course, it works fine as long as there are not many writes. For example think analytic databases, cubes, etc. Those are not written to in real time (like the more common transactional databases)
http://revj.sourceforge.net
There are a number of reasons why not to use FAT: 1. Unreliable 2. Doesn't support large files 3. Doesn't support advanced permissions Since you are running Windows, use NTFS, an external USB drive for backup (also NTFS) and the free Microsoft SyncToy to make periodic backups to the external drive.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
You can buy a PC for less than that, and install OpenFiler, or any of a number of free/Free soft-RAID solutions that support Samba.
If you buy a RAID controller, you move the SPOF to the controller rather than the disks (though admittedly it's not got moving parts and should last longer). If you do use a RAID controller card, though, and you want safety, you need a spare RAID controller of the same sort in a drawer somewhere if you expect to get your data back, unless you're sure that the RAID doesn't use a funny on-disk format to store the options it's using.
If the machine is doing RAID and not much else, what are you saving by buying expensive hardware to offload the RAID processing when you already have a mostly idle box sitting there already?
RAID is not a backup solution.
You should be using mirroring in addition to frequent backups.
RAID does not protect against (accidental) deletion, file system corruption, or user stupidity.
The best way to protect against drive failure is to buy server grade SATA drives, which are designed for 24/7/365*5 operation, and not cheap PC drives which are designed for 10 hours per day for 3-4 years. Buy server grade SATA drives, mirror them using a hardware controller, back up daily, sleep at night.
Err, thats been proven questionable. More specifically, here is an article from eweek and here is the google talk about a large study of drive lifetime characteristics. "Server" drives are just as good as "consumer" drives when it comes to lifetime. The only benefit you get with the more expensive drives is slightly better performance (NCQ, higher rpm, larger buffers, etc). I have several machines at home that run 24x7x365 on the "cheap PC drives" in raid1 pairs (linux md) and a non-raided windoze box, and have had to replace 2 drives out of 12+ over the past 8 years due to failure: I tend to need to upgrade to larger ones before they fail. Even at work we have used the cheaper drives in clusters, next to others running enterprise level drives and found no benefit to the extra cost in most situations.
-Tm
Support TBI Research: http://www.raisinhope.org
No where did he suggest he required Windows 7 for RAID, just that thats what he planned to use for this machine. Personally I would just put the second drive in my server and backup my data to that rather than use any of the two options he lists.
Normal people worry me!
Personally, I haven't yet encountered anyone who really got benefit from those personal Internet backup services like Mozy. In regular use, it always seems like the person exceeds their storage allotment or Internet connectivity issues prevent them from recovering what they need, when they need it.
I tend to recommend people buy an inexpensive external USB or firewire drive, leave it attached and assigned as a backup device, and have some software package run a daily backup of all the relevant folders and files they might need to save.
It's great that your data is stored offline and off-site ... but I'm just not sold on most of the implementations for "home use" being as great a solution as they first appear to be. Many of the providers have come and gone over the years, too. What happens when your offline backup company goes under?
You're in a special situation using Windows 7 RC. So despite the below, you're especially at risk to the possibility of a new (but not yet discovered) bug in Windows release-candidate software, don't use Windows or other RAID capabilities for your "data protection", use backups. Use RAID mirroring for improved performance or to reduce the probability of downtime occuring, and allow for rapid recovery from common drive failure scenarios.
RAID controllers (Software or Hardware), all suffer from various scenarios where the entire array can fail, and data recovery becomes so hard that it may as well be lost. 90% of the bits may be there redundantly, but you can't get to them for one reason or another. Also, RAID will not protect against system compromise, software data corruption, accidental deletion, or any type of volume corruption that occurs as a result of software running on the system.
Don't pick FAT over NTFS on the basis of portability between systems, if reliability is more important, the NTFS filesystem uses a technique called journalling which makes data corruption less likely after a system crash, eg power failure. NTFS _can_ be read by common solutions, if you need to recover data. Recent Knoppix CDs and various rescue disks can read NTFS, and the filesystem checking tools available for NTFS filesystems are better. FAT is more susceptible to certain failures, including excessive fragmentation leading to poor performance.
Research what type of RAID solution your integrated hardware really is. If it is hostraid, or fakeraid, that requires Windows drivers to implement RAID, then don't use that, avoid like the plague as it's SOFTWARE RAID, even though the software is running inside a driver provided by the controller vendor and A FEW functions may be offloaded to hardware, the main RAID code is still running in software, which is bad, mmmkay?
You can often detect this in that there will be Windows only drivers, or the product will be labelled a hostraid solution, but each of the major drive controller/RAID chip manufacturers has a different name for their ultra low-end solution that isn't really hardware RAID, but has hardware offload of just some functions (checksumming, mainly).
(Fakeraid/Hostraid adapters that require special drivers in the guest OS to implement RAID, also generally suffer from the RAID5 write hole if you utilize RAID5. And RAID code may be more susceptible to certain problems, when it isn't running on card firmware.)
I would actually favor implementing RAID in Windows over that. However, there is hardly any point of doing this, except if you are mirroring your boot drive, or you need RAID for improved performance (e.g. You could use RAID1 for all drives to improve read speeds, RAID1+0 to improve both read and write speeds, or RAID5 for redundancy and scalability at the cost of slower write speeds and a read speed penalty).
I mean that: since you aren't mirroring your boot drive, there is little point of utilizing RAID in your case. One of the most performance-effecting files on your disks is the page file on the boot drive. If you were utilizing RAID for improved performance, you should definitely want to maximize read and write speeds to the boot drive.
If your non-redundant boot drive crashes, your system will be down and need to be re-installed on a new system drive. You may as well just pre-image a backup drive with your system, keep a continuous backup on another machine, and in the event of a failure, pop in the backup HD, and start restoring continuous data from backup, to bring your 'spare' up-to-date.
This will probably even be faster, as an OS reinstall and re-up of Windows is not required
Second, your RAID controller can fail, make sure you have a plan. That would mean either two identical controllers with the exact same firmware version, or you use a very common controller that you are CERTAIN you can easily buy anoth
FAT32 has a 4 GB max file size.
This can be somewhat inconveniant if you have say a linux dvd ISO thats larger than that, or if you record HD video, etc.
I would suggest going with NTFS, there are ways now to use the actual microsoft NTFS driver in a wrapper with linux, then you get full native compatability if thats what you are thinking (it used to be that NTFS compatability was an issue with linux).
The cheapest solution for backup would probably be an external USB drive. And you may also want to have 2 of them that you swap out for off site storage, or something like that, depending on how important your data is to you.
controller for your money. I've had bad luck with the cheap ones in Windows, always a blue screen of death that leads to data corruption and other problems.
You want a RAID controller that is properly reviewed, one that has been proven to be stable. One that has good driver support and a good warranty. You want the same with your hard drives that data will be stored on.
I am not sure why you want FAT32 drives, is it because you are running an old version of Windows that does not support NTFS? Most moden Linux systems have NTFS support built in if not an option to load, just read the FAQs for your version of Linux and look for how to add in NTFS support. Unless you have software that breaks on NTFS and needs FAT32 (like old video games and software applications with anti-piracy technology that only works with FAT32 and not NTFS), you should really use NTFS for your hard drive format.
You need more than just RAID drives, you need a backup solution. Either use DVD-RW disks, or buy a Tape drive, or have an older system you can set up a network share and copy data files over to it, or a USB hard drive or thumb drive.
Remember, Slashdot does not have a -1 disagree moderation, and no, troll, flamebait, and overrated are not substitutes.
seems to be one of those buzzwords that are taken for granted, but under duress are not really desirable attributes. The easiest (only?) way to achieve consistency in times of trouble is to destroy all information constent that is not consistent. That means destroying all files (and directories) which are not all there.
You are stranded on a desert island desperate for news. You have a choice between a month-old newspaper which is intact and yesterdays newspaper which has been mouthed over by hungry sharks.
Your network server with all your orders, shipments, invoices etc just went up in smoke. You have a choice between 97.5% recovery of all the current data or a00% recovery of month-old data.
Seems like the people that promote these things are never in the position of actually having to use them. (Sounds like politicians. Lots of plans sound great, as long as you do not actually have to implement them).
There is a basic rule of medicine which seems to not apply to systems analysis. First, do not damage (or something to that effect).
Heat kills drives so avoiding it is a big plus. I've had friends buy neat looking cases that didn't have proper drive cooling and the hard disk died. At three separate employers we had weekend air conditioning failures and each time hard drives died not just over the weekend but the following weeks as well.
My machines are in 4U server cases with a dust filter on the front, a 120mm fan in front blowing on the drive cage, and two 80mm fans in back pulling air through. Temps in the case stay low.
The other key is quality power supplies. A poor p/s can kill your drives or the motherboard with that on-board "RAID" functionality.
Make backups and take steps to avoid hard drive abuse through heat or unstable power. That should be good enough
Run and catch, run and catch, the lamb is caught in the blackberry patch.
Western Digital "RAID" drives feature Time-Limited Error Recovery, and it is a user adjustable setting on many WD consumer drives via a DOS utility.
You are neglecting:
- the rogue app that overwrites your data
- the drive that reads back incorrectly, right into your mirror drive
- the drive that croaks as you attempt to re-mirror from it
I have personally encountered all three of these.
If you don't have disconnected backup, you are just kidding yourself. Any connected media can be trashed by bad software/firmware/whatever.
Your risk factor is never broken hardware. You don't care about hardware. Your risk factor is always lost data.
To begin with, every NT-lineage Windows version ever produced supports software RAID out of the box.
I'm sitting in front of a copy of Vista Ultimate right here that doesn't. Do you have a citation or anything? Or are you using some alternate use of the word "supports" that doesn't mean "supported by Microsoft?"
Comment of the year
I worked for Microsoft for over 6 years, and I swore, when I left, that I would find a better way. I ran into Ubuntu Linux after going from Red Hat, to Fedora, and Suse... I tested it out for some time, setting up a server to handle all those functions I was paying hundreds of dollars for. Then I converted all the machines over that I own, and I have been more than happy. Every machine I set up has a RAID, My "user" machines all get RAID 1, my server and my play machines get either RAID 5 or RAID 6 I have had drives go bad, operating system troubles, hardware troubles, and everything imaginable. My data is still safe. Stuff that I have collected for 10 or more years, files, mails, etc... I don't worry about any of it. I tried raid in Windows, and without purchasing expensive hardware, forget it. It doesn't work, or at least, it doesn't work well. While I worked at Microsoft, I went in for a "blue badge" interview. While I waited, a currier brought in a box for a group in building 53. This box was a copy of Red Hat Enterprise. I thought that was funny, but when the guy came to pick it up, he explained that they had sensitive data on their servers, and they couldn't trust that data to Windows Server Addition, (back in the Win 2K days). If Microsoft cannot trust their own software, why should I?
--E--
RAID 1 hopefully prevents that a server will go down and it makes it possible to easily replace a bad disk.
New things are always on the horizon
Just use some good incremental backup software. Send the backups to the second drive.
FAT32 is not a good choice, it can have file-system corruption more often than hard drive or controller failure occur. If you're running Windows I highly recommend you go with NTFS. Linux can read it pretty reliably if you're looking for portable data, you probably don't have immediate need for portable write support to the drive.
Windows software raid is slow, but it is phenomenally more practical than cheap hardware RAID found for "free" on motherboards. The worse thing about hardware RAID is that it is not portable to different controllers. If your motherboard breaks you can be stuck being unable to use your RAID volumes anymore.
“Common sense is not so common.” — Voltaire
We're talking about an environment that has more then TWO MILLION VIRUSES in the wild, and if your choice of Window-hardening software doesn't stop every one of them, you're screwed anyway, headed for a flush and fill. (And potenitally paying another $100). Welcome to Windows.
Sure, it's been a LONG time since Stacker, the 'wonderful new idea to double drive space', but since it was in Windows, it would last about a week before some memory-hogging virus or poorly-written program would stomp on it.
Hardware RAID can be a disaster; I got in a place and time where corporate data was on a set of drives, there was a failure, and the OS maker decided NOT to make a driver for the replacement RAID card we had. (at great cost and by FedEx!) I had to downgrade the OS, load the data to another device, upgrade it and throw away the raid entirely. What a bitch!
It's even part of the reason I delayed using software RAID on Linux- I was gunshy.
But I tried it. You'd expect the extra overhead to cost access-time, instead it speeds up reads! And the writes (at least in Linux) happen in the background so you don't notice any lag. I've run software RAID in Linux for YEARS, replacing drives and adding spares, etc. It's solid. Not only solid, but (for small applications) the best thing out there.
(If you're gonna approach Amazon.com, hardware RAID, all the way!)
Across the distributions, Redhat's got the lead in RAID-at-install-time, but every Linux out there has the ability. It's worth a shot!
--- For a good time mail uce@ftc.gov
None exactly of that. You have to find cheap RAID controller, with documented or known format, recognized by Linux softraid. There is some. In this case, even whole computer busted (including controller, it can be EOL, when something will happen, and you cannot find new), or system partially corrupted, you can still recover your data with help of Linux (yes it can read Software RAID created by cheap softraid controllers without those controllers. Check dmraid documentation.
"real" raid is much worse. all you need is a backup copy of the software driving fake raid to get it up and running 5 years later when the controller craps itself. when your $500 card lets the magic smoke out and the vendor is 3 years out of business and you can't check your firmware version because the card will just catch fire again if you hook it up.... you are screwed.
Snowden and Manning are heroes.
Huh? Vista Ultimate supports RAID.
http://www.neowin.net/forum/index.php?showtopic=612319
Anyone can "stand up for what they believe", but it takes a very brave individual to change what they believe. - Loundry
If you use RAID0, make sure to "burn in" each of your drives, with full drive surface write and read testing before you start placing data on them.
You'll also want to periodically read each byte on the disk surface while the RAID0 is in operation, to ensure the drive controller will have an opportunity to perform ECC corrections on any sector beginning to encounter errors (so there will be less opportunity for an unrecoverable read error event).
Stress your system at least a week before you start putting any important data on it, and of course, once you start doing so, make frequent backups.
Your backups should be frequent enough that you do not lose enough data to seriously hurt you.
Years ago, when I was a system administrator, we spent a lot of money on a RAID box for our Sun machine.
It came in a really pretty cabinet, and looked VERY impressive.
Some time later, the controller failed, resulting in complete, unrecoverable data loss.
Yes, we had backup, but the array was toast.
And what allot of people don't realize is if you build a RAID array and a drive fails can you replace the drive with the exact make and model? Raids work best when every disk in the array is the same model and revision. If you plan to build a 5 disk raid array you should also purchase a 6th drive to keep as a cold spare.
I built a RAID 5 array using three 500GB disks via mdadm under Linux. I assembled the array and formatted it. Within minutes of testing I was getting mail from mdadm telling me the array was degraded. I then began to test each disk for defects and lo and behold one disk was bad right from the start. I tried to RMA the disk but newegg had informed me those disks were now obsolete. Great. I was credited for the bad disk and purchased a new one that closely matched the other two. It was a nightmare as during some boots the disks went haywire and I would get a "Could not bd_claim sdaX" And it would hang for a while and I would have no array. It happend once in a rare while until it became a real problem. I kept my most precious data safely backed up on different disks I had spread around. It finally got so bad that I would have to constantly reboot the machine for up to ten times before the disks were synced up and the array worked. I purchased a 1TB disk and copied all the data off the array to it and used the 500gb disks in other systems. RAID is great for big fat storage arrays but it can become very sensitive and then one day POOF its all gone.
This is the reason OEM drives from Dell, Apple, HP etc. Cost four times what a retail drive would cost. The cost is no way associated with quality but rather consistency. Retail SATA drives are constantly changing: less/more platters, faster seek and read speeds and firmware revisions. Those costly OEM drives are the same disk every time right down to the inner workings and firmware. So if you buy an Apple 1TB disk on a sled and it takes a dump in three years you can be confident Apple will replace that drive with the EXACT same one. Its not a magical Apple disk of superior quality but a Maxtor/WD/Hitachi disk that is produced for Apple with no revision changes unless Apple orders it. Unlike retail drives which are changed at the manufactures whim.
So if you are building your own raid plan for failures and try to buy a spare for your array. I don't know disk shelf live but it will save you down the line. Also keep a USB or 1394 disk around for backups. Spread your most precious data around like pictures home movies and documents. If you have a few computers around the house keep a mirror of that data one those machines. Music, and downloaded video can be re downloaded but home movies and pictures cannot. Put all the silly stuff on the raid along with the precious stuff for access but keep backups of the good stuff!
I've had several foul run-ins with RAID, JBOD, and lack of true backup solutions. The failure rates of HDDs are unacceptable, and the manufacturers should be accountable for the safety of your data (Insert car analogy here). Because this is not the case, we now have cheap RAID.
Cheap raid isn't worth a copy of Vista, and no matter if you use hard or soft raid, there is still a single point of failure, and no accountability for the array just going poof (I've had it happen, and it's the ugliest thing to ever have to try to explain to a boss/client).
I think it is unforgiveable for manufacturers of drives to get away with some of the poor quality I have seen, but it's twice as bad for makers of RAID controllers to make the garbage they do, and market it as helping to ensure the safety of your data. And I have seen RAID controllers do more harm than if they had just been left as a bunch of disks, even in RAID 1. If they had to foot the bill for the data recovery, they would be a little less cavalier about releasing shoddy hardware.
Again, as has been pointed out, RAID is primarily to ensure high availability, and is NOT a backup solution. If you're not running a web server or other service, while RAID may be useful when one of your hard drives dies randomly for no reason, it leaves a single point of failure, and that single point was probably designed poorly.
--Not to be worried, Pitr fix.
Parallel SCSI is obsolete. New systems come with SAS controllers, and (most?) SAS (Serial Attached SCSI) controllers support SATA drives as well.
RAID != backup but it's still quite useful.
I'd much rather swap a disk when it dies then go through a long and painful restore process from tape or a network. Disks fail.... a lot. Especially cheaper SATA and IDE drives.
Anyone that says "HDD's are reliable, RAID is useless, just use a backup device" needs their head checked. You need RAID AND A DECENT BACKUP.
RAID won't save you from yourself or OS bugs but it is far from useless. I would not trust a single HDD with my data. I also would not trust myself to keep an impeccable backup schedule where I back things up several times a day so without RAID if a drive dies I WILL lose data. Not a heck of a lot but I'm going to lose SOMETHING.
On another note, maybe I should take a look at Time Machine in OSX.
Wrong wrong
Who says that one needs to recover data from the remaining drives? What if I also have the data stored somewhere else on different disks on a different controller?
If you lose data when your RAID controller dies, then your real problem is that you are not backing up your data.
My software RAID6 arrays have higher read and write speed than a single drive. Read speed is limited by bus bandwidth, and write speed is limited by seek time and CPU time.
Do you have any evidence for this claim?
There is no need to go throguh the extra effort to use the same model of disks throughout an array or even keep a spare solely so you can have the same model in the event of a failure.
You don't need a drive with the exact same specs, just with enough blocks to represent the same size as the previous drive.
OEM drives are expensive because A) People are typically stupid morons and B) When you mark down the server so it's not profitable, gotta make the money back somehow!
Mod me down, my New Earth Global Warmingist friends!
RAID 1 hopefully prevents that a server will go down and it makes it possible to easily replace a bad disk.
At the fairly large ISP I work at, all servers run Linux, and wherever redundancy is not easily implemented otherwise, we use software RAID so that if a disk fails, the system can keep going, and the hot-swappable disk replaced without any outage. We use only software RAID — RAID 0 for syslog servers where write speed is an issue, RAID 1 for disk redundancy, RAID 5 for redundancy and larger volumes, and RAID 6 for some larger systems. Even where we use 3ware RAID cards, we still combine the individual disks (exposed as JBOD) with software RAID.
This RAID is totally distinct from backup, which we do to a number of large filers, but we only backup data. Configuration management is the critically important tool that makes it possible to restore the OS and software with its configuration after a failure. LVS is the other important tool for redundancy.
everything I see says only stripping drives is available in XP/2000. Only Windows 2000 Server/Adv. Server/Datacenter, and Windows 2003 Server and up feature redundancy through software.
So it sounds like the originator was onto the cheapest for home, windows solutions.
He wants to mirror the drives. This means he wants RAID 1. Therefore, the failure rate of the array is 1/2 the failure rate of each disk (more, actually, because they're like;y identical drives that will fail at the same time, but you get my point).
"You're posting to Slashdot asking whether you should trust Microsoft?"
Nope, he is asking whether he should trust a software RAID for data protection...
...and no, he should not. RAID (both software and hardwar, all RAID levels except 0) are about minimizing downtime and not about protecting data.
"the only way you'll be able to access that data"
Gee that's funny, because I can read the data from my backups just fine.
"I want any drive and its data to be as safe and portable as possible (that's the reason for choosing FAT32)"
.
Hardware RAID often uses a header to store meta data about the RAID configuration. Software RAID for windows does too (8MB), but that is portable across it's lineup.
In other words if you want this to be portable you had better check. It may not be possible to just pull a hardware RAID disk and slap it into another system due to proprietary meta data headers.
Simple answer: buy more drives. Put the extras in the closet. By the time you run out of spares, it will be time to move on to new drives.
Do you have any evidence for this claim?
A typical RAID implementation writes stripes at a time, by issuing a series of writes to each drive. If your disks have the same geometry, then each write will be at the same physical location on each drive and so complete in almost exactly the same time. If they are not, then the different disks will be moving their heads at different times. The RAID controller (hardware or software) will then be bottlenecked by the slowest drive. To make things worse, the slowest drive can be different for each write. One write may require moving the head sideways on one disk, the next may require moving the head sideways on the other. In both cases, you are limited by the worst-case performance for the disk. The same is true for reads on RAID-5, but not RAID-1, which can just use the result for whichever disk returns first.
I am TheRaven on Soylent News
Fast facts:
I prefer pure software RAIDs, for a simple reason: They do not depend on available hardware. If one controller dies, switch to another one: Other brand, other type, other drivers, and the RAID still works. If you insist, you can even mix an IDE drive, a USB drive, a SATA drive and a SCSI drive into a single RAID. Try that with a hardware or host RAID. Some people even built RAIDs of floppy disks or USB sticks (not for pemanent use, of course).
My faithful old Linux home server runs two RAIDs, both in software: a RAID-1 for the OS (remember: the BIOS does not know about the RAID), and a RAID-5 for the data. The RAID-1 used to run on old SCA drives, but recently, I switched to two small IDE drives due to unrecoverable SCA cabling problems. The RAID-5 is composed of four IDE drives, connected to two IDE controllers, each disk on a single IDE cable. An external USB disk is used to back up my data, rotating through 10 days. All filesystems are ext3, all disks are monitored using SMART, all RAIDs are monitored. If anything wents wrong, I will get an e-mail from the monitoring software.
Until recently, one of the controllers was an el-cheapo non-RAID controller, and the other one was a donated, expensive, well-known brand, RAID-capable controller running in non-RAID mode. The latter one decided to randomly take some free time on the job, and either disconnected from the PCI bus or disturbed it, causing panics in the OS above. Only pure luck protected me from data loss. I ripped it out of the machine, kicked it into the trash bin, rewired the RAID to use two disks per IDE cable, and verified and reconstructed my data. Some days later, another el-cheapo non-RAID IDE controller arrived, the same brand, model and type that already sat in the next PCI slot. So I rewired the RAID again to work with one disk per cable, everything was fine again.
For a new small business or home server, I would use nearly the same setup again: Two software RAIDs, one for the OS, and one for the data. Upgrading the OS is just fun when you can
Denken hilft.
Vista/XP, etc supports stripping, but not a mirror (or raid5) which was what the original article requested. Granted stripping is the lowest form of RAID, so technically true.
RAID is not a backup or archival solution.
If you need a study supply of replacement parts, buy a server from a well known vendor like IBM, HP or Dell.
and it's no good for availability if you have to search ebay for the part to get your system back up. soft raid will run on anything
Snowden and Manning are heroes.
If you are dealing with enterprise hardware and a proper contract, then Sun will come in and fix it right up for you.
If you are not, then you are just wasting time with very expensive hardware.
Or don't use RAID0 for important data, idiot. Use it for games where it doesn't matter if you lose everything.
You may have noticed that some hard drives are marketed as being designed for RAID use. These work slightly differently to most consumer disks. Typically, a small region of a disk is hidden. If the disk discovers a bad sector then it will use one from the hidden region to replace it, so every write to the bad sector goes to one of the spare ones instead. This is very bad for RAID, because two drives writing to the same sector may be writing to two different physical locations (if one is remapped), with the same problems I outlined above.
Disks designed for RAID use do not have this behaviour. If they find a bad sector, they report this to the OS. The RAID controller will then mark that sector, and the corresponding sectors on the other disks, as bad and not use it in future. Note that there is no performance gain from using RAID disks in a single-disk configuration, although this doesn't stop some people trying...
I am TheRaven on Soylent News
There are a number of solutions, not just Windows 7 or a hardware RAID controller.
To begin with, every NT-lineage Windows version ever produced supports software RAID out of the box. Add that to the fact that any major Linux distro today supports software RAID. And so do the *BSDs. And Mac OS X. And Solaris. And probably a bunch of other platforms I can't think of right now.
Perhaps he has already decided to run Windows 7 for other reasons... having done so, none of these options are avaiable to him. Except the NT-lineage Windows software RAID, which is what he's asking about.
Hell, you could buy one of these one of these and throw the drives in it, connect it to your network switch, and presto -- instant RAID+NAS.
My experience is that NAS is a _lot_ slower than local storage for many applications. If he's a developer, for instance, he really wants local storage, otherwise he's going to be suffering substantially slower compile times. Video encoding and DVD authoring likewise is noticeably slowed by access over 100Mbit networks. And he may not want to update to 1Gbit just so he can use a fancy NAS box.
I think we would all like to know why you think Windows 7 is your only option, because if that's what you think, you don't know how mistaken you are.
His options, basically are:
* A hardware RAID controller. Either use the one on his motherboard, or shell out cash for a better one. Summary makes it sound like he's decided he doesn't want to spend any more cash, and I can't say I blame him. It isn't worth it.
* Software RAID as provided by whatever OS he decides to use. Generally it isn't worth switching OS just to get a better software RAID implementation, at least not for a desktop machine, so for him this comes down to Windows 7.
* NAS; this is an expensive solution that provides suboptimal performance. If he has only a single desktop machine there are no real advantages to the approach, either.
* SAN; this is an even more expensive solution, although admittedly it would provide the best performance. Not really a realistic option for a home desktop system, though.
So I don't really see any viable options for him other than the two he asked about.
Again, if you need a stable parts supply, don't go with a whitebox server. Software RAID doesn't help you in that regard - instead of an 8 year old SAS RAID controller you'll be looking for a generic 8 year old SAS controller. It'll still be shitty and not usuable in a production environment.
Is it worth keeping a spare for the sole purpose of having the same model available in the event of a failure when you can get a newer and faster drive in the future? Is the difference in performance between modern SATA drives so significant?
This is the point I am trying to make.
RAID1 "might" protect you from a failed drive.
All three major OSes, Windows, Mac and Linux, support R/W access to NTFS. With FAT32 you're limited to 4GB file sizes and you get NO journaling - which means you actually have a greater chance of data loss when you lose power. RAID won't help you there.
Think about your needs for a bit. Do you want to be able to access your terabyte plus of data from other computers or other OSes? Depending on your needs you might also be able to just use a setup like that of my place: networked Linux based storage over a Gigabit LAN. It won't work if you're grinding away 4 VMs at a time on VMware or editing video, but it's fine for storing downloads and music. Going with a Linux based storage server means you can also access the drive over whatever protocol you want, Samba/NFS for local traffic and even SFTP or HTTP for accessing it away from home. You also get a lot of flexibility with your RAID options. Try picking up a third drive (the same size) or even two more and putting them all in a RAID 5 configuration.
Whether you use local or network-attached storage, go for software RAID. It takes almost no CPU time on modern computers and you can move the array to a different computer. That's an invaluable disaster recovery path to have. And use NTFS, for previously stated reasons.
Nope, he is asking whether he should trust a software RAID for data protection... ...and no, he should not. RAID (both software and hardwar, all RAID levels except 0) are about minimizing downtime and not about protecting data.
Err.. he actually says "should a drive crash, I want the system up and running in no time", i.e. his goal is minimizing downtime. I don't know where you got the idea that this was a replacement for a backup strategy.
All modern disks remap sectors as necessary. The main difference between consumer and RAID drives is the timeout for error correction.
I'm gonna just count on the RAID controller taking out all my drives and all my data when it dies.
This way I will not be disappointed.
Ah, but when you buy a few drives for your RAID array, with the same model and revision, and give them all very similar usage patterns, in the same exact location, then it's a lot more likely that more than one drive will fail on you in quick succession. making your RAID array a lot less safe than it seemed at first.
I think the point the OP is trying to make is that with two different drives, for every task you attempt you're going to get worst-case performance every time.
PocketGamer.org - For the gamer on the go!
I laugh when I see people say,
"If the RAID controller dies, you have to replace it with the same model. blah blah blah"
If my RAID controller dies, I am not going to trust any of the data on any of the drives that were attached to it when it died, so it doesn't matter what brand it is, because there is no way that I am even going to bother trying to recover the data.
You really need to make your backup strategies based on "worst case" scenarios, not just the "what happens when my drive dies" ones.
Is it worth keeping a spare for the sole purpose of having the same model available in the event of a failure when you can get a newer and faster drive in the future?
I would say not, but when one drive fails you should replace all of them. For a home array, expect one drive to fail every few years. I had a disk in RAID-1 array fail last year. It was a 40GB disk which cost around £100 new. For the same price, I can buy two 500GB+ disks now.
Is the difference in performance between modern SATA drives so significant?
It's not a question of performance, it's a question of the difference between a linear access and a seek. The time for a seek is 4ms+. If a drive can read 50MB/s then a linear access is around 10 microseconds. If your one disk is doing a linear access while the other is doing a seek then you are limited by the time of the seek (for RAID-1 writes and RAID-5 reads and writes). If you have to seek after every block, your maximum throughput is 125KB/s. If you do a linear read, your throughput is 50MB/s. If your drives have different geometries, you double the number of seeks you are needing, dramatically reducing your throughput.
I am TheRaven on Soylent News
Cheap hardware (mobo) RAID controllers are actually SoftRAID controllers that implement the RAID functions through the bios and the driver. You don't want those, because if the controller fails you might lose your data, because you have to get the same controller, because the data is stored in a proprietary format of that card or mobo chip.
So it is much safer and saner to use the os for software RAID.
Why you want to go with FAT32 is beyond me, though. Your data is much safer with NTFS. Linux has good NTFS drivers now anyways.
MOD PARENT UP. Good explanation. Statistics often assume randomness, and often events are not actually random.
Who argued otherwise?
That's wasteful when you're using more than two drives.
You are making theoretical calculations. How does this affect real-world usage? (Note: I mix drive models and notice no significant difference in performance.)
Add that to the fact that any major Linux distro today supports software RAID. And so do the *BSDs. And Mac OS X. And Solaris. ... Hell, you could buy one of these one of these and throw the drives in it, connect it to your network switch, and presto -- instant RAID+NAS.
So basically your "other options" are "use a different OS than what you want to" and "buy more stuff"?
Ding! Ding! Ding! Ding!
Not something to be used in a workstation but its currently protecting 16TB worth of MY data and has proved to be damned reliable. I have suffered SEVERAL disk failures that did not result in data loss and I just have to buy disks as they fail or I need more space.
For a workstation though - software RAID for sure. Especially if the controller insists on putting strange formatting or other weirdness in there. The poster wants to be able to have his data PORTABLE so that pretty much rules out a hardware controller for me. Too many folks I know have had those crappy motherboard controllers flake out or do weirdness for me to really trust them with stuff I wish to save.
Mind you if this data is REALLY important than he shoudl also be backing it up elsewhere. All it will take is a careless delete or nasty virii to wipe out his files. There have already been ransomware programs released in the wild that encrypt documents and other programs that trash MP3. Even with my unRAID I backup my MP3s to standalone media. Sadly the movies are just too big, the ISO can be redownloaded thankfully.
Build it, Drive it, Improve it! Hybridz.org
You need RAID AND A DECENT BACKUP.
This is, of course, idea.
That said, if you're going to have only one... go with the backup.
I do not believe you can use Fat32 formatted volumes to make a software raid mirror in any version of Windows, ever. If I'm wrong, tell me, but I have been working with the 'Disk Management' in windows for 15 years and have never been able to do this. Not that I would. 'For portability' is not a good reason to use a file system that was left behind by Microsoft years ago. You can mount an NTFS partition in several O/S's these days effortlessly. I agree with every one else here. Build hardware RAID 0 into your system, use all the space on those disks that you paid for, get some performance out of them. And most of all, PRACTICE RECOVERING YOUR DATA. Do mock-crashes, do some research, READ what other are doing about their problems, make some notes, get acquainted with your hardware, and your backup software, and whatever tools you end up using for data recovery. This is essential for real data loss prevention. Am I preaching? Sorry!
I often use inexpensive SATA RAID controllers from Promise (I do not work for promise). They don't cost very much and they have been absolutely reliable for me (for many years now!). I often stick with RAID 1. I've built several RAID5 arrays and I don't find a lot of value in them for low capacities. Mirroring (RAID 1) is straight forward, and if you ever have a problem you can always read one of the RAID 1 drives using a SATA to USB interface, or if you ever need to clone a hard disk it's easy. Promise seems to use the LAST 64k of the hard drive for it's mirror info, not the FIRST 64k! this makes any of the two drives in the RAID array easy to use out of the array when/if you're in a jam (for whatever the reasons). As far as RAIDing your data only, in my opinion RAID is designed to avoid lengthy recovery procedures - don't put yourself in a disadvantaged position - all hard drives fail eventually - RAID the OS, your data, everything! If your server is a very busy server - start looking at higher end RAID solutions.
I have been running Linux mdadm RAID 5 for years using onboard IDE & SATA and completely different drives (some combination of 300, 500, 500, TB drives all from different manufacturers) and have never had a problem due to differences in drives. General performance is "good," I don't have the numbers offhand but when I was testing it was comparable to ideal configurations with these drives. For the average uses, there's no need to go out of your way.
RAID5 has a nice side effect; if you need to send a drive in for replacement, you don't need to worry about snooping of your data. Not sure how this applies to RAID1.
"should a drive crash, I want the system up and running in no time"
That's a fair reason for RAID and indeed specifically for mirroring. But the most common TSHTF scenario is stuff like power supply or mobo
failing so your goal should be to increase hardware redundancy at every level. Get yourself a good ESATA enclosure for each drive with
independent power supplies and use software RAID.
"I want any drive and its data to be as safe and portable as possible (that's the reason for choosing FAT32)"
I assume FAT32 is for portability (everything reads it) and not for safety (uh, you'd be nuts). This is fair but a better choice is NTFS. It is also
readable by everything (writing used to be a problem but not anymore) and much better for safety.
"even if the OS or the controller screw up big time"
If the OS or controller (or more likely the user) screw up big time, you'll need backups.
I tend to recommend people buy an inexpensive external USB or firewire drive, leave it attached and assigned as a backup device, and have some software package run a daily backup of all the relevant folders and files they might need to save.
Buy two drives.
Every Monday take one to work (make sure your name is on it) and bring back the one you had at work and hook it up to your system. This way if there's a power surge or fire at least one set is offsite.
A backup isn't really a backup unless it's offline (and preferably there's also an offsite).
Hope for the best, but be prepared for the worst.
The one downside of a Drobo is vendor lock-in, if your controller dies. However, this seems to be the case with every RAID controller.
I use a mirror drive in a mobile rack as a backup and an rsync script on a modified liveCD to allow me to point and click to the rsync script to tell it to run the backup. I plug in the mobile rack drive and boot from the LiveCD. When the mobile rack is not in active use, i.e. either being backed up to or restored from, it is stored well away from the computer.
This has served me well for years, including after a hard drive crash. I was up and running in 15 minutes after the crash.
Though since the Knoppix 5 LiveCD it was based on does not recognize SATA running in ACHI mode, I have to rebuild around another LiveCD. The big advantage of this is that a drive in a mobile rack that is not plugged in is completely immune to anything awful happening to the main system, including the events which are most likely to blow up a hard drive.
Tech Public Policy stuff
I would guess that the key word there is "hopefully" - various RAID configurations can help when it's just a disk failure, but they do nothing if the problem was elsewhere - like a motherboard that takes out all the drives.
Also, by the time one disk fails in a RAID, there's a good chance that the others are ready to give up the ghost as well, so you stand a good chance of having another failure while you're trying to rebuild the array - most likely from a previously-unseen error on one of the existing disks. After all, the hardware doesn't know that sector you wrote a year ago is unreadable until it tries to restore it ...
As others have mentioned, I also agree that RAID0 isn't raid. Even if you do consider it RAID, it doesn't address the submitter's request.
Comment of the year
Well, Windows 7 Ultimate does support mirroring. Sadly RAID 5 support remains for server versions only. Hopefully they'll finally wise up and go all the way with whatever comes after 7.
The improvement is huge ...
Yep. Same drives, same source, same batch, same usage patterns = fail around the same time.
Robocopy your data from one drive on the 1st computer to a 2nd drive on a second computer, nightly while you sleep. If you need more assurance, robocopy your data to a 3rd drive on a third computer.
raid will make your data highly available, not secure.
if you want security, you need backups, and backups are:
- off line (viruses, power surge, sabotage...)
- off site (fires, theft...)
- tested (i've got horrors stories of people that THOUGHT they had backups...)
- multiple (... and of backups that turn bad at the worst possible moment)
Raid is none of that. I know plenty of people who thought their data was safe because they had raid. It isn't, it wasn't, it ain't ever gonna be.
The Cloud - because you don't care if your apps and data are up in the air.
I use Mozy. It's cheap and easy. They are also owned by EMC so I'm not too worried about them going under. I only use a laptop so I don't have to worry about having a usb disk permanently dangling from it, or having to remember to plug it in periodically. I get unlimited storage so no worrying about storage allotment. My broadband connection is fast enough that it will backup any large changes overnight at the longest.
It also gives you the added advantage of being off-site, which all reliable backup systems should have as a component. So no worrying about house fires etc. This brings us to the interesting use case they are currently milking where a guy who was on the Hudson river crash used Mozy and his data was perfectly safe, whereas another guy lost both his laptop and USB backup to the bottom of the river.
(http://blogs.usatoday.com/technologylive/2009/03/when-your-lapto.html)
Just use the integrated raid (yeah, softraid or fakeraid) to do it, make sure its the chipset (intel or nvidia) that is doing the work.
Sure the performance isn't there in general (and if you raid0 more than 3 drives on the NV chipsets, not there at all) but it is OS agnostic and is relatively cheap to get the data back if your board dies (cheap mobo + celeron CPU) since all chipsets from each of those two makers support each others raid.
As others said though, what you are doing is not protecting the data, but just increasing your up-time if something bad happens, backups are your only protection.
I would be one of the first to suggest using an areca raid card (or similar quality) should you be looking for performance, but your not, so I won't :)
...
My data is VERY important to me. Therefore I want the cheapest, lamest solution possible. Please recommend the "best" way to protect my data...
>everything I see says only stripping drives
*sigh*
computer porn is everywhere, now :(
hawk
Build a cheap headless NAS that uses ZFS, such as FreeNAS 0.7RC1. Best way to go hands down for a high-availability home file server.
Whoa, hold the boat. I've had a lot of experience with Dell & HP/Compaq(Proliant) provided RAID systems and they are not sensitive to disks with vastly different innards. All that matters is block count and software mirroring doesn't even care about that, because you'll simply be limited to the size of the smaller disk. If you're using mirroring or RAID, try to go with different makes of the same size. This article talks about MTBF. It turns out if 2 drives of the same exact model comes off the line and end up in your PC, there is a chance they could fail within a very close time to one another. So your mirror or RAID could fail permanently while rebuilding from the first failure. But if all your drives are of a different make, chances are they won't fail at the same time and you'll get the critical time needed to rebuild your array.
When I'm going to do mirroring or RAID on hardware that doesn't have high-end dedicated server RAID controller, I use Windows or Linux software RAID. Performance is surprisingly good and I'm not married to a specific hardware implementation. I've had _none_ of the issues you've described with Linux software RAID on several servers for several years. Mdadm has only whined after a power outage or genuine disk failure.
There's no place like 127.0.0.1
I have a hardware RAID 0 setup for the data that I need immediate access to.
Yes I have drive failures and yes they are catastrophic. But I am prepared for them and they don't slow me down for long.
I have the whole thing mirrored on a networked file server. It's software RAID because it doesn't need to be fast, it just needs to be there.
I do the backups from the remote system. I backup onto hard drives, then I unplug them and put them in a safe place. I rotate several drives in this fashion. I check them regularly for hardware issues. I have to trust these drives more than normal, so I give them a really good working over when they are new. If I were rich, I would buy server drives for these. I am not, so I condition them myself.
I only bother to back up my home directory. Why backup data that can easily be restored with an OS install? It only takes me about 15 minutes to get a newly-installed machine set up on my network.
When I do an OS upgrade, I wipe everything and restore from my backups. It's good practice for when I will actually need the backups. I like nice clean systems, not ones with years of accumulated cruft.
If my RAID controller fails, I can mount the backup mirror and continue working, albeit more slowly.
I haven't lost data in years.
raid is for high availability, not backing up your data. Put another way, it's not *data* redundancy, it's *hardware* redundancy: it allows you to store one copy of your data on multiple devices. But it's still one single filesystem spread across those drives, and if you screw it up you're still fscked (pardon the pun).
JWZ (Netscape, DNA lounge) has a very simple guide on using rsync. rsync is really about all you need. Apple's Time Machine is based on it. Now, this guide is not eïxactly windows friendly, but 1) you stated that you only need to back up your data, not your apps, and 2) you sound like you're pretty technically adept so you're probably running cygwin on your windows boxes. Rsync should be able to handle this just fine on win32.
For data serenity, two separate single drives beats raid every time.
"That's nice, but the submitter is asking about RAID 1."
I think he's asking the wrong question anyway.
"Based on past experiences, I have decided that only my data is worth saving"
See? He is asking for backup, not RAID. It has been said one thousand times but it seems it must be said again: RAID is *NOT* in order to protect your data. NOT, NOT, NOT and then NOT again.
RAID (not talking about RAID-0) is there in order to enhance your data's avaliability (as in, say, instead of being able to get to my data 99% of the time, I can get to it 99,9%) but when it's hosed, it's hosed. To protect your data you need backups, not RAID.
"Of course, such a setup should secure my data"
Of course not. Of course you will get quite a funny face when you discover it. Quite more or less the one that had the guy from this story, about six months ago, with the very enlightning title "Why Mirroring Is Not a Backup Solution": http://hardware.slashdot.org/article.pl?sid=09/01/02/1546214
"Even more importantly, I want any drive and its data to be as safe and portable as possible"
Then, *even* if RAID could be considered for data security (which is not) you already answered your question: as a general matter, hardware RAID will only work when using exactly the same controller model, possibly up to its minor revision. You can't count to break a hardware-managed mirror, take one disk to a standard SATA controller and get any data out of it. If your controller dies and miracolously doesn't take the disks with it you can't count on buying a different RAID card (as it will most probably be in about a year for consumer-grade hardware) and get any data out of the mirror. So you should go with software RAID.
AND TAKE BACKUPS.
"What happens when your offline backup company goes under?"
You start using a different one? Personally I have a colocated server where I keep encrypted backups along with a hard drive in an external enclosure attached via USB. I keep backups of all my important files in two locations. Now, if the online backup service went out of business and my house burnt down before I could find a new one, I'd be out of luck I suppose. I think I'm a little more likely to win the lottery though, so I'll take my chances. This also assumes that the backup service shuts down and cuts you off without any notice whatsoever.
Another strike against Vista I didn't know about unless it's just deeply hidden somewhere - however the related Server2008 should have it. I still have an old NT4 box to run legacy software that is using software mirroring of it's drives.
Better yet ... get two inexpensive USB disks, and swap them every day or two, maybe even have the inactive one in a separate location...
"My experience is that NAS is a _lot_ slower than local storage for many applications. "
It's really about storage requirements. If my Internet connection is only 20mb/s and I just want to stream some movies at 50mb/s (1080p) then does it really matter if the network is "only" 100mb/s?
OpenSolaris ZFS
Free and the best.
see http://en.wikipedia.org/wiki/ZFS
Amazing, all these posts and hardly anyone actulay answered his question. Frankly these people who say raid isnt a replacement for backups should just shut the hell up already. The reason people use raid as in mirroring is just for redundacy, and for that purpose it works well. Regardless of what people have been saying about how useless it is I cant say I ever saw a mirrored array fail in a way to lose data when the whole system was fried, talking about file system errors and mailware causing you to lose data is like not driving because you might get carjacked.
Also what is with the people recomending using 2 identical drives? I have seen failues on arrays like that where one will die and then a few hours/days later the identical disk will die as well. Its better to have at least 2 different models so that if you bought a bad run when you got your drives, they wont all die close together.
But to ACTULAY ANSWER HIS QUESTION...
I have never had an issue with a onboard controler. Unless your planning on having a large array instead of a mirror it should be just fine.
What does "fail at the same time" actually MEAN?
Are you saying two drives of the same batch are going to fail WITHIN THE TIME IT TAKES TO REBUILD THE RAID?
I don't think so. They may fail within days, weeks or months of each other, but it's highly unlikely they will fail at the exact same time plus or minus hours - which is all you care about when rebuilding a RAID. Because the next step is to backup the data and replace the drives if you're that concerned about another immediate failure, anyway. So whether the drives you buy for a RAID are the same batch is irrelevant. The issue is WHEN the drives are going to fail - and ALL drives are going to fail sooner or later. That is what the article you reference is saying.
Not to mention that generally people buy a bunch of drives at the same time and therefore it's highly likely that drives will fail at approximately the same time, depending on usage patterns (an OS drive may fail first since by definition it's usually reading and writing more than a data drive, unless the data drive is being used for transactions - for a home user, this is unlikely.) However, if a home user buys one drive and then months later buys another two drives and sets up RAID, it's likely his first drive will fail before his RAID drives do.
And even if the RAID drives fail, one could fail sooner or later - that is in fact the ASSUMPTION of RAID: that ONLY one drive is likely to fail at a time relevant to rebuilding the RAID. Otherwise why bother with RAID at all?
The position of the drives in the box could matter - one could be hotter than the other or vibrate more. This could cause one drive bought at the same time as another to fail first.
You can't predict drive failure based on batch as a general rule. You can't even predict that two drives bought retail come from the same batch. They could be two different revisions of the controller despite being the exact same model.
Richard Steven Hack - This sig is TOO GODDAMN SHORT TO DO ANYTHING USEFUL WITH! MORONS!
A false sense of integrity is worse than useless.
A structured backup system is the only sane answer, where the oldest technology (lowest density) is closest to your daily work, and the newest technology is your backup. Your backup will always have about 2x your online, which is about right. Careful snap-shots, with off-line validation [ ie. ensure that you actually backed up something useful ] is step 1. A well structure pipeline will stage the snapshots for integration into the whole [ ie. yesterdays filesystem ], and archive the accumulated snapshots.
As your storage demands outstrip your storage, discard your oldest [ ie. online ] storage, bringing forward the second oldest, and backfill the chain with what is new today.
In a critical event, you will lose the delta since your last snapshot [ you had 2x, why aren't you snapping more often? ], but it will be online immediately.
If you are paranoid, mirror your backup chain at some point - but don't be too aggressive. A backup is only as good as its validation, so the more redundancy you introduce, the more validation you must perform.
RAID is an accountants answer to integrity - unless storage cost represents something substantial to you, it should be avoided.
I had my but saved by mozy when two drives on my raid 5 array failed at the same time.
Yup RAID is not perfect, but it saved me a couple of times when just a single drive failed. Yes, a 4 drive array had two single drive failures, and a 2 disk at once failures. Suffice it to say I will never buy a western digital drive again EVER!
I've been using Slicehost for a couple of months, backed by Rackspace Cloud Files instead of Amazon. As far as I'm concerned it the best thing since the Original Nintendo.
I have two PC's and a laptop at home and a work computer backing up to it, the resync occurs and night so I don't see any performance issues and its completely encrypted on the backed so I don't have to worry about someone browsing through my collection of unpublished poetry about water fowl.
If you haven't actually used one I suggest you check it out.
That makes no sense. The recommended setup is to actually use a bunch of disks from different vendors which means they're all going to be slightly different. Tons upon tons of people run with this type of setup with no issues.
Not too long ago a disk in my RAID5 went bad and I replaced it with another drive that's not even close to the same size. I just made a partition that was about the same size as the old drive and mdadm rebuilt the array with no problems. Been running fine.
I think your problems stemmed from you not knowing what the hell you're doing. Not totally your fault because the whole RAID system in Linux is confusing (MD RAID, LVM, etc). I stay away from all that LVM and other crap, just pure kernel MD stuff for me.
Anyway, back to the original question the poster asked. Never ever use a motherboard "hardware" (actually it's software) RAID controller. That's just asking for trouble. A dedicated hardware RAID controller card might be OK but what if the controller itself goes bad? Are you sure you will be able to get to your data? For me, an OS software only solution is the only way to go.
yea, its true that raid is not backup. but what am i supposed to do in order to backup 4 terabytes of data? and these 4 teras are rapidly filling up, i will soon expand to 6+. i can store the most critical stuff on a separate machine that's usually powered off, but how can i back up my multi-tera RAID-6 properly without buying twice as many spinning disks? is there an economical tape solution for home use?
Raids work best when every disk in the array is the same model and revision. If you plan to build a 5 disk raid array you should also purchase a 6th drive to keep as a cold spare.
I hate to break it to you, but you're actually wrong.
A RAID array is most effective using completely different drives, but of the same capacity. Five hard disks from the same manufacturer, of the same model, bought at the same time means that you're highly likely to get five drives from the same batch. Let's posit that there was some defect in this batch. Now all five of your drives have a significantly higher probability of failing at the same time. Oops! RAID can only deal with one (or two) drive failures!
Using drives from different manufacturers or model lines means you spread the risk of simultaneous drive failure.
What's the real fear here... that you won't be able to recover data?
Use NTFS if it's windows.
Use a released version of windows - not a beta.
Software raid-1 is fine - perhaps even better in some cases.
Raid is not a replacement for backups. Please repeat that - many times. Raid is not a replacement for backups.
You real solution is probably a proper daily backup solution and whatever raid-1 is easiest to set up.
Scrolling through the first several posts, I see a lot of posts basically saying RAID is unreliable. I couldn't disagree more. I set up a Linux software RAID 5 file server in my house about 8 years ago. It started as 4 Seagate 120GB drives. Over the years, I've had 3 of those drives fail. I've never lost any data.
I can't afford to go out and pick up the latest and greatest in storage every time a drive fails. I replaced the failed drives with refurbished drives of the same size from random manufactures. I've never had a problem. The RAID is just as fast to access and write to as when the drives were matched. RAID is an excellent strategy if your concern is drive failure and uptime.
As others have pointed out, it won't protect you from accidental deletions and overwrites. Regular backups help in this area. However, if you accidentally delete/overwrite your backup, or the backup fails, you are in the same boat.
For data that just can not be lost, I have a copy on the RAID, and a backup on DVD that gets refreshed every couple of years. Everything else just sits on the RAID.
I've also run a file server at work for about 5 years now. It uses the same setup. I've only had one drive fail there. Again, I've never lost data. I have a redundant server set up that contains a mirror of the primary server. However, I've never needed to restore data from it. It is nice to have when I'm doing updates. Everyone uses the primary server while I test the updates on the redundant server.
Request a Linux Shockwave player here: http://www.macromedia.com/support/email/wishform/
Preach Brother! I have had several of my customers ask about RAID, but when actually sitting down and talking to them it turns out they are looking for a backup solution which RAID most certainly is NOT.
Here is what I recommend to my clients-Use whatever you want inside your machine, but get a USB HDD or even better a NAS for backups. Most come with very capable backup solutions provided, and is much better for the purpose than RAID which as you so very eloquently is for access NOT backup. There are several cheap barebone NAS kits where you simply add your own drives, these have the added benefit of being easy to upgrade should your data become larger than the drives. Put a couple of 1Tb drives and a nice multi platform backup solution ( I use Paragon Drive Manager which comes with a nice Linux GUI based boot disc and covers FAT,NTFS,EXT2 and EXT3, but there are several alternatives to choose from) and all is golden.
But please don't use RAID as a poor man's backup, as it will come back to bite you in the ass. Get a USB drive, get a full or barebone NAS, and use a real backup software like Paragon or similar. In the end you will be a LOT better off than trying to use RAID for a job it simply wasn't made for.
ACs don't waste your time replying, your posts are never seen by me.
Perhaps that's the OS that is running on the computer that he wants to use the RAID in?
I tend to recommend people buy an inexpensive external USB or firewire drive, leave it attached and assigned as a backup device, and have some software package run a daily backup of all the relevant folders and files they might need to save.
1. I do this for my digital photography. (1 weekend can yield several GB of data).
2. I am better off manually backing up my files than trusting the computer to get it right. I haven't found backup software I trust. Yes I realize I can miss things too, but I periodically compare the list of backup directories with the ones on the main drive.
3. This whole regime is useless unless you regularly backup your 2nd device (external drive) to a 3rd one and copy it off site. (It gets packed off and sent to my mother's house). This mitigates against losing data if I have a house fire or if malware erases drives 1 and 2. I limit exposure to a few weeks.
4. Once you've backed something up to a 2nd or 3rd drive, you should NEVER overwrite it with another copy. Never copy over the top of one directory if a few files have changed in it. Otherwise if you've damaged/corrupted the files on your first copy, you're propagating that damage.
5. I wish I could get individual drives to show up as read only in Windows. I took it for granted that there'd be something like mount readonly. There isn't anything simple or standard. I think there are utils that do it but they're expensive, non-standard. There are free apps that will change registry settings to block all removable drives but that hardly allows copying from a backup to another USB drive. There are expensive hardware USB write blockers used in computer foresnics. If I were using Linux it'd be trivial to mount a filesystem readonly by default.
These posts express my own personal views, not those of my employer
The first thing to learn about RAID or any other technology that you're using for enhanced reliability: If you use it without first testing it in failure scenarios you might encounter and figuring out exactly what you need to do to fix it (e.g. How do you tell which piece of hardware has failed? How will you know to respond if the automatic failover doesn't work? etc.), you might as well just pretend that instead of spending your time and/or money on a redundant solution you just took a vacation and/or lit the money on fire because that's effectively what you will have done.
With that in mind, although I tend to think I know what I'm talking about, you can take the rest of my advice as help brainstorming those failure scenarios and hints about answering those how-will-I-fix-it questions, but it's no substitute for knowing and testing your own set-up and checking the answers yourself. Okay?
Good.
After having been burned by a not-so-cheap, dedicated RAID controller, I have been pointed to software RAID solutions
Gross generalization for the short attention span crowd: RAID 1 seems simple, but there are lots of options and lots of details to get right. And if you've run into the limits of your knowledge dealing with RAID 1 on a dedicated RAID controller, then going to a software RAID 1 solution (which often doesn't secure against all the same problems and gives you more things to troubleshoot directly when things go wrong) may not make your life simpler...
In particular, remember that although your RAID 1 array stores some metadata, it probably doesn't keep track of which disk has the most up-to-date copy of each block, so if you manage to get two working but out-of-sync disks, take care when you're deciding which disk to restore the set from. Also remember that as we approach 1 unrecoverable read error per disk territory, the same wisdom about replacing RAID 5 with RAID 6 also applies to 2-disk RAID 1.
Moving on... =)
In the world of always-busy transactional databases and server systems, RAID is seen as part of complete breakfast of 100% storage reliability, and people want it to guarantee that whenever a program's explicit request to sync a disk write goes through, the data is definitely going to make it to disk despite a power or equipment failure. This means:
You can test that syncs are working properly by doing plug-pull tests (e.g. see http://brad.livejournal.com/2116715.html)
If uptime is important, they'll also have:
At the other end of the spectrum, RAID 1 is sometimes used on desktop PCs as a real-time 'backup' solution for people who are too 'busy' (i.e. lazy) to make real backups.
In the world of single-user PCs, the conventional wisdom is that most applications operate on documents in RAM (where they are vulnerable to power failures anyway), and only write them to disk occasionally when the user explicitly asks for it, so it has become common practice to leave disk buffers on for better write performance at the risk of data loss if a file is being written during a power failure. Application-level recovery features and journaling file systems have allowed this approach to survive into the era of PCs with modern multitasking operating systems. If you're willing to take the performance hit, you
I sit here, writing this reply, on a motherboard fake raid that's down one HD. All of my data is still here. It's all still easily accessible and I can keep using this system while I wait for the HD that I ordered to show up so I can get back to full data duplication. It's a very handy feature considering that if I had had the HD fail and wasn't using RAID, I'd be SOL as far as this system is concerned. To those who say that RAID isn't backup, they're right. Making any copies of your data and keeping them in the same building as the original data is also not a backup, as a fire can easily wipe it all out. For general usage, keeping your systems operable during disk failures, I think that RAID is a wonderful thing to use and I just use the fake raid of the motherboard. On my linux system, I use a software raid because it's easier than getting the system to recognize the motherboard-based solutions. If that's changed by now, then that's great, but it wasn't an option when I built the system.
You don't have a hardware or integrated RAID controller.
What you have is a non-RAID SATA controller, plus software RAID support in BIOS + Windows driver.
This is easiest to see when booting Linux, whose policy it is to only export your hardware, without any fakery.
See Linux SATA RAID FAQ for a clue...
I bought eight of the mirrored drives from WD that have two 500GB drives that can be used in raid 0 or 1. These drives have been a major disappointment as they have all failed unexpectedly and the mirroring was of no apparent value because when the usb/fw attached unit stopped responding, it was useless whether there was a remaining platter with good data or not. These units weren't cheap and I paid about $3000 for the eight of them. I am really unhappy about this and will not be buying WD products any more. A have pile of these I am hesitant to throw away, but they are little more than doorstops now because I wouldn't trust them at all, even if I could get them going again. What a waste of money. (And my trust)
Yes, Mod parent up. RAID is for providing data resiliency, not data protection. In corporations where either very large data sets would simply take too LONG to restore, or where spindle count for acheiving IOPS is critical, RAID permits a reduction in failure rate. However, is it NOT a replacement for backups, and RAID should generally only be considdered when you;re already considering a multiple disk setup (either for capacity or performance reasons).
A RAID1 setup on your home computer may increase your uptime, and a RAID 0 can imrpove your performance, but generally, it's not that improtant, and on a home PC, typically the OS drive is more critical than the data drive (It's easy to resorte a backup, It'sdifficult to make your machine exactly as it was if you loose the drive and need to re-install).
That said, even a data backup is NOT enough. You also likely need an image/BareMetal backup of the OS and application drives. Rember, it's one thing to limit hardware failure by using a RAID, and another to have good backups of your data, but you also need to take the human factor into account: 1) your mistakes change both RAID 1 disks, there is not rollback; 2) hackers and viruses corrupt data just as easily on your external USB used for backup as it does on your primary drive, unless you're using top nothch backup software that hides the backup[ device from Windows and makes the backups unreadable to the OS (rare); and 3) software installs, bad code, and Windows itself can just as easily render all your data useless.
If your data is important: ...and NO, there is NO SUCH THING as a surge protector that can stop a lighting strike. The EMP alone is good enough to destroy data. (I've seen a montior and PC 4 feet from the nearest outlet get cooked when the lightnig's EMP backfed the CRT's stores static energy into the motherboard).
1) Backup regularly, and please use a real backup application, not Robocopy or some cheap scripting system... Keep miltiuple incremental backups and use software that manages a proper rotation and can search offline data to find files you want to recover.
2) Make image/BareMetal backups of the OS. Vista Business and higher editions have something similar built in, but using a program like Ghost is often easier and quicker to restore from. Make a new image at least as often as you install make major changes to your system, or every few patch rotations.
3) DO NOT leave your primary backup device connected 24x7 unless it's a tape drive or worm device. Your backups are easy fodder for hackers on USB drives. Also, a lightning strike or surge that takes out your primary AND backup is bad, really bad...
4) GET YOUR DATA OFFSITE. Fire or water damage should not be able to take out your "important enough to back up" data. This not only includes your backupos, but critical media, CD keys, and anything else you'd need to rebuild the computer far enough to reconnect to your backup disks...
5) Keep this rule in mind: "Nothing is backed up until it's been restored." Well enough that you're doing backups, but if you have never tried to restore your system, you have NO IDEA what that takes, and NO IDEA what you're missing to do it. Done a firmware patch? many of the original drivers may not work anymore, you might need new ones on CD to reload the OS... Maybe you'll find your backup software doesn't have an open file manager, and your e-mail isn't being backed up properly when it's running... Do you have the backup software stored offsite with your backups???
I have nearly 20 years of important files spread across my 3 main computers (including the wife's machine too) totaling about 1.1TB of actual data and files. My main machine runs a RAID 1/0 (mirrored stripes) with 4x 250GB 7200RPM drives on an AMCC/3ware controller (Soft raids SUCK, onboard RAID is not much better, NEVER opt for the lowest bidder if your data and your performance are important...) That's my OS, Application, gaming, and higly-used data drive. I also have a 750
There is no contest in life for which the unprepared have the advantage.
What happens when your offline backup company goes under?
Further more, what happens if that backup company's hard disk array (that contained your backup) fails at/around the same your hard disk fails?
There's about 50 posts in this thread saying over and over again that RAID is not a backup solution and I've heard it plenty of times before as well. What are some actual solid reasons why RAID is a bad backup?
Your backup solution sounds incredible and very robust but also very very expensive. My data is important, but not THAT important! My ultra important data - my own code - doesn't take up much space and is easy to backup all over the place. The rest of my data took a long time to gather and it is important to me but I can't justify spending that sort of money to back it up properly.
A RAID 1 solution seems like quite a fitting one to me. I recognise that it's not as good as your solution, but on my budget it does the job. If a hard disk fails which has happened to me (damn Seagate firmware) my RAID 1 can rebuild all my data.
"Are you saying two drives of the same batch are going to fail WITHIN THE TIME IT TAKES TO REBUILD THE RAID?"
Statistically speaking - yes - it's much more likely. It's significantly more common. It happens - enough that people warn against the practice.
Given the time it might take to re-build a 1 TB mirror - which is what joe average is trying to do at home - even more so.
There is a reason you still see 73GB SAS drives on the market - more, smaller spindles equals more throughput and faster rebuilds.
Why RAID is not a backup:
1) not fireproof.
2) not mistake proof "oops, didn;t mean to delete that"
3) not immune to file system corruption.
4) not immune to power supply failure/surge/lightning/other destructive forces
5) more expensive than a good backup
6) not protable offsite
7) does not track versionb history or old files (something that should be of critical importance to a programmer...
8) Viruses, mailware, hackers oh my!
9) bad/corrupt install
10) OS failure
I could easily go on. I worked in DR for 4 years...
Nearly all of the above have a higher frequency of occurance over a 5 year typical HDD life. Even if you continually replace drives without a data failure, you're still eventually going to have an issue RAID can not deal with.
My Qnap was a $399 device. The 4 drives in it were $90 each (and the 5th spare too). The HDDs I run the PC off on the RAID 1/0 were $40 each. I only run the RAID 1/0 for performance during video editing. I chose 1/0 vs 1 since 1 halves the reliabiltiy of the drives. Even though I do have a good recovery solution, the downtime, nor the effort involved in recovery, would be welcome, and the extra $80 to mirror the performance stripe was easily spent.
The Qnap is also my iTunes media server, my FTP server, included the price of the DR software, and runs 2 IP cameras I set up at home too (which let me tell the insurance company I have real-time video monitoring, and they knowcked an extra 5% of my homewoners policy cost, which by itself is enough to fund replacement drives as I'll need them).... Oh, yea, and it's a NAS too... It has a lot of value beyond a backup system.
I'm guessing you've not got a child yet, or a large family. You probably don;t value to pictures you take, files you have, and other stuff on your PC. That's fine, someday you likely will.
There are cheaper ways than mine to do backups. I have over a TB, and 3 (currelty, soon to add 2 Macs to the list an decom 1 old laptop leaving me with 4) computers I'm backing up, so centrally makes sense. If you have 1-2 machines, a small amount of data, and don't value most of it, then 2 external USB drives and a safety deposit box (Dad's house) usually suffice... Or, just an online backup account for $5 a month...
RAID 1 might save you from a firmware failure, or a disk going bad, but that's about it... Also, RAID 1 may be cheap, but a backup is cheaper. Also, good luck rebuilding that RAID if your MOTHERBOARD fails... RAIDs are proprietary to a particular controller. Unless your new board usues the same chipset (and firmware too in most cases) you;re screwed without a backup.
There is no contest in life for which the unprepared have the advantage.
Thanks for the kind reply.
I tried everything. Started with checking the volume, kernel revisions, and swapping every bit of hardware. The only other thing it could be is the difference in drives. If I yank the one disk that is different the system boots and the array is initialized but does not get mounted. I did not investigate any further because I did not want to play Russian roulette with my data. I do want to put together another raid array because I need allot of storage but I would rather a file system like Btrfs.
Also, good luck rebuilding that RAID if your MOTHERBOARD fails...
I use software raid (on linux) so I'm not paying for an expensive RAID controller or anything like that.
If I were managing a business with large amounts of critical data, I would obviously do it differently because the points you make about fire and accidental damage are important points. The thing is, I think there are varying levels of data importance. That's all I'm really trying to say.
Good, you want to separate your data from your OS. That make a number of things possible:
- your OS upgrade shouldn't affect your data drives
- you can treat data and OS backup seperate
- you'll be classifying your data (at least somewhat)
Take the next step & put them on a NAS (I don't care which as I'm not supporting you)
- Now you can get your data on multiple machines, regardless of OS (if you do it right)
- Your OS won't affect your data
- Gigabit ethernet is not expensive and it usually beats USB 2.0 speeds.
- You can isolate your data physically
= away from foot traffic with liquid spills, etc
= locate in a cooler environment
= not put in dogy devices into the data device
= less noise & heat at your desktop
For a fixed database of static information, this may be true. The vast majority of database implementation, however, is one that is also being written to in realtime, such as a webserver database, or POS system where the overhead created by decreased write performance will more than negate any advantage in decreased seek times. Really, it all depends on the application you are running, and the specific circumstances of your network/system setup. RAID1 can be useful. I just rarely see it used correctly.
Soft raid? If you looked at the performance impact of that, not to mention the very poor reliability, you'd stop...
Certainly, if your data set is large (lots of video) but of low value (redownload/rip again and your time is of low value itself) then certainly, it might make sense, but still considder that a second hard disk, externally, and rsync run via cron a few times a day, with history enabled, is a better solution and less impact on your I/O.
There is no contest in life for which the unprepared have the advantage.
what are the experiences using intel motherboards hardware raid 1 (for example P45 chipset)? I'm deciding my home network. I was thinking of a PC with Ubuntu Linux for server with two 1TB WD green power disks in raid 1 with the motherboard hardware controller. What happens if the mother dies? can I read the disks in another computer without raid? does software raid take a performance hit?
"I think this line is mostly filler"
a second hard disk, externally, and rsync run via cron a few times a day, with history enabled, is a better solution and less impact on your I/O.
That's good advice. If I were setting this up from scratch I'd probably do that (and may still).
I don't trust this logic. There are failure modes where mirroring makes it impossible to determine which drive is correct. Errors when writing sectors are not always reported, quite apart from the case where the power goes off and one drive has flushed its cache and the other drive hasn't. If one drive says X and the other says Y, which do you believe?
Even if you know which drive to replace, rebuilding the mirror with a replacement drive does a read on every sector of one disk and a write to every sector of the other. What do you think the chances of that operation completing successfully with today's large drives is? Hint: the rate of errors/sector hasn't improved much in the last ten years while the sector count has increase massively.
RAID1 is useless for protecting against hardware errors - people use it for the stellar read-performance and for no other reason.
sheep.horse - does not contain information on sheep or horses.
Well... then I did not RTFA. Another flaw of some of us on /.
I was rather hoping that he already had plans to do backups by some other means. If not then I agree completely that backups are more important than RAID.
A bit of a tangent but what about viruses spreading to your backups on USB or eSATA drives? So if you leave your backup drive attached all the time (or even some of the time) how do you keep malware from being copied to your backups or restored to your fresh machine? What is the best practice for the family PCs?
Not true. Try this: instead of creating a 4-disk raid, split your 4 disks into /, /home, /var, and /srv - you save a lot of overhead associated with parity info on both reads and writes, AND you have the advantage of only accessing one drive for any particular piece of information (whether for reading or writing) so your log file doesn't interfere with your database which doesn't interfere with loading shared objects which doesn't interfere with that image you're editing in your home directory. Lower cpu load, better drive cache usage, less head movement (only one drive needs to seek to write any particular piece of data), less wear and tear, and less heat buildup. Using the same 4 drives in a RAID results in less performance and more wear and tear, and a loss of 25% of capacity ...
If you're worried about data integrity, buy a second computer and back it up. Drives on the same machine are NOT a data integrity OR a backup solution. Neither are RAID - unless it's a distributed redundant RAID - and you'll have a second box in such cases, so what's the point?
The RC does. So did the Vista RC. You point?
You say you care about your data, and yet you are contemplating using software RAID with an RC OS, along with FAT32? This is like wearing a bicycle helmet to protect yourself during a round of Russian Roulette. (Motherboard RAID is actually software RAID done in the device driver... I would trust this even less than the OS's software RAID, given that the motherboard vendor's driver may not play nice with the RC OS.)
If you care about your data but must use the Windows 7 RC for some reason, what you really ought to be doing is putting the data on a separate box which is running a mature OS (WinXP, Debian Stable, Ubuntu LTS... take your pick). And for God's sake, don't use FAT32 -- use a journaling file system like NTFS or EXT3.
RAID is NOT backup
To begin with, every NT-lineage Windows version ever produced supports software RAID out of the box.
Except that it doesn't, really -- Even under vista Business x64 you can't create a software RAID device, you'll either need the Ultimate/Enterprise edition, or Windows Server.
Probably better to leave USB drives disconnected when not in use, to avoid wear, malware/virus infection and electrical surges.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
> I have decided that only my data is worth saving
Then forget about RAID. RAID is designed to protect the integrity of the underlying volume - NOT the data that's on it.
> Of course, such a setup should secure my data; should a drive crash,
Then forget about RAID. RAID will only secure your data under some very specific cases of hardware failure of the drive. It does absolutely nothing towards preventing data loss due to (say) a corrupt file allocation table, virus, accidental deletion, or corruption.
> Even more importantly, I want any drive and its data to be as safe and portable as possible
Then use proper backups - not RAID. Preferably off-site backup. I use Carbonite which backs up to the 'cloud' at minimal cost.
By all means use RAID to protect you from hard disk failure, but don't under any circumstances assume it stops you losing your data. For backups, I always use the rule that at any given point in time, assume that the next time you walk back into your house/office, that NOTHING in that building is still there. Do you have a copy of everything you care about somewhere else?
I'm still amazed by people that carry 12 months of work around on a single floppy disk/USB stick/laptop, then cry when they go to the helpdesk asking what "sector not found reading drive A:" means, or perhaps "A USB device attached to the system is not functioning".
Get your data in as many places as possible - preferably three. A drive which is mounted one inch above the main one is *NOT* a valid second place!
FAT32 is a huge mistake. First of all, there is no journaling. All operating systems, at least reads NTFS fine and with the new fashion NTFS-3G, you can write to them and with ''test disk'' or a Linux recovery boot CD, you can even repair them better than Windows ever can.
Don't use FAT. That is all I can say. Even if you buy a mainframe class IBM storage solution, it won't save you from evil FAT issues.
I would hope that their backup systems and failsafes are a hell of a lot more reliable than anything I could come up with.
Or have them burn it to DVD and posted out to you in a couple days.
(http://support.mozy.com/docs/en-user-home-win/faq/tasks/dvd_restore_faq.html)
In Windows 2003 at least Software RAID is not trustworthy. Especially under high loads, Windows will stop synchronizing the disks and then you have a secondary drive that all of a sudden boots into a 6 month old version of your system (if it boots at all).
If you have no other choice, I would go for the 'firmware RAID' - I have never had any issues except that it's not that fast and you might have issues with Linux bypassing the RAID controller altogether (but then again, software RAID on Linux is at least stable). You can get burnt with any RAID controller, that's why we have backups but any decent setup will usually work.
Custom electronics and digital signage for your business: www.evcircuits.com
Our company finally moved away from windows software RAID because of problems. One drive would fail, then while rebuilding we'd find errors on the second drive. Usually bad sectors that couldn't be read. Windows 2003 doesn't do any SMART monitoring so a drive can start failing and you'll never know until you try to read/write data to that part of the drive.
On my linux server I have smartmontools checking the drives nightly which has saved me once so far. I can't find an equivalent for windows so we've moved to hardware which can check for failing drives.
Outdoor digital photography, mostly in New Engl
In my experience, RAID1 performance gain varies quite a bit from solution to solution.
For a desktop solution, a system builder can get good results by stroking the drive(s) during O/S installation, then carefully choosing which partitions to use for their various needs (outer vs inner.)
And back it up! My solution has been to install a second drive of same-or-greater size from a *different* manufacturer. It stays spun down most of the day, and is started periodically for backups. (Yes, I've been lectured that drives supposedly last longer when spinning all the time, but I don't buy it - they're generating heat, are subject to g-force shocks, etc.)
I've been happy using rdiff-backup, but there are plenty of backup solutions for Windows, if the OP choose to be proactive about that.
O lord, bless this thy holy hand grenade, that with it thou mayest blow thine enemies to tiny bits, in thy mercy.
My experience is that NAS is a _lot_ slower than local storage for many applications. If he's a developer, for instance, he really wants local storage, otherwise he's going to be suffering substantially slower compile times.
Agreed there, but he never said he was doing development and:
Video encoding and DVD authoring likewise is noticeably slowed by access over 100Mbit networks.
Actually, IME, no. Video encoding is mostly CPU intensive so unless you are using some sort of hardware video encoder, the primary bottleneck isn't disk, it's CPU.
NAS; this is an expensive solution that provides suboptimal performance. If he has only a single desktop machine there are no real advantages to the approach, either.
Actually, NAS is not as expensive as you might think. Several devices with 2 bays are available for under $100. And performance may not be important for his needs. Even if it is, the D-Link 323 NAS, for instance, supports RAID 1 and Gigabit Ethernet OOTB and is available in this price range. A GigE switch just ain't that expensive.
My blog
...if you need to send a drive in for replacement, you don't need to worry about snooping of your data...
You are misinformed.
An unencrypted sector on a disk drive is just as readable, with or without a filesystem on a drive. A responsible IT administrator must consider data security when disposing of a disk drive, whether with a hardware vendor, the nearest trash receptacle, or eBay.
I've looked at raid at various times through my IT life, and concluded that, at least in hardware, it wasn't ready for prime time for much of these faults. At least not at the consumer level.
As a sysadmin, almost all of my restore requests for files were from people who shot themselves in the foot: Deleting a file accidentally. (Or for whom Microsoft had shot them instead. The dreaded "Your Powerpoint file is corrupt...")
Most users don't need atomic database consistency as much as they need protection from their own mistakes. Myself included.
So now, I eschew raid. I back up to anohter computer using rsync with some form of versioning. It with great pleasure I see the look of joy on a grad student's face when I tell him that he now has 3 folders in his home directory labeled "Tuesday", "Wednesday", and "Thursday" with the 3 latest versions of this doctoral thesis.
Third Career: Tree Farmer Second Career: Computer Geek First Career: Teacher, Outdoor Instructor, Photographer.
But if you believe the Vinum documentation you should choose a stripe size that is large compared to your average write. This way most transactions use only a single stripe. Alas for all it's elegant theory, I could never get vinum or gvinum to work for me for more than about 3 days without irrecoverable errors.
There is no reason that a RAID system needs to require all copies of a transaction to finish before the next one starts. Indeed the virtue of queued writes is that the file access doesn't even have to proceed in the correct order.
If you are concerned about file system consistency, you write a sequential number as part of each transaction. So if the power fails after copy A is written, but B has not, the sequence number tells the OS which one is the most recent.
Third Career: Tree Farmer Second Career: Computer Geek First Career: Teacher, Outdoor Instructor, Photographer.
Since no one else said it:
If you want to keep your data secure, you sure as hell aren't going to do it using FAT32. The argument in favour of doing so is, it will make it available to multiple operating systems. Thing is, it's probably the least secure file system you can find that will hold large files. It's just not a good idea.
That said, if this is indeed what you want to do, you need to use hardware raid. For no other reason than this: Windows software raid will not be accessible from any other platform than Windows. Going to be highly vulnerable to corruption though.
-1 Uncomfortable Truth
When there is a serious error detected via SMART, Vista will bug the hell out of you trying to get you to back up.
I agree with you, but...
1) not fireproof.
Are you suggesting a tape that is in your tape drive in the PC/Server room is going to be readable after enduring a fire?
5) more expensive than a good backup
Last time I was speccing out backup solutions (about a year ago), high volume DLTs and a drive costed significantly more than an array of similarly sized SATA drives.
6) not protable offsite
External HD cases and caddies are extremely portable.
7) does not track versionb history or old files (something that should be of critical importance to a programmer...
So long as you have a proper rotation of hard drives it does. Also, if you are counting on your backup solution as a code repository, you are doing something very VERY wrong. Using a backup system as a code repository is about as smart as using RAID for a backup solution.
8) Viruses, mailware, hackers oh my!
A corrupted file or virus that gets backed up will be just as potent as the one written to the RAID drive when you pull the data off of it. Mal-ware (I'm assuming you weren't referring to outlook) isn't going to be much of an issue as he should only be backing up data. And I'm not sure why a RAID configuration would make you any more at risk of hackers than a backup solution, although you may have better odds of finding a buffer overrun in one driver than the other.
9) bad/corrupt install
Same can be said for tapes and tape drivers.
Point being, RAID is not a backup solution, it is a tool for ensuring maximum availability of data. Now IF you have a proper backup solution (off site storage, data management, recovery plan, etc...) but you are too cheap to buy a new tape drive and media. Then yanking a mirrored drive every night from a software raid and sending it off to storage can work. But it should only be done with the understanding of the risks that it poses.
The most important part of a backup solution is not the medium. It doesn't matter if you are rotating tapes, disks, thumb drives, punch cards, or what ever. What does matter is that your data is being stored in a secured manner and that you have a TRIED and TESTED recovery plan.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
Sort of like this?
how is babby formed?
Sorry, I don't see this at all. Yes, I use Mozy, so I'm a little biased but if I had seen no use-case scenarios where it was helpful I probably wouldn't.
I pay for an unlimited account... there's currently about 90GB of data up there that I can think of... and yes, I've requested a restore on more than one occasion. Hell, there was a time I found I needed a particular file while I was out in Denver off my personal laptop which I didn't have with me (since it was a business trip). Yes, I can VPN into my home systems, but I also knew my laptop was closed and therefore asleep at home. I jumped onto Mozy and "restored" the file to my work laptop within 20 minutes at the hotel, modified it, used it, and copied it back to my personal laptop when I got home.
And bandwidth / internet limits? Uhm... OK... if you insist. Today is the 6th of July. Being a parent of two kids meant I took a lot of photographs on Saturday of my kids enjoying the day... and some video. Sync'ed to iPhoto on Saturday and then just left my laptop running while ~4GB of pictures got uploaded to Mozy. It's damned nice to know all that data is up there in the event I ever lost it. Yes, I also backup my critical files to a file server at home via rsync, but I find that less and less a necessity since Mozy has fulfilled my backup needs for some time. Oh yeah, and I periodically take an image backup of my laptop so that I can restore the entire system to a point in time... I just fire up Carbon Copy Cloner, plug in my Drobo and go out for the day :)
I am the exception to the rule I think; I keep LOTS of backups. But still, my point still stands; that Mozy and other online services are damned good value and don't suffer from the perceived problems that you mention... at least not in my experience.
If what you want is data security, you have a desktop system with just a handful of disks, and you're worried about an unreleased operating system trashing your files, what you want is not RAID, you want a backup strategy. RAID is for fault tolerance. It protects against hard drive failure and nothing else. It will not save your butt after an OOPS moment, or when your shiny new beta OS decides to write zeros all over your filesystem's critical structures.
If you really want a safe, reliable backup system, build a dedicated backup NAS, use RAID on *that*, and dump all your important files on it via a mapped drive or FTP. If you're really paranoid about your data, you can have the NAS run a nightly task to tar/zip and rsync the whole thing to a remote server or Amazon S3.
Not trusting RAID is the smart thing to do, because it is not about trust, it is about making a sysadmin's life easier by adding a little tolerance to the least-reliable hardware in a server. Trust/security is not a piece of software, it is a methodology.
-Billco, Fnarg.com
I'd say the major one is that changes to your primary device are replicated immediately to the secondary - I could fish for figures on the percentage of 'outages' that are operator error, but I can't be bothered. Suffice to say, it's the majority of them - deleted the wrong thing, let their kids at it, dropped it, that kind of thing.
RAID is fault tolerance - it protects you against a drive failure. That's all. It doesn't allow you to recover your data when it gets deleted/corrupted.
Who said backup? People see what they want to see.
BackupPC server: Linux SW RAID-1 (3 way, leaving a DVD slot) mitigates bad-blocks.. On first smart error, throw the $30 disk out and plop in a new one.
Desktop: Linux SW RAID-1 (2-way).
External USB/NAS for the BackupPC server.
Judicious manual use of subversion for local configuration files in read-only checkpoint mode (so history never changes and thus is safe/efficient to backup). This is slightly redundant with BackupPC's version-checkpoints, but allows a compact infinite history, and visual deltas.
Occasional off-site rotation. (possibly using Amazon storage).
For servers, at my work, we do almost the same thing, but use hot-swappable hardware raid controllers with RAID-01, and we use LVS snapshotting for consistency (every try to rsync actively changing data?) and for apps that support it, explicit dump operations (like mysqldump) to produce wholly consistent point-in-time backups. For our light-but-real-time servers, we ALSO use drbd (block-level network replication) along with heart-beat.
Frankly, I'm rather surprised by the backlash against RAID here.
-Michael
I have. We've got a couple of bit storage arrays which I look after, and it's quite noticable that we sometimes have 'waves' of drive failures - a statistically improbable clumping of drive failures, and then a substantial period of 'quiet'. Upon investigation of this happening it tends to be the case that the drive failures are all from same batch drives - because subjected to very nearly the same access pattern and stresses, it should come as no suprise that they all fail at approximately the same time.
Are you suggesting a tape that is in your tape drive in the PC/Server room is going to be readable after enduring a fire?
No, he's simply saying that you need to store backups off-site otherwise they can get taken out along with the original data. Only an idiot would buy an expensive tape drive then just leave one tape in there all the time.
which is totally what she said
Are you saying two drives of the same batch are going to fail WITHIN THE TIME IT TAKES TO REBUILD THE RAID?
And yet, we had exactly this happen. Actually, the second disk failed when the system attempted to reboot after noticing the array was degraded. Two disks failed within seconds of each other.
You can - if you take two functionally identical pieces of hardware - which 'same batch' tends to imply. And then subject them to very nearly the same environmental and stress patterns - which being in the same array, with the same workload tends to imply - then your probability of multiple drive failures is substantially increased. Mean time between failure is fine for an isolated case, but a large (1Tb+) RAID 5 set will take a substantial amount of time to rebuild. I'm not saying a disaster is guaranteed, but I have seen in real world practice the statistical 'clumping' of drive failures on one of our storage arrays, as all the drives in a batch start to hit the limit of their wear and tear.
Of course, that assumes you hot spare (and we do) - but not all home systems can 'afford' the overhead of a hot spare - which in turn increases the window in which your second failure can occur.
Software-driven RAID 1 (by which I mean RAID driven by the software in your filesystem, as per Windows/Linux, or the software in you machine BIOS and drivers, as per the motherboard-based RAID, as opposed to the software running on a "hardware RAID card" on a separate processor... ultimately, all RAID is software) generally has a very low overhead.... the extra writes do take time, but given the typical overhead of ATA controllers, at worst, you're doubling that (eg, 2% up to 4% or so), with practically no software load over that of a normal driver. Obviously, going to a "real" RAID 5 or some-such, that wants its own auxilary CPU.
But not is all golden, either. On reads, RAID 0 or 1 can actually increase your seek time, since your effective seek time will be the worst case across all drives in the array. You don't notice that streaming, since you seek less often, but for lots of small reads across many, many files (think 32-channel audio mixer), RAID may simply be a bad idea as a project drive.
-Dave Haynie
RAID is good if you're wanting a system that doesn't crash as often. Being able to hot swap, or 'fix' your RAID 5 in a maintenance window is preferable to having it go 'boom' in the middle of the day, and having to fix it NOW whilst your users are breathing down your neck.
We back up all our stuff none the less - we've yet to lose data as a result of a RAID problem, but we frequently have restore requests, due to 'accidents' - some user induced, some microsoft induced (seriously 'file synchronisation' and PST files. Ugh).
We also do a daily snapshot of the filesystem for point in time recoveries - this isn't part of our 'recovery strategy' but having the data online immediately for the last 7 days is valuable - most of our users need files that they have just deleted/corrupted.
But above all, every single restore request we get is not something that RAID would - or indeed could - have protected against - because the user _asked_ for the files to be deleted, and the computer complied.
While not a traditional RAID, I've been really happy with the Drobo unit I put in last year. This delivers RAID-like behavior with as few as two drives, which don't have to be identical (or even the same size). When you're using two drives, it acts like a RAID1 in terms of storage overhead; as you add more, it tends toward a RAID5 in overhead. One drive can always be removed... if you replace a failing drive with a larger one, the system adjusts (usually takes about a day to reconfigure itself). It can run in "supporting old FS limitations" mode... for example, on my XP machine, it configures itself as dual 2TB partitions... if I add enough storage, a third or four partition will appear (it also supports large partition models, but at least under Windows, that would also require Vista... no thanks).
The normal model runs over USB 2.0 and Firewire 800, so no, not as fast as an eSATA or SAS RAID, but good for ready availability of anything you might need that's NOT the current project.
-Dave Haynie
RAID's job is to protect from ONE and ONLY ONE type of failure, DISK failure, NOT controller failure. Any controller, regardless of type, can fail. That's why you should STILL do regular backups. RAID does not protect from viruses, spyware, operator error, fire, theft, or any other type of failure other than a single disk. It relies on the statistical probability that one and only one disk in an array will fail, and that is all.
Backups are floss. You only have to floss the data you want to keep.
-- I am. Therefore, I think!
Where do you get the 1/2 failure rate from?
Supposing the drive has a 10% probability of failure. Assuming no correlation, the probability of both drives failing would be 1%.
The poster seems to be stuck on raid, when he really wants a backup solution.
Apple has really hit it out of the ballpark with usability on their backup solution... but you need Apple hardware to use it.
I'm using a Airport Extreme with an external hard drive and couldn't be happier. It is very easy to go back and find old versions of documents, preview them, and restore them.
My setup consists of local backups to another set of hard drives along with a offsite storage (on a colocated server, encrypted). So if my house burns down or is hit by a meteorite and I lose both of my local copies of my data then I'll need to spend less than 5 days to download 1TB (@ 20Mb/s 4.85 days by my math). Of course personally I have nowhere near a terrabyte. And as you claimed, 600GB would only take about 3 days. I don't see what the problem is? If you need something sooner just download that first. I don't think anyone needs immediate access to the entire storage system, most is for archiving purposes.
I'm definitely open to better alternatives if you've got one.
Actually 70 double-layer DVDs, which would fit in a small CD wallet. I've got several CD/DVD cases on my shelf that are about 3x8x11 inches and hold about 250 DVDs. Not that I'm recommending it, just, as usual, you make it sound much more bleak than it is in actuality.
I personally wouldn't trust either of these environments. The vast bulk of my data is static, with usually only small or minor changes daily. Rsync CRC mirroring is how I handle it.
My setup:
1 TB main drive. All OS, config, and user data.
1 TB backup drive in same box. All OS, config, and user data mirrored from main drive nightly, using rsync with CRC checking.
2 - 1 TB backup drive in second box mirrored once a month. This machine is typically unplugged until in use. Each of the two backups are staggered two weeks apart, so that the oldest recent copy is no more than a month old.
Very important 'real time' data, on the order of 10 GB, are mirrored nightly off-site.
Note - CRC checking, in my anecdotal experience, is important. Modern drives actively look for 'questionable' sectors when reading; calculating CRC nightly ensures that the drive has to actually read every sector that's in use.
CRC checking mails you what has changed, which is easily filtered to make sure that you haven't lost something important.
Nightly cron job should dump any questionable SMART statistics from your drives. If any pre-fail counters increment, immediately consider that drive suspect and have a hot replacement ready.
How important is your information?
This is the level of data integrity risk that I am currently comfortable with. In a particularly nasty scenario involving my building being destroyed in an earthquake while I'm not in it, I have to fall back to my off-site data.
Alter Aeon Multiclass MUD - http://www.alteraeon.com
No, he's simply saying that you need to store backups off-site otherwise they can get taken out along with the original data. Only an idiot would buy an expensive tape drive then just leave one tape in there all the time.
True, but the same is true for any backup solution, regardless of the media. If you use tapes, you have to store them off site. If you use external hard drives, you have to store them off site. How ever you get your data to the media is immaterial if you just leave it sitting in the server room.
A tape back up does not offer any advantage over any other media in that regard.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
Yep, and he was saying nothing about tapes at that point, he was just saying "why RAID is not a backup".. in fact I don't think he mentioned tapes at all in that post!
which is totally what she said
As you said, you're probably an "exception to the rule". The typical computer user I run across finds the entire concept of "backups" to be over their head.
Typically? They don't even understand how to burn a CDR with a selection of files and folders of their choice on it. (They probably only use their burner to make music or video slideshow type CDs using software that makes it essentially a "one click" process.)
They need assistance setting up a backup solution, and whenever someone makes that solution an Internet-based backup package like Mozy, it winds up only working well for a limited time. Eventually, the user copies too much data into one of the folders selected for backup, exceeds their storage limit, and successful backups cease. They aren't savvy enough to figure out anything's wrong for days, and usually can't resolve it on their own when they do figure out backups are failing.
And yes, bandwidth limitations are an issue for many people. Some of the people I've helped out with their computer problems in the past are still on 56K dial-up! One of them is a guy who makes a living selling goods on eBay, out of his basement! You'd think *he* would see the value in upgrading to broadband, but alas - no.
Intel's ICH9R is no different than anyone else's software driven "RAID" controller for motherboards... Intel just didn't like seeing some of these small companies making that money, so they came up with their own chip. Ok, these may do a tiny bit more than a very poorly designed SATA controller.. they have per-channel DMA engines and maybe a few other bits that are useful in RAID, but good ideas anyway. But basically, the RAID driver runs on your host CPU... that's the basic definition of "software RAID". In a "hardware RAID", the RAID driver is running on a CPU dedicated to the RAID controller. It's also likely to be buffering blocks, so it doesn't have to do redundant DMAs for writes to mirrors, etc. (which isn't a big deal, since it's just a low bandwidth these days... the big overhead is in dealing with RAID5 checksums, that sort of thing).
A hardware RAID should look just like any other single device, and not require any OS-specific drivers. Intel's will have software RAID drivers in the BIOS, but since most OSs can only boot from BIOS code these days (they can't run 16-bit real-mode drivers in 32-bit protected mode), you need to load up the drivers to allow the RAID to be recognized by your OS.
Not that I have a problem with software RAID for RAID0 or 1... the overhead really is pretty low. Just realize what it is you have there.
-Dave Haynie
Years ago, I had this computer system without backup. Ordinarily, no big deal, but I had been working at a new startup company, and had our first six months of data on this PC... on a terribly expensive Seagate Barracuda 2.1GB SCSI Multimedia-capable 3.5" drive (this was the mid 90s).
So I set forth to find a backup solution... Travan was all the rage then, particularly on the floppy port, so I bough this TR-3 drive, hooked it up, proceeded to backup and !BANG! The BSOD. Again, with the same results... my PC didn't like the floppy-interface tape at all. So, on back to CompUSA, and I exchanged it for some new, proprietary tape backup, from Sony or someone.... slightly more cash, slighly higher data capacity. Take it home... same thing. Ouch.
So, not quite desparing, I return this unit and shell out the big bucks for a Travan TR-4 drive, for SCSI. Hook that baby up, run backup... and hours later, wow, I'm backup up.
A week later, the terribly expensive Seagate Barracuda 2.1 GB SCSI Multimedia-capable 3.5" drive dies, never to be heard from again. Since then, it may be tapes, it may be CD, DVDs, or BD, external drives, or online, but I have all critical stuff (and even some of the trash) backed up regularly. This also demonstrates that, when you think about doing a backup (if you're not automated), you should do it right then... the universe may well be sending you a message.
-Dave Haynie
You are completely right. I am a complete idiot, and deserve to be modded down. Mod parent up.
As many people point out RAID isn't backup which is something I've taken to heart. But what I haven't been able to find is a good solution to do cross platform backup.
I have three machines which I'd like to utilize in a backup scheme. One machine is a linux box with a large SW RAID5 setup I'd like to back up to. The other two machines are my clients one is running WIndows, the other Mac OSX.
Are there any cross platform backup solutions which will allow me to back up files to the Linux machine from my Windows or OS X machines? I've considered using rsync with the directories which I want to backup, but that will hose files if something gets corrupted on the client box. I'm pretty sure rsync will happily sync my backups to the corrupted file. Plus I don't really have any history as rsync will keep the two directories up to date, but won't generate snapshots. (Well not directly).
I could just occasionally tar/bzip up all my files, but I don't really want to have to manage gigabytes of tar files. Especially as it would be nice to be able to easily recover a particular version of a file without hunting through tons of archive files. Plus tar.bz2'ed files don't really support accessing a single document in the file.
I've also considered just putting all my files in source control with svn, git, or perforce (free for 2 users) then mirror the repo but that doesn't seem to be the best solution either. I'm not sure what the overhead of the various source control systems but at least a few of them support binary diff, and they have versioning built in. But I would like to avoid having to check out, and check in files on the client. This solution also doesn't lend itself to making backups very automatic.
So is anyone aware of a cross platform backup solution (Windows/Linux/OS X)? Which would allow me to mirror/store the backup files on any of the systems that doesn't have the overhead of making a TAR file per day?
Optimally I'd like a solution where I can configure specific directories to get backed up, and have backups occur as some sort of timed cron job. I also would like a way to browse/recover past files without having to mess with manually extracting files from an archive. An open source solution would be nice so that I don't have to worry about losing access to my data in the future, but I have no problem paying for such a backup program.
I don't think the OP believes only Windows 7 is the only OS that supports soft RAID, but rather he just built a new system and is contemplating on the best solution to add RAID to his system... his question is not "what operating system should I use" but "what raid solution should I use on my Windows 7 system".
The Binary Anti-Pattern [http://beyondboolean.blogspot.com/]
I attached the drives to my Macs with a firewire cable configured in mirrored mode. I was using TimeMachine. Things work for a while, then the drive stops responding. The drives were powered with an APC power system and were not allowed to get hot. They were not in continuous use but simply used for backup. When the product showed up on the shelves at my local store, I was very excited about the FW800 and dual drive mirrored functionality. I bought one for each of my computers as a "safe place" for my data. The data remained safe until the drives just quit within several months. I cannot imagine what I might have done wrong with them other than depend on them to do what they were advertised to do.
Soft raid? If you looked at the performance impact of that, not to mention the very poor reliability, you'd stop...
No, if you actually looked at the performance and reliability in comparison to hardware RAID - which you have obviously not - you'd be crazy not to use software RAID.
There are some good reasons to use hardware RAID. Performance and relability are not two of them.
I would highly suggest doing some testing with Windows and raid before you even bother using it for speed (Hint: it does almost nothing in a desktop or workstation). All it does is create more complexity in the system and chance for failure. Windows has never been known for taking advantage of raid for speed unless it is for database use. How is mirroring going to improve speed anyway? Even if you use a mixed raid system, any gains from striping will be lost from mirroring unless you have enough disks and a nice card. Do some investigating you will see. There are a few reasons why I say this creates more chance of failure, beyond the obvious more parts. If one drive fails, yes you can just replace it, drives are pretty stable these days, but what if they are both from the same batch, odds are they will die within a very short time span of each other. Better datacenters will vouch for this. Then there is the quality of the raid itself, if it is a built in setup you may want to experiment with recovery BEFORE you have a failure, I have seen more than one refuse to recover. What if the raid system itself fails? Again this makes recovery even more difficult. One bone of contention I have here as well is not only the threat of badware out there, but where I live, lightning is a consideration. I don't care how good your raid is, a voltage spike or bad power supply and your raid becomes a big lump of melted metal. Yes there are backups claiming lightening protection. I have one, even the manufacturers will admit there is no guarantee, which is why they come with insurance in case of failure. Windows software raid makes some of these problems easier to deal with. The problem is, none of us want to really recommend that method. My advice, unless this is for a server, forget the raid entirely and create a good backup system that lets you recover everything fast.
That's not a very helpful calculation. The OP is concerned with the possibility of data loss due to drive failure, which is a function of MTBF of the drive and his MTTR of replacing a defective drive (MTTR includes the rebuild time for simplicity). In other words, he won't lose data unless the second drive fails during the vulnerable window of time before the first defective drive is replaced and rebuilt. A better question to ask perhaps is what is the MTBF of Windows (and/or the application software and/or the user and/or malware...) not destroying the data partition (and it's mirror). Probably happens 1000 times as often as a dual-drive RAID-1 disk failure. My suggestion would be to have good backups, and for really critical stuff store a copy in a USB drive at some other location.
Having done RAID many times in different ways over the years I'd say that as long as the version of Windows 7 you plan on running supports it, I'd do it. But.. be warned! There may be issues.
For one, I ran into one copy of Comodo firewall that completely blew up the networking stack on XP if dynamic disks (required for Windows software RAID) were present. Also, any BSOD will be followed by a pointless RAID rebuild where it completely copies the contents of one of the drives to the other slowing disk IO for a long time. Also, the entire disks including partitions that aren't RAIDed will need to be configured as dynamic disks which can cause issues accessing them from DOS and Linux. Not usually an issue but it's worth pointing out. One of the things I liked to do was install an OS on each drive that was bootable and have the Stripe/Mirrored partitions accessible from both OSs. If you are going for this type of configuration there is something important in the order with which you create the partitions and convert the disks to dynamic disks. I think you have to convert each disk to dynamic from the OS that boots from that disk in order for it to remain bootable. If OS that boots from disk 1 converts disk 2 the OS on disk 2 will be rendered unbootable (if I remember correctly). If you are going for this type of config, install each OS and have it convert it's disk to dynamic, then create the mirror/stripe partition(s). Other than that, the ability to put the disks in any Windows machine and access the data makes software RAID the clear winner here.
In response to all the posts about how better to safeguard data by backing it up, he's not asking about that. He's asking about which way we'd recommend doing RAID. Your suggestions are off topic. Every time RAID is discussed the same arguments are made. Backups aren't RAID. RAID isn't a backup. Enough already.
I've done software RAID for a long time and I'm a firm believer that it's the right way to do RAID in a lot more cases than it's typically used. Hardware RAID has many proponents and is obviously a profitable industry so there is a lot of money being spent based on it's perceived advantages so I'm in the minority here but that doesn't mean I'm wrong. Expensive RAID array's advantages are getting harder to justify now that Serial ATA has brought dedicated per-drive bandwidth, fast cold (if not hot) swapabililty, and now that multi-core CPUs can handle the load of the software RAID work gracefully. The niche where software RAID makes sense is getting bigger every year. Many people argue that software RAID is too slow, that you need the hardware controller to offload the calculations. I'd like to throw out the fact that I use an 8 drive Linux based software RAID 6 array for my primary storage. Bonnie++ clocks it at 215.2 MB/s on block writes and 263.5 MB/s on block reads. CPU utilization is 45/31% on those respectively but with a quad core machine, using half of one core to do my IO processing when it's writing at that speed is perfectly acceptable. Remember, this is RAID 6 with 2 differently calculated parity chunks, not goofy simple RAID 5 and it's CPU is just a AMD Phenom 9600 Quad with 8 gigs of RAM, not something exotic. I run many virtual machines on top of this array simultaneously and they are nice and snappy. The configuration works quite well and I'd recommend it for a inexpensive, high speed virtual machine server configuration.
For my Windows desktop, I'm currently using my motherboard's Intel Matrix Raid capabilities and have configured half of my drive as a mirror and half as a stripe. The setup has worked seamlessly and I'd recommend it as a reasonable alternative to software RAID. It's slightly cleaner from Windows's point of view and the Intel Matrix controllers are fairly common and, from what I understand, I could put the disks into any machine with Intel matrix raid and access the data in a pinch. I honestly probably would have done software RAID had Vista supported mirroring.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
Amazon's S3 has been around for some time now and will likely continue to hang around for a while. I'm using JungleDisk to provide encryption and a nice interface to it. The nice thing about using S3 is that you don't get an allotted amount of disk space or transfer, you just pay for what you use. A second backup of everything that I wouldn't want to lose (updated nightly) costs me about $5/mo. That's for ~40 GB storage and the incremental transfers.
It came in handy after a theft left me with no physical copy of some of my data. That month, after dumping about half of my stored data back to my home, my bill was a whopping $8.
Being that inexpensive, encrypted, and with an automated backup solution, I find that the WAF is really high. It was her computer that was stolen, too, so she's totally sold on the idea of backups now!
If you want a vision of the future, imagine a youtube comments section scrolling - forever.
What you are looking at is the probability data loss happening in a particular period - the critical window. If you have a single drive that is the probability of that drive failing. If you have RAID-1 storage, then it is the probability of both drives failing at the same time.
The probability of one drive failing is pF. If the correlation is 1 then the probability of both drives failing is also pF. If there is no correlation, then the probability of both drives failing is pF^2, where pF is expressed as a fraction. If there is some correlation, then the probability of both drives failing is somewhere between the two extremes.
I think we understand that software, viruses, stupid users etc see the RAID as a single c: drive or whatever, and any damage they cause will be caused to both drives.
For every idot who's lost data from a RAID array there are 10,000 people who've lost data from drive failures.
Despite everyone's constant loud claims to the contrary, there is a class of data that is important enough that you not lose it to want it on a RAID array but not important enough that you not lose it to want to back it up. For me.. the line between these is gray but a basic rule of thumb is anything on a machine that I didn't create myself (i.e. that can be recreated fairly easily from external sources) does not need to be backed up. This class of data is probably roughly 99% of the data on most people's desktops. For this class of data, losing data is an annoying and time consuming headache but it's not catastrophic. Also, in a lot of cases like installed OS's and programs, backups are of limited use unless you can restore from scratch (which most people can't) In my setup I have a RAID 6 array backed up to a RAID 5 array that holds my unique data. Both my and my wife's primary desktops have mirrors as well though to keep the machines running reliably.
So, to those people who say RAID does nothing to prevent data loss.. you're obviously wrong. Stop talking.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
No, he's simply saying that you need to store backups off-site otherwise they can get taken out along with the original data. Only an idiot would buy an expensive tape drive then just leave one tape in there all the time.
Yep, and he was saying nothing about tapes at that point, he was just saying "why RAID is not a backup".. in fact I don't think he mentioned tapes at all in that post!
And only an idiot would leave a hard drive with backup data on it sitting there all the time. His point is valid in that ANY backup solution is worthless if you don't get the media secured. But that isn't a reason against using a RAID mirror to populate that backup data.
Using a RAID mirror to populate the backup data is a process. The hard drive that is populated is the media. Using hard drives as backup media is not a problem. Using RAID mirrors to populate that data is the problem. More accurately, recovering data from a RAID mirror is a HUGE problem with significant risks.
There are very good reasons to avoid using a RAID system in your backup solution, but the GP missed the mark on many of them.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
And spend a month trying to download it when your system fails and you need a full restore...
Still a lot sooner than "never".
You would "think" Big firmware patch for alot of SCA SCSI drives and some SAS drives were for the TLER issue. Seems even enterprise drives had that number to low:P
I don't know why you're going on about that - he didn't say you shouldn't use it as part of your backup system, only that by itself it does not constitute a backup, and that is IMO the most important point to take from this whole page discussion. Too many people think RAID is a method for securing your data, that's how it was first introduced to me anyway.
Don't think he even said anything about the pros and cons of different media. It's unrelated to the whole RAID situation, you could technically create a RAID setup with any type of media.
which is totally what she said
there's a penalty compared to writing different data on different drives. Example, if your log files are on one drive, and your web server files on another, and your database on a third, you have a lot less head movement. You also are using 3 independent caches, instead of filling the 3 caches with the same data.
Same benefits for reads.
Okay, then RAID just dropped significantly in value if it's primary purpose - to allow recovery from disk failure - is "statistically likely" to be unable to be fulfilled because two drives fail within the time required to rebuild the array.
I find this ridiculous. I think if you did a study on HOW OFTEN THIS ACTUALLY HAPPENS you will find it happens in such a low percentage of cases - probably under 1% or even 1/2 of 1% - that it's not reasonable to be concerned about it.
Shit happens - but not all the time.
If the paranoid truly think this is a problem, then they need to be using RAID setups that allow for more than one disk to fail or they need to be using continuous backup, replication and fail-over techniques with multiple machines. You get the same result as RAID without the headaches - unless of course you think the same disks in multiple machines are going to fail at the same time. In which case, the claim becomes "there IS NO way to prevent data loss".
Add in the problems with backing up and restoring RAID volumes in the event of operating system failures, especially in the case of dynamic volumes, and it would seem RAID is highly over-rated as a useful technique for recovering from data loss.
If you believe the paranoids.
Richard Steven Hack - This sig is TOO GODDAMN SHORT TO DO ANYTHING USEFUL WITH! MORONS!
Linux software raid is fantastic.
The article smacks of false dichotomy. There are a number of solutions, not just Windows 7 or a hardware RAID controller.
Seriously? You have no idea why Windows 7 is the OS he decided to go with so you have no idea if his reasons are valid. Is it too hard a mental exercise for you to perform to take that requirement as a given?
Also.. no desktop OS has supported mirroring for the last 10 years. Vista doesn't, XP doesn't, I'm pretty sure 2000 didn't. Apparently Windows 7 will.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
I dont think you understand what RAID1 is for.
There are failure modes where mirroring makes it impossible to determine which drive is correct.
No there arent. There's always a primary, and unless it fails, its the correct one, by definition.
Errors when writing sectors are not always reported,
This is not a failure mode RAID1 is intended to protect against.
quite apart from the case where the power goes off and one drive has flushed its cache and the other drive hasn't.
If this happens, then that means the system's/rack's/room's battery backups didnt work, and the raid card's onboard battery also didnt work. This is not a failure of RAID1, its a failure of your systems design and maintenance.
Even if you know which drive to replace
You always know which drive to replace. The RAID card tells you. On most systems, its the drive with the red or orange light on it, instead of the regular green light.
What do you think the chances of that operation completing successfully with today's large drives is? Hint: the rate of errors/sector hasn't improved much in the last ten years while the sector count has increase massively.
The chances of it completing successfully before the other drive fails is usually quite high. If your rebuild times are so long that you experience significant risk of failing the other drive during rebuild, then you need to use smaller drives, or some other approach.
RAID1 is useless for protecting against hardware errors - people use it for the stellar read-performance and for no other reason.
This statement shows that you dont understand what RAID1 is for. No RAID solution is, by itself, intended to protect against bus errors, undetected write corruption, cosmic-ray induced bit flipping, or other forms of corruption at that level.
RAID1 provides availability. It allows your machine to stay up and keep going if a drive fails. You also get some concomitant improvement in read speeds, but thats not usually the primary reason, its just a nice side effect.
You can't count to break a hardware-managed mirror, take one disk to a standard SATA controller and get any data out of it.
Actually, that _always_ works, at least with mainstream raid controllers.
RAID-1 mirroring (and only RAID-1 mirroring) does not write the data in a proprietary format on the drive. The only difference between a mirrored drive and a regular drive is that most decent RAID controllers will write the volume configuration to every drive as well.
Now, what you say is absolutely true when dealing with any other raid type.
Note that there may be odd or really crappy controllers that do use a proprietary format for RAID-1, but thats not the case for all the mainstream cards used in intel servers.
hmmm, sounds a little like you are agreeing with me there. Failing to protect against write failure sounds like a pretty big deal.
Look, the whole point of any RAID is to provide continued availability in the event of component drive failure. RAID1 fails to do this reliably for the reasons I outlined. Read speed is the only reason to use it.
However, you have just taught me a new word.
sheep.horse - does not contain information on sheep or horses.
I find this ridiculous. I think if you did a study on HOW OFTEN THIS ACTUALLY HAPPENS you will find it happens in such a low percentage of cases - probably under 1% or even 1/2 of 1% - that it's not reasonable to be concerned about it.
Anecdotal evidence: It's happened to me 3 times in the last 10 years across ~140 Intel-style servers, all with hardware RAID controllers and Direct Attached Storage (DAS). One drive fails, the rest of them get extra-stressed, causing one or two more to fail. The result: Hours of downtime while you restore from backup. Leading to cranky users. Bah, who needs their old business anyway.
IBM's web site even has instructions on how to try to recover as much data as possible from a dual-drive failure on their ServeRAID cards.
Having a distributed hot spare *should* help in theory. Even so, hot-swapping a defunct drive in a hardware RAID on DAS is something I prefer to do outside of prime time hours, and I strongly recommend checking the status of the last backup before proceeding.
--
.nosig
I am agreeing with you here ;) But there are two different schools of thought on the "RAID as a backup" solution. The first, is the completely idiotic, "I'll mirror the drive and call it a backup". This solution is so obviously poor that only the dimmest would ever consider it even worth mentioning.
The second is the "I dun learnt about backup solutions so lets mirror the drive and pull it every night for an off site rotation." And this, while better than the above idea, still fails hard as a backup solution.
If you ever get into the RAID as a backup argument with someone, and you immediately point out the problems with just leaving the mirror in the bay, they will quickly advance to applying the second argument.
So what I'm saying is that many of the GGGGGP's arguments against RAID as a backup fail as soon as the idiot puts the drives into a rotation. So it is important to identify which of those arguments are related to a good backup process and solution and which are related explicitly to RAID.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
He didn't say anything about RAID-0, he explicitly said "mirror" which is RAID-1. And the reliability of N drives becomes 1-(1-r)^N, or in layman's terms the probability of both drives failing at once. If I use your 95% figure, then reliability is 1-.05^2, or .9975. Since the probability of the software or the power supply failing is higher than that, not to worry.
The real point is that backups are to protect the data, RAID is to keep the system up if the hardware fails. In this case I think all the people who said get a backup instead are just right.
My job is is a professional raid engineer. "Burned by a not-so-cheap, dedicated RAID controller" -please explain, usually people are burnt by software raid, because they think its true raid and it isn't. All motherboard raid and even cheap raid cards are software raid. Cheap software cards often only contain one chip, same as found on motherboard raid, so don't buy one if you already have it on your motherboard. Avoid.They are NO GOOD. Trust me. If a drive fails you'll be lucky to keep your data. You're better off using discreet drives and copying date between them than software raid. RAID !- backup as other posters have stated. Good hardware raid such as the 9650se series 3ware or lst 87** series cards are good and will protect data but expensive for the home user. Use single fast drive for os, and proper raid card for your data, if you can afford it. hope this helps.