Slashdot Mirror
RAID Trust Issues — Windows Or a Cheap Controller?
NicApicella writes "My new system has two sparklin' SATA drives which I would like to mirror. After having been burned by a not-so-cheap, dedicated RAID controller, I have been pointed to software RAID solutions. I now stand in front of two choices for setting up my RAID: a Windows 7 RC software RAID or a hardware RAID done by the cheap integrated RAID controller of my motherboard. Based on past experiences, I have decided that only my data is worth saving — that's why the RAID should mirror two disks (FAT32) that are not the boot disk (i.e. do not contain an OS or any fancy stuff). Of course, such a setup should secure my data; should a drive crash, I want the system up and running in no time. Even more importantly, I want any drive and its data to be as safe and portable as possible (that's the reason for choosing FAT32), even if the OS or the controller screw up big time. So, which should I choose? Who should I trust more, Microsoft's Windows 7 or possibly the cheapest RAID controller on the market? Are there other cheap solutions?"
Do you really want to trust Windows with your data?
Comment removed based on user account deletion
You data is most important and you plan to use FAT? Good luck with that!
Seriously, though. No RAID solution that is not totally S/W is portable. But do you really need RAID? It sounds like what you need is a good backup solution with frequent backups. Does you data change so much that losing one day's worth of data would be a problem?
The real "Libtards" are the Libertarians!
RAID is not a backup. Get a backup solution or you'll realize you can be even more frustrated.
If you want data integrity, use NTFS. Using Fat32 is like saying you want a reliable car, so you're buying a Edsel because they've been around a long time-- it doesn't make sense. Every other OS on earth can read NTFS (if not write it), so it won't affect your portability requirement.
Secondly, before you make any decision regarding Windows 7 RAID, make sure the edition of Windows 7 you want to buy ships with software RAID support before you put all your eggs in that basket-- early betas and RCs of Vista had software RAID enabled, only to have it disabled before release. I've seen no guarantees about Windows 7 software RAID support, and which editions will have it enabled. (If any.)
If you're planning to move to a server OS after Windows 7 expires, I can practically guarantee software RAID will be enabled, but that still doesn't mean you can necessarily upgrade your Windows 7 software RAID array to a Windows Server software RAID array. Do your homework.
Comment of the year
You sound like someone that need to be reminded that RAID IS NOT BACKUP! Google for that sentence. All you talk about is saving your data, and RAID will not do that for you. You'd be better off just using the second drive as a backup. RAID will not save you from accidental overwriting of data, corrupt filesystems, broken chipsets, etc. The only thing RAID will save you from is downtime. If you lose that much money on the downtime it takes to recover from a backup, then by all means, use RAID, but don't treat it as a backup solution that will protect your data. That's not what it's made for.
c++;
These motherboard "raids" are called fakeraids.
All that it is is that it writes the metadata on the disk in specific format so that you can see the raid volumes via BIOS. Note: Only "see" their status - in case you replace one drive, the resync is still done by software and you must boot to operating system. One clue is the fact that in Linux the dmraid package uses exactly same driver for accessing fakeraid-mirrored drives and Linux's own software-raids - device mapper just does a bit of magic at init.
However, if faced with choice of Windows-only or motherboard-raid, I'd go with the motherboard-version, because that's at least supported both by Windows and Linux so in case something goes wrong with your Windows installation you can always pop in Knoppix or some other Linux CD for recovery.
RAID1 serves only one function. Increased uptime. If avoiding having to spend 2 hours restoring from a backup is your primary goal, then RAID1 might make sense for you. Do you have an office full of workers that will all lose productivity if you have a system crash? If so, then RAID may make sense. Any other use of RAID1 is fool's gold. It will not protect your data from a system-level problem. It will not protect your data from corruption (especially not on a FAT32 file system, which was never intended for any partition size above 32GB in the first place). It will not even always protect you from a single drive failure, since the rebuild process in a RAID1 setup often kills the second drive while trying to recover data. As many have said already on the thread, RAID is not backup. Backup needs to be a completely independent device. Unless you have serious uptime considerations, RAID1 should not be part of your backup strategy.
RAID is no substitute for backups. RAID is very good at propagating errors and problems very quickly, be they software glitches or human errors.
For consumer class storage, weekly / daily backups might be more efficient than investing a lot of effort into live RAID. Since I'm a Mac guy, I see the best answer to this question as Time Machine to a network / USB attached drive -- hourly (configurable for more or less often) differential backups, almost transparent to the user. To my knowledge, Windows has no similar set of software to allow reinstallation to the last hourly backup -- my wife had the misfortune of having to restore a blank drive from her last backup and it was a flawless process that truly left her where she left off less than an hour before the hardware failure. The reinstall wizard just had to ask where the backup was. Casting aside MacOSX advocacy, there is truly no substitute for a good automated backup solution that is regularly tested. I think the best method would use the fewest common components, like a NAS, followed by an external drive with its own power supply. My least favored option would be an internal drive with every single component shared.
The only way to keep your data secure in any reasonable fashion is to make a copy of it and store it offline, off site. Ideally "off site" would be in another building or city, but it at least has to be on something not attached or accessible to your computer.
Without regard to if you use software or hardware RAID or the quality of the RAID system, RAID only protects you from a physical disk failure. If you as a user screw up (delete or change something you didn't want to) or if some software bug screws up for you, or if you have a non-disk related hardware failure (causing a data corrupting machine crash) then you have lost your data -- RAID doesn't help.
Even if you are only trying to protect against disk errors, if the RAID system fails (even expensive quality ones can), or if you don't know and follow the recovery procedures EXACTLY, you can lose all your data.
The only reliable solution is making a copy or a "backup". Backup does not mean making a copy of the data on the same machine. (Whatever took out your RAID might also take out the other non-RAID disk or directory that you put your copy on.) If you are paranoid (or just prudent) your backup should not be a mapped or mounted drive on another machine. (Viruses can write to the network as well.)
And finally... Backups only count if you have tested your restore process.
That's nice, but the submitter is asking about RAID 1.
Actually, it depends on the reliability. 95% reliability becomes 90.25% reliability. 50% reliability becomes 25% reliability. 1% reliability becomes 0.01% reliability.
So if your drives are very reliable, it's very slightly less than twice the failure rate. If your drives are not reliable, then it asymptotically approaches an infinitely greater risk of failure.
Statistically speaking. :)
Consumer editions of Windows only ever supported Software RAID1. I've made a few experiences with SW RAID1 on WS03, and it's pretty much crap. Linux SW RAID on the other hand worked fairly well.
RAID is not a backup. This is the most important observation. RAID is a high availibility feature. If you lose your RAID array, you shouldn't lose any data. If you do, your backup strategy sucks.
Generally, skip RAID in a consumer setup. RAID is complicated, it's a PITA and especially the low end stuff can do more harm than good. Even expensive stuff can fuck your shit up (I'm looking at you, ServeRAID 8k). Better in invest in a proper backup - to a local harddrive and maybe offsite. Online backups make sense in a home office. For servers, i recommend LTO tapedrives.
RAID is only marginally valuable. In my experience, for all but the most carefully controlled environments, RAID simply adds complexity, the number of things to go wrong increases, along with the likelyhood of lost data. Do it only if you want the *experience* of running RAID, but don't count on RAID to "save your data".
I've worked as a system administrator for more than a decade, in medium-large scale deployments with good success, (think: servicing thousands of users, hundreds of domain names, tens of thousands of email addresses, etc) so I think I have some useful experience you can benefit from.
IMHO, you most likely to lose data from the following things (in order)
1) Aw sh1tz. "I didn't mean to delete that folder"... or "Whoops! I formatted the wrong drive", "I saved the wrong version of the file!", whatever. Although I *myself* don't have this happen often, it does happen. And even in my case I've lost about as much useful information this way as by drives dying. Users delete stuff all the time, and it's usually my job to bring it back, which is why I perform redundant, historical backups EVERY SINGLE DAY.
2) Malware. Don't minimize this - it's real, and it's why I reply to Parent. You are more likely to lose information from a virus/worm/malware and/or b0rked install of something that hoses your filesystem than by a hard disk crash given stable hardware.
3) Bugs. Filesystems have bugs. So do applications, utilities, anything with software. Strange, unexpected conditions, often caused by bugs in applications can cause data to "disappear", files to get corrupted, filesystems to get corrupted, folders to be incompletely written, etc. This is about as likely to cause lost data as:
4) Hardware failure. This is one of the lowest orders of lost data, although when it happens, it can be one of the most extreme.
Let me say this: RAID 1/5 only PARTIALLY protects you from the last one. Actual, bona-fide backups protect you from all of these. If you care about the data, get backups. If you care about uptimes at great expense, RAID *may* be worth it.
My advice is something most people don't want to hear: for personal use, get backups online for $5/month. Mozy/Carbonite/etc. There are zillion vendors, just Google it. In two years, it will cost you about as much as that 2nd hard drive. It protects you far better than that 2nd hard drive, and it's so automatic that you'll hardly notice it until the moment it actually matters: when you just have discovered that your data is gone.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
The article smacks of false dichotomy. There are a number of solutions, not just Windows 7 or a hardware RAID controller.
To begin with, every NT-lineage Windows version ever produced supports software RAID out of the box. Add that to the fact that any major Linux distro today supports software RAID. And so do the *BSDs. And Mac OS X. And Solaris. And probably a bunch of other platforms I can't think of right now.
Hell, you could buy one of these one of these and throw the drives in it, connect it to your network switch, and presto -- instant RAID+NAS.
I think we would all like to know why you think Windows 7 is your only option, because if that's what you think, you don't know how mistaken you are.
My blog
With RAID mirroring, if you overwrite or delete an important file, it's copy on the mirror is immediately overwritten/deleted too, and the file is lost. Wouldn't you rather need a good regular backup?
And as someone pointed out already, FAT is really not a reliable file system. If you are on Windows, use NTFS. It is still portable, having read/write drivers for both Linux and Mac (see this guide).
Since the files you want to keep safe appear to be regular files, not system files, any simple file copy mechanism could do. For an easy and simple system, you can use the Windows robocopy.exe tool in a batch file. For a more sophisticated system which can keep older file versions, and can easily be adapted for use over the network, you could try a Windows version of rsync like cwrsync. There are also a few rsync GUI frontends for Windows.
If you decide you really want RAID mirroring and go with the hardware solution, my understanding is that you need a replacement controller in case yours breaks. Since your controller seems to be embedded in the motherboard, you would need a replacement motherboard.
With the Windows software RAID, you are dependent on that software, and have portability only between machines with this Windows 7 software RAID (possibly even only this particular version).
What RAID is good for:
Better to just throw a disk in an old machine and back up to it regularly.
I would recommend an eSATA RAID enclosure, similar to this. Run the newly purchased SATA disks mirrored (RAID-1) in the enclosure. Power up, run backup, power down. Rinse/repeat.
Fakeraid is software too.
Get a real hardware RAID controller, or don't use RAID. Windows SW RAID or a Fakeraid controller is just plain stupid.
"Safe" FAT32, cheap RAID, RAID implied as backup, Microsoft.
Nice job, you successfully trolled the /. frontpage.
Leben Sie jetzt die Fragen.
The article smacks of false dichotomy. There are a number of solutions, not just Windows 7 or a hardware RAID controller.
Agreed.
As I see it, if you want guaranteed repairability then you basically have two options: enterprise-class hardware with a support contract (and price tag to match), or an Open Source software solution.
Put another way, either you pay someone to take responsibility for fixing it, or you take responsibility yourself. A Microsoft solution doesn't give you enough control to take full responsibility, because you can't be certain that it will be legally or technically possible to recreate your current setup in five years time.
If you're just want a convenient backup of your music collection, porn collection, musical pr0n collection, or your pr0n musical collection then RAID is not a horrible thing. However, if you're backing up important files, like the only existing scans of the now-burned dossiers William Mark Felt left you, then you should not stop at RAID. Statistically speaking, if something happens to one HD in your machine, like a massive power surge or being confiscated by tight-lipped men in black suits and black sunglasses, it has a pretty high probability of happening to the other HD. Offsite backups are, therefore, prudent. Leaving a HD in a box at the bank and giving the key to your lawyer is one of the safer things you can do, but not terribly convenient. There are a variety of online backup services available that are decent. I'll leave it to others to speculate on which ones are least likely to be fronts for the NSA. If you feel that your data might actually be interesting to more than one human being on Earth, don't forget to encrypt it. (Be honest with yourself. You are posting to /. after all.) I'm rather fond of emailing moderate risk files to my gmail account. (Stupid, I know, but very low effort and they're available anywhere you feel safe enough to check your email.)
As for Motherboard RAID chipsets... Keep in mind that your motherboard has a non-zero probability of frying, having it's caps go bad, being peed on by irate government agents, etc.. I once had a RAID 0 array that was hooked up to one of those things. After the Mobo died I had to do without letters K through P of my Japanese horror-comedy-porno-game-show collection until I was able to find a used computer with the same RAID chipset. (I don't know if it's changed, but at the time each different RAID chipset made RAID 0 arrays that were not compatible with anything else on this lump of rock.) If data portability rather than performance is a priority for you, my advice would be to avoid hardware RAID entirely.
Do you work for one of these online backup places?
I would sooner trust a WD drive with my valuable data.
Wrong. You need to buy at least two of these controllers, at the same time, or else when your "real" RAID card dies (and they do), you'll lose all your data unless you can find an identical card (you may even need the exact same firmware version).
Software RAID on Linux is a much better solution, as the underlying hardware doesn't matter. You can mix and match different drive models/sizes (can't do that on HW RAID), and swap the drives to a different system and still read them thanks to the standardized on-disk data format.
Actually RAID1 is quite good for reading data: it minimizes seek time . Of course, it works fine as long as there are not many writes. For example think analytic databases, cubes, etc. Those are not written to in real time (like the more common transactional databases)
http://revj.sourceforge.net
Personally, I haven't yet encountered anyone who really got benefit from those personal Internet backup services like Mozy. In regular use, it always seems like the person exceeds their storage allotment or Internet connectivity issues prevent them from recovering what they need, when they need it.
I tend to recommend people buy an inexpensive external USB or firewire drive, leave it attached and assigned as a backup device, and have some software package run a daily backup of all the relevant folders and files they might need to save.
It's great that your data is stored offline and off-site ... but I'm just not sold on most of the implementations for "home use" being as great a solution as they first appear to be. Many of the providers have come and gone over the years, too. What happens when your offline backup company goes under?
You're in a special situation using Windows 7 RC. So despite the below, you're especially at risk to the possibility of a new (but not yet discovered) bug in Windows release-candidate software, don't use Windows or other RAID capabilities for your "data protection", use backups. Use RAID mirroring for improved performance or to reduce the probability of downtime occuring, and allow for rapid recovery from common drive failure scenarios.
RAID controllers (Software or Hardware), all suffer from various scenarios where the entire array can fail, and data recovery becomes so hard that it may as well be lost. 90% of the bits may be there redundantly, but you can't get to them for one reason or another. Also, RAID will not protect against system compromise, software data corruption, accidental deletion, or any type of volume corruption that occurs as a result of software running on the system.
Don't pick FAT over NTFS on the basis of portability between systems, if reliability is more important, the NTFS filesystem uses a technique called journalling which makes data corruption less likely after a system crash, eg power failure. NTFS _can_ be read by common solutions, if you need to recover data. Recent Knoppix CDs and various rescue disks can read NTFS, and the filesystem checking tools available for NTFS filesystems are better. FAT is more susceptible to certain failures, including excessive fragmentation leading to poor performance.
Research what type of RAID solution your integrated hardware really is. If it is hostraid, or fakeraid, that requires Windows drivers to implement RAID, then don't use that, avoid like the plague as it's SOFTWARE RAID, even though the software is running inside a driver provided by the controller vendor and A FEW functions may be offloaded to hardware, the main RAID code is still running in software, which is bad, mmmkay?
You can often detect this in that there will be Windows only drivers, or the product will be labelled a hostraid solution, but each of the major drive controller/RAID chip manufacturers has a different name for their ultra low-end solution that isn't really hardware RAID, but has hardware offload of just some functions (checksumming, mainly).
(Fakeraid/Hostraid adapters that require special drivers in the guest OS to implement RAID, also generally suffer from the RAID5 write hole if you utilize RAID5. And RAID code may be more susceptible to certain problems, when it isn't running on card firmware.)
I would actually favor implementing RAID in Windows over that. However, there is hardly any point of doing this, except if you are mirroring your boot drive, or you need RAID for improved performance (e.g. You could use RAID1 for all drives to improve read speeds, RAID1+0 to improve both read and write speeds, or RAID5 for redundancy and scalability at the cost of slower write speeds and a read speed penalty).
I mean that: since you aren't mirroring your boot drive, there is little point of utilizing RAID in your case. One of the most performance-effecting files on your disks is the page file on the boot drive. If you were utilizing RAID for improved performance, you should definitely want to maximize read and write speeds to the boot drive.
If your non-redundant boot drive crashes, your system will be down and need to be re-installed on a new system drive. You may as well just pre-image a backup drive with your system, keep a continuous backup on another machine, and in the event of a failure, pop in the backup HD, and start restoring continuous data from backup, to bring your 'spare' up-to-date.
This will probably even be faster, as an OS reinstall and re-up of Windows is not required
Second, your RAID controller can fail, make sure you have a plan. That would mean either two identical controllers with the exact same firmware version, or you use a very common controller that you are CERTAIN you can easily buy anoth
We're talking about an environment that has more then TWO MILLION VIRUSES in the wild, and if your choice of Window-hardening software doesn't stop every one of them, you're screwed anyway, headed for a flush and fill. (And potenitally paying another $100). Welcome to Windows.
Sure, it's been a LONG time since Stacker, the 'wonderful new idea to double drive space', but since it was in Windows, it would last about a week before some memory-hogging virus or poorly-written program would stomp on it.
Hardware RAID can be a disaster; I got in a place and time where corporate data was on a set of drives, there was a failure, and the OS maker decided NOT to make a driver for the replacement RAID card we had. (at great cost and by FedEx!) I had to downgrade the OS, load the data to another device, upgrade it and throw away the raid entirely. What a bitch!
It's even part of the reason I delayed using software RAID on Linux- I was gunshy.
But I tried it. You'd expect the extra overhead to cost access-time, instead it speeds up reads! And the writes (at least in Linux) happen in the background so you don't notice any lag. I've run software RAID in Linux for YEARS, replacing drives and adding spares, etc. It's solid. Not only solid, but (for small applications) the best thing out there.
(If you're gonna approach Amazon.com, hardware RAID, all the way!)
Across the distributions, Redhat's got the lead in RAID-at-install-time, but every Linux out there has the ability. It's worth a shot!
--- For a good time mail uce@ftc.gov
And what allot of people don't realize is if you build a RAID array and a drive fails can you replace the drive with the exact make and model? Raids work best when every disk in the array is the same model and revision. If you plan to build a 5 disk raid array you should also purchase a 6th drive to keep as a cold spare.
I built a RAID 5 array using three 500GB disks via mdadm under Linux. I assembled the array and formatted it. Within minutes of testing I was getting mail from mdadm telling me the array was degraded. I then began to test each disk for defects and lo and behold one disk was bad right from the start. I tried to RMA the disk but newegg had informed me those disks were now obsolete. Great. I was credited for the bad disk and purchased a new one that closely matched the other two. It was a nightmare as during some boots the disks went haywire and I would get a "Could not bd_claim sdaX" And it would hang for a while and I would have no array. It happend once in a rare while until it became a real problem. I kept my most precious data safely backed up on different disks I had spread around. It finally got so bad that I would have to constantly reboot the machine for up to ten times before the disks were synced up and the array worked. I purchased a 1TB disk and copied all the data off the array to it and used the 500gb disks in other systems. RAID is great for big fat storage arrays but it can become very sensitive and then one day POOF its all gone.
This is the reason OEM drives from Dell, Apple, HP etc. Cost four times what a retail drive would cost. The cost is no way associated with quality but rather consistency. Retail SATA drives are constantly changing: less/more platters, faster seek and read speeds and firmware revisions. Those costly OEM drives are the same disk every time right down to the inner workings and firmware. So if you buy an Apple 1TB disk on a sled and it takes a dump in three years you can be confident Apple will replace that drive with the EXACT same one. Its not a magical Apple disk of superior quality but a Maxtor/WD/Hitachi disk that is produced for Apple with no revision changes unless Apple orders it. Unlike retail drives which are changed at the manufactures whim.
So if you are building your own raid plan for failures and try to buy a spare for your array. I don't know disk shelf live but it will save you down the line. Also keep a USB or 1394 disk around for backups. Spread your most precious data around like pictures home movies and documents. If you have a few computers around the house keep a mirror of that data one those machines. Music, and downloaded video can be re downloaded but home movies and pictures cannot. Put all the silly stuff on the raid along with the precious stuff for access but keep backups of the good stuff!
He wants to mirror the drives. This means he wants RAID 1. Therefore, the failure rate of the array is 1/2 the failure rate of each disk (more, actually, because they're like;y identical drives that will fail at the same time, but you get my point).
Simple answer: buy more drives. Put the extras in the closet. By the time you run out of spares, it will be time to move on to new drives.
Do you have any evidence for this claim?
A typical RAID implementation writes stripes at a time, by issuing a series of writes to each drive. If your disks have the same geometry, then each write will be at the same physical location on each drive and so complete in almost exactly the same time. If they are not, then the different disks will be moving their heads at different times. The RAID controller (hardware or software) will then be bottlenecked by the slowest drive. To make things worse, the slowest drive can be different for each write. One write may require moving the head sideways on one disk, the next may require moving the head sideways on the other. In both cases, you are limited by the worst-case performance for the disk. The same is true for reads on RAID-5, but not RAID-1, which can just use the result for whichever disk returns first.
I am TheRaven on Soylent News
Fast facts:
I prefer pure software RAIDs, for a simple reason: They do not depend on available hardware. If one controller dies, switch to another one: Other brand, other type, other drivers, and the RAID still works. If you insist, you can even mix an IDE drive, a USB drive, a SATA drive and a SCSI drive into a single RAID. Try that with a hardware or host RAID. Some people even built RAIDs of floppy disks or USB sticks (not for pemanent use, of course).
My faithful old Linux home server runs two RAIDs, both in software: a RAID-1 for the OS (remember: the BIOS does not know about the RAID), and a RAID-5 for the data. The RAID-1 used to run on old SCA drives, but recently, I switched to two small IDE drives due to unrecoverable SCA cabling problems. The RAID-5 is composed of four IDE drives, connected to two IDE controllers, each disk on a single IDE cable. An external USB disk is used to back up my data, rotating through 10 days. All filesystems are ext3, all disks are monitored using SMART, all RAIDs are monitored. If anything wents wrong, I will get an e-mail from the monitoring software.
Until recently, one of the controllers was an el-cheapo non-RAID controller, and the other one was a donated, expensive, well-known brand, RAID-capable controller running in non-RAID mode. The latter one decided to randomly take some free time on the job, and either disconnected from the PCI bus or disturbed it, causing panics in the OS above. Only pure luck protected me from data loss. I ripped it out of the machine, kicked it into the trash bin, rewired the RAID to use two disks per IDE cable, and verified and reconstructed my data. Some days later, another el-cheapo non-RAID IDE controller arrived, the same brand, model and type that already sat in the next PCI slot. So I rewired the RAID again to work with one disk per cable, everything was fine again.
For a new small business or home server, I would use nearly the same setup again: Two software RAIDs, one for the OS, and one for the data. Upgrading the OS is just fun when you can
Denken hilft.
Or don't use RAID0 for important data, idiot. Use it for games where it doesn't matter if you lose everything.
You may have noticed that some hard drives are marketed as being designed for RAID use. These work slightly differently to most consumer disks. Typically, a small region of a disk is hidden. If the disk discovers a bad sector then it will use one from the hidden region to replace it, so every write to the bad sector goes to one of the spare ones instead. This is very bad for RAID, because two drives writing to the same sector may be writing to two different physical locations (if one is remapped), with the same problems I outlined above.
Disks designed for RAID use do not have this behaviour. If they find a bad sector, they report this to the OS. The RAID controller will then mark that sector, and the corresponding sectors on the other disks, as bad and not use it in future. Note that there is no performance gain from using RAID disks in a single-disk configuration, although this doesn't stop some people trying...
I am TheRaven on Soylent News
Again, if you need a stable parts supply, don't go with a whitebox server. Software RAID doesn't help you in that regard - instead of an 8 year old SAS RAID controller you'll be looking for a generic 8 year old SAS controller. It'll still be shitty and not usuable in a production environment.
All modern disks remap sectors as necessary. The main difference between consumer and RAID drives is the timeout for error correction.
Is it worth keeping a spare for the sole purpose of having the same model available in the event of a failure when you can get a newer and faster drive in the future?
I would say not, but when one drive fails you should replace all of them. For a home array, expect one drive to fail every few years. I had a disk in RAID-1 array fail last year. It was a 40GB disk which cost around £100 new. For the same price, I can buy two 500GB+ disks now.
Is the difference in performance between modern SATA drives so significant?
It's not a question of performance, it's a question of the difference between a linear access and a seek. The time for a seek is 4ms+. If a drive can read 50MB/s then a linear access is around 10 microseconds. If your one disk is doing a linear access while the other is doing a seek then you are limited by the time of the seek (for RAID-1 writes and RAID-5 reads and writes). If you have to seek after every block, your maximum throughput is 125KB/s. If you do a linear read, your throughput is 50MB/s. If your drives have different geometries, you double the number of seeks you are needing, dramatically reducing your throughput.
I am TheRaven on Soylent News
I do not believe you can use Fat32 formatted volumes to make a software raid mirror in any version of Windows, ever. If I'm wrong, tell me, but I have been working with the 'Disk Management' in windows for 15 years and have never been able to do this. Not that I would. 'For portability' is not a good reason to use a file system that was left behind by Microsoft years ago. You can mount an NTFS partition in several O/S's these days effortlessly. I agree with every one else here. Build hardware RAID 0 into your system, use all the space on those disks that you paid for, get some performance out of them. And most of all, PRACTICE RECOVERING YOUR DATA. Do mock-crashes, do some research, READ what other are doing about their problems, make some notes, get acquainted with your hardware, and your backup software, and whatever tools you end up using for data recovery. This is essential for real data loss prevention. Am I preaching? Sorry!
I often use inexpensive SATA RAID controllers from Promise (I do not work for promise). They don't cost very much and they have been absolutely reliable for me (for many years now!). I often stick with RAID 1. I've built several RAID5 arrays and I don't find a lot of value in them for low capacities. Mirroring (RAID 1) is straight forward, and if you ever have a problem you can always read one of the RAID 1 drives using a SATA to USB interface, or if you ever need to clone a hard disk it's easy. Promise seems to use the LAST 64k of the hard drive for it's mirror info, not the FIRST 64k! this makes any of the two drives in the RAID array easy to use out of the array when/if you're in a jam (for whatever the reasons). As far as RAIDing your data only, in my opinion RAID is designed to avoid lengthy recovery procedures - don't put yourself in a disadvantaged position - all hard drives fail eventually - RAID the OS, your data, everything! If your server is a very busy server - start looking at higher end RAID solutions.
raid will make your data highly available, not secure.
if you want security, you need backups, and backups are:
- off line (viruses, power surge, sabotage...)
- off site (fires, theft...)
- tested (i've got horrors stories of people that THOUGHT they had backups...)
- multiple (... and of backups that turn bad at the worst possible moment)
Raid is none of that. I know plenty of people who thought their data was safe because they had raid. It isn't, it wasn't, it ain't ever gonna be.
The Cloud - because you don't care if your apps and data are up in the air.
Whoa, hold the boat. I've had a lot of experience with Dell & HP/Compaq(Proliant) provided RAID systems and they are not sensitive to disks with vastly different innards. All that matters is block count and software mirroring doesn't even care about that, because you'll simply be limited to the size of the smaller disk. If you're using mirroring or RAID, try to go with different makes of the same size. This article talks about MTBF. It turns out if 2 drives of the same exact model comes off the line and end up in your PC, there is a chance they could fail within a very close time to one another. So your mirror or RAID could fail permanently while rebuilding from the first failure. But if all your drives are of a different make, chances are they won't fail at the same time and you'll get the critical time needed to rebuild your array.
When I'm going to do mirroring or RAID on hardware that doesn't have high-end dedicated server RAID controller, I use Windows or Linux software RAID. Performance is surprisingly good and I'm not married to a specific hardware implementation. I've had _none_ of the issues you've described with Linux software RAID on several servers for several years. Mdadm has only whined after a power outage or genuine disk failure.
There's no place like 127.0.0.1
"That's nice, but the submitter is asking about RAID 1."
I think he's asking the wrong question anyway.
"Based on past experiences, I have decided that only my data is worth saving"
See? He is asking for backup, not RAID. It has been said one thousand times but it seems it must be said again: RAID is *NOT* in order to protect your data. NOT, NOT, NOT and then NOT again.
RAID (not talking about RAID-0) is there in order to enhance your data's avaliability (as in, say, instead of being able to get to my data 99% of the time, I can get to it 99,9%) but when it's hosed, it's hosed. To protect your data you need backups, not RAID.
"Of course, such a setup should secure my data"
Of course not. Of course you will get quite a funny face when you discover it. Quite more or less the one that had the guy from this story, about six months ago, with the very enlightning title "Why Mirroring Is Not a Backup Solution": http://hardware.slashdot.org/article.pl?sid=09/01/02/1546214
"Even more importantly, I want any drive and its data to be as safe and portable as possible"
Then, *even* if RAID could be considered for data security (which is not) you already answered your question: as a general matter, hardware RAID will only work when using exactly the same controller model, possibly up to its minor revision. You can't count to break a hardware-managed mirror, take one disk to a standard SATA controller and get any data out of it. If your controller dies and miracolously doesn't take the disks with it you can't count on buying a different RAID card (as it will most probably be in about a year for consumer-grade hardware) and get any data out of the mirror. So you should go with software RAID.
AND TAKE BACKUPS.
Raids work best when every disk in the array is the same model and revision. If you plan to build a 5 disk raid array you should also purchase a 6th drive to keep as a cold spare.
I hate to break it to you, but you're actually wrong.
A RAID array is most effective using completely different drives, but of the same capacity. Five hard disks from the same manufacturer, of the same model, bought at the same time means that you're highly likely to get five drives from the same batch. Let's posit that there was some defect in this batch. Now all five of your drives have a significantly higher probability of failing at the same time. Oops! RAID can only deal with one (or two) drive failures!
Using drives from different manufacturers or model lines means you spread the risk of simultaneous drive failure.
Preach Brother! I have had several of my customers ask about RAID, but when actually sitting down and talking to them it turns out they are looking for a backup solution which RAID most certainly is NOT.
Here is what I recommend to my clients-Use whatever you want inside your machine, but get a USB HDD or even better a NAS for backups. Most come with very capable backup solutions provided, and is much better for the purpose than RAID which as you so very eloquently is for access NOT backup. There are several cheap barebone NAS kits where you simply add your own drives, these have the added benefit of being easy to upgrade should your data become larger than the drives. Put a couple of 1Tb drives and a nice multi platform backup solution ( I use Paragon Drive Manager which comes with a nice Linux GUI based boot disc and covers FAT,NTFS,EXT2 and EXT3, but there are several alternatives to choose from) and all is golden.
But please don't use RAID as a poor man's backup, as it will come back to bite you in the ass. Get a USB drive, get a full or barebone NAS, and use a real backup software like Paragon or similar. In the end you will be a LOT better off than trying to use RAID for a job it simply wasn't made for.
ACs don't waste your time replying, your posts are never seen by me.
Yes, Mod parent up. RAID is for providing data resiliency, not data protection. In corporations where either very large data sets would simply take too LONG to restore, or where spindle count for acheiving IOPS is critical, RAID permits a reduction in failure rate. However, is it NOT a replacement for backups, and RAID should generally only be considdered when you;re already considering a multiple disk setup (either for capacity or performance reasons).
A RAID1 setup on your home computer may increase your uptime, and a RAID 0 can imrpove your performance, but generally, it's not that improtant, and on a home PC, typically the OS drive is more critical than the data drive (It's easy to resorte a backup, It'sdifficult to make your machine exactly as it was if you loose the drive and need to re-install).
That said, even a data backup is NOT enough. You also likely need an image/BareMetal backup of the OS and application drives. Rember, it's one thing to limit hardware failure by using a RAID, and another to have good backups of your data, but you also need to take the human factor into account: 1) your mistakes change both RAID 1 disks, there is not rollback; 2) hackers and viruses corrupt data just as easily on your external USB used for backup as it does on your primary drive, unless you're using top nothch backup software that hides the backup[ device from Windows and makes the backups unreadable to the OS (rare); and 3) software installs, bad code, and Windows itself can just as easily render all your data useless.
If your data is important: ...and NO, there is NO SUCH THING as a surge protector that can stop a lighting strike. The EMP alone is good enough to destroy data. (I've seen a montior and PC 4 feet from the nearest outlet get cooked when the lightnig's EMP backfed the CRT's stores static energy into the motherboard).
1) Backup regularly, and please use a real backup application, not Robocopy or some cheap scripting system... Keep miltiuple incremental backups and use software that manages a proper rotation and can search offline data to find files you want to recover.
2) Make image/BareMetal backups of the OS. Vista Business and higher editions have something similar built in, but using a program like Ghost is often easier and quicker to restore from. Make a new image at least as often as you install make major changes to your system, or every few patch rotations.
3) DO NOT leave your primary backup device connected 24x7 unless it's a tape drive or worm device. Your backups are easy fodder for hackers on USB drives. Also, a lightning strike or surge that takes out your primary AND backup is bad, really bad...
4) GET YOUR DATA OFFSITE. Fire or water damage should not be able to take out your "important enough to back up" data. This not only includes your backupos, but critical media, CD keys, and anything else you'd need to rebuild the computer far enough to reconnect to your backup disks...
5) Keep this rule in mind: "Nothing is backed up until it's been restored." Well enough that you're doing backups, but if you have never tried to restore your system, you have NO IDEA what that takes, and NO IDEA what you're missing to do it. Done a firmware patch? many of the original drivers may not work anymore, you might need new ones on CD to reload the OS... Maybe you'll find your backup software doesn't have an open file manager, and your e-mail isn't being backed up properly when it's running... Do you have the backup software stored offsite with your backups???
I have nearly 20 years of important files spread across my 3 main computers (including the wife's machine too) totaling about 1.1TB of actual data and files. My main machine runs a RAID 1/0 (mirrored stripes) with 4x 250GB 7200RPM drives on an AMCC/3ware controller (Soft raids SUCK, onboard RAID is not much better, NEVER opt for the lowest bidder if your data and your performance are important...) That's my OS, Application, gaming, and higly-used data drive. I also have a 750
There is no contest in life for which the unprepared have the advantage.
Why RAID is not a backup:
1) not fireproof.
2) not mistake proof "oops, didn;t mean to delete that"
3) not immune to file system corruption.
4) not immune to power supply failure/surge/lightning/other destructive forces
5) more expensive than a good backup
6) not protable offsite
7) does not track versionb history or old files (something that should be of critical importance to a programmer...
8) Viruses, mailware, hackers oh my!
9) bad/corrupt install
10) OS failure
I could easily go on. I worked in DR for 4 years...
Nearly all of the above have a higher frequency of occurance over a 5 year typical HDD life. Even if you continually replace drives without a data failure, you're still eventually going to have an issue RAID can not deal with.
My Qnap was a $399 device. The 4 drives in it were $90 each (and the 5th spare too). The HDDs I run the PC off on the RAID 1/0 were $40 each. I only run the RAID 1/0 for performance during video editing. I chose 1/0 vs 1 since 1 halves the reliabiltiy of the drives. Even though I do have a good recovery solution, the downtime, nor the effort involved in recovery, would be welcome, and the extra $80 to mirror the performance stripe was easily spent.
The Qnap is also my iTunes media server, my FTP server, included the price of the DR software, and runs 2 IP cameras I set up at home too (which let me tell the insurance company I have real-time video monitoring, and they knowcked an extra 5% of my homewoners policy cost, which by itself is enough to fund replacement drives as I'll need them).... Oh, yea, and it's a NAS too... It has a lot of value beyond a backup system.
I'm guessing you've not got a child yet, or a large family. You probably don;t value to pictures you take, files you have, and other stuff on your PC. That's fine, someday you likely will.
There are cheaper ways than mine to do backups. I have over a TB, and 3 (currelty, soon to add 2 Macs to the list an decom 1 old laptop leaving me with 4) computers I'm backing up, so centrally makes sense. If you have 1-2 machines, a small amount of data, and don't value most of it, then 2 external USB drives and a safety deposit box (Dad's house) usually suffice... Or, just an online backup account for $5 a month...
RAID 1 might save you from a firmware failure, or a disk going bad, but that's about it... Also, RAID 1 may be cheap, but a backup is cheaper. Also, good luck rebuilding that RAID if your MOTHERBOARD fails... RAIDs are proprietary to a particular controller. Unless your new board usues the same chipset (and firmware too in most cases) you;re screwed without a backup.
There is no contest in life for which the unprepared have the advantage.
> I have decided that only my data is worth saving
Then forget about RAID. RAID is designed to protect the integrity of the underlying volume - NOT the data that's on it.
> Of course, such a setup should secure my data; should a drive crash,
Then forget about RAID. RAID will only secure your data under some very specific cases of hardware failure of the drive. It does absolutely nothing towards preventing data loss due to (say) a corrupt file allocation table, virus, accidental deletion, or corruption.
> Even more importantly, I want any drive and its data to be as safe and portable as possible
Then use proper backups - not RAID. Preferably off-site backup. I use Carbonite which backs up to the 'cloud' at minimal cost.
By all means use RAID to protect you from hard disk failure, but don't under any circumstances assume it stops you losing your data. For backups, I always use the rule that at any given point in time, assume that the next time you walk back into your house/office, that NOTHING in that building is still there. Do you have a copy of everything you care about somewhere else?
I'm still amazed by people that carry 12 months of work around on a single floppy disk/USB stick/laptop, then cry when they go to the helpdesk asking what "sector not found reading drive A:" means, or perhaps "A USB device attached to the system is not functioning".
Get your data in as many places as possible - preferably three. A drive which is mounted one inch above the main one is *NOT* a valid second place!