Slashdot Mirror
Managing RAID on Linux
A person deciding to go with RAID faces a panoply of options and gotchas. Hardware or software? How many controllers? ATA or SCSI (or ataraid)? RAID 1 or RAID 5? Which file system or distribution? Kernel options? Mdadm or raidtools? /swap or /boot on raid? Hybrid? Left or right symmetric? One poster pointed out that putting two ATA drives on the same controller could impact performance. Yikes! Didn't I do that? Upon discovering that O'Reilly had just published its Managing RAID on Linux book, looking at sample chapter , I bought the book and let my blood pressure return to normal.
RAID is one of these subjects that is really not complex; it's just very hard to find all the information in one place. This is precisely the book to solve the problem. Author Derek Vadala, sysadmin and founder of Azurance.com, an open source/security consulting firm, has gathered a lot of information and even personal anecdotes to go through the decision making process when going over to RAID. He goes step-by-step through that process, educating us about hard drives, controllers, and bottlenecks along the way. This exhaustive book may be the first to bring RAID to the masses.
Although parts of the book (RAID types, file system types) may seem already familiar to experienced Linux users, it is helpful nonetheless to have everything in a nifty little book. A section of file systems provided not only a rundown of the merits and drawbacks of each one, but also a guide to their utilities. I learned for example what "file tails" for Reiser are, and why using them causes performance to degrade after reaching 85% capacity. The book compares raidtools with mdadm as well as lovely commands like nohup mdadm -monitor -mail=paranoidsysadmin@home.com (which, if you haven't guessed, causes the system to email you RAID status reports upon boot).
People who use software RAID may skip over the chapter on RAID utilities for the leading RAID controller cards. Still, there was one interesting tidbit: Why, the author asks, do makers of controller cards put all their BIOS utilities on DOS floppies which require us to find a DOS boot disk? Seriously, how many of us carry around DOS boot disks nowadays? The book made me aware for the first time of freedos, an open source solution that solves precisely that problem.
The Software RAID stuff was pretty thorough and clarified a lot of things. The book does an excellent job in helping to identify and eliminate bottlenecks and optimizing hard drive performance (using hdparm and various monitoring commands). The anecdotes and case studies definitely clarified which RAID solution is suited for which task.
I am less impressed by the book's sections on disaster recovery and troubleshooting. Although these subjects are brought up at several places in the software RAID chapter, the book could have discussed several failure scenarios or used a fault tree (such as the famous Fault Tree in Chapter 9 of the Samba book, a marvel for any tech writer to read). The book doesn't even discuss booting with software RAID until the last 10 page of the book and then gives it only a single paragraph (even though the author acknowledges it as "one of the most frequently asked questions on the linux-raid mailing list."). Call me old-fashioned, but isn't the ability to boot into your RAID system ... kinda important? As someone who just spent a significant amount of time troubleshooting RAID booting problems in Gentoo, I for one would have liked more insight into the grub/lilo thing. Also, in the next paragraph in the last chapter on page 228, the author casually mentions that "all /boot and / partitions must be on a RAID-1." Say what? Please pity the poor newbie who religiously follows the instructions in the book but fails to read until the end. I'm not sure what the author meant by this statement, but it required a much more substantial explanation and needed to go into a much earlier chapter.
These complaints don't detract very much from this excellent book, a true O'Reilly classic and a model of clarity and helpfulness. This book provides enough knowledge to avoid the dread and uncertainty that comes with trying to tackle Linux RAID. With a book like this, a sysadmin can sleep a little easier.
Recommended Readings:- Reliable Linux , by Iaian Campbell, Wiley and Sons, Dec 2001, ISBN: 0471070408. Gives excellent information not only about RAID but on general Linux reliability issues.
- Software RAID in the Linux 2.4 Kernel by Daniel Robbins. (Part Two).
- Linux Journal Article on Software RAID by Joe Edwards, Audin Malmin and Ron Shaker. ( Part Two).
- "How to do a gentoo install on software RAID" by Chris Atwood. Gentoo User Forum.
Robert Nagle (aka Idiotprogrammer )is a Texas technical writer, trainer and Linux aficionado. You can purchase Managing RAID on Linux from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Ask him into your heart today!
But seriously, does it play Ogg Vorbis files?
How about a beowulf cluster ...
although personally I prefer spraying it on my weed. That shit gets your SO FAHKED UP!!! (Esp. Flying Insects RAID!)
but the easiest way I've found is to go with hardware RAID. It's easier to setup, doesn't put any extra load on the CPU, and only costs a few hundred dollars extra.
Mind you I'm thinking of RAID used in producion instead of someone RAIDing two drives in there home machine.
And, as a quite note; someone needs to remind IDE users that trying to do striping on the same IDE channel is stupid and if you need to have it explained why - then you are too stupid to have decided to use it in the first place.
Then of course, everything would be easier if the hardware manufacturers also sent along a DOS bootdisk, perhaps with FreeDOS to avoid licensing fees.
...was the use of the word "panoply".
That word simply isn't used enough in the modern vernacular.
Okay, mod me down now...
"It's a tarp!" -- Dyslexic Admiral Ackbar
Is it possible to use Firewire and a service like Rendevous to make an intelligent redundant system? It's a thought at least. My firewire drive I use for my Inspiron works nicely enough. Would firewire be cheaper than RAID for servers, however?
Syr GameTab.com - Game Reviews Database
panoply
n. pl. panoplies
Looks like number one is most appropriate, although I've never referred to my arrays as "splendid".
might not valid in a year or so.
... but some of the information is out of date, and the tricks suggested by people a year ago may be no longer needed today.
re:
this is something you are going to face when you are considering a technology that changes rapidly. a book on the subject isnt going to change the dynamic nature of linux.
-- john
"all /boot and / partitions must be on a RAID-1."
/boot must be RAID1, but / can most assuredly be RAID 5 (or, I presume, any of the other RAID levels). I have this running on an ol' RedHat 7.0 box:
/dev/md1 / ext2 defaults 1 1
/dev/md0 /boot ext2 defaults 1 2
/dev/md0 /dev/sdb1 /dev/sda1
/dev/md1 /dev/sda6 /dev/sdb6 /dev/sdc5
With raidtools, at least,
Hunk 'o fstab:
Similar hunk 'o raidtab
raiddev
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
#nr-spare-disks 0
device
raid-disk 0
device
raid-disk 1
raiddev
raid-level 5
nr-raid-disks 3
chunk-size 64k
persistent-superblock 1
#nr-spare-disks 0
device
raid-disk 0
device
raid-disk 1
device
raid-disk 2
*Shrug* Wonder what the context of that quote was within the book?
the no
Robert reviews the O'Reilly title Managing RAID on Linux below to see how it stacks up to HOWTOs, guesswork and anecdotal evidence.
...
Author Derek Vadala, sysadmin and founder of Azurance.com, an open source/security consulting firm, has gathered a lot of information and even personal anecdotes
So how do anecdotes compare with anecdotes. Can we see a point by point comparison of what anecdotes offer that anecdotes don't? Or why referring to anecdotes might be better than referring to anecdotes when searching for information?
The learning curve on most Wintel software is on the order of the time needed to search through half a dozen menus to find the right command.
Trying to make everyone be an expert before they can operate their machine is how operating systems die.
Does this book talk about the md driver's
multipath personality?
This is the most poorly documented part of the
md driver.
if you read the raidtab man page ("man raidtab")
you will find _no_ mention of multipath whatsovever.
Yet, the md driver can do mulitpath (well, failover) if you set it up right.
It has limitations though... You can't install to multipath devices, or boot from them (lilo/grub, the various distributions installers don't understand md multipath) and, if an hba fails in such a way that interrupts are not generated...commands just go out to lunch... then md won't notice anything is wrong, and so won't failover. Also, it does nothing to notice if the failover path is actually working, so if that path fails you won't have any notice that redundancy is lost....
Well, multipath is not RAID, so maybe this book
doesn't cover it, but any book on software RAID for linux should probably cover all the features of the md driver.
I will be interested to see this book.
I've stepped away from the software RAID idea on my boxes, due to the availability of cheap hardware RAID, such as Promise's SX4000. It will do hardware RAID 5 for four+ drives and has a SDRAM slot for cache expansion. Coupled with LVM, it ended up being a good solution for me, as I had both the reliability, and good volume management if I wanted to combine arrays.
The problem I've had with the software RAID is reliability and expandability. It is a pain in the ass if you lose a drive in the array, and it is next to impossible to add a drive (other than a stand by drive) to your existing RAID 5 setup.
Aah, opinions...
------------------ D. A. Davenport: http://www.firebin.net
...has no use for RAID. RAID level 0 is pointless, increases IO throughput ~40% while decreasing MTBF to 1/2 of its nominal value for each drive. RAID level 1 is just a bit-for-bit backup system, and wastes 1/2 your disk capacity. RAID level 01/10 is both expensive *and* pointless.
If you could find an IDE cheap SCSI controller that can do RAID5, then sure, RAID might be a useful solution to the issue of data redundancy and have a little peformance increase...Too bad cheap RAID5 cards don't exist.
You may as well just do RAID5 in software, really.
------- "From bored to fanboy in 3.8 asian girls" ----------
Please pity the poor newbie who religiously follows the instructions in the book but fails to read until the end.
On the other hand, pity the newbie who cracks a book open and starts setting a server up page-by-page.
The availability of cheap and expensive raid solutions for linux as well as good documentation like this book will no doubt help linux and open source to gain more acceptance in the enterprise.
Jan 26 04:15:02 hostname kernel: hdb: dma_intr: status=0x51 { DriveReady SeekComplete Error }
Jan 26 04:15:02 hostname kernel: hdb: dma_intr: error=0x84 { DriveStatusError BadCRC }
I've looked all over the place for the answer, google, mailing list archives, Usenet, local Linux friends, etc. and haven't been able to find a definitive answer. It's like nobody really knows what that error messages really means.
Newsgroups suggested bad cables, so I replaced those (twice, once with brand new cables bought specifically for the purpose). Some info suggested the drive or the drive's controller was failing, so I replaced it. Other info pointed to my IDE controller, so I installed a new one dedicated only to the RAID pair. I saw info that said the raid tools were to blame, and to see if the errors go away when the mirror is broken. No dice. Other info I found suggested that it was the IDE drivers in the kernel and that the messages were nothing to worry about unless I was seeing data corruption. I'm not seeing corruption so I'm left with this option.
If the book can shed some light on the error message voodoo one sees with Linux's IDE driver, then I'll buy it. I'd pay double what they're asking, even.
-B
Ash and Hickory, straight-grained and true, make excellent bludgeons, dandy for the cudgeling of vegetarians.
You linux hippies should learn how to use RAID, kill off those lice/crabs for once and all.
Now, if only there was a book to teach open sores hippies how to use SOAP, or how to kill -9 BO
I guess it's better to mirror across the channels as well (and my small tests seems to indicate that) but is it so?
Peder
While I'm certainly a proponent of "dead-tree" documentation, I have to take a moment to disagree with one of the statements made -- I'm sorry, but newsgroups, while perhaps containing out-of-date info, are (if it's a good newsgroup) capable of letting you know the current state-of-affairs. This is substantially -less- true with books. Case-in-point is Samba: it's *DARN* hard to know, from the Amazon description (or wherever) which Samba books describe the current state (2.4 and above) of Samba, whereas the FAQs, newsgroups, etc., are fairly obvious on it. Bottom line? I'll take a good book any day, but when in doubt, I'll go with current info gleaned off the newsgroups and other on-line resources.
Why, the author asks, do makers of controller cards put all their BIOS utilities on DOS floppies which require us to find a DOS boot disk? Seriously, how many of us carry around DOS boot disks nowadays?
Well, given Dell's recent announcements, I suppose fewer and fewer of us will be doing so.
But really, the author's point is so moot that it's embarassing: if it's my job to maintain a RAID array, and the utilities are on DOS floppies, of course I'm going to have access to a DOS boot disk. So what ? Just how hard is it to carry such a thing around, and why is this is a worthy thing to rail about, in a book about RAID ? If the author wastes too much time talking about stuff like this, this book can't be that useful - arggh, I've wasted too much of my own time already.
But based on the fact that Hardware RAID for IDE drives has come down so much. I am not sure why you would use software.
Anonymous Cowards - Oh God, How I hate you
It's not that hard.
- Power down the computer
- Remove cover
- Blow out all dust and insect husks
- Spray in RAID
- put cover back on for 15 minutes.
- Remove cover again
- blow out insect husks.
Trolling is a art,
This is the first +3 troll I have ever seen. I bet he's an 'insightful troll'.
In soviet russia, LINUX manages RAID on YOU!
I once had to install chinese win95 and chinese ie and chinese office... and I dont speak chinese. I barely speak my native language of english......
In designing a combatives program, there are many discerning factors that must be taken into considera- tion. The first-ever Special Forces Combatives Course that will be discussed in this article is designed for a special breed of soldier. A Special Forces soldier is the most elite fighting man in the United States Army, and possibly the world. He is trained in all areas of combat: communications, demolitions, light weaponry, medical, and intelligence. All SF members are airborne qualified, with many specializing in HALO (High Altitude, Low Opening), SCUBA, STABO (extraction methods), and SERE (Survival, Evasion, Resistance, Escape).
The Special Forces uses unconventional warfare. The very mention of the name brings visions of Vietnam and John Wayne. The Green Beret symbolizes all that America stands for: truth, strength, and pride in being American. Special Forces teams have high-priority missions, sometimes going weeks or months without contact with friendly units. The Special Forces Combatives Course enhances the physical at- tributes, mental awareness, and self-confidence of the SF soldier. Also, due to the type of missions he is assigned, a thorough knowledge of combatives is often necessary to raise the Special Forces soldier's capabilities to a level whereby he will be successful. The Special Forces Combatives Course provides the medium to build a better fighting man, both physically and mentally.
The SF combatives training program furnishes comprehensive physical development, cardiovascular and aerobic conditioning, and develops anaerobic endurance, muscular strength, flexibility and agility. A combination of these physiological factors develops a soldier who is in overall top physical condition-a requirement for the Special Forces mission.
In the Special Forces Combatives Course, Green Berets receive training in the Filipino arts of kali, escrima and arnis. While the Filipino arts emphasize weapons training-especially sticks and bladed weapons-the techniques are also applicable to empty hand fighting.
From a psychological standpoint, the Special Forces Combatives Course is also quite beneficial. The inner knowledge that, if stripped of all weapons, one still has the skills necessary to effectively defend oneself, greatly increases a soldier's self-confidence. Special combatives training, above that of the average soldier, enhances a Green Beret's self-identity and pride in what Special Forces stands for.
The nature of the Special Forces mission means an increased possibility for close-combat encounters. Special Forces combatives training is designed specifically for close combat and is geared for total annihilation. A soldier untrained for close combat might become wounded and/or die in such a situation and thus fail in his mission. A soldier who is properly trained in close combat will effectively neutralize his enemy and accomplish his mission.
The Special Forces Combatives Course is divided into nine basic components: physical fitness, sentry neutralization, Filipino martial arts, kicking, punching and hand strikes, grappling and throwing, knife techniques, equipment training, and mental/philosophical training.
Physical Fitness
A Special Forces soldier must have outstanding endurance, strength, and skill. He must be able to overcome obstacles, kill or disable the enemy in hand-to-hand combat with or without a weapon, and advance swiftly, silently, and effectively. The Special Forces devotes a great deal of time and effort toward making its fighting men the best in the world.
The physical fitness program for the Special Forces Combatives Course is geared toward achieving high fitness levels in the shortest time possible through strenuous physical exercises. This program is divided into five "gates" or stations: upper body, middle body, lower body, reaching, and aerobics. These stations are further subdivided into variables. The variables are based on data drawn from tests and studies conducted over a number of years, as well as personal experiences of the instructors. The exercises include conventional Army programs, tai chi reaches, Ranger push-ups, Tiger push-ups, martial arts stretching, isometric resistance, SCUBA sit-ups, and boxing reachbacks.
Sentry Neutralization
Sentry neutralization is taught utilizing empty-hand, garrote, and knife techniques. Not only are the actual killing movements taught and practiced, but also the philosophy of close-quarter termination, stealth, stalking, visual domination, spring power, timing, environmental control, and spontaneous reactions (both of yourself and the sentry target). Realism in all situations is stressed to its highest point. Two-man sentry neutralization techniques are presented for absolute control of an armed guard.
A better title for the book would be...
RTFM: RAID - The F*cking Manual!
I recently installed raid out of the how-to and other information I googled for. I really did not have much problem with it. I took a basic configuration and then started playing with it to make it work the way I wanted. Seemed pretty intitive compaired to many other things out there to configure. As for why to use Software Raid? I had an old box lieing around with no HW raid but needed a non-production server as a sandbox, bingo I had one. This turned out to be a really good idea because soon after I built it one of the disks went bad and I was albe to save all my work.
Perhaps what we need is more penetration of Linux into business settings, so people can buy books with their employers' money?
--------------------
Escape geekdom! I did.
Link: Root raid 1 on Debian.
Nuff said.
With disk drives steadily increasing in size, and backup options not keeping pace, everyone has a use for RAID 1. Frankly an extra 100 bucks on another drive is well worth it in comparison to the hassle of maintaining an ongoing backup process. I don't really care that I'm "wasting" a whole drive, since it's still going to be a ton cheaper than any RAID 5 solution.
Ever ripped 500 CDs to MP3 format?
Ever done it twice?
I have, and never will again if I can help it...go RAID 1 go!
That onboard Promise RAID controller you dished out the extra $50 for on that new motherboard is not going to get you a nice hardware RAID 5. AFAIK they can only do 1,0, 0+1, or 1+0. Also, I see people whining about software RAID as compared to hardware RAID. Running a striped set through software was nearly unfeasable a few years ago, but with the resources new machines have these days, the difference is almost negligable, as long as it doesn't have to fight for system resources. let's not forget software RAID is alot cheaper than buying a RAID controller.
At any rate, taking the view that hardware RAID is always the solution and software RAID is never the solution is just bad sysadministration.
Everyone is entitled to their own opinion. It's just that yours is stupid.
Up until now I've bought only SCSI drives because heavy compiles (which I do a lot) just choke IDE down. I now have a 4 x 60 GB RAID-1 and it just screams. With a one time investment in a proper IDE RAID card with escalator scheduling, tagged queueing and big cache I still save a lot of money by being able to buy large but cheap IDE disks.
And why would I buy this book or any book on RAID if I am going to use a hardware solution. If I have hardware then I am going to just make sure it has support & instructions for Linux and be done with it.
I think books like these should be topical oriented rather tha OS oriented.
Or am I asking too much?
I get >160 Megabytes per second off my software striped drives, which is far faster than I've ever gotten off any hardware RAID.
And I've found the RAID 5 overhead is nominal, and very reliable.
That's 10 pages about "booting" to software RAID. Not the same thing.
People who disagree with you are not automatically evil, greedy, or stupid.
I don't know what this writer is complaining about. Software RAID on linux is REALLY easily implementable from the FAQ. And secondly, the RAID mailing list (easily searchable) is filled with answers to all sorts of problems. And thirdly, Neil Brown, who wrote much of the kernel RAID software, is on the list and has always been very helpful. Three cheers for Neil!
The original story sounds like someone who is trying to sell his book.
When I decided to set up a RAID under Linux, I recalled seeing an icon in my webmin. I used Webmin almost exclusively in setting up the RAID. I didn't need any HOWTOs in the process of setting up this thing.
So while there are good collections of information out there, there are also very good tools out there with which to accomplish useful tasks.
I think it's precisely that HOWTOs are rarely if ever needed with Windows stuff that it still has an edge over Linux where the masses are concerned. So it's nice that HOWTOs are out there, I think it's more important that good tools are out there that are easy and self explanatory.
This guy really like to review books.
Tuus crepidae innexilis sunt.
I think that a better title for this book would be,
RTFM: RAID - The Fucking Manual.
Life is the leading cause of death in America.
EVMS is IBM's version of RAID for linux. This is natively available on gentoo linux. I've been running it on a few boxes with great success. The utilities make it a lot easier to set up raid, lvm, etc.. Definately worth looking at for those interested.
I'm not a real doctor, but I recommend beer.
This post brings up an interesting topic. I too have found may how-to's and online resources available to be outdated. Has anyone setup some sort of Linux HOW-TO/Help Desk WIKI where information can be continually updated by the community? If not, I would be interested in hosting something like this personally. Input appreciated.
-Anonymous Coward
All I want is a software RAID 5 array that I can use in both Windows AND Linux. Maybe it's possible. I dunno. I've never found any instructions.
The roots of education are bitter, but the fruit is sweet.
--Aristotle
> RAID level 1 is just a bit-for-bit backup system,
That's exactly why it's good. It also increases your read speed by getting data from both drives.
> and wastes 1/2 your disk capacity.
I don't consider a backup to be a "waste". One extra hard drive is MUCH cheaper than your TIME, and it will take a LOT of time to deal with all the data you could lose if your single drive fails.
My software RAID-1 rebuilds at 48M/sec, automatically, which is a lot faster than feeding in a stack o' CD-Rs, and cheaper than a bunch of DVD-Rs.
(RAID won't protect against data corruption or accidental deletion, however -- you will just get mirrored corruption! So you need an additional backup strategy as well, just like always. Except the relatively common PC drive death won't bother you. I have seen a lot of people's drives fail over the years.)
...but I'm not sure how well it works with RAID arrays. The softly-softly approach may be fine with individual disks, but if you have to manage a small army of them then discipline and the threat of violence is the way to go. Just let them overhear you muttering something about magnets and they'll soon get their act together. If necessary, format one as an example to the others. They're a bit like kids really.
Modest doubt is called the beacon of the wise. - William Shakespeare
I don't really agree with the fact that some info is 'no longer needed'. Recently, I had some problems with an old p133 from an obscure brand and man I was glad when I found that 'out of date' info.
If old stuff is still around, it's because it still helps.
Granted it's on Windows 2000, but I just ordered a SIIG PCI IDE RAID controller and 3 30 gig IBM drives. Total cost: ~$250 or so. The PC runs tape backups for a small workgroup, and we just needed the extra insurance of having a pair of RAID 1 disks plus a spare. The simple SIIG controller can boot from the RAID disks and we don't need online-restores or hot swaps or the like.
Software RAID is plenty good enough in the right situation.
You actually feel good about the Linux drivers that Promise gives you with the SX4000? I bought this card, and I wished I stayed away from it.
I am using it with four 120gb IDE drives with 8mb cache. For starters, if you use anything but the sxcslapp program in Linux to configure the drive, your drives are corrupt. All of 'em. And, your bios will return corrupt information regarding them. This causes DOS not to boot (hard freeze), and Linux to produce keyboard smashings on boot. This is a known firmware problem, and I'll be damned if they have any flashes available, even though the card is four months old. I just checked before writing this review.
Once I figured out that all the work had to be done with sxcslapp in Linux, I started building my RAID5, albeit with caution. Things here went pretty well, except a) performance sucked about as bad as a single drive and b) the closed source drivers rebuild the raid array with no warning if a drive fails and is replaced, even if the file system is mounted. So, this means that if you have a drive that bombs and you replace it, anything you write to the raid array will be wiped out. I could have used some notification.
The Linux drivers are horrible. They are written in 'Engrish', and the documentation might as well have been written by someone who doesn't understand computers. "Select the remove drive from array option to remove a drive from array". This continues for all of the options in their menu-driven app.
I am also forced to use Red Hat 7.3 for this. Great. I now have a cluster of Debian 3 servers I administrate and one Red Hat server.
I would have returned the card if my reseller would have taken my money. It's about equally expensive to buy IDE add-on cards, or maybe a bit less, and the software RAID in Linux seems to be firmly documented. I've used RAID1 in software on servers before, and it works nicely.
3-4 years ago, when we decided to use hardware RAID on our Linux servers, we bought some DPT Smartraid V hardware RAID controllers. Unfortunatly DPT was bought by Adaptec some time after. Adaptec has been really good at getting the driver included in the kernel, but the takeover seemed to delay this proces, so the time in between was a rough ride.
The lesson learned was, never have a production Linux system with (binary) drivers tied to a specific kernel or distro version.
That said, we have been very happy with the controllers, and since at least two disks has died without warning, the expense has easely been worth it. Our systems are used 24/7/365, so every minute of downtime annoys somebody. RAID really makes me sleep better, restoring a server from a slow tapestreamer, at some ungodly hour, while people nervously checks in, asking when we will be up again, is something I really want to avoid too much of.
YMMV, but I think hardware RAID still has an edge over software raid, mostly because I find it simpler to maintain in the long run.
If you are into LVM's, FS tools, and software RAID, go to:
http://evms.sourceforge.net/
and _drool_. Future stuff for now on production servers, but nevertheless.
Tons of Dell's with the PERC (aacraid/megaraid) controllers.
The nice thing about hardware raid is other than the driver for the scsi card, the OS thinks there is just one drive sitting there. No configuration on the OS side.
Also, RAID is going before the OS even starts booting. If a disk dies, so what.
Please correct me if I'm wrong, but if you have software raid and the disk the os/boot/raidconfig files are on goes, you have a dead box.
I'm adding my vote for 3ware. You can get an 8 port IDE card for less than $400 or an 8 port SATA card for under $500. Beware big IDE raids, cabling issues become a major pain. We ended up using 36" (out of spec) rounded ide cables for our last 1.1 TB raid. I'm hoping that next time, we can use serial ata disks and the forthcoming 3ware raid card with native SATA. Their current offering uses the same chips as the 7500 IDE raid controller, combined with SATA bridges.
Do you really need hardware raid? These days disks are so big that you don't need RAID5 (and thus a beefy raid controller) unless you need an unholy amount of storage. For redunancy, RAID1 will do nicely, either in software or cheap hardware.
Hun, I though if you have a RAID 1 mirroring software array running on linux, linux is smart enough to access different files from different drives. Muliple readers is something hardware raid can not do unless they have a special driver, and the os supports it. Writes take twice as long though.
I use a software RAID 1 mirror array on a squid proxy cache box. I would guess that the multiple readers on different drives would increase the throuput for a web caching proxy server since web pages have so many little files.
E-mail the HOWTO writer, or locate a Linux Wiki and document those errors as being unmatched DMA settings or potential hardware failure. And leave a spam-proofed email address.
It was an blatant tech support request.
You got the answer you were fishing for without buying the book. The top post should be modded -5 Lazy-SOB
at O'Reilly, mdadm
and, I'd recommend Enterprise Volume Management System rather than LVM ( Logical Volume Manager ), simply because LVM's seems to be being dropped as
redundant ( ironic, that : ) as EVMS gets more effective, and I don't want the conversion-work from LVM to EVMS, if I can just do EVMS right now, see
Messages to/for me ( in me journal )
For IDE hardware RAID accept nothing less than the incredible 3ware (http://www.3ware.com) Escalade series of RAID cards.
These things are AWESOME!
The 7500-12 (parallel version) is relatively cheap since they came out with the 8500 (SATA version) series.
Very good cards indeed.
Plug the 64 bit controller into the 32 bit slot and let the PCI bus be your bottlneck. See their features under minimum system requirements, which specifiy 32 bit or 64 bit. Also, motherboard compatibility, which lists many chipsets without 64 bit pci.
The reason we went with 3ware controllers last time was that we were maxing out the pci bus. Moving to a 64 bit card solved that problem. It's too bad 3ware didn't opt for 66 MHz PCI instead of the wider bus. I wonder how much this has cost them?
Both systems have their pros and cons, here are some points to consider (HWR=HardWareRaid, SWR=SoftWareRAID):
- HWR with cheap controllers also stress the CPU as only parts of the RAID-Code is executed on the card.
- In HWR, you can only RAID whole disks, not partitions.
- HWR controllers need drivers - for some there are no Linux drivers, some are "alpha", some are not included with your favorite distribution etc.
- Booting with SWR can be quite tricky
- The Firmware of HWR controllers can be buggy - getting a fix is probably not that easy as fixing a bug in Linux SWR.
- What happens, if your HWR-Controller fails and you cannot get a replacement? How do you get your data back? This is no problem with SWR.
- Hotplugging is probably better supported with HWR solutions.
- HWR is not necessarily easier to setup. Managing my Mylex DAC960 was a lot harder than setting up Linux SWR.
I would not say that HWR is the only way to go in production systems. It depends somehow on the application. I think, SWR can be more secure in some way. Probably the performance will not be that good as a $1000.- HWR controller, but maybe it's "enough" performance. Why spend $1000 if you just don't need to?
What about a Linux Firewall? RAID performance is no real issue here - but security for the logfiles is.
What I would really like to see is how Linux SWR performs to various HWR solutions. There are some tests on HWR controllers on Tom's Hardware page, but it seems there are no thorough tests on Linux SWR.
I've just been through setting up a raid system. I set up a file server that automatically backs up data every week that the users on the network put on it via samba. Since I only want to show up at the place every 6 month or so to check on the server it needs to be bullet proof to the max and still cheap, because they don't have much money as social workers.
:-(, since doku is the last thing those guys seem to think about.
I purchased a used p2 system with a stable mb and two ibm scsi drives on an adaptec controller. I installed Debian GNU/Linux stable and upgraded to the latest stable. Then I put up a softraid and opted for xfs in case of a power failure. I decided against an ups, because I hooked the machine up to the local power network, which is very stable, since the server lives in Berlin/Germany, and I wanted to save the cost.
Then I moved the root filesystem over to the raid device. Up until now everything was documented very good, except for the fact, that I heard that reiserfs doesn't work with softraid and I didn't find that info on the net anymore. I would have taken reiserfs instead if I would have had a reliable source, such as the book, telling me that that is OK.
The only thing I had problems with was how to make the system boot off the raid device. Here the howtos and man pages had contradicting stands on how to do this.
I read this Slashdot article with some regret, because I thought it could have saved me a lot of trouble. But the only section that gave me trouble also seems to confuse the auther of the book. Now that is no help at all. So this book is a waste of time if You know how to use google, which I had to learn painfully fast getting into Debian
But since Debian is still by far the best system out there overall I have no choice. If You start to rely on seemingly simple things such as a reliable update of Your system with very low hassle then You are hooked.
Anyway, I would also like to mention that the Multi Disk HOWTO (homepage) is relevant, includes layout examples and also a trouble shooting guide. A fault tree would of course be nice.
The HOWTO also mentions that there are actually 2 different RAID systems available, do make sure you get the right one and the corresponding tools. Additionally it refers to a number of hardware RAID HOWTOs, of which there are many.
And please, please send feedback. That HOWTO author actively asks for inputs, so why don't you?
I have started using the Promise SX and RX 8000 external RAID arrays and have nothing but the highest praise for them. For those who aren't familiar with them, they are self contained ATA RAID chassies with 8 hot-swap trays. Each drive is on its own ATA-133 controller and the RAID (0, 1, 3, 5, 10, 30, 50) is handled by an on-board RAID controller with expandable cache, etc (no software RAID). Hot-swap, hot-rebuild, hot-spare, hot-expand, even on the fly RAID level conversion. Connects to my servers via a U160 LVD cable and is treated as a really really big SCSI hard drive...no drivers required. The array is controlled in one of three ways: 1.) Onboard LCD display with soft-buttons, 2.) Serial console, or 3.) Some Windows utility (haven't used it). For exactly $4,025.xx I put together an SX-8000 with 8 Maxtor 200GB 7200RPM drives. Even comes with an LVD cable and a serial cable (for console work). Simply awesome and great bang for the buck. I haven't used the rack-mount version (RX8000) yet, but one is on the way. If you're in need of such RAID, I would highly recommend it. They also make a 4 drive version (RX/SX 4000) and I am told an RX/SX15000 is on the way.
When you mentioned 4x60GB RAID-1, I think you mean RAID-10. RAID-1 is mirroring between two drives. RAID-0 is striping, that is, the data is spread across two drives. RAID-1 with four drives isn't possible, but with RAID-10 (two pairs of mirrors combined), it is.
Pete