Slashdot Mirror
How Are RAID Arrays Identified By Hardware?
Coward Anonymously Before Me asks: "This is more of a tech/hack question, but recently my highpoint controller forgot my disks were in a raid array. All the Disks still function, and have ZERO problems, aside from being not identified as still in RAID-0. All the data should still be there, but remains unaccessible to me, thus the question how and where would this kind of information be stored? On chip? MBR? and can the data be recovered without 3rd party interaction via free/open source toolkits? or even purchased software?"
...you ought to be able to just dd the data from the drive, and if the chip doesn't use a non-standard data layout, write a program or script to put it back together and back it up, then you recreate the array, and put your data back on it.
A solution to the problem with music today
Did you try Google before submitting that?.
Typically, there's a utility in the RAID configuration that stamps the drive as part of a set, marks the state (good, bad, rebuilding or hot-spare are most common) and some kind of versionig information.
I've ripped a few disks out of the array, mounted them as standard, reformatted and replaced the MBR, threw them back in the array, and still had them recognized as part of the RAID volume. The RAID card didn't like this much, however. :)
I think your best bet is to talk to one of the people who actually wrote the drivers for the card (you've got the Linux source, right :) or possibly see if you can get ahold of an engineer at the manufacturer and discuss ways of getting the information back.
Good luck!
There's so little difference between politics and jihad lately...
...I went to Google and found this:
"Where does the controller store its configuration? Disk, Ram, Rom ? Can it be saved or exported as a backup?
The config is in NVRAM on the controller, but there's also about a 4MB
"partition" (not really a partition, but config area) on each drive in the
array that stores information on things like position in the array, etc.
Alternately, try this link for the whole thread on the subject.
It literally took me 30 seconds to find it. Next time, try Google before posting, mmkay?
Cruising the internet on my TI-99/4A @ a whopping 300 baud!
And we are going to see lots of other flaws in open source software, I'm sad to say. Luckily they will mostly be discovered long before anyone take advantages of them, in contradiction to certain other OS's where on only hears about the holes that are massivly exploited.
Logical volume managers (AIX and Veritas anyway) store a unique ID on the disk, and then keeps track of what volumes are there, how they are configured, etc.
Hardware controllers generally reserve a small slice of disk to store configuration data. Sometimes this slice is marked unusable and can only be accessed by low-level hardware.
One of the big, unadvertised problems with RAID, particularly with new/buggy controllers, is that a controller failure can trash your data.
Unless you have the time & knowledge to reconstruct the data structures, a controller failure that screws up the configuration data on disk effectively destroys your data.
Conformity is the jailer of freedom and enemy of growth. -JFK
You confused?
I think he posted to the wrong topic, note the next story. However, he's a troll, so it doesnt matter either way.
-- 'The' Lord and Master Bitman On High, Master Of All
I saw nearly this exact post a couple threads back! Maybe you should try the slashdot search feature before making a post, mmkay?
-- 'The' Lord and Master Bitman On High, Master Of All
people get annoyed with me that I ask questions on IRC or message boards which are covered thoroughly in manuals. They respond with, of course, "RTFM"
Am I the only one who thinks that a Manual is a pretty lame source of information to reach for first-thing?
I have a few sources I go through, usually the manual is one of them, but I _Always_ first ask a person who might have the answer on hand. Manuals are not often things which lend themselves to answering typical questions such as "Can I blah?". The problem with "Can I blah" being looked up in a manual, among other things, is that often there are numerous synonyms for 'blah', and only one of them is ever used in the book, especially the index.
Perhaps they mean "Read the entire manual before even using the product". The obvious problem here is that manuals are getting longer every day. I've heard that some Linux Distro comes with a 2000 page manual just for getting it installed. Obviously, to read an entire manual before using a product would leave little time for using any products, and leave you more or less unknowledgeable about the product.
Then there's the problem of phrasing. Manuals may answer your question, but only burried in a lot of other information which isnt related to what you're actually working on. A person who knows already, however, can simply answer your question.
Slashdot, however, is far too public and non-specific. There's no reason to ask this kind of question on slashdot, get some friends or something.
-- 'The' Lord and Master Bitman On High, Master Of All
No, I think you need to use the /. search function.
The post you're referring to simply said to check Google.
I have a few sources I go through, usually the manual is one of them, but I _Always_ first ask a person who might have the answer on hand.
So, you're openly admitting you're lazy and would rather rely on asking someone else rather than looking for the solution yourself.
Does your mommy still come over to wipe your bottom too?
Grow up and do it yourself instead of leeching technical knowledge from someone else. If you can't find it by exhausting searches, then ask.
Unless I misunderstand - your system *is* broken without the controller anyways right? So why not just replace it?
.sig wanted: Must be concise, funny, and display my cleverness.
Slashdot is not Highpoint Technical Support. Your question isn't how a software only IDE RAID stores data, your question is how do you fix your Highpoint config.
You can reach the web page at highpoint-tech.com, and reach support via support@highpoint-tech.com
Good luck to you, but ask your question in a support fourum.
If your controller just spontaneously lost it's configuration, it's a problem with the card. Call the manufacturer and get a replacement. If they will configure it for you before they send it (which shouldn't be difficult for them), you should be able to just swap it in and go.
You'll need to do something like this, since you won't be able to get at the data on the drives unless you can hook them up to a RAID controller that will recognize the particular flare code etc of your setup.
If you can stomach losing the data (you backed up the important stuff, right?) then you could try starting over from scratch, but I would not trust your RAID controller if I were you. Replace it or don't use it.
Under capitalism man exploits man. Under communism it's the other way around.
I've worked quite a bit with AMI controllers and Adaptec. At work we looked at using an Adaptec ZCR card but chose not to for the following reason.
AMI Megaraid(and now LSI) write a bit of config info to each disk and to the controller. On these cards, you need to know the drive designations (which is drive1, which is drive 2, etc and the stripe size (how much data to write to the first disk before moving on to the second). On these controllers, if your card goes belly up you can usually put in a new card which will detect that your drives still have a configuration and use it. Otherwise, you can create a new configuration of the drives (same raid level, same stripe size, each drive with same designation) and it will access the data just fine on a reboot. (probably 75-85% of the time. The rest of the time you are just SOL and need to get out tape.)
Adaptec's ZCR card we were testing and going to ship had the unfortunate effect that when an array was created, it immediately initialized (format) all the data.
This is something you should check into. Perhaps the highpoint card will let you make a new array and reboot. Or it might automatically initialize and wipe out all your data before letting you use it.
Hope this helps
--
That's the sort of attitude which makes people prefer to run Windows. I can find the answer to *any* windows problem I have within ten minutes on google - Linux isn't there yet. So surely you should be helping people?
Haha!
... right now I wouldn't consider slashdot my one-stop "info shop", but why not? it's easy enough.
... that's a little too homophobic for me. Rumor has it he was spotted sitting alone in a hotel room wearing a confederate flag like an Aunt Jemima headband, a vest, and nothing else...
This is your third post in this thread that has nothing to do with answering the question (deemed too simple to be worthy of your time) but rather with the proper way of responding to the question, how the question is lame, etc.
Yeah, I'm just adding to that pile - but you're killing the hostages so that they're not killed by the kidnappers in an attempt to "save" em, here.
~~~~
Anyway, here's the proper way of not cluttering up slashdot (gee whiz, isn't it up to those accepting the questions to decide that?):
The answer is: there is some info regarding the drive partitions in the MBR, but most likely this information is stored on your controller itself in what's known as NVRAM.
Besides, I'd ask Jeeves but Jeeves prefers the term "jovial"
-- The truth is the only thing that nobody will believe.
I had a very similar thing happen to me a while back. It was caused by a hardware problem (I was using a pair of IBM deskstar drives), so I never did recover quite all the data, but I did manage to get the RAID array back together. By a lucky coincidence, this happened shortly after drivers/ide/hptraid.h was added to the linux kernel, so I had somewhere to look for inspiration.
This file describes the structure of one sector somewhere near the start of each disk. (Sorry, I don't remember exactly which one.) The magic number had changed on one of the disks from HPT_MAGIC_OK to HPT_MAGIC_BAD. Editing it back again was sufficient to reconnect the drives.
I've got a bunch of these controllers in use. They store the entire array configuration on the disks, that way if the controller dies, you can plug in a new one and it will pickup the config and work right away..
When moving an array to another system that doesn't recognize it, I usually just "add array" and enter the parameters of the config. Then reboot and all of the data is intact.
Is this a HPT370 based controller? I had a very similar thing happen to me a year or two ago. The RAID controller just "forgot" its stripe config, which I think is stored on a very small reserved segment of the disc. Anyway, there's a utility called raidrb (google on raidrb.zip) that I used which fixed the problem without data loss. This used to be covered in an excellent FAQ that was on viahardware.com, but apparently it's gone now.
Low level the drives using the RAID controller card, recreate the array, and restore from last night's backup.
Voila! Complete Recovery in less time than it took for all these responses to be posted.
No backup? Ewww - you are going to hate Monday.
Glonoinha the MebiByte Slayer
The problem with that is that there are very few people who'd like to answer beginner questions. Most people who come to those channels to help have a good knowledge and want to feel useful. I don't know about you, but I wouldn't come to a mathematician to ask her how to compute the square root of something. Surely she's got something much better to do.
This is why things like FAQs exist, people are tired of answering the same questions over and over, when there are far more interesting things to do. #debian has the apt bot, for example.
Also, it depends on how you ask the question. If you think that people in support channel come there for answering every of your questions, you're mistaken and will be rightfully flamed.
BTW, manuals have an useful feature called "index". Maybe you could try using it.
I accidently deleted my first reply to this so I'll make this one brief. It seems there are just as many, if not more, hits on google that deal with Linux than Windows (about 54 vs 59 million for linux vs microsoft. accounting for windows pages not dealing with computers, I'd say they're about even). There are certainly more linux "answers" out there than windows, its just that most of the linux answers that you need are probably basic enough that most of the people running linux don't ask them. Windows is incredibly difficult to find information for. Once you get into hardcore networking and stuff like DCOM and intricate implimentations thereof, you're out of luck in the windows world.
"Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman
1) Backup the drives manually just in case (copy the raw drive data off to tape after booting from CD or floppy).
2) Go into the hardware raid setup and set the array up exactl as before, but dont let it initialize. If your hardware raid controller always initializes new divces by writing over them with zeros or something, this might be undoable or tricky. If you end up initializing, it's not that big a deal.
3) If you did it without initializing, you're probably good to go. If you ahd to initialize in order to configure the array, now boto off of CD again and restore the raw drive images from tape.
Done
11*43+456^2
BTW, manuals have an useful feature called "index". Maybe you could try using it.
Grandparent wrote that many manuals aren't well-indexed: "The problem with 'Can I blah' being looked up in a manual, among other things, is that often there are numerous synonyms for 'blah', and only one of them is ever used in the book, especially the index" (my emphasis).
Will I retire or break 10K?
I went to Google and found this ... It literally took me 30 seconds to find it.
Next time, if you're reporting a Google result, please give the query string that you used. This way, people who read your comment become better users of the Google search engine.
Will I retire or break 10K?
given that a large portion of the documentation for a distribution is available on the computer, one might try searching through the documentation with the computer, if the index is not sufficent.
man and info both support searching (with the / key in both, i believe)
most of the documentation for linux/unix that is not in man or info pages is in html, and most web browsers also support searching, although the key may vary
Need a Catering Connection
Of course not every manual makes it easy, but there are some that are made exactly for that. For example, the Perl Cookbook. The whole book is just "How do I" questions and answers.
About the synonyms problem. Some books I've got here have a short introduction in the form of "foo is also known as bar and baz, but for consistency reasons we'll call it foo". If it's hard to find information in a book then it's just not worth buying. And anyway there's always Google.
Answer: Most RAID (SCSI and IDE) place configuration information at the end of the drive in the array. Typically one or more logical cylinders are lopped off the drive size and used to store the config info. As the config info is not part of the drive's "size" when in RAID mode, the info is inaccessible. Also, the info is drive specific, so copying(dd, ghost, etc) won't solve the problem completely. Don't expect anything out of the config info as it is VERY manufacturer specific.
:-)
Solution:
1. Backup everything.
2. Copy (dd, ghost) the most current drive to the least current drive.
3. Go into the Highpoint BIOS config and re-declare the drives as a RAID-0.
4. At this point the Highpoint BIOS will probably try to initialize the RAID-0. Since you copied the drives using a sector copy program in step 2, the direction of the Highpoint initialization doesn't matter.
5. If this worked, weverything will boot normally with the RAID-0 config.
6. If this didn't work, the drives are probably irrepairably hosed. Restore from the backup you made in Step 1. You did make a backup in Step 1, right?
Good Luck,
Ed
If somebody asked me how to burn CDs in Linux, I'd tell him/her to RTFM
On the other hand, I'd help the user learn to formulate a query: "Go to Google.com, enter [ linux burn cd ], and click Google Search." Or, if a question is answered in the FAQ: "Look in the DJGPP FAQ, section 8.3."
Will I retire or break 10K?
About a year ago the exact same thing happened to me with the built-in HighPoint HPT-370A controller on my motherboard. I was able to recover all my data by simply recreating the array exactly as before, then running an "FDISK /MBR" (I'm uncertain what the Linux equivalent is). This 'undeleted' the primary partition. I then used a program called "testdisk" (try Google), which runs under DOS, Linux, and several other OSes, to undelete the other partitions.
YMMV, but good luck.
--- Remove all references to mud-dwelling quadrupeds to email me.
Did you actually say that you're willing to purchase software in order to perform a task that is worth quite a bit to you?
Someone alert the FSF! You're clearly insane! The software industry's brainwashed you, and you need help!
Only on slashdot can a posting be rated "Score -1, Insightful".
The real question you should be asking yourself is, of course, "Where are my backups?"
Wow, hardware is not software. *BING* *BING* we have a winner!
I'm glad that a genius like you finally made that connection because us average dumb fucks already figured that out 18 years ago.
Since when has slashdot been ONLY about open source software. In fact I thought it was also about FREE SOFTWARE and propetary software like this, this, and this. One of those companies, a produceer of propetary hardware and software, even has its own section here. Then one of the other two has very frequent stories here.
Anyone know where I can find information on the Rockridge standard? I can't find it anywhere and I heard it can burn files to CD including permissions information, I need to get specification though since I need to have each .iso generated by a script.. :D
What? I figure we're off-topic enough already as it is
Though it does bring up an interesting point that some people really just need to know where a manual is, or if one exists. Manpages often tell everything you could ask for regaurding command line switches, but doesnt talk at all about the interface, or even what it is the program does. The most helpful source of information could be burried in a seemingly unrelated HOWTO which you'd never know to search for without knowing the program's intended function in the first place [you just found it in some error report from cron]
I get the feeling that a lot of people who say "RTFM" havent actually read the specific manual in question, they've learned from others, and through experience with related things. I hate being told to RTFM after I just went through the entire thing and have found nothing even related to my problem.
-- 'The' Lord and Master Bitman On High, Master Of All
14 hours of searching I can consider an exhausting search.
So it's really a question of 14 hours vs near instantaneous.
If you dont pick instantaneous you're merely a fucking idiot.
-- 'The' Lord and Master Bitman On High, Master Of All
Buy El cheapo IBM Serveraid 3L Controller,for 10$ each.Buy a second,just for the failure of the Controller (as in these case).If it happen,just boot with the new Controller,insert Serveraid-CD,klick "Import Configuration from drives" and you are done.Its that easy,isn't it?
That's strange.
It's been my experience, especially with simply coding questions, that it's easier to deja and google for competent Linux/Unix answers than it is for competent windows answers.
Of course, I often find more windows answers overall, it's simply that they just aren't very helpful.
If you could be told what you can see or read, then it follows that you could be told what to say or think - BoC
I've had some experience with this, and if you know the exact configuration of the array and recreate it using the RAID configuration utility that came with the adaptor, you have a chance of getting it back.
Let's say you had an 8-disk array with disks 0 to 7 configured as RAID 5, with parity striping and with disk 7, the last disk in the array, assigned as the hot spare, or however it was configured, but you're going to have to know that. You'll have to know whether it was RAID5, RAID4, if there was a hot spare or not, and which disk it was.
Anyway, if you recreate the RAID array but don't initialize or format it, you might get the data back. This has worked for me occasionally and I've done a lot of RAID troubleshooting and hard-disk disaster recovery in my time. In any case you have nothing to lose at this point by trying. If you don't know the exact configuration this won't work. Good luck.
That's also the sort of attitude that makes sane people dread writing software for Windows. Especially for the really odd problems, half the time you'll find other people on Google asking the same questions, and no signs of any answers.
All I want is a kind word, a warm bed and unlimited power.
I have a *far* easier time trying to find answers for linux. If a google search doesn't do it for me, there's almost always a mailing list I can join and talk ***directly to the developers*** There is nothing like that in the Windows world. I've also had much better luck finding code/programming examples for Linux. I recently was trying to work with Audio APIs in both OS's and had much better luck with Linux until I found some open source cross-platform code to use w/windows.
Dear Slashdot Tech Support,
My camputer won't start any more. When I turn it on it just gives an error message. I have Windows 95. Please help, it's urgent!
I'm not familar with the consumer IDE raid stuff although I've worked quite a bit with Compaq SmartArray controllers.
Infact, I have a Compaq Smart array SCSI RAID controller in both my PC's at home (desktop and server). They're available on ebay for very nice prices and are well supported by Linux.
The reason I mention it, is that I can pull all the disks out from one machine and pop them into the other - I don't even have to keep them in the same order! - and the controller is able to read the array configuration data from the disks and the logical volume(s) are immediately accessible. It's pretty sweet.
If it's just your RAID controller that's failed, try putting the disks into another machine with the same type of controller, or try replacing the controller. I'm not making any promises, but if it's a decent controller it should read the saved config data from the disks and make your logical volume available.
Have you tried the manufacturers tech support line? I'm sure they've delt with this situation before. I'm not trying to be a wiseass, but sometimes contacting the manufacturer is going to get you in touch with the folks who are most knowlegable about the product.
I'd rather be a conservative nutjob than a liberal with no nuts and no job.
Well, it seems that a few people had smart ass comments such as "Read the Manual" or "Slashdot is not Highpoint Tech support." Well, maybe you should take your time and use it more wisely. For all you know, your "advice" has allready been attempted. If the person who posted reads slashdot, he/she more then likely knows the nature of the slashdot comunity. If thats the case, then most of your advice, such as read the manual, and call highpoint has been done. When was the last time you e-mailed tech support and got an answer back? I never have besides "We are working on a resolution to your problem..." I have too much time on my hands because I am writing telling you what assholes you are, but maybe it would be nice to help rather then refer. Also, what if the controller is onboard? For a bunch of Linux geeks, I'd expect someone to think of that.