Useful RAID Tools?

msaes asks: "I've got 4 machines now that I'm running RAID5 on. 3 are Dell's with the PERC (Adaptec) SCSI RAID controller, and one is a software (Win2k Pro) RAID. In all 4 cases, the MS defrag program, and the Norton Speedisk program said that the logical drives are horribly fragged. And from disk I/O performance, I'd tend to agree. Running the MS defrag on any of them is futile. It just cranks away for a while and then cheerfully says that it's done, with little or no improvement. I've run Speedisk on the software RAID machine and it's run for about 3 days solid now and performance on the drive is only getting worse. My question is: Does de-fragmentation software get confused by RAID volumes and actually fragment the drive worse?" Which brings yet another question. What tools are out there for the effective management of RAID volumes? Other partition types have a wide variety of maintenance tools, aside from the defrag utility, like a partition editor, an undelete tool, analyzers, and so on. What about RAID? What tools do you use to make sure your RAID volumes are happy and healthy?

Comments

Nothing much...

[MrWinkey]

We have several(5??) Dell 540 Precision Workstations that are Win2K with a level 1 RAID setup for GIS. We run Disk Keeper on them and it seems to keep it up. For the most part they seem to be ok w/o trouble.

We have another one we use for a server with level 1 RAID and we have never ran any type of disk tool on it and it seems to work just fine but it is mostly used for uploading images.

In short. Alot of use Buy Disk Keeper if not dont worry about it or reformat and reinstall.

Wierd

[itwerx]

Sounds like you have something else going on unrelated to RAID. Hardware RAID is completely transparent and shouldn't affect your defragging, (which, by the way, is still effective because the controller still accesses the drives in a consistent manner).
I would look at what's running on the system (virus?) or maybe the driver for the RAID card(s).
BTW DiskKeeper is pretty good...

Re:Wierd

[pmsr]

These Perc controllers are integrated on the hardware it seems and need special drivers that may interact in some weird way with some parts of the operating system. That is why i stay way of internal raid cards. I always try to go for external storage with its own embedded intelligence, and these plug to the servers using standard scsi or fibre adapters. This brings some added advantages to us. In low end servers i can use external scsi raid boxes with ide disks, with a very low cost per MB, and very good reliability. I can make the servers boot easily from the raid boxes with no dependency on special storage drivers. I can exchange servers in a snap if they break or if i want to upgrade, the new server just needs to have the same scsi card, and with Linux the rest is piece of cake. And with the cost for storage with more than one host channel going down, i can plug more than one server to the same storage, both sharing filesystems (Gfs) or raid logical disks.

/Pedro

Re:Wierd

[dfreed]

Who do you prefer for external devices like you described? I am in the market for a unit like that.

Re:Wierd

[pmsr]

For external ide storage with scsi interface I've had just good experiences buying from http://www.raidking.com . They even shipped boxes to Europe for us. They have a full line of scsi hardware, that FYI is rebranded, so you may find the same hardware in other places. But at least you will be able to gather solid information from their website. Their service was awesome though, and you could hardly do better than sticking with them. For high end hardware (scsi and/or fibre everywhere) i have been a http://www.dothill.com customer for years, but that is another story.

/Pedro

Vendor support

[gruntvald]

Why don't you ask your vendor? After all, isn't that the point of buying proprietary software in the first place - that you get "support" ?

My own experience of Dell shows that they may have reasonable 9x boxes, but they have no clue how to build something that will run NT.

Re:Vendor support

[Anonymous Coward]

Dell's PowerEdge line is no worse than Compaq or IBM's NT servers.

they have no clue how to build something that will run NT

Care to elaborate?

Re:Vendor support

[Llama Keeper]

I use Dell for all my Windows 2000 workstations. What problems are you havinf with them. I support 350-400 workstations, all Dell, all runnning windows 2000, and I have only one other person in my shop. What do you mean by Dell Machines can't run NT?

I'm just curious.

Re:Vendor support

[(startx)]

I totally agree. Dell couldn't spot a decent server raid controller if it slapped them in the face. we've got 5(!) e2k dell servers at work for a mere 6000 people totoal, just becuase they go down so often. And it's always been a hardware problem cause database corruption.

Re:Vendor support

[pmsr]

Five servers for a "mere" 6000 people? It doesn't sound much. Some people actually do work you know? What kind of workload is on these servers? Is the workload distributed? Concentrated when during the day? Are these servers you mention all doing the same thing, databases? If your 6000 people put the workload on the servers my 60 or so use to, five servers would be way too little.

/Pedro

Diskeeper

[Halvard]

Like 2 previous posters, get and use Diskeeper. I've used it going back to the 1.x releases and its done nothing but improve. MS Defrag is based on a old version of Diskeeper that is can't be scheduled. It's junk. Further, Executive Software, the publisher of Diskeeper, wrote the API for defragmentation. And it's what Norton uses for SpeedDisk in NT flavored OSes. But Diskeeper is much faster. Besides, it will defragment your directories, MFT and paging files as well.

Re:Diskeeper

[sharkey]

MS Defrag is based on a old version of Diskeeper that is can't be scheduled.

The Win2000 version was current when Win2000 was released. (I don't know about XP.) It was, from the beginning, a stripped-down version of Diskeeper. The plan was to put in a basic tool, then if people wanted more, they would buy a better one. Really, this just brought the NT code-base up to speed with the DOS/Win code-base, which has had a simple defrag for years.

Which is worse?

[Pauly]

Which is worse? Microsoft or Scientology?
Buying anything from the makers of Diskeeper puts money in the hands of one of the world's most notorious cults.

Besides, it sounds like you're facing a grown-up problem. Why not use a grown-up filesystem and/or a grown-up operating system?

Turn Off Read and Write Caching

[DorianGre]

On the Dell PERC(2) Adaptec, not the PERC(II) AMI, turn off read and write caching (can be performed through the software tool) befor defragging. The read and write cache will confuse degrag software. Not a problem with the AMI PERC (aka MegaRaid). Once done, turn back on. This cannot be scheduled, so you have to be there.

PerfectDisk

[mweber]

As far as defrag tools go, I can't comment on Disk Keeper, but I've had pretty good experience with PerfectDisk from Raxco Software.

Among other things, they claim to be the only defragger that defrags all data files and all NTFS metadata files, and they list a couple of other "exclusive" features.

fragmentation beneficial?

[humanasset]

Isn't fragmentation beneficial on a RAID array? Wouldn't the file be distributed over multiple disk spindles?

I would guess that having files in contiguous areas of the physical disks would be disadvantagous.

Re:fragmentation beneficial?

[addaon]

But keep in mind that, to the software, the raid partition looks like a single, linear file. So if the defrag program makes a file use, say, the 4-8th megabyte of 'disk' space, it's actually using the 2-4th megabyte of each real disk. (Of course, this is assuming simple striping, but a similar argument applies to raid 5.)

er..

[Wakko Warner]

If you've got a Dell with PERC controllers, they come with great PERC utilities that will do everything you need. Install them. It's a good idea to be able to monitor your RAID 5 sets.

- A.P.

Container Scrubbing

[SpaFF]

Have you tried scrubbing the container?
I believe this checks and fixes errors with the raid volumes themselves (not the filesystems on the volumes). Using the afacli command line utility for the Adaptec PERC controllers you would just type "open afa0" and then "container scrub".

Re:Container Scrubbing

[sporktoast]

Well, if scrubbing doesn't help, maybe you should switch to Tide!

Questions

[sharkey]

What software are you using to create a software RAID-5 on your Win2000 Pro PC, since the Pro version cannot create RAID arrays on its own. Perhaps you should talk to the vendor about fragmentation, and defrag software issues. The PCs with an actual RAID controller are different. Windows, and thus Diskeeper (the MS defrag program), sees them as nothing but a single disk. I have had no problems defragging hardware or software RAID arrays (1 & 5) using Diskeeper.

Performance-wise, it goes into the toilet while defragging. The defrag operation moves fragmented files to free disk space, then copies them back to the "beginning" of the drive to put the file fragments into adjacent clusters, thus making them contiguous. This chews up a huge amount of disk I/O, stretches out seek times, etc. Also, how much free space do you have on these volumes? The less free space you have, the less space the defrag program has to work with. On a volume that is almost full, defragmentation can take a LONG time, and you will likely need several passes to see significant improvement. If you can, move some of the files off to another volume or tape/DVD/etc., defrag, then move them back. If you get the retail version of Diskeeper, it can consolidate directories at boottime, allowing a bit more optimization.

And remember, this is SCSI you're dealing with. Never underestimate the power of a well-sacrificed chicken.

Unix (AIX) RAID 5

[Anonymous Coward]

I am running a 300 GB RAID 5 array on AIX (Unix) on 36 GB disks. Every time I got about 22 MB/s no matter how fragmented, big or small, are the files. I defrag the file system once a month (yes, AIX has a defrag... sorry Linux...). I never had a problem.

driver issues etc.

[gruntvald]

Well, I quit using Dells about 18 months ago. Sounds like they may be better. The issues I've had with them were a sample of about 30 machines: CD drives not being recognized, internal zip disks causing lockups out the box, all of them had video card issues, modems not working, no recovery CD being provided with the unit. In a nutshell, many, many hardware incompatibility problems that prevented the machines from even getting to the desktop. I still get occasional calls from tech. support at Dell to see if my machine issues are resolved, and the last one was returned (after a long protracted process) over a year ago.

I can't believe nobody has suggested this yet.....

[Anonymous Coward]

Have you tried Spinrite yet? Seriously, M$ defrag is crap, read the FAQ for spinrite here-
http://www.grc.com Cool little site, too BTW, all
kinds of useful utilities for Win32.

Aughhhh, Damn voices writing my .sigs again...

What Models?

[Llama Keeper]

What models were you using? I don't want to order any of those. I usually use 2100's or 4100's. Dell tech support has already been great to me. I buy only Dell workstations (with flat panels now!) and Compaq servers. Its been a killer combination for me for a very long time!

PERC RAID speeds

[icewalker]

These PERC RAID cards are not known for their speed. We have been discussing the speed issues on the Dell List Serve located at http://lists.us.dell.com. Many people have said that software based RAID is much much faster. Though most of our speed problems are related to the linux kernel, specifically the Adaptec brand of controllers, a software based approach has some high points to it. Remember that the processor on that RAID card is no where near the speed of your CPU's. You take a bit of a performance hit, but if your server sits idle most of the time, look into it.

Icewalker