Slashdot Mirror
SCSI vs. SATA In a File Server?
turboflux asks: "I'm currently in the process of replacing an aging file server with something more robust. Company-wide, there will be about 100 people who could be using this server, but I don't imagine there being more than 50 concurrent users. Right now, I'm torn between spending alot on SCSI hardware, much like our other servers, or spending less, but getting more space, with SATA II drives. Whatever I decide, the server will be setup with a RAID 1+0 array for the numerous benefits it offers. Does Slashdot have opinions or suggestions on performance, reliability, and stability?"
Think about the fact that you need a sufficient Backup. You can buy lots of cheap storage with SATA Disks, but Ultrium 3 Tapes (400/800) are still expensive as fuck. Never forget that cost when calculating.
OTOH, there are 300GB U320 Disks now, which you could use if latency is not an issue. Otherwise, go with lots of disk arms (72GB or 36GB U320 Disks)
Serial-attached SCSI. 15K RPM drives, SAS RAID 1+0. Heaven.
If you like what I've said here, and want to read more, go to http://www.krillrblog.com
SAS hardware is currently a little harder to find than SCSI or SATA stuff, but I'm sure there's a good selection out there if you take the time to look.
I was checking out the Sun Fire 4100 a while ago, and it takes SAS drives, however the form factor is 2.5", and I haven't yet seen any 2.5" SATA drives (I wanted that compatibility). Also, I've heard SATA drives don't work with the Sun Fire 4100's SAS controller anyway. Not sure about that, since the SAS spec says they should work, but just something to keep in mind when you're looking for a server or mobo or controller that supports SAS.
You cannot buy the same performance class of drives in SATA as you can with SCSI. Some people call this a market segmentation scheme, I call it catering to the market. People who demand top class drive performance typically also want the other benefits of SCSI as well. Whether those benefits are needed for your requirements, well, depends on your requirements.
SCSI can (depending on which particular SCSI) provide you with more devices per controller without sacrificing (any noticeable) performance. If you need to shove a ton of drives into one server, this will add up quickly. Since you are talking about RAID 0+1, depending on how much storage you are shooting for, this may be a strong factor (but you may be able to skin by on the 4-6 SATA ports you'll find on most mobo's).
SCSI is more mature. So drivers are likely to be more robust, more efficient, and more stable than those you'll find in your garden variety SATA.
You'll typically find that under heavy load, SCSI performs better. Again, this is mostly due to so called "market segmentation" schemes, but that is why you pay more. If your users are going to be mostly dealing with the usual, periodic saving of word processing documents, spreadsheets, and a couple of light media files - you probably don't need to handle really heavy loads. The RAID controller will eat the peaks of write demand in cache (if you get a decent RAID controller - see later), and you should have fairly smooth performance. Then again, if your users are constantly running large installers (development test environment) or working with large remote files - you should really go SCSI.
All that said: I think you would be served best by investing in a better RAID controller rather than investing in top of the line drives. The RAID controllers they integrate on to most motherboards are crap (for what you are trying to do, desktop use - meh). You want something with a ton of cache, and good management soft/firmware. If you buy a real server class motherboard, you may get a better onboard RAID, but however you go about this - pay the most attention to this detail. Unless you really need low latency for high demand, random access applications, top end drives probably won't give you much over the usual network latencies.
I bought into the whole "SATA is the new less-expensive SCSI" and put in two new file servers using SATA last spring. I can say that I'm unimpressed with the SATA servers as compared with our SCSI servers. I now wish we'd spent the extra $1000/server and gone with SCSI. I recommend SCSI -- you won't second guess your decision down the road.
BTW, we used 3ware controllers with WD RAID Edition HDs. We're supporting approximately 75 users per server.
OK, hear mine then.
We have several terabytes of SATA storage at work to hold our main business-critical digital asset archive.
We've been using a ATA/SATA disk-only strategy for over 5 years now. It's worked great, and eliminated our slow and unreliable tape robot, which has greatly improved productivity.
Back in 1999/2000 SCSI wasn't an option for the main archive because a terabyte of SCSI would have broken the bank. We went ATA back then. It was a mess trying to route 24 ATA cables in a case, I admit. SATA fixes that nicely.
We keep three copies of our data, two onsite and one offsite. We use rsync-incremental snapshots to do disk-based incremental backups. Because the cost of SATA is less than 1/3rd the cost of SCSI, we get a high reliability solution for less than the price of a single SCSI RAID.
One more advantage of SATA is that the disks are so cheap, it's easy to just replace all of them every two or three years. The disks you replace them with generally are twice as large after 2 or 3 years, so every cycle your RAIDs get more reliable as the number of disks is slashed in half.
Most companies wouldn't replace every SCSI disk every two years, it would cost way too much. And considering the slow pace of SCSI size growth, you wouldn't see as much gain, a double hit against SCSI.
So basically unless you need the excellent latency performance of SCSI, higher than even the WD Raptor can offer, I see no compelling reason to use SCSI for anything anymore.
I've had enough abrasive sigs. Kittens are cute and fuzzy.
Some more tips for you:
First, make sure you get a SATA2 controller. NCQ is a must for multiuser environments.
Second, whatever controller you buy, grab 3 of them. RAID is great for disk failure, but people rarely think about what they'll do when the controller fails.
Look at some of the stranger RAID options. If you just use RAID5, you'll be selling yourself short. RAID3 is worth a look. I'd actually suggest you put two controllers in a machine. Run RAID0 on 4 drives on a single controller. Run RAID0 on 4 drives on the other controller. Then use Windows or Linux software RAID to run RAID1 between the two RAID0 drives. Very fast performance and fully fault tollerant.
Keep the OS on a small, slow hard drive seperate from the array. You can do funny things there, but I'd suggest you set it up properly and then use a disk clone utility to create an offline backup to store somewhere.
Arrange for testing in the first few months. Unplug drives from the array and see what happens. Verify that you can restore from tape backup to the array. Veryfy that your cloned OS hard drive can actually get the array online. Extensive testing before you go live. Two tests in the first month after going live. One test every 6 months after that.
If you are using commodity servers, get a spare for everything in there.
If you want high avalibility, look into DRBD. It's like RAID1 over a network.
Monitor the damn thing! My last job someone let the server die. It had RAID5 over 5 drives. One drive had failed and no one noticed. When the second failed, that was the end of it. Learn to use SNMP or get some good monitoring utilities that will notify you of problems. You need to know if a drive fails, if it reports SMART errors, drive temp, proc temp and usage, NIC utilization, drive utilization, system temp, and memory utilization. MRTG+RRD Tool and SNMP will give you pretty charts for all that crap. MotherboardMonitor will also give some nice readouts for Windows. If it's commodity server, look at installing a Crystalfontz LCD so that you can walk by and get a quick status without the need to login.
I'd rather you do it wrong, than for me to have to do it at all.
Having replaced sixteen drives and lost two raid 5s with hot-spares no less, in the last eighteen months. I can't recommend SATA for anything that is mission critical.
"think of it as evolution in action"
Serial Attached SCSI takes the best of both world together:
SAS has:
- lean SATA cables
- 3Gbps transfer, soon to be 6Gpbs. Better than U320
- 15,000 rpm disks
- NCQ like SATAII
- RAID-capable controllers
- SATA on SAS possible