Slashdot Mirror
SCSI vs. SATA In a File Server?
turboflux asks: "I'm currently in the process of replacing an aging file server with something more robust. Company-wide, there will be about 100 people who could be using this server, but I don't imagine there being more than 50 concurrent users. Right now, I'm torn between spending alot on SCSI hardware, much like our other servers, or spending less, but getting more space, with SATA II drives. Whatever I decide, the server will be setup with a RAID 1+0 array for the numerous benefits it offers. Does Slashdot have opinions or suggestions on performance, reliability, and stability?"
SATA is fast and cheap; just make sure you spend a little bit more to get the "nearline" storage drives and not just desktop drives. Put them behind a 3ware 9550 and you'll fly.
Fellowship 9/11
It's more reliable, as far as I know, compared to SATA. SATA is good enough for desktop performance, but I have yet to hear any glowing reviews of it in the server market.
"Better to be vulgar than non-existent" -Bev Henson
For any tier 2 storage, SATA is the future. I'd go so far as to say that in about 80% of all instances, good SATA-II drives with NCQ and large caches in a proper RAID setup will be far, far more than adequate for most production servers unless you do tons of non-sequential I/O, need tons of iops, etc.
For a file server, you'll be fine with SATA.
For my tier 2 servers, I am moving a ton of stuff off of my EMC gear (because fibre channel drives are damned expensive) onto SATA-II drives in an iSCSI setup. I'm already running servers with trunked gigabit NICs... might as well let Win/*NIX boot from a local drive and mount block-level iSCSI devices over the gigabit fabric. Save a shit ton of money and get tons of space on the cheap (7.5 terabytes of fast SATA-II in an iSCSI chassis for well under $10K... make your RAID groups, carve up some LUNs, and par-tay!).
Dude, that thinking is the difference between an SA and an Engineer. Thinking like that would have us all running MFM drives, and these newfangled "SCSI" disks would be too risky random equipment to test out on a server.
-- "In order to have power, I must be taken seriously." -Mojo Jojo
Why? Why? Why?
At least give us a 2 line explanation, so I don't think you're speaking crap that you read in an advertisement! Please, if you have some justification, I'd love to know what it is... seriously.
Parent is right, for an admin the scariest thing you'll ever here is.
"I can't find the network drive"...
SCSI will seem a bit more expensive at first but that cost isn't just for the interface most of that cost is for the extra testing and hardware reliability you get with SCSI.
I am a pir8 and I back up everything important so I can run an SATA raid-0.
But you want something with modular controllers hotswappable raid arrays and reliability, hell if I was running a businnes off my home PC that would be something I would invest in.
Find a way to make do with less space on the netword and greater reliability, you will sleep A LOT easier.
I am a huge fan of efficent, cheap systems. The bulk of our server load is handled by dual opteron machines with 3ware cards and a 10k rpm system spindles and 7.2k rpm data spindles. However, even the best sata drives choke under file system and database loads and our primary data stores are U320. StorageReview.com has a good review of the new 150gig 10k rpm WD drive that shows it gettting spanked by SCSI drives under non-linear server loads. Long story short, if you expect a lot of drive activity you might be able to eek by for a while with a well tuned SATA system but you will have to pony up for a SCSI system at some point and you might as well do it now and save yourself the hassle of migrating later.
For some things you NEED SCSI, for others you don't. That much is obvious.
Large files/streams that require heavily mixed-mode I/O beat the balls off of SATA. E.g. Correct me if I'm wrong, but my partial understanding of SATA is that if many writes are cached and a read enters the queue, the cached writes are trashed.
so if you are working with check-in/check-out I/O type such as Samba profiles, SVN stuff, or (Samba|N)FS on a small-medium number of small-medium size files, or web stuff, SATA offers best price/performance ratio, with RAID or whatnot.
If you are working with large files that get a lot of unpredictable I/O, or databases, you really want SCSI.
RAID 5 is noticably slower disk performance for writes, and radically slower performance for reads and writes if you lose a disk. In many cases, the performance during a failure for RAID 5 can reduce system capacity below the required level, and thus RAID 5 is simply not an acceptable technology for those environments.
If just sticking more data on disk is the requirement, and you don't care how slow it gets if you have a drive fall over, then RAID 5 is great. But real world enterprise environments exist where losing half your disk throughput will cause the company's service to go down, and then you're out of business. Those guys don't RAID 5 if they know what's good for them.
The very definition of RAID is "Redundant Array of INEXPENSIVE Disks". Emphasis mine.
I've already read a bunch of posts about how SCSI is more reliable than SATA. Well, they actually mean SCSI drives are generally more reliable than SATA drivers (and some actually say so). They're quite correct for the most part.
Here's what storage vendors don't want you to know: It doesn't matter.
Use RAID. With SCSI or FC disks, you'll have to use RAID5. At that point, two disk failures in a given array and you're screwed. You REALLY care that two disks don't fail at the same time. And when you're using low-end or even mid-range drives, it happens.
Why do you have to use RAID5? Because with SCSI or FC disks, RAID5 is the only economical option. With a 300GB SCSI drive going for at least $1200USD, and FC drives of that size going for $2500USD, even the biggest corporations end up using RAID5.
Of course, RAID5 isn't the only level of RAID. It's the least redundant of any level of RAID, as a matter of fact.
Go SATA with RAID10, at least 4 drives, ideally six or more. With six drives, the likelyhood of having two drives fail before you can replace the first one is somewhat higher than if you're using SCSI, but the likelyhood of that second drive causing you data loss due to a failed array is infinitesimally smaller. It's guaranteed with RAID5, and the chance for RAID10 is inversely proportional to the number of disks in the array. So first the first drive has to fail, then the second drive which fails has to be of the same RAID1 set. Add onto that that drives do indeed "go old", and the heavier you work them, the faster they get old. With RAID5, disks tend to get worked a lot harder (without any cache, or if the cache misses, each write requires n-2 reads, and 2 writes).
Of course, you've pretty much decided that RAID10 is the way to go. At that point it's cost. If you're looking for 50GB of fast redundant storage, SCSI is going to be slightly cheaper. If you need any amount of storage though, SATA is going to be a whole lot cheaper for the same level of reliability (which requires more spindles), and typically better speed (more spindles means more seeks per second and more megs per second, though one needs to be mindful that big SATA disks are only 7200RPM, while the slowest SCSI disks you're going to get are 10kRPM).
Summary? I'm value-concious. I'd go the SATA route. RAID10, four disks minimum to start, a pair of 4-port 3ware SATA cards with 128MB+ of battery-backed cache. I'd do the RAID entirely with software (Linux MD), with each RAID1 set split across two controllers. We get cheap disk redundancy, cheap disk speed, cheap I/Os, and cheap controller redundancy. I'd consider using less fancy controllers, the 3ware jobbies tend to be expensive, but when you're doing big writes the cache makes a massive difference (75MB/s across four disks of RAID10 versus 20MB/s). I've considered putting together a dedicated storage appliance, exporting via SMB/NFS/NBD/GFS/what-have-you, without the battery-backed cache, but with a pair of 1U UPS units (one for each power supply). Then I'd go around turning off all the application-level fsync()ing, and see what happens with 4GB of disk cache. Bet it'd be fast. And with shutdown initiated via UPS trigger, almost as safe as a battery-backed cache. Remember: "Redundant Array of INEXPENSIVE Disks."
God I ramble.
Barclay family motto:
Aut agere aut mori.
(Either action or death.)
You are absolutely correct. In addition, with a tape, you can much more easily take copies off-site for storage. I frequently suggest that people get a safety deposit box in a bank at least 20 miles away from their facilities and store a copy of their backups there.
But if you may need the data 5 years or more from now, tape is clearly far superior.
You have much luck getting data back from a tape five years later?
First you have to find the tape. You can't have misplaced it and you can't have reused it due to the damn high cost of magnetic tape.
Then you have to find a drive that can read the tape. The one you wrote it with died two years ago, its no longer manufactured and oh darn none of the three you picked up off ebay use the same compression format.
Next you need the old backup software. You've been using Acme Archiver for the past three years; It doesn't understand the old SuperBackup format and unfortunately SuperBackup only ran in DOS with an 8-bit ISA SCSI card.
Finally you have to pray that the tape is still good. They're like floppy disks; they go bad just sitting on the shelf.
Buddy, I've been there. It ain't pretty. So for the last 7 years I've stored my backups on hard disks. No pain! No pain!
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
50 concurrent users is a LOT. You may not really mean concurrent, as in "50 people actually reading from or writing to this drive at the same time". If you DO mean that, you desperately need SCSI, the fastest you can find. You'll need seek time more than anything else; the drives need to respond as fast as possible to multiplexed requests for data. Rotation speed, which improves seek time and transfer rate, is good too, but it's seek time that's most crucial in heavy multitasking environments. If by 'concurrent' you mean '50 people occasionally hitting the disk', then yeah, you could probably do SATA.
However, you already have SCSI. Management is used to paying for SCSI machines. If you have 50-100 people depending on something, and it's slow, that's a productivity drag. If you assume that all those people cost $100k/year each (not at all unreasonable with benefits), 50 people are getting paid about 2,500 bucks an hour, or about 20,000 dollars a day. In other words, if you speed them up by just 5% with better hardware, you're saving the company a thousand dollars a day. Even if it's a tiny 1% speed gain, that's still 200 bucks a day. Saving six grand a month for an upfront investment of ten grand is a total no brainer.
Buy SCSI.
If my limited experience has taught me anything about computer reliability, it's that a single mis-set bit somewhere can bring down a system. Maybe the bit got there by user error, maybe it got there because of RAM or disk failure, maybe it got there from a bug in the application, OS, or firmware. Maybe a component on the motherboard shorted out. Maybe it's the climate. Maybe it's the phase of the moon.
I've seen it happen with discount ghetto hardware, I've seen it happen with high end hardware. I've seen it happen on Windows. On Linux. On FreeBSD. On Solaris. I've seen servers go down due to catastrophic hardware failure and I've seen them go down because a $2 fan died. I've seen people come inches from major power supply caused injury working on a desktop PC.
Everything will break.
There's just too much freaking complexity. Now I just buy whatever's cheapest so I can buy way more than I need. Mix up the configurations a bit so you get some bio-diversity; if one drive manufacturer has a bad year, you don't want all of your eggs invested in them.
Most important of all, at the first sign of trouble, throw it away.
Try to resist the urge to fix it. I mean it. You cost more than that piece of junk. Put in a purchase request and move on.
Yes.
That is no problem at all. I keep detailed listing of what backup set is stored on what backup media.
As far as the "damn high cost of magnetic tape", you must be talking about those cheap tape drives that use expensive tapes. We have a couple of those around here, but we don't use them much at all.
Interestingly enough, individual tapes don't really vary all that much whether they carry 4 GB or 400 GB. But IMHO the tape drives that use 4 GB tapes are not trustworthy enough to use for backups and I would never suggest using such.
If it is no longer manufacturered in two years, then you made a very poor choice of choosing a backup system.
Go to LTO Ultrium. It will still be around in 5 years.
Why would anyone use something like that for a backup? The first critera should be that whatever you write to the tape should be readable using any device that can read and write to that tape.
I've had very little problem with bad tapes. But just in case, having just one copy of the file on a backup tape is an amateur error.
I've used tapes, disks, CDs, DVDs, and even in a few cases printouts of really important data. And tapes are still my favorite.
No, choose RAID 5 instead of RAID 1+0. Here is why:
To give you a datapoint, I have set up multiple Linux software RAID 5 arrays on various servers with 10+ SATA disks, and the I/O throughput is over 500+ MB/s (enough to saturate 2 full-duplex GigE links !). At my previous work we had about 200 servers, all using Linux software RAID 5. And we have been MUCH MORE HAPPY than the previous setup where all of them were using hardware RAID 5. Moreover, Linux's software RAID 5 is more flexible (create arrays on ANY disk on ANY SCSI/SATA card in the system), more consistant (one and only one control software to learn: mdadm(8), no need to use crappy vendor tools or reboot into vendor BIOSes), cheaper (no hardware to buy), more reliable (no hardware card = 1 less hw component that can fail), easier to troubleshoot (plug the disks on ANY linux server and it works, no reliance on any particular hw card) and more scalable (spread the load across multiple disk controllers, multiple PCI-X/PCIe busses, or even multiple SAN devices).
It's amazing the amount of misinformation and misconceptions about RAID that is spread around the world. I hate to say it but 95% of IT engineers don't make good choices regarding RAID servers because of all those misconceptions.