SAN, NAS, Cost and Benefits?

luetin asks: "Our company is at the point where our storage and backup infrastructure is ok, but not for much longer. We are looking into SAN, NAS, and variations thereof. We are a small IT department, with two sysadmins and two programmers. Right now we have stored/circulating about 2TB of data, and that's going to increase steadily in coming years. Does Slashdot have experience setting up SANs? Tales of costs and benefits of SANs versus a gaggle of NAS? Can SAN be implemented by reasonably seasoned IT people, or is it too dark an art?"

Comments

See what you can do about getting OTS solutions

[Dancin_Santa]

Don't waste your time doing this kind of thing yourself. The final storage system will end up being cheaper and likely more robust if you go with an off the shelf solution.

Companies specialize in building these types of systems all the time. Hire one of them to help you set up. If you have that much data and expect it to grow at a faster rate into the future, don't bet on your own rinky dink implementation, get a professional.

Re:See what you can do about getting OTS solutions

[Radical Rad]

Maybe they have a few gig's of work data but a really awesome mp3 collection.

Opposing opinion

[GigsVT]

I'd have to go against the well-funded flow here.

Right now you can get 3TB+ of storage in a single SATA RAID5 unit from www.acnc.com, for about $11,000.

You can get it with a SCSI or FC external interface. Use two of them hooked to two computers in two locations (preferably 300+ miles away) with rdiff-backup if you want extra redundancy. We use local and remote mirrors for maximum protection. The space is so cheap, it's easy to keep extra mirrors.

We've finally eliminated our last major SCSI and FC arrays, and I couldn't be happier. We're up to about 6 TB total ATA and SATA storage now. Get cheap storage if at all possible, because it will be obselete in need something that a cheaper system can't offer. That isn't much these days, now that 10K rpm SATA drives are out.

As far as single drive reliability, the first ATA unit we installed has been in service 2 years this month. We've only replaced two drives out of 48, and even then, the drives passed the factory recertification tests from the manufacturer when we ran them on them. And even if you think that's a higher failure rate than your experience with SCSI/FC, keep in mind that the cost is so much lower, it lets you have more mirroring redundancy, so individual drive failures are much less of an incident.

I'd go this route

[Sevn]

Used Netapp brand Network Appliances

We used to use Netapps at MindSpring to serve 80,000+ commercial webhosting clients. They are tough as hell, easy to maintain, last forever, and do everything right. You can mount the shares with NFS or CIFS. It has a web based interface for configuration, or a simple command line interface. You can add drives and change volume sizes, inodes, etc without shutting it off or losing a connection to it. The snapshot feature will eventually save your butt like it saved mine on many occasions. Hope this helps.

You left out a few important things...

[maunleon]

1. What is the use of this data? Who accesses it? How many concurent users? What type of transfer rate? Will you need funky network cards (for example 10GB nics which may not work in some solutions)

2. Can you accept downtime? If not, how much redundancy do you need? How fast can you get replacement parts?

3. Do you need specialized apps running on the machine (such as virus checkers, management tools, etc)?

For a professional installation, I would say you would at least want to ensure some redundancy. For example, an hp Proliant DL-360 G2 or G3 with redundant power supplies, redundant fans and a drive array or two. The server itself is fairly cheap, what will cost you money is all those drives you will need to buy. However it's a sturdy box.

I don't mean to single out hp, you can look for other alternatives as well. We do run an hp/compaq shop, and I am familiar with them.
All this redundancy helps in ways you don't expect. For example tonight I was able to move the server from one rack to another without losing service.. I disconnected one power supply and connected it to the new rack, then disconnected one network cable (the 2 onboard NIcs were teamed) and rerouted it.. dropped the other nic and cable, mounted the server in the rack and connected the remaining cables.. The users at the other end had no idea anything happened.

This may not sound like a big deal to many, but for us to schedule a 30-minute shutdown of a critical server requires up to a month advance notice.

You could of course accomplish the same thing using a cluster setup, but not without some major headaches. Clusters are cool on paper but for most users the bang-to-headache ratio is too low to justify it.

Re:You left out a few important things...

[Zapman]

These are all very important questions. The other thing that people forget about is the service agreement.

My company paid too much money for an EMC array. We don't need the performance, and we don't need all the wizbang features.

That doesn't mean it sucks though. EMC calls us within 5 minutes of any hardware issues on the box. It's fault tollerant to a amazing degree. The only time we've taken downtime was when we needed to schedule a major firmware update. The support agreement is amazing. If something dies, they call us and say 'we can be on site with an engineer and the part in 2 hours. When do YOU want us to show up?'

Asside from our database servers, none of the boxes we've put on EMC come close to pushing the throughput limits. We would have been much wiser to utilize NAS for most of the servers. Much cheeper, good reliability, reasonable performance, and they can talk all the major file sharing protocols: SMB, CIFS, NFS, etc, and we could have spent less on the SAN.

A poster above suggested a mix of either building the SATA arrays yourself, or a small vender. I poked on their website, and their support agreement says 'parts next day'. If the driveplane blows out, 'next day' may or may not be good enough. People scream really loudly if email is down.

The other thing is making sure you find someone who can reverse engineer your 'rsync' solution. This is one reason companies tend to prefer out of the box apps for system level stuff. They can send someone to training for it.

Small SAN vs Big Money

[yancey]

You're on the extremely low end of where a SAN becomes practical... and it may not be practical in your environment. If the SAN is really going to stretch your budget thin, don't do it!

Xiotech's products are very easy to use, but costly. You'd be lucky if you could get the whole setup for under 80,000 USD. The SAN hardware itself is reasonably priced, but you pay a license to use 1-8 servers, you pay more for 9-16, and so on. The software part gets expensive quick. Also, be aware that Xiotech charges about three times the off-the-shelf price for drives and they won't let you use off-the-shelf drives or you void your warranty and get no support. They certainly make their money on the drives.

HP/Compaq has the EVA series of disk arrays that use very similar "virtual array" technology that Xiotech uses. Again, very flexible, but expensive. At least HP doesn't make you pay more to connect more servers and the charges for drives is a little more reasonable.

It certainly seems that someone along the way forgot that RAID means Redundant Array of Inexpensive Disks. The whole point is to make a bunch of relatively unreliable disks into a very reliable whole, unless you just want speed and don't need the reliability.

If you're thinking about a two or three server cluster and just need shared storage where you can add more drives, you might look at some of the small and inexpensive rackmount IDE RAID solutions that are available with Fibre Channel and a FC hub, but be sure to get references and see who's using the things and what their experience has been. You can get two terabytes for under 15,000, but some of these are good yet inexpensive and some of them are just cheap junk.

These IDE RAID solutions do not provide the advanced features of a virtualized SAN, like changing RAID types on the fly (from RAID 5 to RAID 10, for example). However, you could easily spend 60,000+ on a Xiotech SAN or you could spend the same amount and have eight terabytes in four IDE RAID modules. Your choice, but for a small shop, I say get the eight terabytes and setup mirroring across two of these RAID boxes.