Storage Area Networks vs. Local RAID Arrays?

Noxx asks: "My department is purchasing several new servers for an intranet website project. We are under pressure to store our content on an existing Storage Area Network accessed over a fibre connection rather than on a local RAID-5 array, to cut purchasing costs on the new hardware. Have any Slashdot readers evaluated the pros and cons between the two storage technologies, and are there any points of concern we should address? How does performance compare between the two, and is this a proper use of the SAN? If multiple servers access the same content from the SAN, is the possibility of introducing a single point of failure (ie: the SAN crashes) a valid concern?"

Comments

SAN is probably better

[crow]

Generally, you will find that using a SAN is better. The sort of equipment that is deployed in a SAN is typically higher-end than what you would get with a stand-alone RAID array.

You probably also get a number of other advantages. Your SAN is probably already backed up. Your SAN is likely already part of any disaster recovery plan.

And while you could view the SAN as a single point of failure, you could also view your local RAID array as a single point of failure. Any decent SAN implementation has redundancy at every level.

Of course, I'm biased, as I work for EMC, a big SAN company.

You probably need to sit down with your IT people and discuss with them exactly how the SAN is set up. You'll probably find that it has more than enough reliability and performance for any web server application.

Security concerns

[hectorh]

One thing that you should consider when connecting many servers to one shared SAN is the issue of security.

Most security designs involve using "concentric circles" of security.

Each ring contains a set of applications and data that have a common security concern or priority. The closer to the center that you get, the data becomes more valuable and therefore the security measures are stronger and more protective.

The outer layers of the circle usually contain internet web servers, incomming mail servers, etc. The inner layers could contain such things as source code, payroll, billing, R&D, etc.

If you share a SAN across layers of security, an intruder could use the SAN to bypass any security measures that protect the inner layer.

And if you think that this is not possible, think again, I have read the results of a SAN security risk assesment performed by a large security firm, and they were able to plug in a laptop into the SAN and gain access to the SAN by making the SAN controller believe that the laptop had the WWUI (world-wide unique identifier) of a critical server that was down for maintenance.

Can't give any more details, since I am under NDA and I cannot reveal the exact method used, or specific company names or brands.

Depends on what kind of a "SAN" you mean

[foobar104]

The word "SAN" can mean two very different things: switched access to storage, or shared access to storage.

The simplest kind of SAN has a number of computers and a number of storage devices all connected to a fibre channel switch. Each computer gets some of the storage for its own private use. No two computers ever mount the same filesystem at the same time.

The advantages of that kind of SAN are mostly physical: buy a bunch of storage and put it on the SAN, then allocate it to the computers "softly," by changing LUN mapping and such, rather than by running new cables.

If that's the kind of SAN you're talking about, I'd say go for it. The IS group that manages the SAN will take care of some of your problems for you-- maintaining the RAID hardware, namely-- but in all other ways it'll be just like direct-attached storage.

The other kind of SAN allows multiple computers to mount the same filesystem at the same time and access its data over fibre channel. This is a lot more complex, obviously, because your storage software has a lot of work to do: keeping buffer caches consistent, managing file locking, propogating metadata updates, and on and on.

This kind of SAN requires a special driver, like Sanergy or Centravision or CXFS. (Google 'em.)

They're often more trouble than they're worth, especially if you start talking about large storage clusters (8 nodes or more). I'd avoid these.

Re:Depends on what kind of a "SAN" you mean

[foobar104]

Isn't this what NFS is?

Yes, that's EXACTLY what NFS does. Shared-storage SANs try to do the same job in a different way.

Despite what you might think, the primary difference between NFS and a shared-storage SAN isn't the medium; one uses gigabit Ethernet and the other gigabit Fibre Channel. The different is the presence of the server in an NFS environment.

The server listens for mount requests and grants or denies them, and it responds to requests for data by reading the data from the disk, marshalling it, and shipping it off to the client.

In a shared-storage SAN, these functions have to be performed in some other way. A common approach is to nominate one machine on a SAN to be the "metadata server." Any disk operation that doesn't involve reading or writing actual data blocks goes through the metadata server over Ethernet.

For example, if you were doing a "cat" on your workstation, the "cat" program would first do a "stat()" to see if the named file is there, then a number of "read()"s to get the data. The "stat()" call would result in the disk driver sending a set of SCSI commands to the disk to get data out of the file's inode, and the "read()" calls would get blocks of data off the disk.

In a shared-storage SAN environment, these two calls would be handed differently. The "stat()" call would be handled through communication with the metadata server over Ethernet, while the "read()" calls would access the disks directly with SCSI-over-FC commands.

In an NFS environment, the NFS server would take care of both of those things; the NFS client would have to worry about neither.

Some SANs use a dedicated metadata server (like Sanergy) while some have a complex and pretty darn cool scheme for nominating a metadata server dynamically (like CXFS).

Maybe that helps shed some light on why my opinion is that shared-storage SANs are more trouble than they're worth.

NAS is better for many things

[smoon]

NAS, or "Network attached Storage" is often better for maintaing large collections of data to be accessed by multiple computers. You can simulate NAS by exporting some filesystems via NFS (Unix) or CIFS (Windows). Network Appliance "Filers" are said to be very good. On the lower end are the Maxtor MaxAttach and Quantum Snap! devices.

The big advantage to NAS is that dozens of web servers can mount the NAS volume and all serve up the same content. Developers, Administrators, etc. can also mount the NAS volume and do updates etc. Compared to a SAN and buying a fibre channel card, cabling, switch ports, etc. for anything but non-essential components gets very expensive very quickly. Although a previous poster indicated that multiple computers can mount the same SAN volume, It's much more difficult than with NAS since you're essentially operating at the same level as a SCSI bus, wheras with NAS you're operating via TCP/IP.

A Fibre Channel SAN is good for multiple computers running I/O intensive processes, e.g. a SQL database. It's also good as a foundation for clusters since (usually) LUNs can be re-mapped w/out a reboot. SANs really shine for fully redundant storage as well -- multiple loops, switches, controllers, etc.

Many products in both categories suffer in support for backup -- the typical low-end devices require you to mount the data on a server then use a server-attached tape device. Some products feature built-in tape drives or offer ways to back up the entire storage unit to a fibre channel attached tape drive, however this option tends to get very expensive very quickly.

One major bonus in the backup arena is the "snapshot" feature many products have (SAN or NAS). This lets you freeze 'the drive' so that no updates happen to the drive for your backup, but the system still stays up and allows updates. See vendor propaganda for more details.