Storage Area Networks vs. Local RAID Arrays?

Noxx asks: "My department is purchasing several new servers for an intranet website project. We are under pressure to store our content on an existing Storage Area Network accessed over a fibre connection rather than on a local RAID-5 array, to cut purchasing costs on the new hardware. Have any Slashdot readers evaluated the pros and cons between the two storage technologies, and are there any points of concern we should address? How does performance compare between the two, and is this a proper use of the SAN? If multiple servers access the same content from the SAN, is the possibility of introducing a single point of failure (ie: the SAN crashes) a valid concern?"

Comments

Depends on what kind of a "SAN" you mean

[foobar104]

The word "SAN" can mean two very different things: switched access to storage, or shared access to storage.

The simplest kind of SAN has a number of computers and a number of storage devices all connected to a fibre channel switch. Each computer gets some of the storage for its own private use. No two computers ever mount the same filesystem at the same time.

The advantages of that kind of SAN are mostly physical: buy a bunch of storage and put it on the SAN, then allocate it to the computers "softly," by changing LUN mapping and such, rather than by running new cables.

If that's the kind of SAN you're talking about, I'd say go for it. The IS group that manages the SAN will take care of some of your problems for you-- maintaining the RAID hardware, namely-- but in all other ways it'll be just like direct-attached storage.

The other kind of SAN allows multiple computers to mount the same filesystem at the same time and access its data over fibre channel. This is a lot more complex, obviously, because your storage software has a lot of work to do: keeping buffer caches consistent, managing file locking, propogating metadata updates, and on and on.

This kind of SAN requires a special driver, like Sanergy or Centravision or CXFS. (Google 'em.)

They're often more trouble than they're worth, especially if you start talking about large storage clusters (8 nodes or more). I'd avoid these.

Re:Depends on what kind of a "SAN" you mean

[foobar104]

Isn't this what NFS is?

Yes, that's EXACTLY what NFS does. Shared-storage SANs try to do the same job in a different way.

Despite what you might think, the primary difference between NFS and a shared-storage SAN isn't the medium; one uses gigabit Ethernet and the other gigabit Fibre Channel. The different is the presence of the server in an NFS environment.

The server listens for mount requests and grants or denies them, and it responds to requests for data by reading the data from the disk, marshalling it, and shipping it off to the client.

In a shared-storage SAN, these functions have to be performed in some other way. A common approach is to nominate one machine on a SAN to be the "metadata server." Any disk operation that doesn't involve reading or writing actual data blocks goes through the metadata server over Ethernet.

For example, if you were doing a "cat" on your workstation, the "cat" program would first do a "stat()" to see if the named file is there, then a number of "read()"s to get the data. The "stat()" call would result in the disk driver sending a set of SCSI commands to the disk to get data out of the file's inode, and the "read()" calls would get blocks of data off the disk.

In a shared-storage SAN environment, these two calls would be handed differently. The "stat()" call would be handled through communication with the metadata server over Ethernet, while the "read()" calls would access the disks directly with SCSI-over-FC commands.

In an NFS environment, the NFS server would take care of both of those things; the NFS client would have to worry about neither.

Some SANs use a dedicated metadata server (like Sanergy) while some have a complex and pretty darn cool scheme for nominating a metadata server dynamically (like CXFS).

Maybe that helps shed some light on why my opinion is that shared-storage SANs are more trouble than they're worth.