Storage Area Networks vs. Local RAID Arrays?

Noxx asks: "My department is purchasing several new servers for an intranet website project. We are under pressure to store our content on an existing Storage Area Network accessed over a fibre connection rather than on a local RAID-5 array, to cut purchasing costs on the new hardware. Have any Slashdot readers evaluated the pros and cons between the two storage technologies, and are there any points of concern we should address? How does performance compare between the two, and is this a proper use of the SAN? If multiple servers access the same content from the SAN, is the possibility of introducing a single point of failure (ie: the SAN crashes) a valid concern?"

Comments

Security concerns

[hectorh]

One thing that you should consider when connecting many servers to one shared SAN is the issue of security.

Most security designs involve using "concentric circles" of security.

Each ring contains a set of applications and data that have a common security concern or priority. The closer to the center that you get, the data becomes more valuable and therefore the security measures are stronger and more protective.

The outer layers of the circle usually contain internet web servers, incomming mail servers, etc. The inner layers could contain such things as source code, payroll, billing, R&D, etc.

If you share a SAN across layers of security, an intruder could use the SAN to bypass any security measures that protect the inner layer.

And if you think that this is not possible, think again, I have read the results of a SAN security risk assesment performed by a large security firm, and they were able to plug in a laptop into the SAN and gain access to the SAN by making the SAN controller believe that the laptop had the WWUI (world-wide unique identifier) of a critical server that was down for maintenance.

Can't give any more details, since I am under NDA and I cannot reveal the exact method used, or specific company names or brands.

Depends on what kind of a "SAN" you mean

[foobar104]

The word "SAN" can mean two very different things: switched access to storage, or shared access to storage.

The simplest kind of SAN has a number of computers and a number of storage devices all connected to a fibre channel switch. Each computer gets some of the storage for its own private use. No two computers ever mount the same filesystem at the same time.

The advantages of that kind of SAN are mostly physical: buy a bunch of storage and put it on the SAN, then allocate it to the computers "softly," by changing LUN mapping and such, rather than by running new cables.

If that's the kind of SAN you're talking about, I'd say go for it. The IS group that manages the SAN will take care of some of your problems for you-- maintaining the RAID hardware, namely-- but in all other ways it'll be just like direct-attached storage.

The other kind of SAN allows multiple computers to mount the same filesystem at the same time and access its data over fibre channel. This is a lot more complex, obviously, because your storage software has a lot of work to do: keeping buffer caches consistent, managing file locking, propogating metadata updates, and on and on.

This kind of SAN requires a special driver, like Sanergy or Centravision or CXFS. (Google 'em.)

They're often more trouble than they're worth, especially if you start talking about large storage clusters (8 nodes or more). I'd avoid these.

Re:Depends on what kind of a "SAN" you mean

[foobar104]

Isn't this what NFS is?

Yes, that's EXACTLY what NFS does. Shared-storage SANs try to do the same job in a different way.

Despite what you might think, the primary difference between NFS and a shared-storage SAN isn't the medium; one uses gigabit Ethernet and the other gigabit Fibre Channel. The different is the presence of the server in an NFS environment.

The server listens for mount requests and grants or denies them, and it responds to requests for data by reading the data from the disk, marshalling it, and shipping it off to the client.

In a shared-storage SAN, these functions have to be performed in some other way. A common approach is to nominate one machine on a SAN to be the "metadata server." Any disk operation that doesn't involve reading or writing actual data blocks goes through the metadata server over Ethernet.

For example, if you were doing a "cat" on your workstation, the "cat" program would first do a "stat()" to see if the named file is there, then a number of "read()"s to get the data. The "stat()" call would result in the disk driver sending a set of SCSI commands to the disk to get data out of the file's inode, and the "read()" calls would get blocks of data off the disk.

In a shared-storage SAN environment, these two calls would be handed differently. The "stat()" call would be handled through communication with the metadata server over Ethernet, while the "read()" calls would access the disks directly with SCSI-over-FC commands.

In an NFS environment, the NFS server would take care of both of those things; the NFS client would have to worry about neither.

Some SANs use a dedicated metadata server (like Sanergy) while some have a complex and pretty darn cool scheme for nominating a metadata server dynamically (like CXFS).

Maybe that helps shed some light on why my opinion is that shared-storage SANs are more trouble than they're worth.

GunBroker.com site outage - email I received

[cymen]

Here is an email I received on the 21st of this month:

These last two days have been the worst two days we have ever experienced at GunBroker.com. Our EMC disk array, which is supposed to guarantee 100% uptime, failed. It took EMC 24 hours to get it back online, and when they got it back online they corrupted our database. Although we have tape backup the tape runs at regular intervals, the crash occurred at the worst possible time. Everyone here worked 48 hours straight to restore the damaged data as fully as possible.

We are extremely sorry that this happened. Downtime is extremely rare at GunBroker.com because we spend a lot of money on an extremely high quality infrastructure. This never should have happened at all because EMC disk arrays are never supposed to fail. We have a contract that guarantees that the EMC will not fail and that it will not cause us data loss. We are of course going to find out why this occurred and do whatever it takes to make sure that it never happens again.

To ease the financial loss of our sellers who use this site as business income, we are waiving the Final Value Fee of any item that is listed Monday, Jan 21 through Wednesday, January 23 (eastern time). The FVF will appear on your real-time account info but will be removed before you get your monthly statement so you will never pay it. If you had not already noticed we did away with the fraud insurance fees (but not the Fraud Insurance) starting Dec 1, 2001.

We have extended all auctions that were to end in the down period to be fair to our buyers and sellers. If you placed a bid or listed and item on the morning of Jan 18, please read the following and check to make sure that you bid or listing is still there.

As a consequence of the crash it is possible that a small amount of data was lost from the time frame immediately before the crash. If you placed bids or listed an item after 10:00am on 1/18/2002, please check 'My Auctions' from the top of any web site page to make sure that your listing or bid is there.

Once again, we are extremely sorry for the downtime. If you have had a problem that has been caused by this incident please 'Reply' to this message and tell us what we can do to help you and we will do everything we can to resolve the issue.

GunBroker.com
The Web's Largest Hunting and Sport Shooting Auction and the ONLY firearms trading place that guarantees your purchase against fraud.
http://www.gunbroker.com