Storage Area Network Solutions?

TJPile asks: "I work for a larger advertising company with offices all over the US and soon Europe and Asia. Due to our growth in the past year, our current archive/storage system cannot fulfill our needs. Others in my IT team have been talking with Dell about a storage area network. I will be the one administering this system and I was wondering if anyone in the Slashdot community has ever dealt with this before. I know we will be needing some heavy metal along the lines of an SMP Sun or SGI box. We need a system that can support (at max) about 100 simultaneous users working on large image files stored on the server. We also need cataloging software that will allow PC/Mac users to browse documents via thumbnails and job numbers. What do you guys think?" The previous two articles that touched on this subject didn't get much traffic, and were posted at least 6 months ago. Has the intervening time provided more advancements in this area?

Comments

Think NetApp.

[jfrisby]

We have a bunch of NetApps where I work (don't ask) -- they are blindingly fast (something like 6,000 NFS ops/sec with 10ms latency) with oodles of space (hundreds of GB or more) and loas of redundancy. You can get them with dual fiber channels from each head unit to the disk shelves, and you can cluster two of them together so that if a head unit fails, the other just takes over...

Very impressive, relatively cheap... And oh yeah -- no Sun box required since they just hook straight up to your LAN. (100Mb, or 1,000Mb Ethernet...)

-JF

Re:Think NetApp.

[krismon]

NetApps are NAS (network attached storage) not SAN (Storage attached network) different things, a NAS has all the data stored in one node, whereas a SAN has the data shared between multiple nodes. SAN provides redundancy in that data is duplicated within the nodes and if one node goes down, the data is still accessible.

However, I think NetApps are GREAT, and should work well in the setting in question, provided that lag from the other side of the ocean isn't going to be an issue. NetApps are Alpha boxes, with tons of disks on them, they can be clustered for redundancy and are fast and reliable. I've gotten 12,000 ops/sec from them.

But since the situation involves 2 continents, I believe that a true SAN should be the solution. maybe outsourcing to a company like StorageNetworks might be an option.

Re:Think NetApp.

[krismon]

OK.. I guess I wasn't clear enough...

The big difference is that a NAS is basically a fileserver, and a SAN is a network of disks. A SAN needs to be attached via a traditional disk attachment(scsi or fibrechannel), while a NAS is ethernet based(be it 10/100/1000/etc).

The advantage of a NAS is definitely cost. it's cheaper and way easier to install and maintain.
The advantage of a SAN is that it is a network of "disks" or nodes, and data is replicated between those nodes, you can have a nodes all in one space, or in separate datacenters, london, ny, la, etc. The SAN takes care of making the data available to the servers attached to it, no matter where the location is, but the server -must- be attached to the SAN in some way to access the data or to share the data.

A cheaper alternative to netapps are the new IDE NAS's maxtor, quantum, have made these with IDE disks, based on linux, RAID5 in most cases going up to 480GB in a 1U or 2U case. pretty cheap and quick storage considering it's plug and use.

The advantages for the netapps is that they have a very cool filesystem/operating system with snapshop capabilities.. this means that it literally takes a snapshot of the disk at a given time, say every midnight, and keeps the snapshot accessible from anywhere in the filesystem in .snapshot, this provides for easier backups (no file locking problem) and easy file retrieval if things change.

Buy this, don't build it. You will save grief.

[human bean]

Designing a storage-area-network that has better performance than a traditional client-server storage model is not a trivial task. Yes, you can put all the pieces together in the right ways, but can you make it go faster and hold more?

A major part of the operation of this sort of system is tuning it to the actual data and use patterns of your particular users. Note that not all of your users are the same, and that you will probably have to compromise on the tuning. Even though systems like this are supposed to be redundant, don't forget to build in enough network capacity for data backup servers.

Best bet is to buy a Network Appliance or one of the SUN arrays, and get it over with. Of the two I would go with the Net App boxes.

The other factor is the networking outside of your SAN, that is, now that you have the data online how do your processors get to it? If you are doing image-at-a-time load-work-save sorts of work patterns, then you will have less of a traffic load. If you are doing batch image processing one-right-after-another, then you are going to need all the bandwidth you can get. Doing larger geo models, we found that gigabit ethernet was needed from workstations (SGI Octanes) down to the storage farm. Trunked ethernet just didn't do it.

YMMV. Get a sniffer and do a study on your traffic to get a feel for what you need.

Re:Cliff

[Prolog-X]

The most closely related articles to this story could be Affordable Backup Hardware for Today's Systems? and Hardware To Archive/Manage Large Collection Of Images? .

Re:It depends on how serious your firm is...

[weave]

The biggest thing about EMC I didn't like was the fact that they won't let you near the box. You have absolutely no access to the configuration of it. Need to rebind your disks into different RAID sets? Gotta call EMC to come down and do it (and a hefty cost too).

An alternate solution is their Data General division, which makes the Clar iio n disk arrays and SAN. We actually bought one of these puppies and it should be delivered sometime next week (at which time no one will ever read this thread again since /. threads have a half-life of about two hours so me following up with status is useless...)

SGI + XFS = schweet

[The Iconoclast]

A friend of mine tells me a story that a gyy he knew worked at a TV station in Florida when that plane (ValueJet?) crash happend about 4 years ago. Anhow, their SGI server that housed all the video they were streaming over the internet was filling up and the admin had a flight to catch. The TV people wanted him to add new diskspace and grow the XFS after the newscast. Well since the guy had that plane to catch he did it DURING the newscast. Dynamically grew the XFS while it was mounted and sending data all over the net under heavy usage and NO ONE NOTICED! Pretty schweet.

A 486 with Loads of SCSI Storage -- Seriously

[InitZero]

we will be needing some heavy metal along the lines of an SMP Sun or SGI box. We need a system that can support (at max) about 100 simultaneous users working on large image files stored on the server.

The newspaper I work for has 75-85 ad builders (30 or so a shift) working on Macs. They regularly work with full page ads that are more than 70 meg each (color doubletruck runs 230 meg or so). For the past four years, they've been using a single processor (486DX-66) Novell server (hardware by Tricord) with 270 gig of SCSI disk space and 512 meg of RAM. It has a pair of 10mbps NICs. It has an uptime of more than two years. This machine is probably half of what you need. It's slow but rock solid.

We're replacing it before the end of the year with a big IBM Netfinity with four PIII processors, 320 gig of disk space, four 100mbps NICs (one per ad subnet and a hot spare) and a gig of RAM. I suspect that this would do what you need it to do and then some.

We also need cataloging software that will allow PC/Mac users to browse documents via thumbnails and job numbers.

CCI's AdDesk is your overall solution. We (the Orlando Sentinel who I am not speaking for) have used it for several years now. If you look at the top 25 newspapers in the world, more than half will be using CCI's products for either Editorial or Advertising.

AdDesk ain't great but it's the best available in terms of a full-featured, highly-expandable, highly-customizable solution. It's built on top of standard applications (Photoshop, Illustrator, etc.) held together by common (Oracle, TCL, etc.) running on either AIX or Solaris.

What do you guys think?

I think you have two choices. You can go cheap, buy some heavy hardware and put an operating system on it. Or, you can go with an AdDesk-like solution, spend a bunch of money and have a real advertising creation environment. It all depends on the size of your budget.

InitZero

SAN != NFS,CIFS

[d_engberg]

Strictly speaking, Storage Area Networks are not the same as TCP/IP attached NFS or CIFS storage (which are typically referred to as Network Attached Storage - NAS).

NAS is nice since there are a lot of simple off-the-shelf solutions that allow you to put a bunch of disks up with a server that can be accessed by many computers at the same time for both reading and writing. NFS is simple old technology with support in any $500 Linux box with a $20 ethernet card. The disadvantage is that it is slow ... as much as 100x slower than local hard drives due to all of the networking overhead.

True SAN gets rid of the TCP/IP and NFS and just directly attaches disks to computers using something like a Fibre network (SCSI-3). This is blazingly fast (approximately local HD speeds), but requires more complex networking. Since each computer is basically mounting SCSI devices, you also don't have any easy way to have multiple computers that can read and write from the same SAN storage. Shared-SAN software is in the pipeline from Tivoli and Veritas, but you might want to take a look at Global File System, which allows you to have multiple Linux boxes on a Fibre SAN (or SCSI bus!) read and write from the same disks.

You don't want a SAN

[sighup]

Like several other people posted, you do not want a SAN. SAN requires a really high speed, low latency link. It's almost always a fiber setup.

What you're -really- looking for is a combination of technologies. You need fileservers at each of your locations to serve the local clients, then you want replication between the fileservers. You do not want to try and run NFS over a WAN. Especially not the distances you are talking about. The timeouts will drive you mad.

So, I reccomend NetApp. They're a finalist in our search for home directory storage, but we don't own any. However, I've read a lot. They have a technology that will keep multiple NetApps in sync over long distances. Go this route and you'll have high speed storage for your local clients while keeping the data in sync between your locations (they send block level deltas, if you care).

So, check NetApp. Read their white papers. This has already been solved. Don't reinvent the wheel.