Cross-Platform Company Storage Architecture?

Eric^2 asks: "My company is preparing to implement a major network storage upgrade, and I'd like to get some ideas from Slashdot about what devices should be considered, and hopefully some experiences with some of the offerings that are available. What types of storage are you using and what would you recommend?" "We are currently using approximately 2TB of storage space, and will need to expand to over 10TB in the next two to three years. We have a mix of Windows, Mac OS X, and Linux clients and servers. All of our authentication is presently done through an Active Directory. If possible, we would like to centralize all of the storage into a single namespace, such as OpenAFS or DFS. Anything we purchase will have to be under maintenance contract for hardware such as failed drives or controllers. Ideally, whatever system we choose would allow us to purchase both high-speed SCSI spindles for our transactional needs and lower-speed SATA high capacity drives for our archival storage needs."

Comments

NetApp

[ZESTA]

Depends on what your budget is, but I would look into Network Appliance (http://www.netapp.com). Their systems are top notch, and have some very cool software features. They support NFS, CIFS, iSCSI, and Fibre Channel as connection methods.

-Randy

Re:NetApp

[weaselprince]

The OP doesn't say much about the selection criteria - scalable? performant? manageable? cheap?

If it's cheap, then Netapp might not qualify... :)

What about technologies - NAS? Host-attached? Gateway/NAS? Grids?

Other companies/products to consider:

EMC (The Celerra is a nice product)

Onstor Bobcat

HP

IBM

Hitachi

Panasas

Exanet

Yotta Yotta

StoreAge

If you want basic raid devices look at Infortrend/Transtec. Their S-ATA offerings now support RAID-6 and are dirt cheap.

Re:NetApp

[Jim_Maryland]

In some cases, going cheap isn't always a good idea. A group that manages one of our labs decided to buy a device from Excel Meridian and in my opinion, it's a piece of junk. We found limitations that affect both a mixed OS environment as well as scalability (not in disk space but handling larger Active Directory structures). The Excel Meridian device we have has about 6 TB of storage so space isn't an issue but the ability to execute files on it from an NFS mount fails. We also find that it can't join a domain that has a large number of user entries (don't recall the exact number but want to say it's around 1000 users). For a small workgroup this might be fine but not for a larger corporation.

NetApp is by far my choice but if I need to get a cheaper device, Dell PowerVaults are generally adequate unless you are looking for highly transactional file activies (we've run into the file lock problem on the device occasionally if we process files, in our case image processing). To avoid the file lock though, we process locally on our UNIX boxes and transfer the results to the PowerVault. One limitation we find on the PowerVault (and likely it affects all MS Win32 based file systems) is case sensitivity. I believe you "can" change to allow differentiation of files based on case but Microsoft doesn't recommend it.

Consider Apple's XServe RAID and XSan

[Alowishus]

I went through this decision process at a civil engineering firm with 3TB of data. We wound up building a SAN out of XServe RAID cabinets, QLogic SANbox switches, XSan software for the OS X machines and ADIC's StorNext FX software for the Linux and Windows. XSan/StorNext is a shared filesystem, which inherently gets you a combined namespace without any DFS-type machinations. All servers literally mount the very same volume and access it simultaneously.

XSan is really the deal of the century - you can build a full-blown StorNext system starting with ADIC's software, but that approach can be exceptionally expensive. Instead, start with XSan (which is a functional but slightly stripped version of StorNext) and then use ADIC's much less expensive StorNext FX client licenses for each non OS X server that needs to join.

Redundancy can be everywhere. Start with a pair of redundant XServes as metadata controllers. Add a pair of redundant SAN switches. Apple's FiberChannel HBAs are all dual channel, as are the XServe RAID cabinets. For any non-Apple hardware, buy dual channel QLogic HBAs.

Apple provides a variety of maintenance contracts for all their hardware, as does QLogic. ADIC and Apple provide support maintenance agreements for the software. The only missing piece of your equation is SCSI-based storage. But since this whole system is entirely standards-based, all you need to do is find a favorite vendor of SCSI FiberChannel cabinets and drop a few into your SAN and then partition them accordingly, right along with all the SATA storage.

It's a beautiful system, and a raging bargain compared to every other comparable solution I've investigated.

Venti

[DrSkwid]

Either through plan9 port or the real thing

Venti is block level and, as such, coalesces identical blocks, a bit like LZW, so backing up 100 Windows machines doesn't take up 100x the disk space of backing up 1 windows machine.

http://cm.bell-labs.com/sys/doc/venti.html

http://cm.bell-labs.com/magic/man2html/8/venti

http://swtch.com/plan9port/man/man8/venti.html
http://swtch.com/plan9port/man/man8/vbackup.html

Sean Quinlan (one of the 2 Venti inventors) moved from Bell Labs by Google.

08:56-10:13
News for nerds, stuff that matters
Post Comment
Database maintenance is currently taking place. Some items such as comment posting and moderation are currently unavailable.

MySQL r0x0rs

Application ?

[pwet]

you forgot the most important : what are you using your storage for ?

Re:create by yourself?

[walt-sjc]

I really don't understand why people keep pushing Fedora Core for production systems. It's not appropriate. Not that FC is bad, it's not, but in a production systen you need a level of stability and consistancy that FC by design does not provide. This is especially the case when it comes to things like SAN's and such. Centos is MUCH more appropriate. The Fedora legacy project was supposed to help, but has proven to be ineffective.

it depends...

[therus121]

You need to see how you want to access your storage, and what is going to be running on it, as to how you go:-

SAN - block level data access to storage. Good for databases; low client counts (because SAN ports are expensive relative to ethernet) - but with high IO demands. EMC are good, but pricey - a low to mid end Clariion would probably be the right range to aim at.

NAS - file level data access to storage. Good for situations where there are many clients connecting, and their IO demands are not excessive. Netapps filers are very good at this (if youy can find information on their new OS (10GX) then it's VERY interesting. ILM use them in their render farms.

iSCSI - a blend of the best of both, but it's still looked upon as an emerging technology. You get (or did) free iSCSI licenses with netapps filers.

O'Reilly have a good book on this. "Using SAN's and NAS" which is vendor agnostic http://www.oreilly.com/catalog/sansnas/index.html

Re:it depends...

[TrueKonrads]

Don't forget that there is ATA-over-ethernet You can buy the 10 disk arrays make them RAID5 and offer as SAN solutions to linux machines with ease, without expensive fiber switches.

Samba, Dfs, and NFS

[skroz]

At my company we've used a number of Dell PowerEdge Linux servers running Samba. All of the servers are then tied together using Samba's Dfs implementation to "stitch" individual components together for Windows clients and NFS/AutoFS/symlinks for Linux clients. This is all accomplished with some very simple perl and shell scripts.

This likely won't work in all environments, however. Our data is divided into thousands of discrete and manageable chunks stored in individual directories, so stitching it together via an automated process is relatively simple. Part of the job of the scripts mentioned above is to "rebalance" these chunks (move them from server to server) to prevent any one volume from becoming full. If your data "components" are large, or if your data is too active to move regularly, this won't work.

It's the poor man's cluster, and there are better solutions out there, but it works extremely well in our case.

etherdrive

[Apreche]

Here are some guys my friend was looking at for a storage solution. Basically they just ethernet-ify as many hard drives as you want. How you configure them is up to you. It's a bit expensive, but it's incredibly simple and flexible.
http://www.coraid.com/

How we skin a cat...

[gurutc]

Here's a pretty OS-nonspecific example of cross-platform storage implementation. Some of it is about backups and may seem off-topic but is valuable as an example of how much you can mix platforms and OS to get what you need in network storage solutions.

We protect 3 Terabytes per night from 250 remote servers with a backup strategy using RSYNC. These include both Windows and Netware servers. Our centralized backup file server is a single Dell PowerEdge 2850 with dual Xeon CPUs which runs OpenSuse 10 and has a combination of both Dell Powervault RAID SCSI enclosures and LaCie Big Disk USB External drives attached. Using a fast server with an OS that we can tune gives us incredible multistream-capable throughput for network storage. Think about the speed required folks, 3 Terabytes in 12 hours from 250 hosts at 75 sites. (Well RSYNC means we don't send all the data, but still! ;-0 )

Then, each day, we back up the Linux box using a Windows server installed on a Dell Optiplex workstation box with a tape jukebox attached and running CA ArcServe. That way we get a daily snapshot to tape allowing us to do a scheduled rotation.

This means we are following the Golden Rule of Backups, which applies no matter how much data you back up, which is this: Always have 2 separate backup copies of important data. And it's better if they are different types of media. And with SANS and NAS solutions redundancy is critical. These acronyms should be called AIOB which stands for 'All In One Basket'

RSYNC has done what no commercial software seemed to be able to do: give us a good working backup system for our enterprise. It uses very efficient synchronization and compression algorithms to move the changes from our distributed servers. If you want this rig to do backups too I recommend considering it. Here's a link to the RSYNC Project:

http://rsync.samba.org/

Here's the Novell RSYNC forum:

http://forge.novell.com/modules/xfmod/newsportal/t hread.php?group_id=1148&group=novell.forge.rsync.h elp

And here's a good resource for RSYNC on Windows:

http://art.wilderness.org.au/software/help_cygwin- rsync.shtml

Here are two more good RSYNC Windows links:

http://www.itefix.no/phpws/index.php?module=pagema ster&PAGE_user_op=view_page&PAGE_id=6&MMN_position =23:23

http://www.nasbackup.com/

The NASBackup Project is a neat Open Source effort to make a gui-based RSYNC client for Windows. It works very well.

More info: RSYNC uses an algorithm that only sends the changes in the file systems. This algorithm is so efficient that i can even get down to only sending the changed blocks in an individual file without having to send the whole file. It works very well for us even over DSL/Cable speed connections. You want to optimize your entire I/O schema including all network layers as well as the way you read, write, and cache file and database operations on all connected hosts.

I hope this little bit of info helps you.

FreeNAS.org

[gurutc]

Free NAS Wireless, secure, open source, multi platform, easy to configure, etc etc etc. For free! I've used it. Compared to the LaCie network devices (not the USB LaCies, they're great) it is FASTER! But a dedicated Linux box you config yourself and with a tuned IP stack is quicker. However, for the effort of downloading a teensy ISO, burning a CD, and spending 5 minutes to install and config, this solution is really astounding. www.freenas.org

Google's Appliance? Storage is cheap.

[twitter]

Google's Search Appliance has been on the market for years. They have a page of user stories, which includes National Semiconductor, Nextel, Universities, government agencies, large and small companies.

Given an effective search, you can store the information on anything. That means you can deploy many cheap and fast servers close to the source of information creation, and have that information available everywhere. With 250 GB drives going for $50, you could have all 10TB of storage taken care of twice for $4,000.

Netapp

[TopSpin]

Netapp has a new division called "StoreVault" that is about to release new products that might be ideal for your purposes. There isn't much information publically available yet, but what is available is:

o Data OnTap OS
o NAS and iSCSI
o Optional FC interface (yes, NAS, iSCSI and FC in one device)
o "simplified" web interface
o Based on FAS250/270
o $5000 entry level price
o Scalable to 12TB

Presumably the products will launch some time in June.