Slashdot Mirror
Ask Slashdot: Linux and Fibre Channel Storage Systems
"The connection to the Clariion storage sub-system will be made using point-to-point Fibre-Channel via two Emulex LP8000 64-bit PCI host bus adapters over copper connections/cables. The FC interfaces and a Gb NIC will be plugged into a triple-peer 64-bit PCI bus. The whole thing should be able to sustain approximately 120 simultaneous MPEG feeds at 1.2Mbps/feed on our 100baseT LAN."
Eventually, we will be clustering several servers in order to increase the number of simultaneous feeds, and also plan to expand video-on-demand service (educational only, primarily for foreign language curriculcum) to the dorms and other locations on campus, possibly the Internet at large if permission is granted from the video title's author (sorry guys, no free StarWars d/l's). Our initial storage capacity will be 180 GB (approx. 144 GB w/ RAID 3 striping). But we expect to scale up to 2 TB in the next two years.
Another advantage to using Linux would be its clustering support (under Caldera for example). We will need that capability soon, and Microsoft still seems to be dragging behind, with only a handful of 3rd party solutions available."
http://www.cs.wpi.edu/~claypool/mqp/raid/
Please moderate this up. They used the Emulex LP6000 not the 8000, but depending on how different the card is the driver may work or may be easily tweaked to work.
I wish you all the luck and love Linux but did you look into getting a Sun instead?
Err, he's using a gigabit Ethernet card, not multiple 100bt ethernet cards. So multiply your available bandwidth by 10. (And gigabit by definition is switched).
One thing to consider: NT is not good at handling large filesystems. According to many people, once you get past 100 gigs of storage NT becomes massively unstable (gee, and it isn't for less than that?!). Linux is not ideal here either, especially for video on demand, where you will run into the 2gb file size limit.
The "safe" bet here, that would avoid the massive instability of NT when dealing with large data sets and the incapability of Linux, would be to go to Sun. You say you are in an educational environment. Sun has approximately 50% discounts in that environment for many of their products. In one case at CUNY, Sun quoted a system for the exact same price that an equivalent high end Linux system would have costed from VA Research or Penguin. Sun also has very good support in the educational environment, because they want to "seed" the field.
So what happens when the machine goes down?
What happens when something... blips.
What happens when you need to do system upgrades?
What happens when you need to do maintenance?
Building one big fancy server is -- in my opinion -- a really bad approach to solving this problem. Sure, I'd love to have a box like the one you describe. It sounds cool. But having the coolest box on the block does not necessarily build the best solution to the problem.
Suppose instead that you bought a small farm of rack-mount PCs. Equip each with a nice 10K RPM SCSI hard drive and a 100BaseTX network interface. If running Linux or *BSD, such boxen could get by with CPU and memory that is modest by today's standards. NT, well, will require more to do the same job. Such boxes can be put together for around $2k a pop. My guess is that you'll get much more aggregate horsepower building it out of a cluster of such boxes than the same amount of dollars would buy you in a big fancy server.
Then you simply make your software able to cause the video to stream off the box that has it. You might also add the cability for the boxen to automagically clone frequently used streams from their home machine to others and load-balance.
If a server goes down, you just restore its data off of backups onto some of the other ones and shuffle where things are homed.
If you need more capacity, to a pretty high degree, you just buy more inexpensive boxes.
This would of course require some clever software to make it all work. Your application sounds like one where some custom software is involved anyway, and it won't be that hard to do the stuff needed to pull off a clustered solution.
At work we've been looking into a method of setting up a large amount of fast network storage. We know the traditional methods of doing it (one way would be by attaching a RAID-5 unit to our Enterprise 450.)
/*' moments. Because it's just storing file deltas, it doesn't take as much space as you'd think.
:)
We've been talking with Network Appliance. Their NetFilers look pretty good. They claim to be very fast, and are highly scalable. They natively speak both SMB (NT's file system) and NFS (for Unix). ALL they do is storage. You can't program them; they're highly specialized rackmount machines that tie into your NT domains (and presumably your NIS domains, though I didn't ask that) pretty much seamlessly. You can organize your available space as Unix, NT, or shared.
It has one very, very cool capability in the filesystem. You can take a snapshot on a given day/time, which basically means they copy all the inodes at that time. If anything changes subsequently, the original drive data is preserved, and only the new data is written. The inodes in the fresher filesystem are updated to point at the new data, where the old data is still there, being pointed to by the old inodes. It gives you an online revision history, which would be exceptionally nice right after one of those "*doh, I just did rm -rf
Another real advantage we see to it is that because it's a specialized device, if something else on the network needs to be taken down for maintenance (say, the backup server), the network storage is unaffected. The only thing that's at all likely to take your storage down is, well, storage problems. And it's fully hot swappable and running fibre channel internally. It looks really good.
I think you're talking around $100K - $120K or so for a midrange box with 150GB of storage. I believe they presently scale to 1.5TB, and are building ways of scaling further -- they claim much further. Tons of options for high speed networking on it, too.
It's just a thought -- instead of doing what you are doing, which is using PC hardware, switching to a specialized device, with its custom, highly reliable hardware, might be a very appealing solution -- as long as all you're doing is streaming the data off the server. If you need the server to fiddle with the bits before they arrive, you'd have to have another server in the way, which is probably defeating most of the purpose in a specialized unit anyway.
My $0.02. I don't have actual experience with these yet. If anyone does, please chime in.