Slashdot Mirror
Ask Slashdot: Linux and Fibre Channel Storage Systems
"The connection to the Clariion storage sub-system will be made using point-to-point Fibre-Channel via two Emulex LP8000 64-bit PCI host bus adapters over copper connections/cables. The FC interfaces and a Gb NIC will be plugged into a triple-peer 64-bit PCI bus. The whole thing should be able to sustain approximately 120 simultaneous MPEG feeds at 1.2Mbps/feed on our 100baseT LAN."
Eventually, we will be clustering several servers in order to increase the number of simultaneous feeds, and also plan to expand video-on-demand service (educational only, primarily for foreign language curriculcum) to the dorms and other locations on campus, possibly the Internet at large if permission is granted from the video title's author (sorry guys, no free StarWars d/l's). Our initial storage capacity will be 180 GB (approx. 144 GB w/ RAID 3 striping). But we expect to scale up to 2 TB in the next two years.
Another advantage to using Linux would be its clustering support (under Caldera for example). We will need that capability soon, and Microsoft still seems to be dragging behind, with only a handful of 3rd party solutions available."
I guess i don't quite see WHY you would need to do such a thing... You could buy an awful lot of TV's and DVD players for that kind of cash. Put one in every classroom that might need it, or make them mobile. As for getting it into the dorms, I'm sure there is an existing cable tv system that could easily carry a few extra channels. Get some DVD players and video modulators and pay some student $6 an hour to swap discs when necessary. As a student myself I am sick of seeing my tuition money thrown away on useless crap... If we got along fine without it last semester, we can do without it this semester. A hundred grand could build a whole LAB of computers for students to use.
http://www.cs.wpi.edu/~claypool/mqp/raid/
Please moderate this up. They used the Emulex LP6000 not the 8000, but depending on how different the card is the driver may work or may be easily tweaked to work.
I wish you all the luck and love Linux but did you look into getting a Sun instead?
Err, he's using a gigabit Ethernet card, not multiple 100bt ethernet cards. So multiply your available bandwidth by 10. (And gigabit by definition is switched).
One thing to consider: NT is not good at handling large filesystems. According to many people, once you get past 100 gigs of storage NT becomes massively unstable (gee, and it isn't for less than that?!). Linux is not ideal here either, especially for video on demand, where you will run into the 2gb file size limit.
The "safe" bet here, that would avoid the massive instability of NT when dealing with large data sets and the incapability of Linux, would be to go to Sun. You say you are in an educational environment. Sun has approximately 50% discounts in that environment for many of their products. In one case at CUNY, Sun quoted a system for the exact same price that an equivalent high end Linux system would have costed from VA Research or Penguin. Sun also has very good support in the educational environment, because they want to "seed" the field.
So what happens when the machine goes down?
What happens when something... blips.
What happens when you need to do system upgrades?
What happens when you need to do maintenance?
Building one big fancy server is -- in my opinion -- a really bad approach to solving this problem. Sure, I'd love to have a box like the one you describe. It sounds cool. But having the coolest box on the block does not necessarily build the best solution to the problem.
Suppose instead that you bought a small farm of rack-mount PCs. Equip each with a nice 10K RPM SCSI hard drive and a 100BaseTX network interface. If running Linux or *BSD, such boxen could get by with CPU and memory that is modest by today's standards. NT, well, will require more to do the same job. Such boxes can be put together for around $2k a pop. My guess is that you'll get much more aggregate horsepower building it out of a cluster of such boxes than the same amount of dollars would buy you in a big fancy server.
Then you simply make your software able to cause the video to stream off the box that has it. You might also add the cability for the boxen to automagically clone frequently used streams from their home machine to others and load-balance.
If a server goes down, you just restore its data off of backups onto some of the other ones and shuffle where things are homed.
If you need more capacity, to a pretty high degree, you just buy more inexpensive boxes.
This would of course require some clever software to make it all work. Your application sounds like one where some custom software is involved anyway, and it won't be that hard to do the stuff needed to pull off a clustered solution.
At work we've been looking into a method of setting up a large amount of fast network storage. We know the traditional methods of doing it (one way would be by attaching a RAID-5 unit to our Enterprise 450.)
/*' moments. Because it's just storing file deltas, it doesn't take as much space as you'd think.
:)
We've been talking with Network Appliance. Their NetFilers look pretty good. They claim to be very fast, and are highly scalable. They natively speak both SMB (NT's file system) and NFS (for Unix). ALL they do is storage. You can't program them; they're highly specialized rackmount machines that tie into your NT domains (and presumably your NIS domains, though I didn't ask that) pretty much seamlessly. You can organize your available space as Unix, NT, or shared.
It has one very, very cool capability in the filesystem. You can take a snapshot on a given day/time, which basically means they copy all the inodes at that time. If anything changes subsequently, the original drive data is preserved, and only the new data is written. The inodes in the fresher filesystem are updated to point at the new data, where the old data is still there, being pointed to by the old inodes. It gives you an online revision history, which would be exceptionally nice right after one of those "*doh, I just did rm -rf
Another real advantage we see to it is that because it's a specialized device, if something else on the network needs to be taken down for maintenance (say, the backup server), the network storage is unaffected. The only thing that's at all likely to take your storage down is, well, storage problems. And it's fully hot swappable and running fibre channel internally. It looks really good.
I think you're talking around $100K - $120K or so for a midrange box with 150GB of storage. I believe they presently scale to 1.5TB, and are building ways of scaling further -- they claim much further. Tons of options for high speed networking on it, too.
It's just a thought -- instead of doing what you are doing, which is using PC hardware, switching to a specialized device, with its custom, highly reliable hardware, might be a very appealing solution -- as long as all you're doing is streaming the data off the server. If you need the server to fiddle with the bits before they arrive, you'd have to have another server in the way, which is probably defeating most of the purpose in a specialized unit anyway.
My $0.02. I don't have actual experience with these yet. If anyone does, please chime in.
Rule #1 - what is the limiting factor?
Bandwidth? Machine costs? or your time?
It really puzzles me how people wish to skimp on the hardware then set themselves up for later hassles and risk of (expensive) failure.
Lets take a look at a baseline video server from SGI, Origin 200 with MediaBase with say 100 seat configuration, web management tools + 200 Gbyte FibreChannel and network bits would cost about $50K minimum upfront (guesstimate here based on educational discounts and extrapolation of bits and pieces we've purchased over the years) + 10% maintenance/year. Extra for their FailOver system.
OK, now the hand-rolled Linux version. You need to look for
a) streaming software (Darwin?) for multiprocessor
b) decent high-end file system (port SGI XFS?)
c) tuning the sucker for the best SCSI and network parameters
d) video library management software (none as yet, perhaps someone port SGI OpenVault?)
e) system management to monitor the whole thing
Lets assume you've got a collection of genius hackers at 1 man-years worth at each task, working for nothing except glory, you can probably get it done for $20K and 5 man-years worth of pizzas and coke.
Cheap at that price.
Rule #2 - If you don't know what you're doing, make sure you get damn good advice from people who've done it before.
Rule #3 - You pay peanuts, you get monkeys.
LL
You have a machine here that is way into six figures and you want to try Linux to save money? Guy, the cost of NT is neglible in your case. By all means ivestigate your options, Linux, NT, Solaris, etc, but dont be an idiot and try to save an extra 10 or 20 grand. Choose the best OS (whatever that is) and you'll end up with the cheapest solution.
This is not the greatest sig in the world, this is just a tribute.
Just thought you might want to make sure your company doesn't care about maxing out ram.
Windows NT currently supports 4 GIGs of RAM.
Linux only supports 1 GIG... but I have heard some people say that it supports 2.
For a high end video server, RAM is just as important as HD space or CPU power. NT may just be the better choice in this situation.
BTW, using NT for this solution won't increase your overall costs as much as you may think.
Often, the upkeep for Linux can be so much more expensive than NT (if you build your machine correctly) that it more than covers that initial costs of the OS.