Slashdot Mirror
RAMdisk RAID?
drew_92123 asks: "I've got a friend who does a LOT of video editing but is on a limited budget. He is currently using a raid array and while quite large, it's not as fast as he had hoped. I had an idea and wanted to know if there is a way to make it a reality, so of course I though of all the brilliant minds here. I have about a dozen Pentium II computers with 1GB of RAM. I would like to upgrade them to 2GB and throw in some gigabit NICs and create a 1.9GB RAMdisk on each one. Then I want to use one of the computers to RAID the RAMdisks together to be shared via Samba most likely. They are all 1U systems, with no HDD's, just a 64MB IDE flash disk. Any ideas out there?" Has anyone successfully put together such a system? How well did it work for you and are there any caveats that you would like to share with others who would do the same?
Wouldn't the bottleneck of the NICs be an issue?
You might just try reconfiguring the raid to be Raid 0+1 (Striped and Mirrored) That would give you the redundancy and speedy access. If the RAM is already maxed out on the video workstation it might be more cost affective to get a better motherboard that supports more RAM.
~george
Before you buy a bunch of hardware, set up one ramdisk with a network link and find out what your real-life tranfer bandwidth is. I'll bet that the gain, if any, would not be worth the effort.
"Eve of Destruction", it's not just for old hippies anymore...
Just remember, he's SCREWED if the power goes out and he hasn't flushed that /huge/ RAMdisk to a real disk.
At first glance, this sounds like an incredible waste of time. RAID RAMDisk? Why? Are you crazy? What's the point?
But, if you give it some thought it is an interesting idea. Basically you are trying to build a clustered RAM disk..
There is however, a major drawback to this idea. The whole advantage of a RAM disk is speed/performance. Locally, the RAM disk is MUCH faster than a normal disk drive. But, the problem arises when you connect your "RAID RAM disk". You must network the machines in order for them to communicate with each other and suddenly, your performance has dropped to nothing. In fact is is below the performance of a normal disk drive.
In order for your RAID RAM disk to perform equally with a good disk drive you would require a switched gigabit network between your nodes. This will cost more than the "normal" disk. Additionally, even with a switched gigabit network the performance is highly unlikely to exceed the performance of highend disk drives.
So, when you get right down to it, the RAID RAM disk is an interesting idea, just to see if you can do it. But, there isn't really any advantage to it.
Assuming that you'd have to buy at least some of the Gb networking hardware (switches, cables, etc.), you're really not going to be saving much. Assuming at least $100 per 'RAMdisk server', you'll be spending $1200+ for a ~20GB RAID array that will lose everything the minute the power blinks, not to mention drawing several kilowatts of AC.
On the other hand, if you just throw four 100GB ATA-100 drives in a standard tower case with a decent IDE RAID controller, you get five times as much storage for probably about half the money.
Also, remember that most low-to-mid-range PCs can't actually fully take advantage of a gigabit network link, since the PCI bus and CPU get saturated long before the network does.
Sounds like it nbd may be your ticket if you are using Linux. nbd is designed to take a block device, like a hard drive and make it available over a network on a different host. It will also do RAID 0,1,5. Perhaps it will work with a ramdisk. I can't swear that this will work but is sure might, since after all, a ramdisk is implemented as a block device.
RAM is cheap. If you are unconcerned about high electricity costs and need a large *F*A*S*T* device for storage, stripping a number of ramdisks could be the thing to do. PC133 1GB DIMMs are currently about US$200 and are on their way down. Sure, it's expensive compared to RAID 5, but I'm sure it's a lot faster. Just make sure you write out anything you need prior to downing the whole array.
Buy the RAM and use it with a few of these solid state disks. 4GB per PCI slot. But don't be disappointed if it still isn't as fast as you want it to be: The disks are probably not the bottleneck. I'd be surprised if a properly configured RAID array couldn't deliver adequate performance for video editing. Even single disks are fast enough to work with uncompressed video these days.
I'd bet money that he doesn't even have a 64-bit PCI slot in his beefy video editing client. Even marginal IDE RAID cards with 4+ 7200RPM drives can saturate that with no problem. Get a nice 64bit SATA RAID controller and pick up aobut 8 of those new WD SATA 10KRPM drives due out this month and you'll have a local solution that will easily max out a 64bit PCI bus.
This isn't a complete answer, for sure, but for linux RAID you need to present the RAM on computers A, B, and C to computer M (the mux) as block devices. You'll probably need to write a device driver for machine M that presents a block interface and speaks a UDP protocol to machines A, B, and C, where your server stores blocks on the local RAM disk according to whatever scheme works for you. Then 'just' edit the raidtab and build your md0 from the block devices. The reason 'just' is in quotes is because who knows if it'll work with a non-disk block device.
Don't forget to deal with lost UDP packets, but you don't even want to go near TCP's latency on this. If you put them all on a switch your packet loss should be negligable anyhow.
I don't think it's practical for your application but it would be a very cool hack. Good luck!
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
I do customer service repair and testing for high end video servers, particularly the RAIDs attached to them. Based on my experience, what you're proposing seems like a waste of time and money. I think your friend would be much better served with a more traditional RAID setup. For single-user editing station a 4 drive IDE RAID-0 should be able to handle the load, and a similar SCSI array should be more than capable. I recommend SCSI.
In typical storage situations you have 2 issues you need to consider: bandwidth and space. With video you add a 3rd issue which can easily eclipse the other 2 in importance: latency. Latency can cause hiccups in your data stream which would be unnoticable in any other application, but become painfully obvious with video, and any networked sollution is going to add latency. The more network is involved, the more latency will be added, which is why I would absolutely advise against a distributed solution. For that reason even if your network has the same theoretical bandwidth as your RAID, it will be slower, and that will kill the video stream.
Anyway, your friends needs should be taken care of with an older (cheaper) SCSI RAID controller and some older SCSI drives, say 9-18GB 7200RPM. In a RAID-0 configuration they should be able to handle simultaneous record and playback of 12Mbps per drive, with a 3 hour capacity at that maximum bandwidth. For example: my test fixture has space for 5 drives, so it can handle 3 hours worth of 60Mbps video, which is decent for HD and ludicrous for SD. You should be able to pull together something like that for less than what you're planning to spend on RAM and Gig-E network gear, and will be more reliable (minimized data loss in case of power outage) and a hell of a lot cheaper to operate (unless your friend lives in the magical land of free electricity).
Bear in mind that what I've described is the test I use to verify the fitness of our drives, and we use it because we've found that it is more strenuous than any commercially available SCSI test setup. Most new drives are able to handle it, but used drives can be a different story even though they might be perfectly good for any other application. With used drives you may have to drop your expectations to 10 or even 8Mbps per drive, so plan accordingly.
That said, you also want to take a close look at your encoding/decoding hardware, as that can be a source of problems. Don't just look at the hardware specs, either, as all to often the driver capabilities fall far short of what the hardware can theoretically do.
Under capitalism man exploits man. Under communism it's the other way around.
RAID disks and multiple network connections in parallel are equally fast. The PCI bus maxes out around 4 disks or the equivalent in network cards. Who cares how fast it's getting blasted to the machine if you can't do any processing on it.
FCAL controller: $100
36G FCAL disk (10k RPM): $125
cabling: $50
misc parts: $50
enclosure: $75
for say $500, you could have a 36G RAID-0 array setup, local to the machine, and very, very speedy.
..one or two of these :-)
Briefly said: Kicks any RAID (SCSI or not) and your RAMdisk solution up and down the street.
It could be a tad pricey though, as you might wanna suspect.
We suffer more in our imagination than in reality. - Seneca