RAMdisk RAID?

drew_92123 asks: "I've got a friend who does a LOT of video editing but is on a limited budget. He is currently using a raid array and while quite large, it's not as fast as he had hoped. I had an idea and wanted to know if there is a way to make it a reality, so of course I though of all the brilliant minds here. I have about a dozen Pentium II computers with 1GB of RAM. I would like to upgrade them to 2GB and throw in some gigabit NICs and create a 1.9GB RAMdisk on each one. Then I want to use one of the computers to RAID the RAMdisks together to be shared via Samba most likely. They are all 1U systems, with no HDD's, just a 64MB IDE flash disk. Any ideas out there?" Has anyone successfully put together such a system? How well did it work for you and are there any caveats that you would like to share with others who would do the same?

Comments

Eh??

[foooo]

Wouldn't the bottleneck of the NICs be an issue?

You might just try reconfiguring the raid to be Raid 0+1 (Striped and Mirrored) That would give you the redundancy and speedy access. If the RAM is already maxed out on the video workstation it might be more cost affective to get a better motherboard that supports more RAM.

~george

Do some tests first

[HotNeedleOfInquiry]

Before you buy a bunch of hardware, set up one ramdisk with a network link and find out what your real-life tranfer bandwidth is. I'll bet that the gain, if any, would not be worth the effort.

Also...

[foooo]

Ever thought of just upgrading your SCSI controller? You can get RAID controllers that have insane amounts of RAM in them. That might patch up any access issues.

If your concern is extended duration throughput the multiple rack computers with ram *might* be an option, but most normal users wouldn't consider it due to the latency involved with going through the southbridge then the nic then the nic then the southbridge (of the other computer) then the north bridge. And that's just a one way trip.

Just don't shell out a bunch of money before you do a proof of concept.

~george

First reactions.

[FreeLinux]

At first glance, this sounds like an incredible waste of time. RAID RAMDisk? Why? Are you crazy? What's the point?

But, if you give it some thought it is an interesting idea. Basically you are trying to build a clustered RAM disk..

There is however, a major drawback to this idea. The whole advantage of a RAM disk is speed/performance. Locally, the RAM disk is MUCH faster than a normal disk drive. But, the problem arises when you connect your "RAID RAM disk". You must network the machines in order for them to communicate with each other and suddenly, your performance has dropped to nothing. In fact is is below the performance of a normal disk drive.

In order for your RAID RAM disk to perform equally with a good disk drive you would require a switched gigabit network between your nodes. This will cost more than the "normal" disk. Additionally, even with a switched gigabit network the performance is highly unlikely to exceed the performance of highend disk drives.

So, when you get right down to it, the RAID RAM disk is an interesting idea, just to see if you can do it. But, there isn't really any advantage to it.

Re:First reactions.

[Harik]

Sayeth CounterZer0:

Sorry, but I don't think so. RAM -> RAM across a network (assuming at LEAST 100mbit ethernet) will be FASTER than accessing a RAID of local disks. It's all memory to memory transfer at that point - no spin up, no seek time. The disk's may get close for a very long sequential write/read, where the multiple drives can actually come close to using the bandwitdh available via the RAID controller.

I, however, beg to differ.

harik@taz:~$ ping -s 1492 192.168.100.99 PING 192.168.100.99 (192.168.100.99) 1492(1520) bytes of data. 1500 bytes from 192.168.100.99: icmp_seq=1 ttl=64 time=2.80 ms 1500 bytes from 192.168.100.99: icmp_seq=2 ttl=64 time=2.77 ms 1500 bytes from 192.168.100.99: icmp_seq=3 ttl=64 time=2.77 ms 1500 bytes from 192.168.100.99: icmp_seq=4 ttl=64 time=2.77 ms 1500 bytes from 192.168.100.99: icmp_seq=5 ttl=64 time=2.77 ms

This is two machines sitting side by side on a seperate, completely unloaded switch. Don't just go by the 500ns ping time, you actually have to transfer data. You're talking at LEAST 3ms PER BLOCK... and thats with some insanely optimized code.

Now, for video editing 99% of the effort is linear (unless you are horribly fragmented) so you're talking ONE 6ms seek ONCE then thousands upon thousands of linear reads.

Secondly, his "raid array" sucks if the performance is bad. I buy low end LSI Express 500s (Ultra 160 LVD) and they have stellar performance. For doing AV, this is my reccomendation:

Buy a multi-channel Ultra160 or Ultra320 SCSI Raid controller (160s are pretty cheap now that 320s are on the market) Load it up with 5 large drives. Set the stripe size to the maximum. Buy a cheaper IDE RAID and set it in mode 15 (Mirror two RAID5 arrays together, harder to lose data that way.)

Use the SCSI for your working set, and reformat it frequently (or at least delete all files) to defrag. Use RAID0, it's faster. Save your finished projects to the IDE raid, burn to DVD, DLT, whatever.

It will _STILL_ be cheaper then putting 2gig of RAM unto a pile of boxes, AND faster. single-channel ultra-320 can hit you with up to 40 megaBYTES per second, all on a measly 5ms initial seek. (Remember, ALL the drives seek in parallel) Putting drives on the second channel can whollup you with 80MB/second. You're talking around $1500 for the card, of course. But have you priced out a 1U server with 2gig ram lately?

Closer is better

[Anonymous Coward]

Buy the RAM and use it with a few of these solid state disks. 4GB per PCI slot. But don't be disappointed if it still isn't as fast as you want it to be: The disks are probably not the bottleneck. I'd be surprised if a properly configured RAID array couldn't deliver adequate performance for video editing. Even single disks are fast enough to work with uncompressed video these days.

Re:Trading disk latency for network latency

[GoRK]

I'd bet money that he doesn't even have a 64-bit PCI slot in his beefy video editing client. Even marginal IDE RAID cards with 4+ 7200RPM drives can saturate that with no problem. Get a nice 64bit SATA RAID controller and pick up aobut 8 of those new WD SATA 10KRPM drives due out this month and you'll have a local solution that will easily max out a 64bit PCI bus.

Waste of time

[MrResistor]

I do customer service repair and testing for high end video servers, particularly the RAIDs attached to them. Based on my experience, what you're proposing seems like a waste of time and money. I think your friend would be much better served with a more traditional RAID setup. For single-user editing station a 4 drive IDE RAID-0 should be able to handle the load, and a similar SCSI array should be more than capable. I recommend SCSI.

In typical storage situations you have 2 issues you need to consider: bandwidth and space. With video you add a 3rd issue which can easily eclipse the other 2 in importance: latency. Latency can cause hiccups in your data stream which would be unnoticable in any other application, but become painfully obvious with video, and any networked sollution is going to add latency. The more network is involved, the more latency will be added, which is why I would absolutely advise against a distributed solution. For that reason even if your network has the same theoretical bandwidth as your RAID, it will be slower, and that will kill the video stream.

Anyway, your friends needs should be taken care of with an older (cheaper) SCSI RAID controller and some older SCSI drives, say 9-18GB 7200RPM. In a RAID-0 configuration they should be able to handle simultaneous record and playback of 12Mbps per drive, with a 3 hour capacity at that maximum bandwidth. For example: my test fixture has space for 5 drives, so it can handle 3 hours worth of 60Mbps video, which is decent for HD and ludicrous for SD. You should be able to pull together something like that for less than what you're planning to spend on RAM and Gig-E network gear, and will be more reliable (minimized data loss in case of power outage) and a hell of a lot cheaper to operate (unless your friend lives in the magical land of free electricity).

Bear in mind that what I've described is the test I use to verify the fitness of our drives, and we use it because we've found that it is more strenuous than any commercially available SCSI test setup. Most new drives are able to handle it, but used drives can be a different story even though they might be perfectly good for any other application. With used drives you may have to drop your expectations to 10 or even 8Mbps per drive, so plan accordingly.

That said, you also want to take a close look at your encoding/decoding hardware, as that can be a source of problems. Don't just look at the hardware specs, either, as all to often the driver capabilities fall far short of what the hardware can theoretically do.