Slashdot Mirror
Large-Scale Video Archiving?
BondHeadGuy asks: "Ok, say you have 1000+ cameras emitting 30 frames/second worth of 640x480 grayscale video...and you have to store it indefinitely. What do you do? This is a real question, believe it or not. 30 frames/s * 300 KB/frame = 9 MB/s per camera. 100:1 video compression brings that down to ~90 KB/s. But 90 KB/s * 1000 cameras = 90 MB/s, or ~8 terabytes/day. Retrieval, though, can be essentially arbitrarily slow. Reliability should be good enough to not be annoying long term. Is there a solution that: has 8 TB/day storage capacity, can handle the 90 MB/s write speed, and lets you save some bucks on the (slow) read side?"
Although you are probably looking for a digital solution, don't overlook the solutions that already exist. Security camera VCR's (available at RadioShack et al.) can put 24 hours (or more) of video on a single VHS tape. Get a few VCR's (at $200 each), and a pallet of VHS tapes at Sam's club, and you could record all the video you want!
Like a security camera in a stairwell or something? In that case you can use motion detection to start/stop recording and save well over 100:1. The choice of video codec is going to be important if it's for security (so faces, etc. can be recognised), but if not, you can crank the compression ratio up quite high on most codecs, especially the video codecs that do frame-by-frame motion differencing (i.e. not MJPEG).
Be pragmatic and only archive 15fps. This cuts your archive media costs by ~50% no matter what solution you choose. 15fps should be adequate, although who knows your exact parameters.
100:1 video compression brings that down to ~90 KB/s.
Very interesting problem, with one more very interesting challenge that hasn't been raised yet:
Because the video is streaming in 24/7, you'd have to build a real-time compression system that could handle the 9MB/s and produce a 100:1 ratio. You could perhaps distribute that across multiple machines/CPUs, or build a custom parallel hardware setup to handle the encoding, but at this scale, the overhead of everything might prevent you from reaching the essential criteria of real-time.
Does anyone know what the hardware requirements are for real-time encoding one 640x480 stream? Now, multiply by 1000.
Since you did not state a retrieval time or storage/retention needs, I am going to offer to scenarios; one for long term, fast access storage, one for short term and/or slow access storage.
Storing 8TB/day for a long time with quick access would probably require a tape silo, which is essentially a tape library the size of a small house. StorageTek is one of the leaders in silos (And might be the only vendor making them these days.), and they make some pretty nice stuff. Their PowderHorn 9310 is a nice model for bulk storage and quick recovery. A downside to the silos is that they do not often handle DLT tapes, which can make it hard to use tapes outside of the library.
If you do not need fast access to the data, and have time to root through tapes for restores, just get a smaller tape library (Anything in the 50-100 tape range from ATL/Quantum Adic or Qualtstar running SuperDLT drives controlled by Veritas Netbackup would give you an easy way to handle all the data. NetBackup has excellent archiving capabilites (IE record data, wipe data from disk.), works on just about any platform out there, scales well, and keeps files in GNUTar format for easy access. As for storing the tapes themselves, if you have a small retention time just keep around a few hundred tapes to cycle through. If you need to store the data for a long time, get a few thousand tapes and a set of nice shelves to keep them on. If you do not have somewhere to store them, Iron Mountain does a great job storing data, I have worked with them before and toured one of their facilities, and I can vouch that they do a great job storing data.
This is actually a pretty easy question to answer:
Don't Do It.
This is someone either playing a theoretical game (in which case, the answer is "outsource it") or its someone who has no idea what they really want. You have, ultimately, many conflicting specs here.
You may as well ask for a space shuttle that can fly to pluto in two minutes with no fuel.
Any system that is recording a thousand video inputs is unlikely to need 30 fps for 24/7 (I can't think of anything short of national security installations that would even desire to record 30 fps 24/7, and you'd still have trouble justifying 1000 cameras to cover every building in Washington, DC). Not to mention the logistical implications of DELIVERING 1000 full-frame video feeds to a central location -- you could saturate the entire radio spectrum for the eastern seaboard or have to build the largest gigabit LAN ever deployed.
If you have a real question, please ask it, but this is as bad as a pointy-headed boss spouting off insane specs as the "requirements" for a project because he wants to be on the cutting edge.
And BTW, you won't need 300k per frame for a grayscale 640x480 video image (except that you desire insane specs, which point we've already covered). A fine quality image could be stored in 25-50k, even less depending on the real needs (of which this project seems to lack).
Recursive: Adj. See Recursive.
The Casino industry is probably the most advanced in the business of surveilence... the average Vegas casino probably approaches the scale you're talking about already, however they probably don't archive indefinately.
However, any information I've seen shows them to still be mostly analog capture for any storage, or at least digital-to-analog conversion for storage.
Although they probably won't talk about their security systems, they'd be a great resource.
MadCow.
I used to have a sig, but I set it free and it never came back.
That'd be a storage nightmare.
I don't think so.
Let's assume one camera per VCR, full 30 fps. That's 3 8-hour tapes per day per camera, 3000 tapes a day from 1000 VCRs. 1000 VCR's should cost you $100,000 and take up one
medium sized room (power and AC will need to be enhanced). 3000 tapes per day shouldn't cost more than $3000, or $1 million per year.
You'll only need a few tape monkeys at any given instant, because they'll be around one tape needing changing every 28 seconds. A days's worth of VCR tapes (assuming we pack them in boxes with NO room to spare and stack the boxes in blocks) will take up about 1.5 cubic meters or 50 cubic feet (based on 1x4x8 inches per tape, my rough estimate). That means for a year's worth of tape you need 550 cubic meters or 20,000 cubic feet, which is 3300 square feet if piled six feet high. 3300 square feet is about the floor-size of one big house.
Question to original: Are you still sure you want to do this? If so you might be best off "spreadking the load around". IE: Don't do it all in one place. There are a million convenience store camera's and vcr systems in the world, but they're not all in one place.
Off-hand I can only think of one thing that would handle 3,000 terrabytes per year, and that's if the half million people using Morpheus donated 6 Gigabytes of space each year to your cause.