Large-Scale Video Archiving?

BondHeadGuy asks: "Ok, say you have 1000+ cameras emitting 30 frames/second worth of 640x480 grayscale video...and you have to store it indefinitely. What do you do? This is a real question, believe it or not. 30 frames/s * 300 KB/frame = 9 MB/s per camera. 100:1 video compression brings that down to ~90 KB/s. But 90 KB/s * 1000 cameras = 90 MB/s, or ~8 terabytes/day. Retrieval, though, can be essentially arbitrarily slow. Reliability should be good enough to not be annoying long term. Is there a solution that: has 8 TB/day storage capacity, can handle the 90 MB/s write speed, and lets you save some bucks on the (slow) read side?"

Comments

Analog video systems still work

[jgrider]

Although you are probably looking for a digital solution, don't overlook the solutions that already exist. Security camera VCR's (available at RadioShack et al.) can put 24 hours (or more) of video on a single VHS tape. Get a few VCR's (at $200 each), and a pallet of VHS tapes at Sam's club, and you could record all the video you want!

Re:Analog video systems still work

[rho]

There are specialty security camera/VCR combinations that are built to handle this kind of storage.

If you are willing to sacrifice full 30fps, you can use these. The way VHS is laid down on the tape is the data is written diagonally across the width of the tape. Look at your VCR heads -- see how they're cockeyed? that's intentional.

These specialty camera/VCR combos lay down the data from each camera on successive stripes, and can display 9 different cameras at once on a regular TV screen. You can choose a single camera to fill the screen, but at a reduced framerate.

This will reduce your storage needs by a factor of 9, but at the cost of framerate. This wil not be able to be digitized, unless there are specialty capture cards that can handle this.

From your description, it sounds like you're setting this solution up for a casino, or something very similar. If that's the case, you NEED the full framerate.

You are at the fork in the road -- fast, cheap, reliable: pick any two.

Are they looking at normally still settings?

[RobKow]

Like a security camera in a stairwell or something? In that case you can use motion detection to start/stop recording and save well over 100:1. The choice of video codec is going to be important if it's for security (so faces, etc. can be recognised), but if not, you can crank the compression ratio up quite high on most codecs, especially the video codecs that do frame-by-frame motion differencing (i.e. not MJPEG).

Re:Are they looking at normally still settings?

[stilwebm]

If it is a casino, you want to be able to see finer resolution when a blackjack player slides a card out of his sleeve in under 1/5 of a second.

Store half as much

[Terry Cumming]

Be pragmatic and only archive 15fps. This cuts your archive media costs by ~50% no matter what solution you choose. 15fps should be adequate, although who knows your exact parameters.

A (very) large DLT array.

[j.e.hahn]

A suitably large DLT library with a fairly large number of drives would probably do this. Couple it with some HSM (Hierarchical Storage Management software) and you're probably all set.

In terms of sizing, assuming you get 6MB/s per DLT drive, you'll need at least 15 drives. Go for 20. This gives you room to do cutovers, and the like. I'd recommend fronting this with a LARGE disk for scratch space (preferably solid state, but if that's not in the budget, a big old SCSI disk'll do.) You'll need a pretty hefty server to handle all this (at least a pair of Sun E450s for redundancy). You'll also chew through at least 200 tapes a day at a native capacity of 40G/tape.

HOWEVER, this is by no means cheap. The virtue of the fact that you're talking about 8 terabytes a day should be a clue to that. The sort of tape archive, tape supply, and tape library you'll need is... vast. You're talking very high-end hardware here. You'll need a good cataloging system, and some serious software to maintain all this. You'll need to keep about 75% of your drives streaming all day every day. Tape costs alone will run to about 10k/day, let alone electricity, storage, maintenance and initial outlay. I'd venture a project like this is probably a $15 million dollar outlay to do it right, with at least 2 full time support staff and budget on the order of $40k/day . But if you've got the money, go for it.

Real-time 100:1 compression?

[image]

100:1 video compression brings that down to ~90 KB/s.

Very interesting problem, with one more very interesting challenge that hasn't been raised yet:

Because the video is streaming in 24/7, you'd have to build a real-time compression system that could handle the 9MB/s and produce a 100:1 ratio. You could perhaps distribute that across multiple machines/CPUs, or build a custom parallel hardware setup to handle the encoding, but at this scale, the overhead of everything might prevent you from reaching the essential criteria of real-time.

Does anyone know what the hardware requirements are for real-time encoding one 640x480 stream? Now, multiply by 1000.

Re:Real-time 100:1 compression?

[Coq]

You wouldn't necessarily need real time compression.

If each camera or small cluster of cameras had a storage unit with 2x the space needed for one day, then you could store one day and compress the other, then send the data over to central one day late. Of course, I don't know a whole lot about compression, and I'm working under the assumption that real time compression is slower for some reason. If it takes more than a day to compress a days worth anyway, then you still have the same problem.

Storage silos...

[supabeast!]

Since you did not state a retrieval time or storage/retention needs, I am going to offer to scenarios; one for long term, fast access storage, one for short term and/or slow access storage.

Storing 8TB/day for a long time with quick access would probably require a tape silo, which is essentially a tape library the size of a small house. StorageTek is one of the leaders in silos (And might be the only vendor making them these days.), and they make some pretty nice stuff. Their PowderHorn 9310 is a nice model for bulk storage and quick recovery. A downside to the silos is that they do not often handle DLT tapes, which can make it hard to use tapes outside of the library.

If you do not need fast access to the data, and have time to root through tapes for restores, just get a smaller tape library (Anything in the 50-100 tape range from ATL/Quantum Adic or Qualtstar running SuperDLT drives controlled by Veritas Netbackup would give you an easy way to handle all the data. NetBackup has excellent archiving capabilites (IE record data, wipe data from disk.), works on just about any platform out there, scales well, and keeps files in GNUTar format for easy access. As for storing the tapes themselves, if you have a small retention time just keep around a few hundred tapes to cycle through. If you need to store the data for a long time, get a few thousand tapes and a set of nice shelves to keep them on. If you do not have somewhere to store them, Iron Mountain does a great job storing data, I have worked with them before and toured one of their facilities, and I can vouch that they do a great job storing data.

Seriously non trivial

[CharlieG]

I've looked into almost this exact problem (we had about 100 hours of full color video/day - broadcast quality)

Your going to have to get VERY friendly with your local "Storage Area Network" vendors. What we came up with as a best SHORT term solution was this - Store the video on Video tape or DVD (depending on quality requirements - DVD is NOT broadcast quality), and then use multiple players - things like DVD jukeboxes/tape changers. They can either be manually loaded, or a robot. You then use a cache to store the vidio on a last in/last out basis if you need fast playback (assumption here - the most recently used tapes are most likely to be used again)

Encoding isn't that bad a problem - you just use multiple encoding stations - You say you have 1000 cameras - you're probably going to need better than 1000 encoding stations (don't forget spares) - you batch up 1/2 hour (for example) files and write those out to the SAN when your done - while one station is encoding, the next is recording, and you batch the encoded file up into Near line storage, so you don't NEED real time

Storage is going to take space/money BIG MONEY - your talking about 30 DVDs worth of data/day depending on your robots. Figure 1000s/day

Charlie

Casino

[Sixty4Bit]

Obviously this is some sort of security system that watches a large amount of space. So we are talking either a Casino or park of some kind. If not, then these are the people to ask.

Also, is keeping all of the footage forever a requirement? Or just some of the footage? I would think you may want to keep the footage for a couple of days or weeks at most. If something requires footage to be kept longterm then you would move that from the harddrive to cdrom or dvd.

This is a job for a cluster of iMac's if I ever saw one :)

Easy

[NMerriam]

This is actually a pretty easy question to answer:

Don't Do It.

This is someone either playing a theoretical game (in which case, the answer is "outsource it") or its someone who has no idea what they really want. You have, ultimately, many conflicting specs here.

You may as well ask for a space shuttle that can fly to pluto in two minutes with no fuel.

Any system that is recording a thousand video inputs is unlikely to need 30 fps for 24/7 (I can't think of anything short of national security installations that would even desire to record 30 fps 24/7, and you'd still have trouble justifying 1000 cameras to cover every building in Washington, DC). Not to mention the logistical implications of DELIVERING 1000 full-frame video feeds to a central location -- you could saturate the entire radio spectrum for the eastern seaboard or have to build the largest gigabit LAN ever deployed.

If you have a real question, please ask it, but this is as bad as a pointy-headed boss spouting off insane specs as the "requirements" for a project because he wants to be on the cutting edge.

And BTW, you won't need 300k per frame for a grayscale 640x480 video image (except that you desire insane specs, which point we've already covered). A fine quality image could be stored in 25-50k, even less depending on the real needs (of which this project seems to lack).

Re:Easy

[NMerriam]

30 fps for 24/7 is what our customer wants. End of discussion.

Hey, I want a lot of things I can't have, either.

Part of your job is to make sure the customer isn't making life harder for himself than it needs to be (at least if you're a good consultant/engineer and not just trying to get the bucks regardless of outcome).

I suspect the end of this particular discussion is going to end up with:

- the customer not getting what they want
or
- the customer spending a hundred million dollars to custom-build a system that in three years will cost ten grand, be available by mail-order, and fit under your desk.

SO, the real answer to your question always has been and still remains to call up someone who has a clue, not slashdot -- we can't spec out custom hardware installations for you. This is not a software problem.

If this really is necessary, then call up a video company and get a VAR in your office to figure out how to build a thousand+ cameras with 100:1 hardware codecs that can transmit video over whatever arbitrary distances to whatever arbitrary equipment you have.

Then get THAT VAR and a storage VAR together and figure out how the hell you can store terabytes a day -- they'll build a nice online/offline disk and tape mixture that will cost enough to fund a third-world country, but it'll work, but you'll probably be able to buy the entire storage company for the budget you'll be spending, so look into building your own company or buying one out in order to save some money.

Then call the contractors to install all the cameras and network links and build a control room with monitoring equipment.

THEN -- figure out how to back up all this data, since its clearly very valuable stuff that you've spent a couple million on already, you don't want to lose it. Have that storage VAR get a system that'll automatically dupe all the tapes before storage for redundant storage.

Re:Easy

[monkeydo]

30 fps for 24/7 is what our customer wants. End of discussion.

Your customer could have saved themselves your fee by posting this to Slashdot themselves.

Two questions:
1. How did you get this customer if you have no idea how to meet their needs, and
2. What kind of consultant doesn't work with the customer to refine project goals and requirements?

Your customer has a need, 30fps is not a need, and they belive the only way to satisfy it with full framerate video 24x7. If they are wrong it is your job to show them why and how you can satisfy the need without 30fps.

Gigabit ethernet is becoming common in network cores. Unfortunately GigE is still very expensive, and to go a reasonable distance you need costly single mode fiber. Even 100MB goes much farther over fiber than copper, and I can't imagine your 1000 cameras will be very close together. Just running the fiber to 1000 cameras could easily cost you several million.

If this projecet is going to happen it will probably cost more than 10 million so your customer is wither a government agency or someone with really deep pockets. Either way if you want to get good information here you will need to give more information. If you are worried about disclosing too much you should go talk to some people at vendors like IBM and EMC. They do this kind of thing everyday. They can handle the data storage part and you evidently can handle the video part. Talk to the folks at Cisco about putting it all together. Of course when you go to these vendors they will need more information as well. "That's what the customre wants," doesn't cut the mustard when developing specs for cutting edge projects. Sorry.

Re:Easy

[|guillaume|]

the AC is right. He's asking a "homework" question.

He want suggestions, but will never reveal anything usefull or use his discoveries to educate the masses.

Re:Easy

[onyxruby]

Good post, interesting dilemma, I would ignore those morons who complain that this is a hardware issue and not a software issue.

Now, I have some idea of what your talking about. Before choosing my current job I was interviewing with Dictaphone as a support tech for their systems. I have also worked with a lot of techs from Vegas (and spent a bit of time working out there). I have also worked in security, including a nice high profile skyscraper that housed a very large banks headquarters. Their system comprised over 600 cameras, with 95% of those pinhole. So I might have some idea of what I'm talking about.

I think several assumptions can be made from your other posts. First, you are going to run this over a Gigabit backplane, probably fibre. Second, the cameras are sending their signal back to a central (or one of several sub-central locations) location. From here you want to archive the data. I am assuming that the problem of getting the digital camera feed from the camera to where the data is processed is already solved. Your probably wanting to convert an existing tape system over to a digital format.

Since this isn't a gov job there are only a few customers it could be. Most skyscrapers don't require this kind of coverage, with a very few notable exceptions. Even so, they typically only archive data to 90 days, 1 year or three years depending on the location being filmed.

It can also be assumed that since you want to record everything, that you don't want motion activated systems and with your frame rate of 30 FPS you are looking for quick subtle movements that would be easily missed with a standard security camera system (filming once every so many seconds). This requirement rules out something like an airport, as they would not need that speed. This leads me to believe you are working for a large casino since they like to keep their data a nice long time (lawsuits etc), and they would need a frame rate of 30 fps for sleight of hand issues and the higher resolution that you listed. Regardless, even if that is the case, there is nothing wrong with submitting an ask slashdot article about your needs.

As for solutions,

Re:Easy

[onyxruby]

Hit submit too soon.

As for solutions, you only have a few practical solutions. Certain high end tape jukebox silos (StorageTek) etc can do this. You could also record to DVD with similiar jukebox silos from Dictaphone. I am guessing that there is going to a centralized processing and collection point.

From there you are probably looking for long term storage at an off site location that is accessable from the central location. This is where you can enter the real problem. Jukebox silos are large, and take up a lot of prime (expensive) real estate that you probably don't want occupied for long term storage. This means you need a way to archive the data but still search it from a centralized location (what happened at door #117 on Sept 23rd 2001). This means that you have to have fibre to an offsite location where additional jukeboxes are stored.

Since the speed that you would need makes telco solutions ungodly expensive you make need to lay your own fibre underground to an offsite storage center. You couldn't get the speed you need through solutions like microwave. This also rules out solutions like iron mountain. Make sure your data backbone for this system is completely seperated from normal building IT traffic. Best yet, put in an air gap for this system from the normal IT system, good for security too.

Assuming you lay your own fibre (or already have it as many large financial institutions do), than you would simply add in additional jukeboxes at the other end of the fibre. I also have to assume that any such fibre would have a redundant connection no less than six feet parallel from the primarly line (Norwest Airlines learned that lesson the hard way when their Sonet ring lost an argument to a backhoe - cutting the redundant fibre line right along with the primary).

Storage media that could suit your needs would include tape and dvd solutions. CDr's would be too impractical with the standards you dictated that must be met. Fibre hard drive solutions are a possibility but would be very expensive now, getting cheaper as hard drive capacities grew (assuming the number of cameras and data collection rates stay the same), so might be more favorable from a long term cost analysis than a short term one. Since you want to keep data "forever" this would weigh in more than normal.

Hope this helps

Consult with a Casino if you can...

[MadCow42]

The Casino industry is probably the most advanced in the business of surveilence... the average Vegas casino probably approaches the scale you're talking about already, however they probably don't archive indefinately.

However, any information I've seen shows them to still be mostly analog capture for any storage, or at least digital-to-analog conversion for storage.

Although they probably won't talk about their security systems, they'd be a great resource.

MadCow.

Big Brother needs you...

[MADCOWbeserk]

The guy who wrote the post, is working on face-recognition technology. (thanks for researching him, cwhittenburg)

This exactly the type of thing that could be done, but shouldn't. Stay away from this, we can only hope that any competent people will stay away from this and they will never get it to work.

Sleep with one eye open.......

640x480?

[hatless]

Why 640x480? That's higher resolution than broadcast TV. Do you need that? Broadcast TV is 460x360. Capturing at that resolution will lose you detail, of course, but if it's detail you can lose, your storage requirement just dropped by 40%.

And since you said retrieval can be "arbitrarily" slow, I'd look into using VHS videotapes--even if you store compressed digital on them--as a storage medium. They're slow as hell for rerieval, but the media might be cheap, especially compared to the likes of AIT and such.

Re:Are you sure about those numbers?

[j.e.hahn]

It's unlikely that any digital conversion device(s) would be able to handle the input from 1000+ cameras and then be able to get that data to a central storage location through a local network.... the bandwidth needed for something like that would be incredible (90 MB/s ??), perhaps even requiring it's own seperate gigabit fibre network. Even in a high-end server with several devices connected to it you'd be lucky to capture 20fps.

You're thinking too centrally. Each camera (in a setup such as this) is likely distributed. The cameras can likely be grouped into clusters with a local digitizing system for each. These can transmit back to a central server for archival via a standard switched 100MB/s network (since the total aggregate bandwidth is 90MB/s, the local nodes can't reach that.) Assuming 10 cluster distributed homogeneously, you get 9MB/s from client to server. You still have to deal with the aggregate bandwidth at the server, but the network is no longer clogged (most switches have backbones around 12Gbps+). I'd go with either fiber Gig-E to the server, or channel bonded Fast Ethernet (4 channels should do it, yielding 400MB/s local bandwidth at the server.)

But if you were able to get it done, storage options would probably consist of some type of RAID array (with a HUGE number of disks to be able to hold 8TB/day).

Storing that much data indefinately would require enormous rooms dedicated to storage devices, which may not be feasible. Storing data for a week or even a month would be a challenge in itself.

Not really. Rooms dedicated to tape archival maybe. But for the most part it could be done with several large tape cabinets and off-site storage for tapes older than about 5 days.

The first question is, WHY are you doing this?

[sphealey]

That's an awful lot of data. Why exactly are you doing this? What is the application? Who are you working for? If you are working for/in the United States, does this application meet the requirements of the 4th and 1st Amendments to the Constitution?

Just a few minor questions.

sPh

Re:compression!!

[marcovje]

Still compression might be the way to go, except not compression of the single frames, but some way to only store each x'th frame, and between such a store, and the next xth, only store differences, and again compress the entire stream (stored frames +differences).

The efficiency of the differences storing should be improved by a preprocessing (to compression) step to try to reduce small variations in color,
iow to make two surfaces match
in color in the digitized picture, if they are equal in color

If one, while designing the difference-detection
algorithm, is able to differ between background and foreground, one could try to further increase
compression, while maintaining quality by using lossy compression for the
background only, while keeping the foreground (e.g. faces, important with security cams) sharp.

Since some security camera's send home nearly static, or a set of static images recurring after
a certain time (moving cams), this should increase compression.

I can't imagine that something like this is not already available, e.g. as a sideproduct of all
the research that went into the DVD/DivX/MPEG
standards.

Re:Face Recognition Application

[Anonymous Coward]

I'm now afraid of canadians. Maybe their beaver is out to get me.

Anyways outing the guy was poor form. Especially if it's the other Rob McCready in the USA working at I think Caltech. He gave a great lecture on the GPL and it's ins and outs last year. He's also big on analysing large amounts of data theory through mathematics and computers.

Doesn't scale up very well

[DickBreath]

Only 1000 cameras?

I mean 1000 cameras is only enough to put one camera into each home in a fairly small community. Most of the solutions I'm seeing posted so far don't scale up very well. What if you need multiple cameras per home? And what to do about large cities? Maybe this should be a seperate Ask Slashdot question?

Re:Archiving on the cheap.

[Zurk]

with time delay you dont get full frames.
heres a solution :
3 Breece Hill (or SpectraLogic OEMed) 2.75TB AIT-2 autoloaders with 2 sony ait-2 drives (66Mb/s throughput per autoloader) each - $30,000 each x 3 = $90,000
3 x Sun Ultra 2's @ $2000 each = $6,000 (yep they can do 66MB/sec and max out the breece hills and they can handle the compression/video..total throughput will be 66MB/sec x 3 = 198MB/s which should be more than enough to handle the video feed.)
AMANDA tape backup software (free download) and Sun solaris + a shell script.
30 AIT-2 50GB catridges (about the size of a cigarette pack) x 3 = 90 catridges (should fit in a small briefcase) per day for 2.75 x 3 = 8.25TB/day @ $3000/day

So..it can be done for $100,000 and running costs at $3000/day (plus 1 guy 10 minutes to swap the catridges from the autoloaders and cycle em at the end of the day)...which is fairly cheap for this setup. A small standalone sony AIT-2 drive in a PC can read back the tar files of the video if necessary. One large room should be able to hold all the tapes for a couple of months or so. still at $90,000 its not exactly cheap.

What are the real requirements?

[GreyLightning]

The original question is basically meaningless without a description of the real project requirements.

You've put some big numbers up there, which all the hardware-heads have been happy to answer for you. But without real information, this is all just big-clock-speed hardware masturbation.

The real question is: what kind of single system/application would produce 24,000 hours of unedited high-quality video per day and storing it until the end of time?

Most respondents seem to assume that you're running a network of security cameras. If so, other posters have indicated that your quality or recording time requirements are probably 1-2 orders of magnitude too high.

If you're producing something where this video is actually going to be watched by people who care about beautiful full-color full-frame-rate production quality, where is your 200 million dollar production staff that will be watching and cataloging this data? And if you can afford that, surely you can afford at least one knowledgable systems engineer who knows how to design a storage system!

My absolute favorite bit of your post: "Reliability should be good enough to not be annoying long term". So how much lossage is "annoying"? Will you save space by randomly culling videos from the previous 7 days? If the whole system breaks down once a month, will that be annoying? If you lose one minute out of each hour, will that be annoying?

This is the sort of problem that can be designed and solved if you've got the need and budget for it. But it's not a turnkey solution. That's why you hire engineers.

Analog is the way to go

[CKW]

That'd be a storage nightmare.

I don't think so.

Let's assume one camera per VCR, full 30 fps. That's 3 8-hour tapes per day per camera, 3000 tapes a day from 1000 VCRs. 1000 VCR's should cost you $100,000 and take up one
medium sized room (power and AC will need to be enhanced). 3000 tapes per day shouldn't cost more than $3000, or $1 million per year.

You'll only need a few tape monkeys at any given instant, because they'll be around one tape needing changing every 28 seconds. A days's worth of VCR tapes (assuming we pack them in boxes with NO room to spare and stack the boxes in blocks) will take up about 1.5 cubic meters or 50 cubic feet (based on 1x4x8 inches per tape, my rough estimate). That means for a year's worth of tape you need 550 cubic meters or 20,000 cubic feet, which is 3300 square feet if piled six feet high. 3300 square feet is about the floor-size of one big house.

Question to original: Are you still sure you want to do this? If so you might be best off "spreadking the load around". IE: Don't do it all in one place. There are a million convenience store camera's and vcr systems in the world, but they're not all in one place.

Off-hand I can only think of one thing that would handle 3,000 terrabytes per year, and that's if the half million people using Morpheus donated 6 Gigabytes of space each year to your cause.

Re:Analog is the way to go

[Phoenix_SEC]

There's one small problem with your space calculation though; how do you retrieve anything?

When moving any media to shelf (digital or analog), you need to be able to retrieve it. The type of setup you are speaking of does not satisfy that requirement. Typically, when storing tapes like that, you need to set up shelves on wheels (any idea how heavy those tapes are?) that allow dense storage while allowing the tapes to be cataloged and retrievable. This is expensive to setup and maintain.

Finally, a few points to keep in mind with this type of setup...

1) The 'tape monkeys' get expensive fast. They need to catalog and properly place the tapes. They need to move the tapes from the room(s) with the vcr's to the storage location (at the very least down the hall); it takes time to move them there. Also don't forget that most people you are paying at the 'monkey' level don't really care about your organization, and will lose/misplace/damage tapes (on accident) from time to time.

2) VCR's break down rather frequently when recording 24/7. You will be replacing them each at least once per year (extremely conservative). Granted, they are cheap, and easy to replace, but the numbers do add up.

3) Your footage is analog. Seeing as how this was posted on slashdot, I'm guessing they want to actually do something with the video, and I'm guessing it will have to be digital for that.

If you're interested in storing a few petabytes of data, take a look at my other post (and you are right about distributing the storage).

if this is an airport there will ALWAYS be motion.

[Anonymous Coward]

so the motion/change detecting algorhithms may be of no use. Face-recognition in a crowded walkway in an airport would require 30fps or close to it. The compression can be done at the camrea so that is not an issue.

There seems to be (2) issues at hand:

1) How to store this data given his original paramters (I would want to work with worst-case numbers myself anyways)

2) SHOULD we be helping without knowing what we are helping with (we already have in some ways)

My personal view is that if we are helping someone bid on the recently beta-tested face-recognition systems (in Miami I think it was) that will be part of the new anti-terrorism work we should know that up-front before we help..

Re:Face Recognition Application

[darkonc]

That system rocked. But it has nothing to do with this post.

So then, what IS the system being used for?

Tackling the wrong problem?

[chivo]

Instead of trying to store 8TB a day of video, why not look at ways to reduce the video. If the video is for security reasons(that is my assumption) then why do you need 30fps? Why do you even need one frame every second?

Another solution may be to take an initial image and then simply record the changes, similar to what CVS does. It's much more efficient to store just a few changes than an entire image*30 every second. This solution would probably require a lot more computing power, then it's easier to add computing power than infinate storage space.

MPEG-4 may do some of these things, if so, you already have a solution. If not, get cracking a "fun" algorthim :)