Slashdot Mirror
Afterlife Will Be Costly For Digital Films
Andy Updegrove writes "For a few years now we've been reading about the urgency of adopting open document formats to preserve written records. Now, a 74-page report from the Academy of Motion Picture Arts and Sciences warns that digital films are as vulnerable to loss as digitized documents, but vastly more expensive to preserve — as much as $208,569 per year. The reasons are the same for video as for documents: magnetic media degrade quickly, and formats continue to be created and abandoned. If this sounds familiar and worrisome, it should. We are rushing pell-mell into a future where we only focus on the exciting benefits of new technologies without considering the qualities of older technologies that are equally important — such as ease of preservation — that may be lost or fatally compromised when we migrate to a new whiz-bang technology." Here's a registration-free link for the NYTimes article cited in Andy's post.
I cant help but relate some personal experience here. I know its not production quality, or lots of information, but I recently pulled out my Apple IIe from storage. It included the original 5 1/4 floppy disks and drives.
There was also a cardboard box with ~150 floppy disks, some as old as 20+ years. NOT A SINGLE ONE WAS BAD. Yes, "Zork" still works!
Could it possibly be that the quality of media just isn't up to the demands of a longer life of storage anymore? We all know how Cadillac runs that racket, as in sell the crappy car, and make the money off replacement parts. Has media storage gone the same way? As in 'sell the media, but just good enough to work for x years' before being replaced. And with the demands to increase revenue year over year for public companies, perhaps that time-frame has become shorter and shorter over the years to keep the money flowing in.
Or am I just being too cynical? But you know, a world where such works as "Zork" can survive and "Legally Blonde" can not, on their respective media, might not be that bad.
The original negative is rarely ever touched, except to make more intermediate positives. Even when they remastered the star wars trilogy, they did so from the intermediate positives made from the negative. The original negative should stay in good shape for a very long time, as it's really only accessed to make intermediate positives, usually 3 or so after the negative has been cut. You can always make more inter-negs and release prints from these, which means that the negative will probably NOT degrade due to usage, but from the natural wear of the dyes.
Also, the line in the article regarding digital editing is incorrect. Films are edited in digital form on the computer, but the edit decision list is given to a negative cutter who cuts the negative. There is no loss of quality editing digitally.
The reasons are the same for video as for documents: magnetic media degrade quickly,
The myth of bit rot on hard drives is just that- a myth. It's been perpetuated for two decades by the idiot Steve Gibson, selling his own snake oil (Spinrite), and unfortunately, not enough people are calling him on it. I thought it actually did something too, until I read that post from someone who actually knows how modern drives work. As the author points out, there's a track that can only be written at the factory, and if what Gibson claimed were true, ALL drives would be dying left and right after a few years. Funny how I've found drives made almost a decade ago working just fine now...
The problem hasn't changed; it's mostly obsolescence in drive interfaces, and the drives themselves (for tapes.) PATA is common these days, but everything is going towards SATA, for example.
Both DAT and 8mm were in common use as little as 6-7 years ago...but you'd be fairly hard pressed to find a place to but either now save eBay. And...do YOU want to entrust a backup to an ebay drive?
Please help metamoderate.
People who think raid is about backups are idiots. Raid is useful and legitimate for increasing MTBF. RAID is about keeping servers running in the face of hardware failures, not redundant file backups. Given the failure rate of harddrives, each backup site should be raided for system component redundancy.
If you've ever seen the classic film Metropolis, chances are you didn't see a good 1/3 of the film, with digital, if they had been taken care of that poorly chances are you'd have nothing to watch.
Actually, the damage to Metropolis is due in large part to editing rather than damage of the film stock. Metropolis was edited early and often; the only time the whole, original film was viewed was during its original (and brief) German first-run. Subsequent German, US, and other world-wide releases contained major deletions, reordering of scenes, and other changes which significantly changed the storyline of the film. The only reason that we now know the original order the scenes were meant to go in, and just how much has been lost, is due to the discovery of the original score and title cards.
the no
The standard motion picture format is MJPEG2000. It's not a very efficient format, but it's well defined and going to be around for a long time: there's both a lot of hardware and software that relies on it, and it scales up to high resolutions.
The consumer format wars between Microsoft, Apple, Sony, and other companies have no influence on this.
DVD's have a fraction of the resolution of the original digital video and have already undergone lossy compression (e.g. MPEG2 video, AC3 audio). HD DVD/Bluray is also lower resolution than the original, and the compression is still lossy. As some others have mentioned, you ideally want to store all the film's components (unedited footage, audio, etc.) at the highest quality possible for re-mastering to new formats in later years.
Beyond that, single-bit errors in encoded data streams (e.g. MPEG2, AVC, MP3, AC3) can lead to large distortions in the decoded data. You really have to store everything raw in order to reduce the chances of severe corruption and increase the chances of recovery.
Current movies are already printed to film for viewing in theaters, so the problem isn't at a crisis point yet. The problem will come when major film manufacturers quit making movie film.
If the major studios demand it and are willing to pay higher prices for low manufacturing runs, film manufacturers will still make the film. I predict this will happen for the forseeable future.
By the way, nothing but cost says you can't take each element in a digital scene and print it out to its own frame in addition to or instead of printing out the movie frame-by-frame. Also, nothing says you have to use 35 or 70mm format: If your original digital image has more resolution than you can store on 70mm film you can use a larger format.
You can also use microfilm techniques to print technical information such as the descriptions of camera angles and even computer data files and computer code in human-readable, hex, or some other form to film for archiving, along with the computer code for the programs and enough information to build a virtual computer to interpret that code. Sure, it's a lot of information but remember, the goal is to put all of the information in a storage box and be able to retrieve it in 100 years and make use of it.
If they had done this level of preservation with old NASA computer data and data-descriptions we wouldn't have some of the problems we are having today with un-interpretable data.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
we'd be missing a lot of dr. who episodes if it weren't for folks who copied the original broadcasts. not bad for the 60s.
http://en.wikipedia.org/wiki/Doctor_Who#Missing_episodes
DPX or TIFF image sequences. (These are the standard formats for high-end digital post production already.)
This space unintentionally left unblank.
Hypothesis: it can be as important to lose data as it is to retain it. If all knowledge were preserved, the human specie would be incapable of processing it (with our current technology) in a meaningful way. The problem becomes more advanced when you change 'knowledge' to 'data'. Natural selection occurs as fundamentally in our pursuit of knowledge, our collections of art, and our collective memories as it does in the survival of species. Data must be sacrificed for information to be gained. That we desire to preserve as much as possible is as admirable and honorable a goal as any, and that pursuit should continue unhindered; however, milk will be spilled, movies lost, and species vanished. Without a way to meaningfully index all of the data in all the genres of all the mediums present to date, the goal of preservation is somewhat hollow. This Utopian world in which the Library of Alexandria is preserved is without virtue unless I can find "What Would Jesus Wear to a Funeral on Thursdays?" in a timely fashion.
If it looks like a duck, let's call it a moose.
Two things:
A: That's an enormous load of bull. The average visual impact of a single block error on uncompressed video compared to theoretical 10x lossless compression would be effectively nil even before you take into account that the increased data loss from an error is canceled out by the increased likelihood of errors when you archive a larger amount of data.
B: Even if you don't compress it, that kind of data can be archived indefinitely (including making new copies every few years) for prices in the sub-$5000 range. sub-$10000/year, it could be continuously and readily accessible on a mid-range SAN storage device including electricity.
Even without lossless compression, the numbers you describe simply don't add up to very much data in today's terms. (310MB/sec * 3600 (seconds in an hour) * 3 (hours in a long movie) / 1024 (megabytes in a gigabyte) = 3270 / 1024 (gigabytes in a terabyte) = 3.2TB) A 6TB mid-tier (EMC Clariion, for example) SAN array will run you in the $8000 range after discounts. Archival storage only gets cheaper than that.
This is just another bullshit number Hollywood can spit out to include in the shady math it does to tell everybody they're not making any profit on their $300million blockbuster.
Beaucoup. It is spelled Beaucoup.
mod it 'informative', mes petits.