Slashdot Mirror
Digital Media Archiving Challenges Hollywood
HarryCaul writes "Movies are moving to digital, but what about long-term archiving of the master source materials? Turns out it's harder for digital media than for contemporary analog. Data is being lost, and studios have to learn to cope. Phil Feiner of the AMPAS sci-tech division says when he worked on studio feature films he 'found missing frames or corrupted data on 40% of the data tapes that came in from digital intermediate houses' How to deal with it? Regular migration from old media to new media. Grover Crisp, says Sony has put in a program of migrating every two to three years. Other studios are following suit, but what about indie features? Will we lose films like we lost the originals of the 20s?"
Sounds like a good archival method to me...
If they are concerned about digital data being lost: why not introduce redundancy? Make sure that the data is stored at many locations as possible (and also with a high quality). Luckily the Internet already has a solution for this problem: BitTorrent.
"... only wimps use backup: _real_ men just upload their important stuff on ftp, and let the rest of the world mirror it."
I think it'd work well for the MPAA.
Go somewhere random
They can manage to "lose" the digital masters for every film Nicholas Cage has been in?
Movies that suck will be lost.
Movies that do not suck will be turned into AVI (or MP4 for the quality nitpickers who watch movies in front of their PC).
Hollywood produces them. Piracy sorts them.
Archive final cut to 35mm film.
Personally, I'd like to see methods like OpenAFS with a RAID/SAN data store. A great benefit of AFS is that it's ideal to work over a large IP network. Every night issue a update for all the nodes, a little like rsync I suppose in this respect, but it's ideal for a large infrastructure. Of course things like MD5 sums should be used on the files, perhaps split the large files with RAR or something, maybe use a .PAR file also. You know.. I think the pirate world has this sort of thing sorted already. Why don't the media giants take a leaf out of their book and see how others in the volatile world cope? Maybe they could use newsgroups for data retention?
Why UNIX?
This will only get worse because they insist on the stupid DRM schemes. If a drive crashes you can usually recover a fair portion of the data, if the drive is heavily encrypted and the crash takes out the key to your cipher, then you are fairly fucked. Sure, it is fine today when everybody and his mother has a HDMI compliant player, but with the amount of key-revocations that will likely be necessary as the scheme is cracked over and over again, sooner or latter the increasing complexity of key-management will cause them to start getting lost. The issue is further complicated by having the "plain-text" all in a central place rather than in everybody's home, a hurricane could easily take out a decade's worth of art that way. Of course none of this will happen because the people who make decisions about where the unencrypted originals are stored have a good understanding of how cryptography works, which is why we have DRM to begin with ...
...but codecs are. Chances are we'll have the information in another hundred years but not the means to access it.
Tape backup has a limited shelf life. There is an effect known as "bleed through" that degrades data simply because of the physical layout of magnetic tape backup. This maximum shelf life of data on quality DDS media is about five years, therefore data on tape must be renewed in intervals less than the "best before" date.
Additionally, daily backup tapes (differential or complete) have limited write cycles and must be replaced well before the manufacturers recommended maximum write cycles is exceeded.
Obviously optical media is not an acceptable backup solution, due to its many failure points. Tape is still the best, despite the requirement for periodic renewal.
Digital archiving 101 - get with it people.
There are two problems:
Data loss, where the data is actually lost. This is the equivalent of a scratch on a frame of the master negative. The cure is redundancy.
Obsolescence, where the format becomes difficult to read after a period of time. The cure is lossless copying to new formats over time and/or keeping old equipment around.
Another possible cure the the 2nd problem is to convert it to analog in an "easy to digitize" way.
For example, simply "printing" the movie to 3 black-and-white filmstrips, one for each color, is considered archival. These can be rescanned later if needed. For better archiving, use larger film formats.
Preserve each audio track in an archival analog format as well.
Of course this doesn't preserve all the data that a digital filmmaking process has, but you aren't any worse off than you would have been with an analog film.
If you want to, you can preserve each element of each scene separately, in an analog format or a completely-documented digital format but on an archival media, such as a "paper printout" stored on microfilm. I don't think most movie studios will go to this expense.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
"Turns out it's harder for digital media than for contemporary analog"
The negatives of the original 'Wicker Man' movie were either burnt or buried under the M3 motorway. From what I remember, some of the original 'Babylon 5' negatives were eaten by rats. They're gone, nothing will ever bring them back, because they're analogue media which can't be copied without quality loss.
The problem is the whole idea of a 'master copy' of the movie on media that goes obsolete. The benefit of digital data is that it can be copied any number of times without quality loss, so build a big RAID system and stick the movies on there. Over time it will be upgraded but the digital data will remain... the only time you'll put the data on tape will be for backups, though even then you'd probably be better copying it to other RAID servers at remote sites.
This isn't news -- at least not to those of us who deal with data.
The typical procedure is to do a media refresh (ie, copy it) every few years, and to check for damage. There are concepts like LOCKSS (Lots of Copies Keeps Stuff Safe), so those joking about BitTorrent aren't that far off, but it's a little more structured than that.
Dan Cohen gave a talk recently on "Can Today's Scientific Data Be Preserved? The Specter of a 'Digital Dark Age'", which touched on not only the issue of media failure, but also the loss of the knowledge to extract the encoded information. (much like the 'lost languages' that we don't understand now, how do we make sure that future generations have the necessary hardware and software to get the data back out?)
What's disapointing is just how fast the media is failing. Vendors give a 'mean time to failure' estimate that's based on perfect storage, and that they have no real ways of testing (because, well, if you say it's 40 years, are we going to have to wait 40 years before using it?). Even if you're duplicating your tapes, what happens when all of the copies were put on the same potentially bad batch of tapes?
Quite likely, we're going to lose data. And some of it's going to be because we no longer have copies of the data. The rest is going to be lost because there's so much crap being saved that doesn't need to be that we can't find stuff that still has value in the future.
Build it, and they will come^Hplain.
Publish them on piratebay and let the world be the backup
... and also need to ensure no parts of Jar-Jar are lost/modified due to data corruption. Hot Pink Jar-Jar, anyone?
Put it on the p2p networks and it will be available forever :)
to the original Star Wars. Supposedly he has no original copies from which to return the original classic to us (Laserdisc work-arounds notwithstanding)
I doubt his word on this, but if true, he's a bigger fool that Ep 1 made him appear. In any case, its a great case for multiple digital back-ups.
Cost may play a role as well - as important as it is for film history to save as much as possible, how may film makers in the early stages of a career have the money to produce high quality, redundant backups? And then maintain their viability over the years?
Sure strage is cheap - but who can be sure the hardware will be usable in say 50 years? Can a disk last that long without being spun up regularly? Is optical disk / flash memory archival over time? Will the hardware be readable on whatever computer is in use then or will it be like trying to read an 8" CP/M disk today? Of course, then there is the codec issue as well.
Lucas was dealing in analog which make it even more difficult to properly archive copies for posterity.
I'm a consultant - I convert gibberish into cash-flow.
I think the countless lost Doctor Who-episodes is a good example of how analog video storage isn't perfect either.
GAAH! MY PRINTER IS ON FIRE!!! PUT IT OUT! PUT IT OUT!
Have Google build them a redundant cluster like Google's own, couple of thousand machines, couple of petabytes.... shouldn't be too hard :)
Mix in some redundancy and use RAID-like computations to recover bad segments of tape. The pirate world has had this solved for quite some time.
I have an archive of most of your _good_ material stored on my hard drives.
-William Brendel
A better question might be, "Will anyone really care that they can't watch a high-quality cut of 40-year Old Virgin in the year 2087?" If we are really worried about losing the content of a movie, then archive it to film and accept the faults (loss of image quality, cost of storage, risk of damage, etc.).
So they're going to be using equipment that utilises the analogue hole?
Sounds... hypocritical. The movie industry balks at us for archiving movies we already own, but they're doing it at a massive scale just to save their own ass.
52 52'23" W 47 32'07" N
I was an early adopter of digital cameras and have almost 10 years worth of images. At first I backed them up to multiple gold CD's. Later when external hard drives went up in capacity and down in price I invested in several of those. However, with all this, I have no program that will let me know when I've lost a bit in one of those many files. I'm afraid I won't find out that some data has been damaged until I try to retrieve a specific image, and this, of course, is too late. I've been keeping my eye out for a program that will compute a check-sum (CRC, MD5, etc.) for my files so that when I rerun the program it will let me know if any data has changed so that I can recover files from an alternate backup. Seems to me that this kind of program should be fundamental to any digital archive. Any ideas? --Carey
A good start would be to not use tape.
I don't know what actual the percentage of tape failures is (and they're not telling), but in my own experience it's pretty high.
Hard disks and PCs are cheap enough that every movie could have its own little RAID array somewhere.
No sig today...
The article doesn't give a figure (or maybe I missed it). How much data does the digital original of a feature film comprise? My wild guess is a terrabyte or two, but I guess I can imagine it's much more. It would kind of have to be a lot for this to be a serious issue, or else redundancy would be cheap enough to make this a non-issue.
If long-term data-loss is such a problem, there's an eminently fillable niche for utterly immortal data storage. There are some definite tyranny-of-numbers problems here, but the movie industry has money.
The BBC lost many, many TV shows because video tape was really expensive in those days and the tapes were reused after a show was broadcast.
No sig today...
raising arizona was good. don't get too overzealous, now.
Please stop stalking me, bro.
Is there any way we can accelerate this process for Gigli?
On the subject of disks, RAID etc I remember and article about Google's data storage, and how more could be added to the pool and have redundancy etc automatically dealt with.
Why can't the archive make use of a similar, media independent system? As long as there is some capability in the system to talk to a) the old media and b) the new media (Which can easily be achieved as long as the system is used, because hardware and software are easy enough to build 'bridges' into) then updating the archive is no more complex than adding whatever the latest readily available mass storage is, and letting old and defunct hardware vanish off the far end.
The system itself does all the data integrity, moving data around to make sure it's always available in a couple of places etc so in the event of an old array failing, it just goes "Ah, I've lost that copy. Better make a new one on this new empty array here to make sure I've still got X copies".
On the codec front, again the automation should help make this easier. Presuming the studios are too stupid to archive in a straight file format (Won't surprise me) the system could still be programmed to do the conversion. As long as it can read the old filetype and write the new one, it can be set to automatically update archives to the latest format.
A massive amount of work certainly, and of course it'll need some ongoing support to make sure the support for new hardware/codecs is implemented, but once it's up it just makes sure the archived content is always in a readable format on readable media.
How many people can read hex if only you and dead people can read hex?
As it stands, there is little to no incentive for movie studios to archive their movies beyond the period of their copyrights. Upon entering public domain, their value decreases, and they fail to cover the costs of maintaining them. We then have to rely on other shady operations, like *shudder* P2P, or *bigger shudder* Archive.org (the maintainers of who, I happen to know for a FACT, regularly funded Al-Quaeda and frequently molest our children). They are unreliable and fail to compare with the safe and extremely competent hands of the MPAA.
This is why I advocate unlimited copyright periods for all works retroactively!
Come on people, we've got to save our culture! Do the the right thing and keep our movies profitable! Don't be un-American! Support our movie industry! Even you foreigners can be loyal to America! You just have to keep importing our culture! Resistance is futile! We will rule the world! MWAHAHAHAHAHAHAHAAAAA!
(Mods, if you can not tell that that was a joke by now, you probably don't deserve those mod points.)
You know, there is a difference between trolling and pointing out the flaws in your reasoning. Just saying.
"Slashdot - News and Chat Sites Deviant". (Click "homepage" link above for details).
You could laugh back at me and point out that you are using Linux.
You know, there is a difference between trolling and pointing out the flaws in your reasoning. Just saying.
(What are the odds) They can manage to "lose" the digital masters for every film Nicholas Cage has been in?
Not good enough.
morcego
fortunately nothing made since hollywood started using digital recording is worth archiving.
delete it all and the problem is solved.
Snowden and Manning are heroes.
Feel free to disagree but don't tell us, tell the BBC who've relied on the assistance of private collectors to reconstruct their archives.
Jeez, don't hold it in, tell us how you REALLY feel!
You know, there is a difference between trolling and pointing out the flaws in your reasoning. Just saying.
The current trend in the archival industry is to convert everything to digital. Unfortunately scanning is often a destructive process. In my experience, I have scanned documents that were written over 200 years ago and were still legible. In order to scan these documents, we had to cut all the pages from the binding which effectively destroys the document. The data was then burned onto dvd-r and sent back to the company. If there is any problem with a single disk, there would be a permanent loss of over 100,000 documents.
DVD is fine for the consumer market, if the disc is damaged you can just buy another one. This is not the case with the film industry, once these original masters are gone, they are gone forever. Microfilm, however can last for generations. Even if there is some degradation in the film stock, you can recover almost all the original data. Film can be split into their primary colors onto different reels of microfilm and later be re-joined.
One of my duties in the scanning industry was to operate the microfilm scanner. In this case, these were documents, but any type of information could be theoretically stored. Current models are capable of scanning at least 600 dpi. One of the hardest things would be to rejoin the frames later on and make sure they are all in sync. The way a microfilm scanner works is that on traditional microfilm, there are small squares that mark each frame. The scanner scans continuously and the software searches for these squares known as blips and it will know where to capture the image. With the addition of medium blips for keyframes and large blips for chapters, you can be fairly certain that you will be able to retrieve all the information later. If there is a missing frame, you will only be missing 1 channel of color for that particular frame. This data can be digitally re-created later.
Unlike digital media, microfilm has been around for over 100 years. The images are stored optically rather than digitally so there is a minimal amount of equipment needed for retrieval. Reproduction of microfilm is relatively inexpensive and multiple copies can be produced from the master and can be stored in multiple off-site storage areas. If the master is digital, you can produce multiple copies that are all the same quality so there isn't a single original master. It may be possible to store the sound on microfilm as well. Software would have to be developed to encode and decode the data, but it is possible.
The cure for obsolescence is to simply come up with a lossless format and stick to it. It doesn't matter what it is, just stick with it. If movie formats are in flux, then go with a standard that has endured the test of time, TIFF (it lasted 20 years and is still going strong) and convert each frame into a TIFF image. If the movie industry is worried about "leaking movies", they can just encrypt each frame with another old standard, and stick to that encryption.
There really is no need to keep changing formats for your archiving. When you have billions or trillions of movies and music to manage and preserve, you have no time to monkey around with the digital fashion business. Leave the fashion industry to the consumer side.
I've never regarded CDs to be as durable as either analog tape ...
You have GOT to be kidding. I have had too many tapes fail because of drop-outs, runners, and breakage. Cassette tapes are horrible.
The main problem is quite simply overpopulation. I've often wished I could wake up one morning and discover that around 70%-75% of the global population had simply disappeared during the night. The sociological improvement that would be experienced by the 25% that were left would be astronomical.
Globally, that would leave us at about a 1900 AD population level. In 100 years, we'd be right back where we are today.
Okay. We lose the original elements to Grindhouse or 300. Is it really such a big cultural loss?
--
Franklin Brauner
You really think they'd have DRM on the masters? Is that a joke or are you that crazy?
Who knows, maybe Sun can sell a bunch of 'Thumper' boxes to Hollyweird for preservation of the digital masters.
Just back from Tinsel Town after talking to some of the dudes in the article. Still jetlagged, so it feels a bit unreal reading about it in Slashdot. Still, I'll do my best to explain why things are the way they are...
A film is not often made by a single body. If you are shooting to film, then this will get handled by an editorial department. You may have a fast telecine scan for reviewing the material as dailies. Some of these scans may be used as low-resolution proxies for initial grades. Some chosen bits of film may get re-scanned on a slower pin-resolution scanner for inclusion in the final film. Artificial rendered scenes and special effects may be done by specialist houses, then composited in a post-production house. Your film may have 25 4K images per second in the final version, but the data used to generate it is scattered over the place - if you think a good IT department should be backing all this up, then you haven't worked on a film, my friend. As deadlines approach, people may be working stupid hours, and filling up all the available storage. Then the film gets released, and either makes a billion dollars or doesn't. Either way, the tension is off, people take holiday and zonk out. Nobody will be picking over the cutting-room floor or its digital equivalent looking for things that might be useful twenty years from now. By the time people are back from holiday, they don't know or don't care.
Your end product may be big reels of negative film that you send to a film lab to make prints for cinemas. The lab should keep the golden master clean, and make most of the prints from a second copy. This would be a sensible time to make an archival print of the film. The lab can transfer the whole thing to black and white film. Black and white film does not fade, like conventional colour film does, even in the can. You are getting the print lab to do a pretty full backup of the released film when your people have all gone on holiday. These days you need to back up other stuff. The soundtrack is digital. You will have extra data for the releases in different formats (5:4 TV, 16:9 widescreen, IMAX, etcetera). Still, it is a lot better than nothing. But it is not often done.
The other think is to know what to archive. Very little of the newsreel film I had to sit though as a child to get to the cartoon has survived. Key stuff like the Queen's Coronation or the outbreak of WW2 was clearly history, and put on a special shelf, but little of the day to day stuff survives. There is one cache that survived when a cinema closed, and the tins of newsreel went into landfill. The cinema was in Alaska; the landfill was permafrost, and the film was kept in near ideal refrigetrated conditions. Apart from this fluke, it has probably all gone.
There will probably be digital solutions in time. Increasingly, as we have to manage more different sorts of digital data, there is a need to organize and track everything, which ought to mean it is possible to archive all the essential bits that go into any production. Many other people have posted on the problems of knowing what is on (say) a FAT16 Windows 3.1 disk in some 1980's image format. You can keep copying the data to overcome the degradation of the physical medium, but you still have to know what it means. I know of a system for archiving film images, where the people who did the archiving left the company, and one of them took the laptop with them that had the archiving software, so the ability to read the archives went with them. Do you archive the archiving system? Then, do you archive the system that archived that? Yes - basically, that is exactly what people are proposing to do. But it takes a bit of organizing, and we are not there yet.
Film, on the other hand, has visible images. The 35mm format has remained readable for over 100 years. Even where nitrate stock has flowed over time, we still know what shape it ought to have been. Sometimes we can get something back if we want it badly enough.
A simple analogue solution may be to
It's a bigger problem than most people realize. Twenty years ago, the original footage shot for a film might be 3x what finally appears on the screen, maybe more for a really big-budget film. Today, not only will there be more raw material, there's far more intermediate work product. A big project might have twenty layers going into a final frame. During the project, all that stuff is stored. But then what? Where does all that stuff get archived? There will be terabytes of stuff for any major film. Studios would like to keep it around. It might be profitably reused some day. But how?
Not only is there a huge amount of information, it's in a format used by some content management system that probably won't be runnable ten years from now. Worse, each subcontractor will have their own systems. Look at the list of effects companies involved with any major film today. Will they have the intermediates of a project from ten years ago?
And who's going to archive it? Many production companies are ephemeral, lasting only for the lifetime of the project. There's no ongoing operation responsible for the work products. The major studios may be involved with financing and distribution, and may provide some facilities, but they are no longer the organizations who directly make films. Hollywood is a rental business.
I've often wished I could wake up one morning and discover that around 70%-75% of the global population had simply disappeared during the night.
Chances are that you wouldn't wake up.
You have GOT to be kidding. I have had too many tapes fail because of drop-outs, runners, and breakage. Cassette tapes are horrible.
I have the feeling he's talking about half-inch or one inch analog tape masters, which are quite good, and last a long time with little (perceptible) loss if stored properly. If he'd meant cassettes, he'd have said "cassette tapes".
Compact cassette tapes have always been regarded as one of the first true bastard inventions of the copyright-obsessed recording industry. Mechanically balky enclosures, thin tape that stretches easily...you name it - it took ten years to truly catch on and was subjected to a slow death by CDs.
Saying that having 70-75% of the global population disappear would make the remaining population all share the same idea of morals is ludicrous.
You know, it's odd...but I don't recall having said that. What I said was that I believed that a reduction in the population to that degree would lessen the intensity of problems caused by overpopulation to a corresponding degree. I didn't say anything about whether or not the people left would all think the same way.
I *have* observed in the past however that people do tend to behave in a manner that is more directly conducive to their wellbeing when there are less of them around. Aside from anything else, it's a simple issue of scale. Less people, less crime, less pollution, less of all the bad stuff that by default goes with having groups of people around.
Then what motivation would we have to expand to space?
I've often wished I could wake up one morning and discover that around 70%-75% of the global population had simply disappeared during the night. The sociological improvement that would be experienced by the 25% that were left would be astronomical. Every human problem that you can think of would be either completely solved or radically reduced in severity by that one incident.
Well, I agree with your statements that CDs are rubbish as an archival medium, but this is a bit extreme, doncha think? At first glance, it reminds me of how everyone complains about all the bad drivers out there, but 80% of people proudly consider themselves good drivers.
What I do think is invariably true is that if you wait long enough, chances are you will wake up and the problems you had will have taken care of themselves. No need for disposing of anyone. If one of those problems was a concern about the longevity of CDs, for example, wait long enough and you won't remember or care what was on them. Problem? What problem?
You need to go back and study history, seriously.
Go home, nobody loves you. .
- The Blog
...Furthermore, what makes you think you're personally worthy enough to be in the lucky 25% that would remain after your hypothetical cataclysm? And who the fuck made YOU so high and mighty to determine which people would get to live and die, anyway? Suffering from hubris and delusions of grandeur, are we? Get off your fucking high-horse, buck-o!
'He who has to break a thing to find out what it is, has left the path of wisdom.' -- Gandalf to Saruman
I'm involved in some preservation of history & documentation, and the issue is both huge, and far ranging, not just movies. Here are a few issues, beyond those already stated:
- volume of data. Not uncommon for people to go on a vacation, and come back with 1,000 or 2,000 or 5,000 images on 2 or 4 gig SD chips off thier digital camera. Who has the time to catalog them all? When film cost you - oh, say for arguement a dollar a shot, most people were very careful what they took pictures of to begin with, and keep them safe after the fact. We do not see very much of that attitude today, so paradoxiaclly, the more images a person has, the more likely they are to be lost over time. When it becomes too big an issue to sort the chaff from the wheat, many people just walk away from the job.
- archival materials very rare. Most color films have dyes that fade, most papers today have acid in them and will degrade over time. The list goes on, as it is not just CD's or DVDs that degrade.
- war on terror. A lot of information that is or was public domain is or has been disappearing from public libraries, web sites, etc. For example - not that I ever want to look at them, but the blueprints for the Detriot-Windsor tunnel used to be in the public domain, but not anymore.
There are many more stories, but we are creating a huge, huge memory hole at this very point in time. Maybe now that hte issue has caught the attention of th emovie industry, something might be done about it. Maybe not. Maybe 10,000 years from now the only thing left of our society to prove we were here might be our left over waste form nuke reactors that is still hot.
What about all that material from decades past, where we only have copies left? I could easily envision in that 100 years into the future, some movies may only exist as some disks found in someone's attic that no one knows how to decipher the DRM on them.
Recently an article concerning high efficiency (~250GB on an A4 sheet) paper based storage was posted on Slashdot. Assuming that the article wasn't a scam, then this would provide a good solution for long term archival. Long term archival of paper documents is well understood, provides massive redundancy through easy duplication and requires minimal maintainance.
Nothing sucks like a Vax, nothing blows like a PowerMac G4
Break the data into manageable chunks and use SmartPar (or equivalent) to make recovery files that can repair any one of the chunks. The up-side is you don't need many redundant full backups of the entire data set.
The wide-spread adoptage of SmartPar has been significant in enabling the error-free transmission of huge files via USENET News.
http://parchive.sourceforge.net/
"Because this new approach doesn't benefit from like sized files, it drastically extends the potential applications of PAR. Files such as video, music, and other data can remain in a usable format and still have recovery data associated with them.
The technology is based on a 'Reed-Solomon Code' implementation that allows for recovery of any 'X' real data-blocks for 'X' parity data-blocks present. (Data-blocks referring to files OR much smaller virtual slices of files)."
AC
and uh.... conair.. KIDDING!
On the subject of disks, RAID etc I remember and article about Google's data storage, and how more could be added to the pool and have redundancy etc automatically dealt with.
Why can't the archive make use of a similar, media independent system? As long as there is some capability in the system to talk to a) the old media and b) the new media (Which can easily be achieved as long as the system is used, because hardware and software are easy enough to build 'bridges' into) then updating the archive is no more complex than adding whatever the latest readily available mass storage is, and letting old and defunct hardware vanish off the far end.
That certainly would be a good way to do it - the electronic equivalent of stories passed generation to generation; which brings me to:
On the codec front, again the automation should help make this easier. Presuming the studios are too stupid to archive in a straight file format (Won't surprise me) the system could still be programmed to do the conversion. As long as it can read the old filetype and write the new one, it can be set to automatically update archives to the latest format.
Here's the rub - unless each codec is lossless each conversion losses some of the original; much as retelling stories introduces subtle difference that, over time, result in a very different version than the original.
I'm a consultant - I convert gibberish into cash-flow.
So: what media have survived for centuries or more, not this puny 5 to 10 years' worth for digital tapes or discs? You got it: etch in stone. So it will be a little bulkier than the 10 Commandments (the original ones, not the movie :-) ) but it'll last thru everything except a major volcano.
https://app.box.com/WitthoftResume Code: https://github.com/cellocgw
Eight tracks. The official tape format of the IMF.
People have always been greedy. I laugh when so-called audiophiles swoon about the virtues of vinyl. Record pressing plants were notorious for doing anything to save a nickel, like using stampers well past the point that they had worn out, and adding crushed rock, or something that sounded like it, to the vinyl when oil prices shot up. Their quality control was horrible. A large percentage of the product was defective, and they didn't care.
Mea navis aericumbens anguillis abundat
It would be an absolute bitch, but if you really want it to last, engrave the data on a superhard oxide ceramic. Think Sapphire CD engraved ever-so-slowly by laser ablation. It'll never, ever "rot," it'll never get scratched unless you blast it with diamond powder, and it's stable forever at room temperature. Then it'll play back in an ordinary CD drive. If you're smart, engrave plates with pictures depicting the encoding method and data format, starting with basic physics of light diffraction.
What it comes down to is, "How long do you want your data to last and be readable?" The more work you're willing to put in, and the lower the data density you can tolerate, the longer it'll last. Use stronger/harder materials, use as much material as practical to represent each bit, and make sure they're 100% stable compounds/arrangements. Think Egyptian hieroglyphs from 5000 years ago still being readable today, despite all the crap in the intervening millennia.
...just aren't that great for preservation, but making it digital is still a sound practise
1) Digitize it
2) Create parity data
3) Write to known persistant media
For example, microfilm as we know that will last much much longer than a HDD or tape. Various forms of etchings, glass disks and whatnot are possible and also far more durable. In any case, I think everything that's been in public distribution gets recorded and kept by someone these days, sure we might lose the pristine 4k master copy but an image of any retail CD, DVD etc. should be more than doable.
Live today, because you never know what tomorrow brings
Not if the *right* people disappeared ;-)
I so wish I had some points to hand you, but I don't. :-} Let's not forget the compromises made in the audio signal just to make vinyl sound passably good, eh? Ah, yes; there was never anything quite like bringing a record home only to have the damn thing skip like mad, then take it back when you discover it's a defective record. And to have that seem normal and even acceptable.
Audiophiles rightly hated cassette tapes--but were records really that much better? I don't think so.
I doubt that there'll be an easy solution to the movie master backup problem, at least not anytime soon. In that case, I'm betting we'll never see any Star Wars Prequel Special Editions. Thank God.
Stating on Slashdot that I like cheese since 1997.
After reading a number of the comments it seems that some don't quite understand what needs to be archived.
Firstly, I work at a sound studio for post up in Canada, British Columbia, so I think I've learned a thing or two about the industry and even some of it's archival methods.
Now, for film wise, what needs to be archived in a whole junkload of stuff, theres the raw dailies (or what they shot during the filming days), and the seperate versions of the edited prints. Sound wise, there's the edit sessions, then the re-recording versions. Which can add up to a hell of a lot of space (and yes I know it's a simplified version of it).
So just in that if all that is backed up theres at least 2 levels of redundancy, and even more if you consider all the slightly older versions which could be used to recover the final version.
The other thing is that even if you've lost parts of the one and the other, as long as there are only different parts lost they can be regenerated so to speak through the use of an EDL (Edit Decision List) which (should in theory) says all the SMPTE time-codes of all of the edits the picture editor did.
There's another thing to, is that frequently if it's a video studio working with a sound studio, the sound studio will have a copy of the video on hand as well, merly for their backups incase the video ever comes back for fixes (yes, it does happen that 2, 3, 4 even 5 years down the road a client will come back with fixes.)
At the studio I work at we have two layers of current version redundancy already when working on a project of each part. Tape backup for our final mix, hard drive (shelved after)of our final mix, tape backup of all of our working sessions, and a hard drive of our final mix sessions. On top of this we do daily backups of everything to tape.
So it's fairly difficult to see how information can really get lost persay and seeing as we do the bottom of the barrel (uwe boll anyone?) and we have a backup system that has redundancy out the wazzu, I don't think the major studios should really have that much of an issue with loss.
Have you watched Turner Classic Movies lately?
There is no such thing as an obsolete film, not as long as there are film fanatics. The studios will remove less-than-profitable films from the market, at least temporarily. They'll destroy physical copies, but they don't usually destroy all the physical copies of a film: after a few years off the market, a new generation of film fanatics will be curious about that film, and the studios can make another small profit then by reissuing it. Repeat this cycle often enough, and even a film like Liz Taylor's Cleopatra can make a profit.
Studios are usually good about shifting formats for films they own. They can advertise that this new format shows the film better than it has ever looked since it was in the theaters. They can resell the film in the new format--hey, Slashdot considers that an ulterior motive any time new formats for films come up. It doesn't hurt that if there's been any change in the package since the last issue, or any real remastering, then the whole edition often gets a fresh copyright.
There is a fine line between recklessness and courage... -- Paul McCartney