Slashdot Mirror
MXF+JPEG-2000+HDD = Future of Video Preservation?
Anonymous Archivist writes "Media Matters, a technical consultancy specializing in archival audio and video material, recently completed a Mellon Foundation funded Digital Video Reformatting Preservation Project for the Dance Heritage Coalition. They conclude that MXF is the recommended container format, JPEG-2000 is the recommended encoding format and HDD is the recommended storage media. It's a very valuable series of experiments and offers a strong indication of where the archival preservation of analogue video is heading."
JPEG standart defines several encoding formats, which include lossless compression as well
" The Material eXchange Format (MXF) is an open file format targeted at the interchange of audio-visual material with associated data and metadata. It has been designed and implemented with the aim of improving file based interoperability between servers, workstations and other content creation devices. These improvements should result in improved workflows and result in more efficient working than is possible with today's mixed and proprietary file formats." -- What is MXF
To make laws that man cannot, and will not obey, serves to bring all law into contempt.
--E.C. Stanton
If you just mirror it on two hard drives and then put them into storage, they will last for a very long time. HDDs only die when run via wearing out and not just sitting on the shelf.
JPEG2000 also supports a lossless wavelet compression mode. But yes, the 'lossy' version of JPEG2000 is supposedly better quality than traditional JPEG.
http://www.jpeg.org/faq.phtml
https://bugzilla.mozilla.org/show_bug.cgi?id=36351 (no link for obvious reasons) is the bug report, which has been around since April 2000 but has not progressed much due to licensing issues (copyright ones fixed, patent ones not?).
Avoiding inter-frame compression means that, if you have some small amount of data corruption, you only get one, maybe two corrupted frames of video.
When there isn't patent litigation surrounding the format.
and no you cannot " build a machine to project film out of junk". do you know how much film projectors cost? (hint a good lens alone, is over $5000,00.
I think it is possible. Such an expensive lense isn't necessary. The best film projectors cost a lot, but I don't see it as that difficult to fabricate a basic one from scratch. It won't be the best but it could actually be watchable on a small scale.
It might look hard to the monkeys that assembles ATX computers but I think a decent one could be made from scratch as a small senior engineering project for college, and probably could be adjustable with different sprockets and such. A little more complex than just shining a light through it. It may be hard to imagine, but there was a time when people had portable film cameras for home videos. It wasn't fancy and didn't need to be.
Kodak announcing they'll stop producing film has little to do with anything, IMO. Five years is a lot of time but thus far, the drive to push digital projection is going much slower than people expected. Lucas wanted his Episode III to be exclusively projected in digital video, but it's not going to happen unless he wants to drastically cut the number of screens, I'm thinking a tenth of the screens is not an unrealistic figure.
Of course, part of that is political and economic, because it saves the film distributors from major costs, but they refuse to pass on the savings to the theater companies that must invest as much as a quarter million dollars just to get started.
I wouldn't bother with ArsTechnica. For the definitive guide to capturing analog video and digitally archiving it, you would want to read this guide on Doom9. Plus, they have many other video-related guides on that site and a forum that is second to none in terms of the sheer amount of expertise exhibited by the users there.
Software piracy is victimless theft.
There is more to jpeg2000 than a compression scheme offering scaleable quality and resolution within a single losslessly compressed file. There is also the interactive delivery mechanism offered by the JPIP protocol. Now there is something really useful...
In theory, theory and practice are the same. In practice, they're not.
If you mirror across two disks and put the into storage, and one develops some minor errors, it is not possible to tell which one has the errors
Exceptionally incorrect, prepare for smackdown.
All data on a hard disk is protected by very sophisticated error detection and correction elgorithms. The chance of getting "some minor errors" is effectively nil - either they are corrected by the disc's controller, or the controller returns a "sector unreadble" error - which is what keys any effective mirroring system to go get the data from the second disk. You just don't get bad data from modern hard disks.
This is why God RAID-5 was invented.
No, raid-5 was invented to maintain the I in RAID. Mirroring doubles your costs, RAID-5 only increases them by one disk out of the N disks in the parity group, where N is usually but not limited to 4-5 drives.
When information is power, privacy is freedom.
Wrong. Here in Finland, a new 160 GB hard disk ( Maxtor DiamondMax 10) costs 89 euros. An empty 700 MB cd costs 1 euro. Assuming 1 GB = 1000 MB, it would take 160 GB / 0.7 GB = 229 CD's to get the same capacity as that one HDD. So, if you use CD's, you pay 229 euros, if you use HDD's, you pay 89 euros.
The cost per gigabyte in my example HDD is about 0.55 euros, while the cost per gigabyte in CD is 1.43 euros, which is over twice as much.
Please note that this is by no means the cheapest disk of this size you can find (or the cheapes price for this particular disk); the cheapest price for a 160 GB disk I found was 74 euros for a Seagate Barracuda. For that, the price per gigabyte is 0.46 euros, one third of the price with CD's.
Add the fact that HDD's are much more convenient, and it becomes pretty obvious why HDD's are recommended :).
Hmm. The lowest price for empty DVD-R's (4.7 GB) seems to be 1 euros, which would make the cost per gigabyte 0.21 euros... However, the same source also claimed that the lowest price for empty CD-R's is 0 euros, which puts it's trustworthiness into some doubt. And in any case, HDD's keep on getting bigger, and are still more convenient (no constant CD switching).
Forget magic. Any technology distinguishable from divine power is insufficiently advanced.
"Archiving in film is still the absolute highest quality you can achieve."
Nope.
"Scanning it once does not deteriorate the analog copy for later scanning."
Yes, it does. Scanning usually involves physical damage and dye fading.
That was classic intercourse!
Dear kind AC,
MPEG-2 has nothing to do with wavelets, MPEG-2 is based on DCT. In general, there are four methods for compression, discrete cosine transform (DCT), vector quantization (VQ), fractal compression, and discrete wavelet transform (DWT).
MPEG codecs (1, 2, 4, H.26x) all use DCT. Have a nice day.
There's a good idea in there, but it needs to be clarified.
We're all familiar with RAW files being smaller than TIFF while containing equal or greater content, in the case of digital cameras.
The various RAW formats preserve a camera's individual RGB sensor element values, while a TIFF file is the product of processing on each element's neighbors to simulate (for example) 5 megapixels of 24 bit color from a matrix of 5 million 8 bit monochrome sensors. TIFF is a bit of a waste for digital camera pictures. It amounts to puffery for that application, basically inflating the size by 100% and that is done primarily for the unfortunate purpose of avoiding the camera manufacturer's proprietary RAW file format.
But in other aplications, such as the case of a flatbed scanner, each "pixel" is a real multicolored pixel with its own R, G and B sensor values... so there may actually be 24 (or more) bits of true raw data per pixel... and so correspondingly you don't see a proliferation of RAW file formats for that application. TIFF files are actually not really a waste for scanned images. They may be large, but at least they do not expand the size of the dataset for that application.
Now when in the case of video, the scanning hardware will be quite different. There will be no optical sensors to encode RGB data (since the video camera did that already.) Instead, you have this composite video signal, with embedded luminance and chroma values constantly being sampled by a high-speed Analog to Digital convertor chip (ADC). Any subsequent "RGB" values would only be only the result of processing... which of course expands out the video data somewhat analogously to how the TIFF format unnecessarily puffs out a digital camera sensor matrix. JPEG-2000 or any of the other compression formats are basically adding processing on top of this processing. I'm sure it would make plenty of sense to explore the feasibility of a "RAW" video format consisting essentially of the data stream from the ADC.
That would truly be a raw digitized version of the video signal, on top of which compression could be added.