Slashdot Mirror

Taking a quick glance at Microsoft's HDPhoto standard it looks like it is not really suitable for capturing raw image data for cameras.

In a digital camera, a pixel is red, green, blue and sometimes additional colors laid out in a pattern that can differ from camera to camera. A pixel is not RGB (unless it's a Fovon sensor), so standard lossless formats like PNG or TIFF won't work. HDPhoto supports N color channels and more than 8 bits per color, but I do not see support for the raw CCD data, which is usually not RGB, but R, G, or B (sometimes with additional colors).

I like to preserve my pictures in RAW format since as time goes by, the algorithms to convert the image to a RGB image suitable for displaying keep improving. Also, when editing my photos, some of the processing is done on the raw data before converting it to RGB. Raw data helps for things like noise filtering, for example, since the noise filtering software can be aware of the camera's CCD properties (Noise Ninja, for example, has profiles for my camera at different ISO settings).

The only problem with current raw photos is that each manufacturer seems to have their own format which is incompatible with other manufacturers, or even incompatible between different cameras. It would be nice if they could standardize on something like OpenRAW.

Now, as much as I dislike Microsoft, I think this could be good for regular photos since the compression is about as good as Jpeg2000 (assuming Microsoft isn't spreading FUD) but with a much faster encoding/decoding speed. This could also be a good format for most people taking pictures (who are happy with JPEG).

-Aaron

Well, there isn't any problems in making a TIFF from a RAW. RAW gives you the best post-processing options, which is why it is the preferred way by many. Sadly RAW is problematic (it is not an open format) as well which you can read about on http://openraw.org/

If even a few people splurge on a $29.95 USB device and keep copies of every electronic text they're given, then changes in such things as the "official version of history" and the "current stance of the scientific establishment" can be tracked and compared with similar collections from around the world.

In theory, that's nice idea an that how stuff should be done.
The problem, is that, in practice this is going to be much more complicated. DRM (or Digital Restriction Management, as RMS calls it) is a nice exemple of a technology that is getting more popular in the industry each day. For the keeping by a few individual of the originals - as you suggest - to be useful, the data must be accessible. If the data is crypted by some stupid DRM scheme, a century later, when this data could be found again and should be used to study the past, the DRM keys or even the knowledge of the particular scheme may be long lost ago with the company that produced them. If the crypto algorithm is kept documented, maybe, by the time the historical backup is discovered, flaws in the algorithms will be known or brute force will be easily possible with future hardware (like Enigma). But maybe that won't be the case. Maybe future historians will be left puzzled with some data that looks like garbage and they don't have any clue how to decipher it.
Or maybe the data couldn't be copied in the first place. If the trend to enforce DRM "from-end-to-end" (like in the near future Vista will bring between the graphic card and the monitor, or between the softwares and the output of the sound card) with TCMP and whatnot gets popular, maybe the drive controller will just plain refuse to save "protected intellectual property" to a backup medium (this was already somewhat attempted in Sony's memory stick. The option also exist in the SD design, but was never user yet).

Format will be another complex stuff. If it data is saved using an obvious and/or documented format (be it file format or medium format), it may be retrievable in the future. On the other hand, if data is stored in some weird proprietary format, the ability to read it will be gone when the company designing it disapears, or even if the company decides to stop production and move to a newer format and/or medium. (The keeping of photography is a nice exemple : amateur photos are usually stored in JPEGs, a format that is quite well documented and for wich a lot of documentations and decoding libraries exist. It can be used years later to be visualised again. Given the abundant documentation, a decoder could be rewrotten in a future were the format has moved out of fashion but when picture must be viewed again for historical reasons. On the other hand, professionnal camera often use proprietary RAW formats. Documentation is poor or inexistant, multiple incompatible variants exist within the same brand or even within similar product series, company abandon support for older models and documentation about them is lots. All this leads to the fact that what is shot today in this format may not be viewable in the future, as critiqued by the openraw website) (Texts are another exemple. Old stuff typed decades ago on the first home computer, but that mostly consisted of ASCII Text may still be accessible today, as long as the data was regulary copied from one medium to another, to avoid rotten medium (I doubt audio tapes used back then are still in good shape today) and the lack of modern hardware (no more audio cassette produced today. On the other hand some popular and more modern word processor could have become graveyards if there wasn't enough reverse engeneering, specially given the fact that each version may introduce subtle incompatibilities with previous ones). (On the medium compatibility side of the question, the Domesday Project is a nice exemple of why you should try to avoid to lock yourself in a dead format like LaserDisc).

Sure, this is just the usual proprietary file format game and an attempt to monopolize the market. This why it is important to name the manufacturers who do encrypt data or do not disclose the file format.

Personally I would not count on Adobe. Adobe's stupid PDF update sagas serve exactly the same purpose as encrypting parts of the camera's raw data. You may also contact the makers of xpdf and gs. They certainly can tell a few stories about Adobe.

There is an open raw format. OpenRAW gives more details about this issue. I think it is important that people know how well manufacturers support customers (or not for that matter).

Just did, thanks for the tip, I wasn't aware of it.
Others might want to look at that site as well, it has some interesting pages on it (dealing with the closed RAW formats issue). Of course taking the survey can't hurt :)

Uh, no. If you have a real digital camera and you want REAL control over your images, you use RAW. That TIFF file has been processed by the camera already. The RAW file is unprocessed -- working with RAW is the digital equivalent of working in a darkroom with a negative -- you have lots of choices in how to process the image. Though I do agree with you that there needs to be a RAW standard. Check out OpenRAW. I should point out that one piece of software that's incredibly similar to the Aperture is Bibble, which is also a non-distructive RAW workflow application. I find the interface to be kind of clunky, but Bibble works well. MUCH cheaper than Aperture, and also available for Windows and Linux in addition to Mac. (Full disclosure: the creator of Bibble is my girlfriend's brother-in-law.)

My solution to the backup problem is quite simple, two SATA drives in my work computer in a RAID 1 configuration, and two external 250 GBytes harddrives which I use as backup drives, for redundancy I have two, not one. The ideal would be to take one or both of the external drives off-site, but I haven't gotten that far yet.
It is very unlikely that all of my drives will fail at the same time, unless there's a fire or something of that order - and obviously, off-site drives would remedy that.

When new technology arrives, or I need greater capacity, it's a simple and fast procedure to move data to the new technology, at worst I'll have to add a new interface to the PC.

And most importantly for me, I can afford it; as this solution is quite inexpensive. You don't need any special hardware or software, just regular standard drives which are around 200 each, and that's it, you're about as safe as you can be.

Summa summarum, in the digital age you shouldn't count on archiving anything, instead focus on keeping 'live' backups, store data on media whence it can quickly and easily be copied to new formats or media, as they arrive or become cost efficient.

Oh, and for the DSLR RAW images, you might consider looking into the longetivity of the RAW formats, what guarantee do you have that there's going to be any software to read or process those proprietary files in 10 years time?
See http://www.openraw.org/ for more information. The same applies to any digital format, but the more specialized the format, like e.g. a particular model DSLR from a particular manufacturer, the less likely it is that you're going to have software to read it in the future.

Thank you for your support! You could have used HTML to make the links work.

But I ask you, what the bigger risk? That I delete the mailing list I created, maintain and finance or that Nikon, Canon or all the other camera makers stop supporting some older cameras, leaving you alone with your collection of abandoned RAW files, even you paid for the camera?

And posted by Anonymous, that's so lame.

"My concern with OpenRAW is that Juergen Specht abused one stewardship -- the Japan Photography Mailing list -- by uniaterally deleting it. Who's to say he won't do it again?" (plus your links)

I would have to bet that Juergen Specht is a very sensitive individual and doesn't like criticism.

"My concern with OpenRAW is that Juergen Specht abused one stewardship -- the Japan Photography Mailing list -- by uniaterally deleting it. Who's to say he won't do it again?" (plus your links)

I would have to bet that Juergen Specht is a very sensitive individual and doesn't like criticism.

And this is *exactly* what I'll do from now on and for the foreseeble future; I will *not* entrust the future accessiblity of my visual data to such a company and its formats, and I will not render myself under their mercy given their manifest chauvinism. Does anyone know what suppliers are cooperating with openRAW? Those will get *all* my business.

You can not just change the supplier, because everyone does the same thing and after a while you'll be obliged to return to Canon and ask them for pardon. That's why they don't care about saying you such things.

A more effective way to push them into opening their RAW formats or adopting an open one is to take part to the Act Now! initiative.

Personally I already mailed Canon representatives in the country where I live (BTW you can cut&copy my letter in French from my blog) to urge them into changing their format policy. Obviously it will not be enough, unless all major producers in all countries will be bombed with similar requests from their customers.

This effort is being set up by a guy (Juergen Specht) who hosted a mailing list and then deleted it without notice when some of the posts offended him.

See:
http://www.vudeja.com/04/09/mailing-list

http://www.esthet.org/blog/archives/001294.html

http://www.wirefarm.com/archives/004186.html

http://www.easterwood.org/hmmn/archives/001111.htm l

http://openraw.org/about/

Don't be surprised if this site just up and disappears one day, taking all of the data with it.

http://www.openraw.org/ OpenRAW is a group of photographers and other interested people advocating the open documentation of digital camera RAW files.