Image Preservation Through Open Documentation

OpenRAW Group writes "The OpenRAW Working Group launched a website today at http://www.OpenRAW.org designed to solve issues crucial to the future of photography. Digital technology is revolutionizing the photography industry, and an emerging part of that technology is the set of RAW camera file formats. Most professional photographers prefer using RAW image capture because it offers the highest quality and the greatest creative control. The grass roots OpenRAW group arose out of photographers' frustration with camera manufacturers' refusal to openly document their proprietary RAW file formats. That lack of file format information inhibits innovation, limits image processing choices, and endangers the long-term accessibility of millions of photographs. The goal of the new website is to obtain complete documentation by manufacturers of their RAW file formats."

Comments

I have the solution...

[LegendOfLink]

...a camera that will etch the image into a chunk of granite!

Let's see somebody try to encrypt stone, baby!

*off to the USPTO

Re:I have the solution...

[pavon]

1. Patent encryption method in which, you
   
   • Cut stone into million pieces.
     
   • For each piece pick a randomly assigned unique number between 1 and a million.
     
   • Carve number on back of piece.
     
   • Record order of numbers on seperate rock (key).
     
   
   
2. Start rock etching service.
   
3. Profit.
   

Nikon White Balance Encryption

[Hulkster]

For those interested in the recent related Slashdot stories that are very relevent to OpenRAW's efforts, there was discussions last week and also this weekend about Nikon encrypting the white balance information. Michael Reichmann asks a rhetorical (IMHO) question of Do You Really Own Your Own Raw Files.

Enjoy my fun little christmas hoax - help me do it for real in 2005! ;-)

Re:Nikon White Balance Encryption

[Anonymous Coward]

Acording to the author of dcraw it's not only Nikon who are encrypting their metadata...

What about ...

[foobsr]

...taking the position that manufacturers deprive photographers of the proper future use of their IP if the format is not open? IANAL etc ...

CC.

Adobe DNG

[shirai]

I know the story is about getting manufacturers to open up their RAW formats but I think the preferred goal is to have camera manufacturers standardize on ONE format.

Note that Adobe has already developed an open raw format called DNG (Digital Negative). They have a good track record with open formats with PDF files. You may or may not like them, but you they certainly can be generated by non-Adobe products and as far as I'm aware, nobody pays any license fee for that.

Another plus for DNG is that Adobe has a free DNG converter which will convert RAW files from many popular cameras to the DNG format.

You can find more info here about DNG.

Note that Photoshop (the most common photo processor) supports RAW formats for over 80 cameras. You can See a complete list here

Re:Adobe DNG

[myc_lykaon]

but I think the preferred goal is to have camera manufacturers standardize on ONE format.

How can camera manufacturers standardise on one raw format, unless they all agree to use exactly the same technology to capture the image in the first place? I thought the idea of raw was that it's what is pulled off the CCD (or whatever other technology is there) with no preprocessing? Unless all manufacturers agree to have a set of given 'constants' in camera manufacture it ain't gonna work.

Professionals GO HOME!

[disposable60]

These camera makers obviously don't want professional photographers buying their equipment. If I (hobbyist) can't use the Industry Standard photomanipulation package (PhotoShop; my own money, too) with my prosumer camera's highest-quality-mode's files, I ain't buying the camera.

Pinhead control freak MBAs have ruined everything.

Re:Professionals GO HOME!

[dazedNconfuzed]

Nikon's plugin costs $100. While that's a pittance compared to the cost of the camera, it's an extra cost I probably can't afford after buying a camera that expensive.

OpenRAW?

[Anonymous Coward]

Somehow I'm feeling a little uneasy about clicking on a link which points to www.openraw.org.

Film versus Digital?

[John Seminal]

Most professional photographers prefer using RAW image capture because it offers the highest quality and the greatest creative control.

I won't argue the second point, that there is more creative control on a computer, be it a jpeg or anything. To do minor editing in a film lab takes great skill, anyone can edit with photoshop.

But what about quality? Will digital ever come close to the quality film when blowing up an image to full page size or more? Will digital ever be as true as film, can an algorithm on a camera that converts colors and images to zero's and one's be as good as film which reacts naturally to the light?

Re:Film versus Digital?

[Em Adespoton]

This is the same argument that goes on in every other area where there are digital and analog ways of storing data.

Just remember -- you don't see the same colours with your eyes that I see with mine; even the colour skew will be slightly different.

As for quality of enlarged photographs, digital images have pixel halos, but these can be compensated for with digital algorithms; with a digital camera, *every* aspect of what has been recorded is a fixed known value.

With Film-based photographs, there are many uncontrollable variables that go into the recording process; not all films are identical, film is not 100% even across its surface, and most importantly, film is not your retina. When enlarging images recorded on film, there will be a grain effect caused by lack of information in the film. To combat this grain effect, many people *digitally scan* the film and use a computer algorithm to reconstruct the lost pieces. Sound familiar?

The main thing is that film and digital imaging are *both* lossy, and store different bits of visually captured information. Both can be of exceptionally high quality (much higher than the human eye can detect), but both have different limitations on what data actually gets recorded. Film has been around long enough that we accept it, with all its flaws, as "standard". Eventually, this spot will probably fall to digital imaging/storage, as a new generation of people who aren't used to seeing film-based images grow up.

Re:Film versus Digital?

[BrianJacksonPhoto]

But what about quality? Will digital ever come close to the quality film when blowing up an image to full page size or more? Will digital ever be as true as film, can an algorithm on a camera that converts colors and images to zero's and one's be as good as film which reacts naturally to the light?

Uhh, Yes. Most professional photographers shoot digital now. Photojournalism, sports, wedding, editorial, even the commercial studio guys. Full page size? You mean a single page or a double truck at 11x17? Oh, you'd be suprised as to how many of the images you see published are shot digitally.

Digital is being used for billboards and large (4' x 5') display prints. I personally have around 15 20x30's for display prints and they are amazing. And I shoot with a camera that 3 1/2 years old (the original Canon 1D)

Digital doesn't mean printing on some home inkjet. Most pros, don't produce images that way, they still send them to a pro lab to be printed if making physical prints. Offset printing does their own thing.

The capture on the latest crop of cameras is amazing. The colors are beautiful, and crisp.

You think film captured colors true? Wow, where have you been? Film manipulates color all over the places. Ever wonder why there are so many types out there? Provia, Velvia, Astia, NPS, NPC, NPH, E100G, E100GX, E100VS, etc, etc. Each capture the image in a different manner.

Just like there are different types of paper. Each delivers slightly different results. All present color differently. Which one is true? Which combination of paper and film is true? Seems rather subjective if you ask me. But what do I know, I just do this for a living ;p

Re:Film versus Digital?

[Inkieminstrel]

I can't wait do buy one though, simply because of the great learning effect instant feedback can provide.

I have a friend who is a photographer and refuses to use digital (he's also a programmer so it's not because of some fear of modern electronics). His claim is that with regular film you have to learn how to take a picture and get it right the first time, whereas with digital you get instant feedback and can therefore afford to be sloppy.

He laughs about the behavior of digital photographers which he calls "chimping," that is taking a few of the same shot until you get the right one, then hovering over the LCD screen going "ooh oooh oooh."

If you're going to pose every shot, then by all means go digital, but if you want to be sure to catch a particular moment just right, film is the way to go.

What's the matter...

[chill]

...was the domain "nikonsucks.com" already taken?

Shooting RAW is not so great anyway

[cpuh0g]

Good article here on why RAW is really unnecessary for almost all photographers, no matter how "advanced" you think you are.

Re:Shooting RAW is not so great anyway

[smcavoy]

much of his criticism is based on the fact that current cameras use proprietary RAW formats. Open RAW eliminates that.
Granted a large percentage of photographers will never need RAW, but there are plenty that would prefer processing the image themselves and not have worry if they will be locked out of their collection next adobe upgrade or whatever.

Re:Shooting RAW is not so great anyway

[Hays]

That guy doesn't know what he's talking about. Well, most of what he says isn't factually wrong, it's just very misleading. He makes it sound like it's a bad thing to process your RAW files later instead of having your camera apply some half-baked conversion on the fly. It's like arguing that film camera's are crap because polaroids can do the conversion for you on the fly.

Shooting in RAW is very powerful.

Re:Shooting RAW is not so great anyway

[Hays]

Sure, for a lot of shots it won't make a difference... I can accept that. But at the same time I don't see any harm in it. It's twice as much space. Big deal, so I can take 130 pictures instead of 260. Memory is cheap even for my 8 megapixel camera. Also it's not any trouble to convert them. A couple of clicks in a raw conversion program and it will batch convert them all to jpegs.

But the beauty is in the exposure control. You can't expect your camera to properly meter all scenes. It's an AI-hard problem. Where to clip the highlights and the shadows depends on your subject matter. It's so nice to be able to take some time and think about it later.

It's not entirely different from the considerable amount of skill that can go into developing a negative (versus using a polaroid).

Re:Shooting RAW is not so great anyway

[n6mod]

Umm, he has a lot of opinions on the subject. That is NOT the same as knowing a lot about it.

"RAW is NOT a digital negative. Unlike a real negative, it still has restricted resolution and dynamic range..."

Whereas negatives have *un*restricted resolution and dynamic range? Bzzzt.

RAW is a digital negative in that it is as close as possible to what the sensor captured.

Most of his arguments come down to the time spent waiting for the conversion process. If you can't figure out how to use one of the myriad tools out there to do a basic RAW>PSD batch conversion at least as well as the camera, (and then walk away from the computer while it works) then you should stick to wet processes.

Having the raw image is insurance. If something is wrong with a critical shot, you might be able to do a little more with the RAW than you would with a JPG.

Unless you're a sports photographer working on a deadline measured in minutes, shoot raw. Storage is cheaper than a reshoot.

Re:Nits...

[AviLazar]

Actually, RAW is also an acronym "Rules As Written" :)

Guess it depends on what you mean

[Sycraft-fu]

Good digital cameras already exceed teh resolution of 35mm film. You can blow their images up to 8x10 or larger and they look flawless.

Well digital ever look the SAME as film? No, probably not. They deal with light in different ways. However that doesn't mean film is better, just different.

Double Edged Sword

[DumbSwede]

I generally applaud anything to do with opening up data standards. I have heard that several camera manufacturers include things like focal distance and exposure time, etc. to their image formats. Perhaps this is the norm, I'm not really into the specifics of digital photography. We now live in an age where it is trivial to retouch photographs for propaganda or to tamper with evidence. With adequately safeguarded setting info it becomes much harder for digital forgers to do their work. If you can easily get to all that data you can easily alter it.

Granted those with enough motivation, time, or money can circumvent any protections against forgery, but in trying to open up the standard it should be done in such a way to make it an nonreversible process, such that you can manipulate the images, but not be able to push them back into the original format.

I predict that at sometime in the future Digital Camera manufacturers may taught their cameras has having "evidence quality" data integrity. Perhaps some already do.

Granted this evidence integrity argument almost certainly has nothing to do with why most manufactures might choose to close up their data formats.

prior art

[morcheeba]

Here are some examples...

I first saw this on the Korean war memorial in Washington DC (see images at top of that page). That one is low resolution, but a really neat effect. closeup of surface

When?

[John Seminal]

Yes. High end digital exceeds film for resolution

When does digital exceed film? 5 megapixels? 6 megapixels? More? It seems when digital cameras came out, the sales people said 2 megapixel is better than film for 4 by 6 prints, and 3 megapixels is better for a full page.

Then they came out with the 5+ megapixel cameras, and they changed their docs to say 3 megapixels for brilliant 4 by 6 prints, 5 megapixels for a full page.

The quality of film was never measured by how large the print would be, the way they do with digital cameras. Instead film is more concerned with lighting conditions, the time of the exposure.

So I am asking, at what point does film do worse than digital? And who is programming those digital programs to say what "ones and zeros" equals an image. With film it is all natural.

One last quick comment. What will last longer? Film or digital content? What can you be 100% certain to be able to view in the future? CD's get rot, and go bad. Many programs and games that used to run on my 386 will not run on my PIII. Technology changes, maybe we will need some emulator to view those digital images. Or maybe the standard will change and our old 3 megapixel jpegs will be considered crappy, like it came from a childs toy. Film will always have it's place as the elite method for taking quality pictures.

Re:When?

[Inkieminstrel]

My understanding was that 35mm film ~= 10 Megapixels resolution.

How much of that resolution you actually need for a 4x6 print is up in the air, though.

Re:When?

[jtev]

Realy, then why is slide film more expensive than 400 speed color negitive? Eventual print size DOES matter, even with film. The film captures a certain amount of information based on the size of the film grains. The faster the film, the larger the grains, and the lower the resoultion of the picture. The way around this is to use medium format or large format cameras instead of 35mm, in other words more film to get more inforamtion. No, digital cameras do not come close to matching a 12x8 negitive, and I don't realy know how they compare to 35mm, because that also depends on speed. If you have subjects who stand still, it may still be preferable to use that 35mm slide film. Anyway, your argument about the size of the print is utter bullshit. The film may not be advertized that way, but it is one of the factors one must take into account when determining the film to use.

Re:When?

[LurkerXXX]

Certainly, someday 3 megapixel jpgs will be thought of as something from a kids toy. We will all be using holographic displays then ;)

Seriously though, I just got done scanning in a few thousand old family photo's, mainly from the 60's and 70's. A lot of the photos were very faded and overall the colders on many had shifted toward red or yellow. It took a good bit of photoshop'ing to bring things back closer to the way they originally had been. Old prints don't last forever, and neither do negatives. So if you've got old family photo's, I'd start scanning/rescuing them now into electronic versions. If jpeg becomes outdated, you can always keep converting them to the next big file type.

Re:When?

[BrianJacksonPhoto]

When does digital exceed film? 5 megapixels? 6 megapixels? More? It seems when digital cameras came out, the sales people said 2 megapixel is better than film for 4 by 6 prints, and 3 megapixels is better for a full page.

Megapixel does an image not make. Judging based on megapixels has ZERO bering on the quality of the image. That's like comparing Mhz/Ghz in CPU speed. Does the higher the number always equal a faster system? NO.

I have 2 different 4 megapixel cameras(from the same manufacturer even). Canon 1D, and Canon S400. Now the battery of my 1D is larger than the entire S400. You want to tell me that these 2 cameras will produce the same images? I think not! I'm not saying that battery size determines the quality of an image, but the sensors on these 2 cameras are no where near each other. Megapixel is NOT a good indicator of image quality.

I'll put my 4 megapixel 1D up against a 6 megapixel P&S camera anyday.

Instead, film is more concerned with lighting conditions, the time of the exposure.

So, you're saying that I don't care about lighting conditions nor my exposure? Are you NUTS!! Properly exposing digital is the same as exposing for chrome. Just because you're shooting digital doesn't mean you no longer care about lighting and exposure.

What will last longer? Film or digital content? What can you be 100% certain to be able to view in the future? CD's get rot, and go bad.

Never hear of negatives crumbling? I have. Heck, I had a printfile get stored in the wrong place and those negatives are gone. That was only a 15 year old roll of film. Negatives are useless from that shoot. Now, they're gone because of my fault granted, but still...they're gone.

Sure, CD's and DVD's will eventually need to be replaced, but that's easier to convert than the thousands of rolls of film I've shot over the years, esp when they start to degrade. Film is NOT permanent. Not saying that digital is, but film not either.

Standardized RAW = non-sequitor

[dazedNconfuzed]

Standardized RAW files don't make sense precisely because they are "raw".

Each camera, particularly as technologies progress, has its own peculiar nuances regarding how the image is captured. It's up to the manufacturer to decide the appropriate way to store that data in a "raw" format. Complying with a standard for unprocessed data will add unnecessary bulk and/or change data values (wrecking the point of "raw" image files).

I don't want a standard RAW format; I want the camera to give its data unmodified. If I need a camera-specific driver to interpret that data into a useable form, fine. If I want the camera to produce standardized formats, pick TIFF or JPG or such from it's menu. There is a place for standards; unprocessed data is not it. I want the unprocessed data unprocessed.

Re:Standardized RAW = non-sequitor

[shirai]

I answered this in a separate post but don't confuse a standardized format with a format that cannot adjust to the capture parameters of a camera.

For example, DTS (for home) is a digital encoding system for sound but it is very flexible. You can specify the bit depth, the encoding rate, the number of channels and the amount of compression. In other words, you can encode anything from AM radio to 6.1 (and higher I think) all in the same format.

In the same way, a RAW format could easily support multiple bit depths to match the bit depth of the camera. It could also support multiple color square patterns (though almost every camera manufactured uses the RGBG square with the exception of Sony's new camera and the Foveon sensor in Sigma cameras. Don't flog me if I missed one.).

The rest of the data could be encoded as meta data and basically are *hints* on how to decode the image anyways and are not part of the bitmap image. By hints I mean readings from ISO, shutter speed, etc.

Re:Standardized RAW = non-sequitor

[mrchaotica]

It can't just support "multiple" bit depths and patterns, it would have to support arbitrary ones. In other words, the bit depth and pattern would have to be described in the file, not just identified.

RAW

[Anonymous Coward]

RAW IS WAR!

Batista is a false champion!

Re:Adobe DNG and GPL compatibility

[acb]

The DNG specification may be patented. Adobe grant a license to those wishing to implement DNG-compliant code, though the license (in particular, the revocation clause) may be GPL-incompatible. (Disclaimer: IANAL)

Websense says OpenRAW is porn

[Valleye]

Access to this web page is restricted at this time.

Reason:
The Websense category "Nudity" is filtered.

URL:
http://www.openraw.org/comments/?id=47

DNG not RAW for all cameras

[SuperKendall]

If you read the DNG spec carefully, you'll note that some camera data is normalized into plain old RGB values - still logarithmically arranged to keep the most of the sensor data, but still not the exact RAW data obtained from the sensors.

This is because the DNG file format can essentially hold two kinds of sensor data - Bayer grids, and RGB values as mentioned before. If you start to do anything different (like the diagonal arrays of the Fuji cameras or stacked sensors of the Foveon chip, the format just has no way to hold the "real" RAW data and has to transcode it.

For that reason I think the OpenRAW group is a much better idea, because as sensors evolve open specs are the only way to get real raw processors built. DNG is just not enough to handle a space that is still evolving very quickly.

You don't seem to be aware

[^BR]

That there is more than bit depth that vary. Bayer based camera and Foveon based are so different that they can't really share a format...