Adobe Releasing New Photo Format

salmonz writes "Toronto Star just posted a story that Adobe is releasing a new digital picture format; the Digital Negative Specification,or DNG. " Supposed to be use in raw photo formats; without the lossyness of JPEG.

Comments

I can't remember...

[datadriven]

Are we supposed to hate Adobe?

Re:I can't remember...

[alanxyzzy]

Are we supposed to hate Adobe?

Well - Adobe were the ones that set the FBI onto Dmitry Sklyarov

Remember, however, that Elcomsoft are associated with spammers

Re:I can't remember...

[Beautyon]

How' bout just thinking for yourself and not worrying if things should be black or white?

You go first.

Re:Why?

[I_Love_Pocky!]

Don't be silly, Adobe doesn't own PNG.

Maybe

[nonameisgood]

The kicker is, IF the camera companies decide to use it. Standards are only standard is they are used. My questions is, can existing cameras be updated to the new format, or will the manufacturers just want to sell the new ones.
--

Re:Maybe

[polecat_redux]

My questions is, can existing cameras be updated to the new format, or will the manufacturers just want to sell the new ones.

I'm not horribly concerned if Nikon doesn't release an update for my particular camera since Adobe will be providing an image conversion utility that supports many of the proprietary raw formats.

Re:Maybe

[polecat_redux]

So why do we need this solution from Adobe?

Perhaps because it's an effective method to get everyone playing nice. There *are* other lossless formats that camera manufacturers could support (and I'm sure some do), but with no real direction, everyone would just do what suits them and there would likely be no common format between them. Adobe is just trying to help give the camera manufacturers something to agree on (barring other possibly nefarious intentions).

Re:Why?

[avandesande]

This is to provide a unified RAW format for digital cameras. Right now all the manufacturers have their own formats and you cannot process them unless you have special software on your machine. I dont know why they mentioned JPEG because it is not a raw dcamera format.

Re:Why?

[remahl]

PNG is not very good for photographic data. The compression phase also requires significant processing (zlib), so it would be impractical for a digital camera to PNG compress a multi-megabyte image on the field.

Great idea. Some info.

[Thinkit4]

A raw image is what directly comes out of the CCD. In fact it uses less storage than the bitmap that can be produced from it. But it's even better, as with it you can customize white balance and such after the picture has been taken. I use the raw images exclusively on my Canon S45 (it's a difficult feature to find). The problem appears to be in standardization.

Re:Great idea. Some info.

[rgmoore]

Of course, since the nature of the data will be very different depending on what kind of image sensor you have

Which shouldn't be a worry for the vast majority of cameras that use square pixels in a RGB Bayer array. I'd expect that it would be pretty easy to deal with other square pixel Bayer arrays, like Sony's RGBE and the occasional CMY. Non-square arrays, like the ones used in some Nikon pro-series SLRs, and non-Bayer setups, like Foveon's X3 system, might be harder, but I'd expect that they could be accomodated without great difficulty.

Re:Great idea. Some info.

[rbrunner]

No, its 12 bits/pixel. Each pixel represents one of the three color channels. Color info is interpolated to produce an output TIFF or JPEG.

Re:Great idea. Some info.

[laird]

"it would be pretty simple, actually. The RAW format just needs a header which describes the data it contains."

The problem is that the fundamental data isn't standardized -- the RAW data is the signal straight from the CCD, and that's very different depending on the specifics of the CCD. For example, some high-end cameras have three separate CCD's (much, much clearer image, particularly in low light), Fuji CCD's are just weird, etc. So the RAW formats are all very proprietary, not only per manufacturer, but even for each specific camera model.

So TIFF is a good starting point (it's very flexible), but all of the details would have to be defined.

Customer oriented naming

[tod_miller]

People are comfortable with the idea of 'negatives'. If Adobe can make a market for this format, it will tie people into using thier tools (or thier tools will have an additional 'incentive')

I have read up on how using the raw format of the camera, and using the software on the PC you can use the additional information the camera would have thrown away, to do things such as save areas that would have been captured to dark otherwise.

Of course, each cameras format for RAW is basically that, RAW format, and this proposed file format should be nothing more than making sure each software can access it seamlessly.

So in fact, reading the article, it woudl seem like a good idea...

until you look at PDF. I just hope they don't try and put some tagging / watermarking / superflous junk into it.

*cough*

Re:Customer oriented naming

[Nexx]

until you look at PDF. I just hope they don't try and put some tagging / watermarking / superflous junk into it.

Why is that superfluous? I bet you law enforcement would JUMP at a digital file format where they have an encapsulated proof within an image that the image has not been adultrated, as would insurance companies and such, and many other uses I can think of. Heck, as a photographer, I'd like it for protecting my own copyright.

Re:Customer oriented naming

[Anonymous Coward]

It's like DRM -- it would require the camera to slip messages to the court past its owner and against its owner's will.

*Sigh* - you can do reasonably tamper-proof harware, so that it would be pretty much impossible to get at the cryptographic photo-signing module without letting it destroy itself. But you miss the point.

Court evidence is all based on a chain of custody. Cop A stands up and says "I collected empty casings from the scene and placed them in a bag labelled "blah", and gave them to examiner B. B says " I recieved 3 casings in a bag labelled blah, examined them under the microscope, and saw something else".

If you can produce a reasonable chain-of-custody method for digital photos, you don't need crypto in the camera.

Re:Customer oriented naming

[Lumpy]

exactly!!

my digital rebel I take a "black mask" shot every night I start a sky photographing session. I'm looking at 25-60 second exposures (multiples to get even longer exposures) and when I subtract a black mask from the images I remove most of the unwanted ccd noise. (simple script under linux in C to work with the RAW files.) I can not accomplish the same quality by doing a black mask on Jpegs or in photoshop with the RAW plugin I bought for it.

The same goes for stacking images to get a better exposure. performing this on the native RAW images produces crisper results.

Re:what are the benefits?

[polecat_redux]

I suppose the benefit would be that (ideally) when all cameras support the new format, there will no longer be a need to install and use your camera's proprietary software package just to be able to access your images.

Re:Why?

[dmayle]

Does this format offer anything that couldn't be done with PNG?

They key to this format is that it's in a format that's given off by the CCD and CMOS sensors, not in a processed colorspace of any kind (like RGB)

What really concerns me, however, is this:

which Adobe is making available for free

Is this a free-to-all? Or just free-to-camera-developers so we can force user to use photoshop or license from Adobe?

Re:Why?

[Ed Avis]

A conventional 24-bit image file will have eight bits for red, green and blue for each pixel. But very often digital cameras don't have separate sensors for each pixel; they have alternating R G B sensors in a kind of chessboard arrangement, and then interpolate the missing values. This interpolation happens when you go from raw format to the final output, and it can be done by the camera itself or by a photo manipulation program on a PC.

A raw format file, while still storing all the data that has come off the image sensor, can be one third the size of a PNG because it knows that the first pixel has only red channel information, the second only green and so on.

JPEG-2000?

[warpedrive]

What about using the new version of JPEG, for 'digital negatives'?

There are no royalties, no licencing, it has 2x to 5x the compression efficiency, and it's inherently multiresolutional. One file, all resolutions, no reprocessing.. It supports hundreds of component layers, data embedding, lossless encoding..

So.. why would you use some new proprietary Adobe format?

Re:JPEG-2000?

[GigsVT]

We don't know that it will be proprietary.

Adobe has traditionally understood the value of releasing full file specifications under a non-restrictive license, as they have done with PDF and PS.

They have no motivation to make this standard proprietary, if they did that, digital camera makers wouldn't use it! All signs point to them making this one completely open.

Re:JPEG-2000?

[The Old Me]

Also, there is a true-lossless compression option in JPEG2000, which is even more to the point for this story. For background info, old JPEG has a true- lossless option, but it has never been widely implemented-- Getting lossless compression right is one of the various things that JPEG2000 fixes.

Re:JPEG-2000?

[Florian Weimer]

What about using the new version of JPEG, for 'digital negatives'?

There are no royalties, no licencing, it has 2x to 5x the compression efficiency, and it's inherently multiresolutional.

Actually, Adobe did a very similar thing: they took the TIF format (the industry standard storing images with lossless compression) and added a few special fields, using the extension mechanism already provided by TIFF. As far a I can see, Adobe doesn't intend to charge royalties for DNG. It looks quite open -- even the DNG guide for manufacturers doesn't mention any licensing requirements.

(Adobe's DNG web site is already online.)

Re:JPEG-2000?

[radish]

But BMP is not raw. It may be raw from a display device's point of view but an actual RAW file is just the data as recorded by the CCD. This allows for all sorts of interesting post-processing (like adjusting white balance etc) on the PC which isn't possible after you translate to any bitmapped format (lossy or otherwise).

Re:JPEG-2000?

[insac]

At a first glance of the specs, it seems that Adobe is proposing something more than a "RAW" format.

They're filling the format with lots of structured metadata that can give you all the information you need (and probably also some you don't need :-) about the photo AND the settings of the camera (which lens you used, which model, and every other setting I don't know anything about).

You could look at a photo and say "Hey, I like the way the details are emphasized in this photo. Let's see which settings he used".

Compression: they're proposing a "lossless JPEG" compression. Since they cited the DCT-based compression (for lossy compression), I guess that the "lossless JPEG" will be the "old" one and not the JPEG2000 wavelet based format (probably to avoid heavy modifications to today firmware).

My opinion is that the main feature of this format is the presence of photo-specific metadata (while JPEG2000 is "general-purpose"), but it probably could have been substituted by a "sub-specification" of the JPEG2000 metadata.

Re:JPEG-2000?

[Naikrovek]

JPEG-2000 probably wasn't even considered because its not just about lossless images. There is a lot of data in a RAW image besides the image itself. The camera settings, the zoom, aperture, simulated film speed, time of day & date, shutter speed, and a lot of other information that isn't really translatable into human-visible imagery. It is the raw sensor data plus all of those things above and more.

The greatest advantage of photographing in RAW is that you can adjust the white balance long AFTER the image has been taken. My GF (at the time) took photos of an entire party (this was a paid gig) with the incorrect white balance, which would have effectively ruined the entire set if i'd had her shoot in JPEG (or even JPEG-2000), but since she shot in RAW we could easily adjust the white balance after.

JPEG-2000 simply couldn't work in that scenario.

Re:JPEG-2000?

[crawling_chaos]

Except this:

Patent Clarification Notice: Reading and Writing PDF Files Adobe has a number of patents covering technology that is disclosed in the Portable Document Format (PDF) Specification, version 1.3 and later, as documented in PDF Reference and associated Technical Notes (the "Specification". Adobe desires to promote the use of PDF for information interchange among diverse products and applications.

Accordingly, the following patents are licensed on a royalty-free, non-exclusive basis for the term of each patent and for the sole purpose of developing software that produces, consumes, and interprets PDF files that are compliant with the Specification:

U.S. Patent Numbers:

5,634,064
5,737,599
5,781,785
5,819,301
6,028,583
6,289,364
6,421,460

In addition, the following patent is licensed on a royalty-free, non-exclusive basis for its term and for the sole purpose of developing software that produces PDF files that are compliant with the Specification (specifically excluding, however, software that consumes and/or interprets PDF files):

U.S. Patent Numbers:

5,860,074

Unisys never did the same with their submarine patent. They simply said nothing and then enforced it. Adobe has already precluded themselves from that.

Re:JPEG-2000?

[tyrione]

Clarification:

PDF 1.5 is Adobe's current standard and soon followed by PDF 1.6.

Mac OS X utilizes PDF 1.4 and soon PDF 1.5. Adobe licenses prior versions of PDF to be royalty free which only makes sense--give them a taste to what PDF does and encourage them to purchase the latest version to leverage all that PDF can do.

We had a huge pain in the rear issue with Adobe at NeXT and then Apple dealing with Display Postscript--even though we co-developed the standard and actually perfected it. After months of futile negotiations the DPS model for then Rhapsody, now OS X was switched to a modified PDF- Display PDF with custom additions for OS X only.

Adobe is not in the business of free software. They leverage it, yes, but they are a consulting firm.

I like my photo formats RAW

[BitWarrior]

RAW is the way to go for professional photo stuff. From my Nikon D70 I can get RAW format pics which contain lots of extra info about the camera settings and ALL the digital data from the camera, not just what the JPG compressor decided I should have. This is critical for later processing of the photos. Without this extra data, lots of detail in the shadows and highlight regions will likely be lost. I for one want to choose what data to keep and what to throw away, I don't want a compression algorithm making that decision. But, here's the catch... 98% of the people won't give a rat's ass about this. This kind of format is for professional photographers or serious enthusiasts. So for most people, it means nothing, but for me it may be another great format to use since I already use all the Adobe products.

Re:I like my photo formats RAW

[Keeper]

Unfortunately, "get it right the first time" isn't possible because you don't have a laptop sitting next to you to examine the image in every fine detail.

The biggest advantage digital cameras give you over film is the flexability to NOT get it right the first time. Aside from the power to take a large number of shots to experiment with ideas you wouldn't otherwise give the time of day, it also allows you to fix things you didn't notice when you did take the image. If you just want to shoot once and hope it turns out the way you imagined, don't bother getting a dslr.

A lot can be done to process an image after capturing it, and the extra few bits of information make a huge difference in the final quality of the image. Add into the equation a sharper image and better color (jpgs seem to 'compress' the red channel more than others...). Not to mention the advantage it gives you if you decide to work in a colorspace other than sRGB (ie: more accurate color when transforming to the printer's colorspace).

Hopefully...

[erwin]

this makes dealing with RAW files less of PITA. However, has anyone other that Adobe been involved in the spec's creation, or is this just another case of the brilliant minds a [insert company/organization name] coming up with the "ultimate" solution to their corner of the world's problems, without really considering the broader context.

I await more information and a working open-source library...wake me when it's ready.

Re:Hopefully...

[Binary Boy]

Many well known photographers, manufacturers, and developers were consulted. However, in a case like this, in the end it takes someone like Adobe taking the bull by the horns - the proliferation of RAW formats was not (and is not) going to be solved by slow-moving standards bodies - this will take a market force demanding adoption by the many stakeholders, who have not even shown interest in the problems let alone investing in a solution.

Preservation of digital photography in RAW formats is an ugly challenge and kudos to Adobe for taking the lead in a very serious issue. This is not a marketing ploy - in fact, if you understand the effort you'll see it's a very open attempt, and in some ways will be subsidized by Adobe - for instance, their DNG Converter will continue to provide the capability to convert any RAW format they support into DNG, leaving other DNG developers to focus on the act of processing DNG images and not on reverse engineering every new model camera's RAW format.

Re:Why?

[Binary Boy]

This format is not about replacing PNG, and no PNG does not provide the capabilities to do what DNG is about.

DNG is about unifying the mess of "RAW" formats - camera-specific proprietary file formats containing raw dumps of unprocessed sensor information and shot metadata.

Furthermore, DNG is not immediately about getting camera manufacturers to use it themselves - though that would be the ideal. DNG is a bridge format - something you can convert all of your RAW files to for the purposes of long-term preservation/storage. It is open and documented, and based on TIFF so there are existing reader libraries that can handle the basic format (they will need extensions to do anything with it of course).

Adobe has provided DNG Converter which will enable anyone - even non-Adobe users - to benefit from the ongoing R&D Adobe does to support the variety of RAW formats out there. This will simplify the task of building quality RAW converters by allowing small developers to focus on excellent RAW processing and not have to exert to support the many camera RAW formats out there.

Sorry, I just woke up so I'm not going to touch on everything - but this is a major announcement whose importance will become more clear in time.

Re:I fail to see...

[ClippyHater]

Here's another article.

Yet it will be up to camera makers to support the specification, which Adobe is making available for free.

So it looks like they aren't charging for it. And if everyone can standardize on a single format, that'd make EVERYONE'S life a lot easier.

Re:Newsflash

[JessLeah]

A few remain behind and write about why they don't use the new "standard". They get branded "communists". Historical revisionism takes over, and the creators of the useless file format standard get lauded as "innovators". Anyone who complains is tagged as "just jealous".

Full press release link from DPReview

[buro9]

http://www.dpreview.com/news/0409/04092711adobe_dn g.asp

Adobe Systems has today announced a new unified public format for raw digital camera files and a free software tool, Adobe DNG Converter, for translating raw photo formats into the new .DNG format, which is compliant with the Digital Negative Specification. There is no standard format for raw files, which vary between manufacturers and cameras. Digital Negative Specification will introduce a single format that can store information from a diverse range of cameras. An updated Adobe RAW File Converter adds support for DNG as well as several other cameras.

Click here for more information on Adobe DNG

Press Release:

Adobe Unifies Raw Photo Formats with Introduction of Digital Negative Specification Free Converter Tool Kick Starts New Digital Negative File Format by Translating Raw Formats into Easy-to-Use, Archive-Ready Files

SAN JOSE, Calif. -- Sept. 27, 2004 -- Adobe Systems Incorporated (Nasdaq:ADBE) today introduced the Digital Negative Specification, a new unified public format for raw digital camera files. The company also launched a free software tool, Adobe DNG Converter, which translates many of today's popular raw photo formats into the new .DNG file format, compliant with the Digital Negative Specification.

Raw files, which contain the original information captured by a camera sensor prior to any in-camera processing, have become popular due to their promise of greater flexibility and image quality. Until today there has been no standard format for these files, which vary between manufacturers and individual cameras. The Digital Negative Specification solves this problem by introducing a single format that can store information from a diverse range of cameras. Technology leaders, major customers, and professional photographers today also endorsed the new specification (see separate quote sheet).

"Professional photographers and other creative professionals are moving to raw camera workflows because of the outstanding creative control they get over digital images," said Bryan Lamkin, senior vice president of Digital Imaging and Digital Video products at Adobe. "However, clients and publishers have difficulty working with disparate raw file formats and nobody can be sure that today's raw formats will be supported ten years from now. Adobe customers asked us to work on a unified, public format for raw files and that's what we've delivered with the new Digital Negative Specification."

Serious photographers want to store raw files in long-term image archives, because -- unlike standard JPEG's and TIFF's -- these files represent the pure, unaltered capture. Current raw formats are unsuitable for archiving because they are generally undocumented and tied to specific camera models, introducing the risk that the format will not be supported over time. The unified and publicly documented Digital Negative Specification ensures that digital photographs can be preserved in original form for future generations. The new .DNG file format also simplifies digital imaging workflows for creative professionals who today have to juggle multiple file formats as they bring raw images, from different cameras, into print and cross-media publishing projects.

New Specification Built on Existing Standards

The Digital Negative Specification is based on the TIFF EP format, an accepted standard, and already the basis of many proprietary raw formats. The power of .DNG format lies in a set of metadata that must be included in the file to describe key details about the camera and settings. .DNG-compliant software and hardware can adapt on the fly to handle new cameras as they are in

Re:Why?

[mookie-blaylock]

It's very simple: People who deal with content creation (that is, content created for commercial consumption), especially print, don't like JPGs or other lossy formats. PNG, being an indexed-color format, is not the end-all of graphics formats, slashdot ranting aside. Most designers want TIFF files (and PCD wil do, in a pinch). JPGs, no matter the quality, tend to have a nasty habit of exhibiting some noise in their output. That's totally unacceptable for print.

And, again, PNG is totally the wrong format for this. You'd be taking a huge hit upfront in terms of indexing -- or your images would be outrageously huge.

How DNG differs from TIFF, I don't know. I would have thought TIFF would be the obvious answer. (TIFF, for those who don't know, aren't compressed but can be losslessly compressed)

Re:Why?

[Tenebrious1]

I dont know why they mentioned JPEG because it is not a raw dcamera format.

They mention JPEG because that's usually the options you have on a digital camera; proprietary RAW format, which Adobe is trying to standardize, or standardized JPEG, which professionals don't want to use because it's lossy.

It's a good idea, as long as the standard isn't "owned" by Adobe.

Re:Newsflash

[Albanach]

Corporation creates useless file format standard. Everyone grumbles, but switches over.

Actually no it's not useless, this would be very useful, especially if the format is open. Remember Adobe also cerated PDF - they know about making money from open standards.

You see proper digital cameras - especially the ones that cost $10,000 and are used by photojournalists and the like all let you save the image in raw format - that's a copy of the actual data that was captured before any processing. By doing so, you can take the image home and adjust it - white balance, satuaration and everything else - with photoshop et al. Rather than letting the camera make the adjustment and possibly messing things up, you know you still have the raw data so you can undo your changes. Trouble is, all the camera manufacturers ahev their own standard for raw data, so to get it into photoshop, the gimp or whatever you want to use, you must first run the raw image through software provided by your camera manufacturer - and you can bet that software won't run on Linux.

So this is good, 1 because it encourages interoperability and 2 because it further opens up proper image processing to Linux users.

Re:Yep.

[DoubleEdd]

Postscript is probably subject to more controls than PDF. Take the use of Display PDF rather than Display Postscript in OSX for example - Ars Technica mentions the licensing fees that Apple would have had to pay. Surely we're better off with pdf than gzipped ps?

Re:Why?

[Nexx]

Yes. Currently, CCDs and CMOSs support 12-16bits/channel. That can be encompassed in PNG's 48bits/pixel. However, newer generation gear already samples at 18-24bits/channel of RGB, which superscedes what PNG can do.

Plus, correct me if I'm wrong, but doesn't PNG assume all pixels have RGB information? Oftentimes this isn't the case in digital cameras (unless you're using a Foveon sensor). Google "Bayer Pattern" if you're interested.

The article is light on details, but I don't think Adobe is aiming this solely towards digital cameras (even expensive digital backs for medium-format cameras), but also towards medical imaging and what not. There is a reason why ImageMagick can be built with 24bits/channel and up.

Re:Why?

[theparanoidcynic]

A good reason for it is that currently the only commonly used lossless image format (BMP) is uncompressed and enormous.

Supplied software

[Anonymous Coward]

Adobe supplies castrated versions of photoshop which get shipped with some digital camera's. They are in a very good position to pull this off.

Re:Why?

[gabuzo]

Is this a free-to-all? Or just free-to-camera-developers so we can force user to use photoshop or license from Adobe?

Looking at Adobe's history on postscript & pdf format I guess we should reasonably expect this new format's spec to be free (as beer) and usable by everyone

What a stupid name

[JustNiz]

The connotations of 'Negative' are purely historical and bear no relevance to modern (i.e. digital ) photography.

The images stored in ths format will not be negatives (i.e. inverted) anyway, contrary to what the name means and suggests.

Re:What a stupid name

[lachlan76]

The idea is to make it easy for the general public to understand what the format is for - transferring the original image out of the camera.

In this sense, negative is the ideal name.

Re:What a stupid name

[ceoyoyo]

Actually, most professionals and advanced amateurs who tend to shoot in RAW call the RAW files "negatives" and refer to processing them as "developing."

Re:Why?

[GigsVT]

What's wrong with TIFF?

It's lossless, high end cameras already support it, and it's the gold standard for lossless transfer of bitmap data already.

So why make something new when TIFF does the job?

If I see one more "what's wrong with PNG" post...

[MustardMan]

I'm going to vomit. That's liks asking "what's wrong with the mini cooper" in an aritcle about jumbo jets. PNG is not what this format is designed to work with, RAW data from the camera is. RTFA before jumping on the open source bandwagon and screaming that everything should be PNG because you saw a blurb about it on ESR's website. Fuck, I like open source and masturbate every time I see a linux login prompt, and you zealots are starting to piss me off.

Pronounciation?

[NegativeFX]

Dong?

This corresponds to another format release...

[JRootabega]

The Picture Outline Object format, or .POO

Re:Why?

[GreyPoopon]

Sorry, I just woke up so I'm not going to touch on everything

Maybe, but you appear to be more on top of things than all the people who didn't bother to do a shred of research before accusing Adobe of just inventing a new format for no other reason than to control the market. I wouldn't put that kind of tactic past them, but people should at least do some verification for their evil market domination theories....

Re:Why?

[n3k5]

Does this format offer anything that couldn't be done with PNG?

Well, I would hope so, because otherwise it would be incredibly retarded. As the article says, "Raw photo files contain all the original information captured by a digital camera sensor before any in-camera processing occurs [...]" Many cameras have sensors with not just sensors for red, green, and blue, but also a fourth colour. PNG can possibly contain a fourth colour channel, but can it also save meta-information about that channel, which colour it represents? Surely not in any standardised, widely compatible way. In the future, there could be CCDs that don't have square pixels, but hex ones, like an insect's eye. And then there already are cameras with a laser sensor that adds depth information to every pixel. A truly universal file format would have to handle this. Adobe's current version probably doesn't support that kind of funky stuff, but they could include it in a future version and make it backwards compatible. I'm not sure if PNG can even save basic information that's vital for a photographer, like aperture or exposure time.

Re:Why?

[Pirow]

PNG was designed as a replacment for the GIF format, it's not designed for storing photos, PNG is good for images with large areas of contiguous colour and sharp edges such as cartoons and logos. PNG can handle true colour fine, but in my experience it creates massive file sizes for photos, which you don't want in an environment where space is extremely limited such as on a digital camera.

Adobe's take on the issue

[sammy baby]

Adobe has put up a page regarding the new format on their site. But for those who couldn't be bothered to read the original article and are posting questions like, "Why bother..."

There are currently two image formats in wide use for high-end cameras. RAW is the format of choice for people who demand high-quality shots with no compression artifacts. Unfortunately, different camera manufacturers have implemented their RAW encoding differently, which means that two cameras that can save to RAW don't necessarily use the same format. As a result, professionals often have to convert between their vendor's RAW format, and that used by their software.

The other format is good old JPEG, but as you probably know, JPEG is a lossy compression algorithim, making it unsuitable for those who demand a certain level of quality in the shots as captured.

The new format is designed to provide the same advantages of RAW, without the cross-vendor incompatibilities. Adobe is calling it "a publicly documented and readily available specification," although I didn't see any kind of license data around the download of the spec (which is on the Adobe page listed above).

Re:Why?

[Binary Boy]

DNG *is* TIFF.

First, PNG is not always indexed. However, it would have required massive extensions to PNG to turn it into something capable of being DNG, and TIFF is already well placed for extension (TIFF is a container format - most people think of it as a simple image format, but it is very flexible and capable of adaptation).

TIFF supports a huge variety of compression modes, including uncompressed, JPEG, LZW, and ZIP, and a variety of color modes.

DNG is an extension to TIFF, to allow the additional properties of a RAW to be expressed without losing the efficiencies of RAW (linear data, typically one color channel per pixel until processing). Just as a for instance - you can take your DNG into most any TIFF reader today and it will at the very least be able to read the preview embedded in the DNG without any mods to your TIFF library.

Re:Why the need to diss Adobe?

[theskeptic]

Did you even RTFA? Oh, I forgot.. this is slashdot.

For more details about this announcement, go to dpreview.com

Adobe announces new format for raw files

The Digital Negative Specification is being posted to the Adobe Web site free of any legal restrictions or royalties, enabling integration of the .DNG file format into digital cameras, printers, and software products.

Re:Why?

[JimDabell]

standardized JPEG, which professionals don't want to use because it's lossy.

The JPEG standard includes a lossless option too; professionals don't want to use JPEG because lossless JPEG is inefficient, not because it doesn't exist.

This actually does make sense, odd as it sounds.

[foxtrot]

The idea for this spec is not to replace JPEG or PNG. Higher-end digital cameras have a mechanism by which to save images in a lossless format. It used to be this was generally TIFF, but when you're looking at six megapixel images, TIFF nets you pretty monstrous file sizes.

Most digital camera manufacturers came up with their own lossless compression. And, of course, they're all incompatible.

Now, why Adobe? If you're shooting high-end digital photography where you care about it being lossless, and you're doing post-production on your images, what are you using? Adobe Photoshop. So instead of having to have input routines for Photoshop for seventeen different specs, Adobe would much rather the manufacturers have one standard-- can't say as I blame them. Standards are good.

Now, most of us will still keep our cameras set to shoot JPEG, but the folks who do this stuff for a living, this will benefit them. This isn't a case of trying to create a new standard to replace one that already exists to try to get market dominance, a-la Microsoft (or, heck, Acrobat/pdf for the most part...), this is a new standard to make up for the fact that there simply isn't one in this segment and there desperately needs to be.

Now, this doesn't mean Adobe won't leverage the spec and make piles of cash off of it, but at least in this case they're actually inventing something that people need instead of trying to push something on them that they don't.

Re:What does the "G" stand for?

[ari_j]

And, in PNG, the G stands for Graphics; Portable Network Graphics. I know it's a stretch, but possibly Adobe meant the G in DNG to also stand for Graphics. I know it seems to have nothing to do with the file format at hand, but it's possible. I mean, they made PDF, which has nothing to do with Firearms even though the ATF's F stands for that. It's just an Adobe thing, I guess.

Re:Why?

[Binary Boy]

It's an extension to TIFF to support, in a standardized way, the many unique variables that go into a RAW file. No existing specification could capture this range of unique data encoding and metadata without extension - DNG as TIFF was the logical choice for many reasons.

This is not processing RAW into a TIFF - you can do that now with many tools. This is repackaging a RAW file into a new, universal RAW - this should open the RAW processor world to a new level of competition (as the greatest amount of R&D time was always wasted on reverse engineering RAW formats - something Adobe is now doing for you with DNG Converter).

More info available ... specs too

[Midnight Thunder]

DP Review has the press release, which includes the following description:

The Digital Negative Specification is based on the TIFF EP format, an accepted standard, and already the basis of many proprietary raw formats. The power of .DNG format lies in a set of metadata that must be included in the file to describe key details about the camera and settings. .DNG-compliant software and hardware can adapt on the fly to handle new cameras as they are introduced. The new file format unifies conflicting raw formats, enabling the preservation of a pristine version of the original raw image and the metadata associated with it. .DNG is also flexible enough to allow camera manufacturers to continue to add their own "private" metadata fields.

Adobe already has a page on DNG. Its is a free format and the specs are right there on the page, so GIMP won't lose out.

I believe the format is a) to save Adobe money long term (they don't have to support yet another specific sensor) and b) reduce headaches and complaints from the user. We'll just see how the camera companies and digital photography professionals react.

Re:More info available ... specs too

[Binary Boy]

Adobe can save money with the adoption of DNG - but more money will be saved by smaller developers who cannot do the ongoing reverse engineering that Adobe does to support new RAW formats.

Adobe is leveraging their reverse engineering work in providing the free DNG Converter - this will actually benefit smaller developers more than Adobe, as they will only have to tell their users to download DNG Converter to move their RAW into DNG - the third-parties can focus on supporting DNG and providing excellent processing tools, while Adobe will continue to do the hard work of camera support (until the cameras produce DNG directly - which is of course the long-term goal).

Re:Why?

[eddy]

A few years ago it was unpractical to decode MPEG1-Layer 3 in realtime. A blink of an eye later it was merely unpractical to encode in realtime. Now we have ~100g devices that can decode and encode in realtime for hours on end.

If they (Adobe) don't want any kind of compression then, as we all know, TARGA would do.

If they in fact wants to use compression, but to use different models from the ones provided in the PNG standard, wouldn't it make more sense to extend PNG with said models?

Re:Me too.

[magefile]

Except that they don't control DNG either. It was released "for free" (legally & royalty-wise). So it's either NIH, stupidity, or something we don't get.

Re:Why?

[lucifuge31337]

Because TIFF doesn't do the job. This isn't another direct display format. It's a raw format. For example, the raw files from my Nikon equipment require processing before they can be displayed (FWIW, Photoshop CS already supports the nikon raw format - .NEF). Raw format files are nothing more than the the CCD saw....it doesn't take into account the dot screen or any filtering that is integral to the CCD...your processing software (whether it's in the camera or 3rd party like Nikon View, Bibble, or something else) needs to apply color correction and actually interpolate the sensors on the CCD into the correct colors.

Re:Why?

[dspeyer]

So what's wrong with JPEG 2000? It's a lossless, free compression scheme with similar file sizes to JPEG. There's already an open source implementation even if it's not quite good enough for embedded use.

For that matter, why aren't we seeing J2k everywhere? It looks like a great format.

Re:Why?

[magefile]

I don't understand it either, but RTFA - they released it with no legal or financial strings attached.

Re:Why?

[Anonymous Coward]

This DNG is apparently TIFF-EP with metadata added. The metadata content seems quite reasonable and contain information about the camera which is basically irrelevant in any other context.

The spec. appears to be unencumbered. Although there aren't bright yellow "FREE FOR ALL TO USE!" disclaimers, it does not bear any mention of a patent and it does state that the spec. is "free". Since it would be in Adobe's interest (as a market leader in photo editting software) to minimize writing kludgy compatibility code in the future, I don't see why anyone should assume that this offer is being made "in bad faith".

Oh yes, there might not be all of this "OMG PATENTZ!" hysteria, if the posting had included this link to the Adobe site, which features a pdf of the specification itself.

Even more happily, the pdf was a simple one & rendered promptly under xpdf/gs.

Re:Why?

[Zocalo]

Because my DSLR produces ~10MB RAW files with 12bit/hue colour resolution which convert into ~36MB 16bit/hue colour resolution TIFFs? Or how about that those TIFFs only have a fraction of the flexibility offered by the RAW versions in post processing. It should be obvious that you need 3x the storage space, but if you've got used to rattling off shots at a rate of several a second, expect that to get slashed too.

Re:Why?

[UnknownSoldier]

> But very often digital cameras don't have separate sensors for each pixel; they have alternating R G B sensors in a kind of chessboard arrangement, and then interpolate the missing values.

Couple of notes...

That chessboard layout is called a Color Filter Array, usually arranged in a Bayer Pattern.

Digital cameras these days are 10 bit in RAW mode.

And some even have 4 color sensors.

dpreview is THE site for camera buffs, much the same way avsforum is for us audio & vidio philes. Now if only I could find sites for other categories....

--

"Geometry is frozen music"
- Pythagoras

Re:Why?

[Too Much Noise]

Tiff is HUGE (at least in uncompressed form, which seems to be what cameras use these days). On the other hand (quote from here)

DNG is based on the TIFF-EP format with extra metadata and scope for lossless JPEG compression, Adobe said. The format supports both mosaiced (CFA) and demosaiced interpolation.

Re:Obligatory Simpsons Ref...

[Aceto3for5]

All your JPEGS are belong to Adobe?

Something about Soviet Russia, im not sure how it goes, but your mother was a whore.

do you know what you're talking about?

[SuperBanana]

How about people work on something we actually need?

How about you don't tell us photographers what we do/don't need?

We DO need a standard raw file format. Canon, Nikon, Fuji, Sigma, etc all have their own raw formats. This makes developing good raw-file handling software difficult, because you either have to dump lots of time into supporting several file formats, or settle for a small piece of the market and only support certain brands. It's also a royal pain in the ass for media companies who, to maintain some order in their workflow, "standardize" (fancy word for "get locked into") on one camera system maker.

Adobe is the defacto tool for processing digital images; nothing comes close. Knoll and his team have, after several years, picked up quite a bit of experience with what works and what doesn't; what customers need and what they don't, etc. Adobe's status puts them in the position to push a common raw format, and it's likely many of the companies that make decoders will add it in; it will be a case of software support before hardware support no doubt- but eventually camera makers will grumble a little and add it in. They've long since given up trying to make money off their raw format decoders.

Most media companies will no doubt be thrilled, because now they can handle Joe Shmoe's D4X raw file just like they handle Bob Smith's 1Ds Mark 3 raw file, save maybe for some image size differences.

By the way- RAW = Canon, NRF(I think?) = Nikon. Confusing that the style of file is called "raw" but Canon has a format called RAW. Please use capitalization to distinguish between the Canon format and the general style of compressed image.

Why this is a good thing

[Anonymous Coward]

Since many people here are not photographers and don't deal with RAW formats, let me but it in simple terms.

Each camera has there own RAW format. Read each Manufacturer has their own proprietary format. Some even have multiple formats. For example, Nikon uses .NEF files. Canon uses .CRW and .CR2.

Photographers work with RAW because it is lossless and can be recorded with 16 or 12 bits of data per pixel, where JPEG and TIFFS tend to be 8 bits per pixel. Also, as mentioned already, settings such as white balance, tone, sharpness, color, and even exposure compensation can be applied after the shot was taken.

BTW: Post-Processing is a HUGE part of Digital SLR photography for those that are only used to the Point and Shoot cameras.

Now for why it is a good thing to have a unified RAW format. I recently purchased the Canon 20D. It included a new .CR2 format. However, none of my existing programs could work with it, even though it is similar to the .CR2 format found in the Canon 1D Mark II. There was a hack for Photoshop CS that worked, but the "As Shot" white balance was wrong. Adobe released the new Camera Raw plug-in today and it works good.

With each new camera release, all software writers will have to update their program if they want to support the new cameras. At the rate at which DSLR's seem to be announced this could be a huge pain. If a company like Adobe could convince the market the their DNG file is the way to go, your software would only have to work with that format.

Re:Why?

[Zocalo]

Did anyone actually see a good reason for the creation of this particular format?

Hell yes. Unlike with a raster image such as JPEG or PNG, the data from a camera sensor is most likely a Bayer array - alternating lines of Red/Green and Blue/Green sensors, rather than RGB triplets, so it's not so much RGB, as RGBG. (There are some varients/exceptions in the in sensors from Foveon, Fuji and Sony). There is also a lot of data specific to the exposure; duration, ISO, lens details, etc. which would need to be applied in camera before a raster image could be produced. With RAW, you can apply these settings after the event in Photoshop or whatever. Exposed the sky correctly, but got the ground off by a stop? No problem; "develop" the RAW twice and use the sky from one shot and the ground from the other for a much better result than "enhancing" the ground in an image editor.

Yes, you could have most of this with a tweaked version of PNG and a bunch of ID3 type tags (and maybe that's exactly what Adobe has done, I haven't looked at the file format yet). The main benefit though is to make it very easy for data exchange and solve the nightmare situation whereby each new sensor has it's own RAW format. The state of play at the moment is a nightmare for vendors like Adobe who need to update their software for almost every new high-end camera release. Likewise for the makers of those "digital photo stations" that are cropping up like Starbucks, or their little brethren; the printers you can plug a camera into directly. With a standard like DNG to support you gain the much larger colour gamut of the RAW format and more flexibility in tweaking the image for a better print.

Anyway, you can read the actual Adobe press release, or download a free (beer) DNG converter here to find out a little more.

Re:Anyone who uses JPEG...

[sweede]

I would say that upwards of 90% of the magazines, books, and other printed materials use JPEG images in their books for ALL non-photographic images.

photographs are usually TIFF documents, but we've ran accross many customers who supply all of their graphics and photos as high-res JPEGs

Adobe lockin v.s. Quark? you obviously don't work in a printing house. Quark is basicly the only tool used in creating anything for print. Quark can create Adobe PDF Files without issues from any other software we use.

Now that i think about it, most of the software we use is based on Open standards from Adobe (Postscript, DSC(Document Structuring Convention), PDF, PJTF, JDF)

YIDIWIP (Yes I Do Work In Prepress)

Re:Why?

[Binary Boy]

Yeah, well, color me shocked. People on Slashdot jumping the gun? Spouting off on issues they haven't researched? Unheard of.

Re:Why?

[metamatic]

PNG is also a container format. You can add whatever data chunks you like to it. They could easily have defined a raw data chunk for the PNG format.

Re:Why?

[eddy]

There are actual lossless implementations outside of the research community?

I thought that died The Death Of A Thousand Software Patents :-\

Re:Why not lossless JPEG/JPEG2000 then?

[fyonn]

well, jpeg is a processed image, what this format is designed to address is unifying every manufacturers raw image dumps. most medium to high end camera's offer a raw modew which gives you the data directly off the CCD or cmos sensor without any processing applied. this means that you can apply and vary things like white balance settings after the fact rather than getting it right when you take the shot.

the RAW images basically give you direct data off the sensor and all the data about the cameras current settings, ie shutter speed, apeture, meter settings, focus settings etcetc.

having a single format would allow all the image processing apps to only have to load up the DNG file, rather thn having plugin's for nikon's raw file, sony's raw file, canon's raw file etc. right now if you don't have a plugin then you have to process the image through the manufacturers image application which can be a pain and seems unnecessary. also what if you have another platform, are all the raw input filters available for a mac for example? for linux? with a standardised free format then all the apps will have access and no companies will have to pay licence fees to load in the data (which soemthing like the gimp isn't going to do).

dave

Re:Why?

[Tenebrious1]

professionals don't want to use JPEG because lossless JPEG is inefficient, not because it doesn't exist.

Photoshop and PSP allow you to save files as lossless JPEG, but the only camera I know is the newest Canon digital SLRs. Most other professional digital SLRs and prosumer cameras only store proprietary RAW, then some level of lossy JPEG.

Less why, more how

[Builder]

I see a lot of posts asking why bother. What I'm more concerned about is how this could work ?

Ignoring the differences in the various RAW formats between manufacturers, what about differences between two cameras from the same manufacturer ? What causes that and would DNG cater for it ?

As an example, look at the Nikon D70 and the D100. Adobe had full support for the D100 with their ACR (Adobe Camera Raw) plugin. But when the D70 came out, we waited months for true compatibility with Photoshop (yeah, yeah, you could use the nikon supplied plugin, but that was worthless really).

I'm guessing Adobe want this because in order to keep selling Photoshop to photographers, they have to keep amending ACR everytime a new camera comes out. But can a fixed standard cater for everything that Canon and Nikon will be putting in their cameras, and want to store in the RAW files 2 years from now ?

Oh great, now we have to deal with new terminology

[Klowner]

So they're gonna be pronounced as "Ding" I'm assuming, consider the possibilities:

Dude2: "Hey Dude, I have a funny picture to send you"
Dude1: "Dude, wait, I have dialup and it's gonna take forever"
Dude2: "No way dude, it's a ding, it'll only take a moment, here it comes"
Dude1: "Okay, got it .. Dude, is that a ding of my dong? HTF DID YOU GET THAT?"

... I shall stop here

Key benefits from DNG primer

[SuperKendall]

Adobe has posted a DNG Primer online, describing some slightly technical details. Here are the key points from the document that helps to understand what makes the format useful:

Image format: DNG is based on the TIFF-EP format, but DNG specifies the inclusion of a number of additional tags that let the converter properly interpret the raw file.
Metadata: DNG enables inclusion of metadata in EXIF, IPTC, and XMP formats.
Compression: Files can be stored as uncompressed (either bit-packed or padded to 16-bits per pixel) or with lossless JPEG compression.
Color space: DNG fles are stored in a linear, nonwhite-balanced color space (usually the native color space of the camera).
Interpolation: DNG enables file storage either in mosaic (CFA) form or in demosaiced form. Generally, a mosaiced file is preferred because it represents the original data the sensor captured and enables maximum conversion fexibility. It is also smaller than a demosaiced file. In some instances, however, saving a demosaiced file can improve compatibility, particularly if the camera sensor contains an unusual mosaic pattern that all converters do not support.

Re:Why?

[akpoff]

As others have mentioned this is about creating one RAW data format. The RAW data from the CCD or CMOS sensor is basically unprocessed information unlike PNG, JPEG and basic TIFF.

When you shoot in JPEG and even in TIFF with any current camera the internal processor applies white balance, color, contrast, sharpening and other algorithms to the data and saves it in the chosen format, typically JPEG at anywhere from a 1:2 to 1:8 loss compression ratio.

This is great for point, click, print. But for hobbyist and professional photographers this eliminates a lot of the post-shot production that can be done to the image later, even if it's a TIFF file. With a RAW image format you can apply those effects afterward, unapply them and generally tweak the output. My Nikon came with some software that allows me to set the white balance using the same algorithm the camera would do internally if I shot JPEG. But with the in-camera option if I don't like the results, tough. Sure, I can tweak it later in Photoshop but I can't fundamentally alter the effect. This is particularly frustrating for effects like white balance and sharpening. Having the RAW format, also known as the "digital negative", gives the photographer much more flexibility.

Why do we need a universal file format? Because every camera manufacturer out there has a different one that programmers have to figure out and photographers have to put up with (not that professionals change kit all that often). With Photoshop CS Adobe helped photographers out by putting in an excellent RAW reader but someone at a Adobe has to keep up with all these specifications. If all the camera manufacturers would adopt ONE format then Adobe AND open source developers could focus their time on the important business of digital photography -- producing excellent final photos.

Another benefit of the RAW format hinted above is that for programs like the GIMP developers waste a lot of time trying to reverse engineer these formats. As you can imagine a lot of the camera manufacturers aren't out there sharing the love with their specifications. The one format that's been best reverse engineered is the Canon RAW format with varying levels of success going down from there for Nikon, Minolta et al.

Lastly, let's not forget the benefit of having a common format for long-term archival and retrieval. Anybody out there have any binary files they can't get into any more? A common RAW format will reduce the chances of being orphaned with files you can't read.

In summary, then, this is a huge benefit for everyone as we can all focus on digital image production using a shared format and libraries and less on figuring all these formats out. The only question the article doesn't address is whether Adobe are releasing the format into the wild, as it were, or plan to setup a "toll" on it via patents.

Adobe technology is well thought out

[tezza]

I've seen a lot of people ranting against Adobe. I deal with Adobe tech all day, postscript, pdf, fonts. They had a big hand in the SVG spec.

In my opinion, working with the bare bones of their technology, ALL of it is well thought out, comprehensive and well explained.

They consider all of the difficulties of the problem domain. For instance, see how easy it is in PDF to create changes to an existing document, great for low powered CPUs. Just append the changed object and add on a new footer to the file. 95% of the file retained, which is a lot less expensive than re-generation of the whole file.

I think Adobe will do a good job here and post the specifications ala PDF and Postscript.

Not mentioned in the other comments is the run time hardware cost of saving this Digital Negative. I think Adobe will put effort into making this as friendly to integrated hardware capture as possible. A large portion of this has to be very little re-ordering of data as it comes from the CCD, as these usually require an in memory buffer. This fundamentally changes the nature of the format.

Re:Why?

[Atzanteol]

PNG, being an indexed-color format, is not the end-all of graphics formats

No, it is not. It *can* do that, but PNG supports full 24-bit color, with 8 bit alpha. And possibly higher color depths if necessay (there's a byte or so in the header for bits-per-color).

Finally Raw interoperability.

[guidryp]

This is a major boon. For those too lazy to read. It is 100% free and open for anyone to implement. Adobe is also providing a free converter.

This provides a common RAW format for cameras. This is not a png or jpg replacement, but a RAW replacement.

There are a number of third party RAW converters on the market right now. Many have limited camera support. You can bet they will quickly moving to support DNG. Which will instantly open up their usage to almost all current cameras.

With DNG support and Adobes converter you will soon be able to open just about any RAW image with any converter.

Even without camera output this is a benefit. As you can get one converter to support all your cameras.

You can archive all your RAWs as DNG and not have to worry that you kept all the software that came with the camera that generated the original RAW.

Camera support would be even better, but that may be slow as the manufactures may suffer "Not Invented Here" syndrome, or see value adds to their own format quirks.

Parent is uninformed

[Archimonde]

Who modded this up?

This format is about putting all RAW files under one (DNG) format.

Eg. Nikon has NEF, Canon has CRW, Olympus has xxx, adsf has yyy....

Isn't it better to have one open/standard format which all manufacturers support/endorse?

If you are skeptical read this.

Re:If I see one more "what's wrong with PNG" post.

[soulsteal]

Fuck, I like open source and masturbate every time I see a linux login prompt

That can be very embarrassing in a lab environment. :|

Re:Nope, just have to get photographers using it

[kzinti]

Adobe is very smart about this. It's based on TIFF, so it's a pretty easy to read standard as there are a lot of libraries to read TIFF...

TIFF is also a format that most camera makers are already comfortable with. The files that come out of most cameras conform to the Exif spec. Exif is the JFIF (JPEG) file format with metadata embedded using TIFF tags. It will be interesting to see (I haven't read the specs yet) whether Adobe wants to keep any or all of the Exif tags, or wants to ignore them and invent their own. It would be smart of Adobe to try to design something that can live happily with what users and manufacturers are already familiar with

Re:Why the need to diss Adobe?

[Anonymous Coward]

Because the raw format is the data straight off the CCD plus the camera settings.

Those two together let you play with the image composition before its set into any format. Don't like the exposure? Check what the exposure setting was, then recalculate the pixels based on the original source data. Bad aliasing effect? Try again from the raw data at a different resolution or different interpolation between the CCD sensors and pixels and see if you can save the picture.

Rant much?

[Anonymous Coward]

In your patronizing rant you forgot to explain what is wrong with advocating PNG over extending TIFF. I don't see why you feel so "holier than thou" about this, given your apparent inability to argue your case.

PNG can handle anything that TIFF can. I checked, you could make a color format with >4 256-bit elements if you wanted to. (that'd be a "1024-bit format"). There'd be no problem at all adding whatever model and optional compression scheme Adobe wishes to use.

When you explain, feel free to get as technical about it as you want.

Re:Rant much?

[MustardMan]

In their idiotic kneejerks, 99% of the posters forgot to explain why PNG would be a better choice. Just because it's PNG doesn't automatically make it suited for everything.

It's quite simple why TIFF is a better choice for adobe: they own it and are familiar with it.

Your troll tried to use the "technical" argument to throw off the point of my post. My post wasn't about technical merit, it was about typical slashdot groupthink. You don't need a technical argument, you need an argument of simple economics. Adobe owns TIFF, therefore they used it. Simple as that. I nowhere said TIFF was technically superior to PNG, nor will I now. Neither will I argue the technical merits of each. There's a much simpler reason to use TIFF.

NAME RECOGNITION. When you go to a website, and they post their "publication-quality" pictures, almost universally those pictures will be posted as TIFFs, and not PNGs. By basing your raw data standard on TIFF, you throw out a term that is familiar to publishers and photographers alike, neither of whom really care about the nitty-gritty details of how the data format works. Familiarity breeds a warm fuzzy feeling and will make adoption of this format more likely.

There's an obvious reason to advocate PNG over GIF, and that's the ridiculous issues that we all know about. However, there's very little reason to reccomend PNG over TIFF for the pure sake of "BUT ITS PNG AND PNG IS 31337 LOL LMFAO U FAG" which is pretty much the gist of the /. arguments.

Supports EXIF and more

[SuperKendall]

The spec not only allows the metadata to be held in EXIF, but two other formats as well - IPIC and one other I forget.

this is a sensor data format specification, really

[coult]

We really shouldn't even call it an image format. Most people think of image formats as a way to compress and store image data for viewing or printing; things like JPEG or GIF or PNG.

DNG is a format for storing the data recorded by the CCD's in a digital camera. This data can of course be processed and displayed as an image, but DNG really isn't an image format exactly.

DNG is TIFF -- Dont Panic !

[Sigfried]

If you actually go to adobe's website and RTFM,you will see that Adobe did the Right Thing (TM):

A DNG-format file is fully compliant with the TIFF 6.0 Specification Standard and the ISO TIFF-EP codification of that spec, which was designed from day one as a fully extensible raw, processed, or whatever image / metadata annotation spec.

BTW, TIFF was originally designed for offset printing folks, and in the 6.0 standard already supports a huge number of colorspace models besides RGB, and has an extensible mechanism for specifying color-data encoding and compression schemes (you can even store JPEG encoding in TIFF).

When I worked at the ground-data processing section of the Jet Propulsion labs, TIFF was occasionally used to store and transmit raw multispectral satellite data, which consisted of over 256 separate color-spectra bands from far infrared to ultraviolet, stored spatially in separate tiles.

Working together with Spot Image and other satellite providers, NASA also helped develop the GeoTIFF extension to TIFF, which annotates an image with exact georeferencing information.

It looks like Adobe went the route of using SubIFD's to define the extended data. A little bit unfortunate, since that data will not show up in a "tiffdump" listing of the file, but in any case I have no doubt that folks are already taking the spec and writing "libtiff" extensions to parse the stuff.

For more information on TIFF, see my old, clunky website that is chock full of invalid links,but still has a few useful things to say:

http://home.earthlink.net/~ritter/tiff

--Niles (original GeoTIFF and TIFF webpage author)

Re:Why?

[TRACK-YOUR-POSITION]

So are the instructions for how to do this processing embedded within the DNG format, or do programs viewing and manipulating DNG files have to be constantly updated for every new camera release, in order to properly process additional metadata?

Or, to paraphrase, is it possible for camera manufacturers to produce "standard" DNG files that aren't actually viewable on anything other than that camera's included software without reverse engineering proprietary metadata?

Re:What has not been said

[halivar]

Adobe does not have the support of any major camera vendor yet, that's where this really counts

Maybe that's because it's brand new and it's just been released.

Adobe in it's traditional greedyness did not release this as Open Source, they released a spec but not code. They just don;t get how to work with the open source community.

The way to work with the Open Source community to to release non-proprietary specifications for which we can write code ourselves. SO FAR they've done a pretty good job with PDF and DNG. What? Do you think no one is going to write an Open Source library for it?

Sheesh, talk about greedy; they could've colluded with camera manufacturers to monopolize on a closed, proprietary format, and you complain because they didn't give you the frickin' code? Get off your rear-end and write it yourself. That's the Open Source way.

PS: Adobe sucks for entirely different reasons (e.g. Dmitry Skylarov). This, however, is not one of those reasons. Adobe did right this time.

Insert AOL! here.

[Glytch]

Amen to that. At the risk of sounding like an elitist asshole, it's obvious that a good many folks on Slashdot don't know much about photography, and think that just because they bought Sony's newest fucking Cybershot that they're the next Helmut Newton.

As for myself, I've been eagerly waiting for an influential company to propose something like this; I work in a pro lab, and having to master and keep up to date on a dozen different raw converters is very stressful. A single standardized open format that I can use right inside Photoshop (at work) or the Gimp (at home) is like the holy grail to me!

It would be interesting if this could somehow be adapted to 35mm or medium format negative scanners, too. Being able to do big corrections after the scan would save me a hell of a lot of time.

Re:Irony of life, DNG can be lossy too

[JimDabell]

So in other words, the Slashdot writeup that stated this was a new format that was better than JPEG was completely incorrect, and in actual fact this is simply a container format that uses existing JPEG algorithms? Sounds about usual for Slashdot these days.

So where is the (more) efficient method?

According to the JPEG FAQ, PNG is more efficient than lossless JPEG for most images. Unfortunately, this specification doesn't allow for that; as far as I can tell this has little to do with picture quality and more to do with metadata and interoperability.

RTFA

[p3d0]

This is an instance of the TIFF-EP format. It specifies things like values for certain tags, byte ordering, etc. It's not a completely new format.

Re:Not right now, keep the gnus of war at bay for

[Wanderer2]

Dogs or Gnus, people?

Guns. Lots of gnus.

Re:Why?

[lucifuge31337]

Based on being a user of several pieces of raw image processing software, I can tell you that choosing event eh wrong camera type (even if its int he correct family) produces crappy results. Also, different raw post-processing engines give different results (some better, some worse, some depending on the raw file). Based on those observations, I'm going to have to guess that any raw processing engine is going to need to know quite a bit about the camera in order to actually be any good.....so even if the file format is standard, I'd say that you'll still need some data about the camera the file came from. And probably a whole lot more than what woudl be reasonable to put in the file itself.

Re:Why?

[afidel]

Pro's don't like to use JPEG because JPEG doesn't allow the level of post processing that RAW does. Hell Canon RAW's include a normal quality JPEG as part of the RAW image for faster previews.

And my whole point was...

[SuperKendall]

They are using TIFF-EP as a base - a thing they didn't even develop. So how would they have a submarine patent on anything here? If there were a submarine patent related to TIFF we would have seen that long ago. Since it's really TIFF and metadata, there simply is nothing to torpedo with the submarine.

This is just a graphic file format like any other, using bog-standard compression and metadata standards. It's just a more standard bag for raw camera data than what we've seen before.

Re:And my whole point was...

[arose]

Don't underestimate the ability to patent things you haven't invented. :-D

Calibration images for astronomical photos

[ChrisDolan]

In astronomical work, there are usually two calibration images you use: the dark frame and the flat. The dark frame is an image captured with the shutter closed. It lets you identify the hot (i.e. broken) pixels. The flat is an image of a uniform field exposed just long enough not to saturate any pixels. This lets you measure the relative light sensitivity of the pixels (which is a function of both the lens and the CCD).

To get a corrected image, use this formula for each pixel:
newimage = (image - dark) / (flat - dark)

Better yet, take a bunch of darks and flats and median-filter them to get rid of cosmic rays which can introduce spurious glitches in the images.

"Raw" scanner files

[flybrarian]

Actually, there's a scanner program called Vuescan that will let you do something like this. It's not an open format, but the program will allow you to save the raw output from the scanner to a file and then manipulate it after the fact.

It's not bad, but...

[Thagg]

I've looked over most of the information Adobe has published, and it's not bad. It's true that a typical RAW format file is difficult to interpret. I've reverse-engineered a couple of RAW formats just for fun, (it's pretty easy if you can tell the camera to output a RAW and TIFF image of the same shot) and the Adobe propsed DNG format does have flags for most of the issues that I've come across (I have to say that there were some new ones for me, too -- the flag that specifies how closely the G in the RGRG rows compares to the G in the GBGB rows is something I've never even thought of.) It's good that Adobe has considered the possibility of more-than-three-channel cameras.

But -- I think that digital cameras are still *way* too new for this kind of standardization. Significant true innovation is happening at a frenetic pace, and if we limit RAW formats to a preconceived format we may inadvertantly (or advertantly, I suppose) squelch that innovation. Fuji's spectacular sensor with separate sensors at each pixel for dark and bright values is an example -- how would that be encoded here? One might well have a camera with vertical and horizontal polarizers on every other cell, to allow post-processessing to reduce or enhance specular highlights. Cameras could be built with psuedorandom placement of cells, to eliminate aliasing artifacts (Why not? It's not as if the semiconductor masks are laid down by hand anymore.)

In short, I think that this format could end up being a Procrustean bed that we force camera makers into, and that it's not worth it at this point.

Thad

Re:Why the need to diss Adobe?

[HughsOnFirst]

" I read the article and I still don't see the point. How is a "raw" file format any better than any other lossless image format (like PNG or TIFF)?"

Well, a raw file is a lot more compact, and it does not do anything to change the artifacts and or "lossyness" of the sensor itself.
For instance the Canon cameras use a Bayer sensor. Imagine a checkerboard where half the sensors, are green , arranged in a checkerboard pattern, and the other half the sensors alternate between red and blue. The 12 bit value of each sensor site is encoded as the eight bit difference between it and the nearest same color site to its left. ( Amiga developers will find this vaguely familiar) The theoretical case of there being a full 12 bit difference between 2 sites never happens due to the optical properties of the camera and lens.

Re:Nope, just have to get photographers using it

[kzinti]

A lot of programs can read it. I think the latest versions of PHP have Exif support built in. The latest GIMP might have Exif support. I wrote my own software that was in Sourceforge as Exif-tools, but I no longer maintain it and I believe the project has been removed. If you're looking for a command-line tool, jhead might read Exif tags. Even Windows XP can read them.

The most useful tag is DateTimeOriginal, which tells you when the photo was taken. Some cameras also record info like shutter speed, flash setting, aperture, focus distance, lighting conditions, and other interesting info.

Re:Nope, just have to get photographers using it

[rgmoore]

If you're looking for a command-line tool, jhead might read Exif tags.

Jhead does read EXIF tags. This can be very useful if you want to analyze something about your photo usage. As an example, I've been considering getting a fast prime lens for available light photography in a range that I currently have covered by a slower zoom lens, but I wasn't sure whether to get 20 mm, 24 mm, or 28 mm. I used jhead to extract the focal length for every picture I've taken and found that I use 24 mm a lot more than either 20 mm or 28 mm. Now I know which lens to buy.