A RAW repository, The Internet Archive and OpenRAW

Stan writes "I just read this in the OpenRAW mailing list, OpenRAW plans to create a RAW repository, a final resting place for RAW file documentations of current and already abandoned digital cameras. The RAW repository will be hosted in the Internet Archive, which describes themselves as a digital archive of the Internet and other cultural artifacts. And they have all reasons to support OpenRAW, they currently photograph billions of book pages with cameras and store them in RAW format. Unfortunately the camera makers think different (which is not always a good thing)."

Comments

Digital == Loss of freedom

[Dancin_Santa]

I'm not going to say I told you so, but I told you so. The minute you give up the physical artifact and rely on a digital representation of your data, you are at the risk of any company who wishes to exert some control over the format of that data. That's why all those RAW file formats for each camera are different from company to company. They gain the most benefit by locking you into a certain piece of software and forcing you along their upgrade path.

If you stick with film, you are only limited in your ability to develop your own negatives. If you can do this, you will be able to continue with film for as long as you want. Scan the negs and save them in whatever format you want. It doesn't matter because the actual physical artifact is still in your possession.

Not so with Digital.

In many ways, digital is superior to film. However, when it comes to ownership of your data, you are far better off with film than you ever can be with digital.

Re:Digital == Loss of freedom

[phidipides]

They gain the most benefit by locking you into a certain piece of software and forcing you along their upgrade path.

Just to nitpick a bit, most of the professional photographers I know use the various plugins to Photoshop to work with RAW images, so technically they aren't being forced into an upgrade path by the camera manufacturers. I personally use a Canon 10D, and the Canon software is so awful that I always use other tools to convert and manipulate the images.

I fully agree with your point that it would be better if the camera manufacturers fully opened up their file formats, and I fail to see how keeping them closed provides them an actual competetive advantage. However, so long as there is no constraint against converting RAW images to another lossless format I'm not sure that this is a battle in which the camera companies can be accused of trying to pull a fast one on consumers; I think it's merely a case where they need to be educated about the further benefits of opening up their formats (ie open source developers can build free tools, etc).

Re:Digital == Loss of freedom

[Hast]

Oh you mean you'd rather use propriatary film with propriatary developer chemicals? There is nothing particularly "open" about that besides that the chemicals are available to buy in most photography stores. If they go out you will no longer be able to develop your old exposed film.

Just to be clear, RAW is like the undeveloped exposed negative. After "developing" it to a TIFF16 or whatever format you want to have. You might think that RAW is equivalent to the undeveloped negative, but it really isn't.

Besides, there is always DCRAW which allows you to "develop" your RAW files in an OSS fasion.

Furthermore the reason RAW formats vary between makers is because it is raw data from the CCD/CMOS. So it's not strange at all that different manufacturers use different formats.

I do agree with you though that we need open standards as far as RAW is concerned. I don't agree that the film world is any better though.

Re:Digital == Loss of freedom

[smithberry]

There is always some risk in machine readable data.
In 20 or 30 years time you might have trouble getting hardware which reads your data and runs an OS which runs your software.
Ok, you might be clued up enough to always copy backups to newer technology, but joe public is one day going to bring a CD out of his dad's attic and find he cannot even look at the photos on it.

Re:Digital == Loss of freedom

[Phanatic1a]

There is always some risk in machine readable data.

There is always some risk, period. If I keep stacks of negatives in boxes in my house, then in 20 or 30 years time my house might catch fire and burn to the ground.

Okay, *I* might be clued up enough to always keep my negatives in a fireproof safe, but Joe Public is one day going to use a penny as a fuse replacement and find that he can't look at the photos on all that celluloid ash.

Re:Digital == Loss of freedom

[eggz128]

On the other hand, in the analogue world, in 20-30 years Joe Public pulls his prints and films down from the attic and finds they've been damaged due to damp etc.

There's always some risk no matter what you do. Call it the 'shit happens' principle.

Re:Digital == Loss of freedom

[Eivind]

That is a risk you can't fix anyway, I mean even if you where using a film-based camera the authorities could still take your pictures and negatives away by force.

If anything digital makes it easier to guard against that, because it makes it trivial to ensure mutliple, backups. If you're *really* paranoid you make you've got atleast 5 backups in 5 different jurisdictions.

Re:Digital == Loss of freedom

[theAtomicFireball]

What bullshit. The lengths that some people go to to "defend" film. If you like film, use it, but don't come up with these lame arguments, just say you like it and be done with it. I've NEVER lost a digital file since I went digital back in 1998. I keep my photo library on a computer and do periodic backups to DVD, which I store offsite. In my years of using film, I did lose negatives (only one copy of them, remember?) had many negatives that could only produce poor quality images because they got scratched, faded, creased, etc., or required a lot of work to get a good print. At many times, I didn't have room to set up my darkroom (and most people don't have one), so I had the negatives, but no way to make copies except to pay someone else - making me dependent on someone else, which is far worse IMHO than being dependent on a file format you don't own. I only periodically shoot in RAW, but I have an "official" converter that will turn it into any number of open formats, plus Photoshop supports the RAW format. If in some weird stroke of luck, both companies revoked my licenses to use that software, I'd convert them all to an open format. Otherwise, my images are JPEGs or TIFFs. I can do anything I want with them and there's absolutely no doubt that I own them. I can make unlimited copies with no image loss without a darkroom, make backup copies to store elsewhere, and print copies from pictures taken years ago at exactly the quality they were when I took them. How in the name of Dog is film "better" in terms of image ownership? And how did this lame argument get a +5 insightful?

RAW format

[James_Duncan8181]

The RAW format wars seem an odd competiton between camera manufacturers, who are actively hurting their presence in the professional space by making their imafes less useful for archive purposes and less interoperable for press agencies to sell. The thing that is particually noticible is that manufacturers are now being actively co-opted into sharing this information under NDA with MS to allow the hardware to work seamlessly with Longhorn. This mass move from open to propriatory standards (something MS will, of course, encourage) is meaning that the camera manufacturers are seeing their poduct become commoditised, and apparently feeling unable to compete on hardware quality alone.

Thanks Canon, you just made me finally feel confident about buying Taiwanese.

Re:RAW format

[Uruk]

I think though that most people buy cameras for the functionality of the camera, rather than the image file format. Besides, a lot of the cameras that have strange formats come with ready-made software to convert that format into something reasonable, (even if such software doesn't run on minority OS's)

They aren't hurting their presence in the professional space - those folks still are going to buy the camera for its camera features. That provides an opportunity to sneak in other stuff that the camera company thinks will distinguish it from other companies, including different file formats with extensions to do particular things.

is meaning that the camera manufacturers are seeing their poduct become commoditised, and apparently feeling unable to compete on hardware quality alone

If what you say there is true, you've identified the reason for the strange formats and differences between cameras; they are trying to differentiate themselves.

Re:RAW format

[DigicamGuy]

The thing that is particually noticible is that manufacturers are now being actively co-opted into sharing this information under NDA with MS to allow the hardware to work seamlessly with Longhorn.

Actually, that's incorrect. The Longhorn interface is binary-only (no source code or format information is communicated to Microsoft or to the OS). Basically, the manufacturer (or third-party developer) writes a driver with an API that makes processed RGB data available to the OS. This is the same basic mode of operation as Canon and Nikon (and probably others) have implemented already in their free SDKs. Here's a brief interview with a Microsoft exec about the Longhorn interface and the shortly forthcoming "powertoy" RAW thumbnailer/viewer that's coming for XP. -- Not likely the level of detail /. people would want, but more than I've seen elsewhere, may help dispel some of the misconceptions.

Of course, this means that the proprietary RAW formats remain entirely proprietary in the Longhorn era.

For the record, I personally think that some level of open documentation of RAW formats makes a whole lot more sense than trying to come up with a common standard. A number of people (Adobe prominent among them, of course) have proposed Adobe DNG as a "universal" format. This sounds like a wonderful idea until you look at the assumptions underlying the format: It assumes a rectilinear pixel array, with a Bayer color filter array pattern (a checkerboard of RGB color filters on the pixels, with twice as many green pixels as red or blue). This is indeed the format used by the majority of cameras out there, but it completely misses innovations such as Foveon's full-RGB-in-every-pixel sensor, Fuji's hexagonal-pixel/diagonal-array "SuperCCD", and Fuji's latest "SR" sensors, which combine low- and high-sensitivity sensors in each pixel.

While a "universal" RAW format would help with the issue of access to the underlying data, so would simple documentation of the structure of various proprietary RAW formats, and the latter wouldn't have the negative effect of stifling innovation in sensor technology.

What horseshit

[loraksus]

The manufacturers are just opposed to working together to create some sort of standard.

But can you blame them? Really, think about this for a second - people (scumbag fucks who should hang from lampposts, call them what you will) from Rambus sat in standards groups for years and then turned around and secretly patented the standard and then had the balls to demand royalties. You saw more or less the same bullshit with .gif and in hundreds of other similar cases in the last 20 or so years.
I think it is (sort of) understandable that companies would be hesistant to work together to develop a standard way of doing something - especially in a cutthroat business such as photography.

And by the way, using Canon is a fairly shitty example, Nikon is far worse when it comes to the RAW format (ok, its not really a format) bullshit that flows through the world of pro photography.

That all said, this smacks more of the petty bickering that is involved in cameras more than than anything else (See Also, "Complete lack of lens interchangability" et al), but as always, we (or those who buy $600+ cameras) get fucked.

Don't get me started on how "using the DMCA to "protect" the super complex almost but not quite encrypted raw format". I don't need a stroke at this age. . .

Need for a broader approach?

[archeopterix]

Hm... Everytime I read something similar to the article (that is about twice a week), I think that there should be a non-profit organisation to oppose taking away customer rights under the guise of "intellectual property".

Something powerful enough to organise boycott that would cause *pain* to the offending company. Something that a congresscritter would be afraid to piss off. EFF comes close, except that it a) has a broader scope and b) sadly is not powerful enough.

Too bad that the existing consumer organisations are focused on making money from their "consumer reports" and the general population doesn't care (the frog is half-boiled and still comfortable).

Re:why?

[Anonymous Coward]

why does the article link directly to some sort of blog?

"Blogs" are the new informercial. Take something said somewhere else, post it on "your blog". Get a bunch of other accounts and link to it, and presto lots of page hits for your ads.

Then of course, you can get it posted to slashdot (for free, I'm sure), and rake in the Rolandbucks.

I wish google had a "blogs:no" option when searching.

its not so much digital thats the issue...

[petermgreen]

they could just use an unecumbered format like png

BUT the advantage or "raw" is its the closest you can get to what actually came out of the cameras CCD. because of the way CCDs work this will be about a third the size of the resulting image (assuming they are uncompressed or compressed using a lossless algorithm that gets roughtly the same compression on both).

Re:storing raw digital images is stupid

[jgordon7]

Most raw formats on modern cameras is compressed using a lossless format. Take Nikons nef format, a 6.1mp image in raw is only about 4-5 MB, if this was not compressed it would be closer to 10 MB.

You can not tweak certain settings as easily once the image is convert to another format, even a lossless one like tiff. Best example is white balance.

There is no real alternative..

[shaanX]

when it comes to doing any sort of real editing JPG is *NOT* an option. I see people here that are obviously not photographers or have not used a digital slr camera saying just to use JPG. If you plan on doing little to no post processing sure use JPG but for true manipulation RAW is essential