Slashdot Mirror
Reuters Bans RAW Photo Format (petapixel.com)
grcumb writes: Reuters is the latest agency to join the ranks of the technically clueless who think that ethical problems can be solved using technical means. They recently issued a circular to their contributors, stating in part: "In future, please don't send photos to Reuters that were processed from RAW or CR2 files. If you want to shoot raw images that's fine, just take JPEGs at the same time. Only send us the photos that were originally JPEGs, with minimal processing...." The problem they claim to be addressing is doctored images, but they don't explain how they plan to ensure that the JPEGs weren't simply exported from RAW files with their EXIF data altered, or heck, just altered as JPEG. They also assert that getting JPEG files straight from the camera is quicker, which is fair enough. Lots of professionals shoot with RAW+JPEG at newsworthy events. They can send the JPEGs off quickly to meet the first deadline, then process the RAW files at leisure for higher quality publications.
A lot of executive decisions boil down to demands to solve a problem (e.g. photos may be doctored) and an executive deciding he has to "do something", else when it does blow up he did NOT do "something". For example, if an unknown terrorist might strike, it doesn't matter whether the action (ban refugees at a state level) actually matters, it's insurance that when something did happen that you demonstrated precaution. CYA
Gently reply
As a former photojournalist, I can saw that you simply blacklist them and/or fire them from being a contributor/stringer/staffer at that image bureau. There are ethical standards in the professional photography world, and it is nothing bad to those of us who upheld our high ethical standards to see someone get fired for unethically altering images and cheating and breaking the rules. I doubt this is as much a problem from a "who altered their photos?" problem as it is the photographers are submitting larger files (even if lossy down converted into JPG from RAW) and Reuters is having problems handling so many large files in their infrastructure and pushing photos out in distribution to their newsroom client "on the wire" servers. I know in my past when dealing with AP, if you uploaded a file that was too large they either rejected it, or WORSE, applied their lossy compression using whatever software they saw fit. When what your image looks like is everything to a shooter, and when a perfect images is ruined by crap third party compression due to file size, the lesson is hard learned and PJ folks are pretty savvy getting the best bang per MB.
Care to enlighten me as to how one sets jpeg compression to 0%? Also, no, RAW formats are not simply uncompressed, but largely unprocessed data as well (certainly less processed than what you get from an out of camera tif or jpf.)
The main reason to shoot in raw is to take advantage of the larger sensor dynamic range so you can adjust exposure and color balance without harming image quality. I don't see every photojournalist suddenly asking people to hold still while they adjust their camera.
Bet you anything they've a managed workflow system and their solution can't deal with raw files.
Not surprising since each camera and sensor can have a different format that needs to be accounted for. Supporting raw natively would mean having to effectively deal with hundreds of different image formats, with new ones introduced with each camera model. They probably don't want to pay to support that.
One method of identifying modified images is to glean information from how the jpeg encoder constructed the file. It leaves a sort of fingerprint and it gets modified if an image is modified or re-encoded.
Of course a RAW file is a dump from an image sensor and that is NOT the same thing as an uncompressed planar bitmap as many assume. It's more of a signal and contains much information that is thrown away when creating something a human eye can intemperate (Processing). In fact, I think it would be harder to doctor a RAW format because all image sensors have random imperfections, their own physical "fingerprint" that can be traced back to a specific camera. (These imperfections are fixed in processing. All serious cameras have a built-in imperfection reference map created during manufacture and testing. More serious cameras let you update this manually too) Not to mention doctoring a RAW would require inanimate knowledge of the imaging sensor.
What's a JPEG at -90% compression? Whoa. Mind blown.
They're not trying to prevent "doctored" images.
The original memo reads:
I’d like to pass on a note of request to our freelance contributors due to a worldwide policy change.. In future, please don’t send photos to Reuters that were processed from RAW or CR2 files. If you want to shoot raw images that’s fine, just take JPEGs at the same time. Only send us the photos that were originally JPEGs, with minimal processing (cropping, correcting levels, etc).
And a follow-up quote reads
While we aim for photography of the highest aesthetic quality, our goal is not to artistically interpret the news. [...] Speed is also very important to us. We have therefore asked our photographers to skip labour and time consuming processes to get our pictures to our clients faster.
Which doesn't mean they're trying to prevent people from faking photos; as that line is clearly referring to the "minimal editing" part of the above guidelines, and the "JPG not RAW" is just for workflow-related reasons.
No. Processing raw files involves more than just compression, it includes things like demosaicing and setting white balance.
Is a JPEG at 0% compression a RAW image?
Not even remotely. JPEG does not have the native sensor data regardless of how little compression you apply. It isn't the native sensor bitdepth, it isn't the native sensor resolution without interpolation, and even the best quality JPEG is crap compared to a RAW original.
Reuters doesnt want to have to dig so deep to discover manipulated photos. They dont want it to be quite so easy to manipulate those photos without it being easily discoverable. Its an entirely separate issue from having punishment for misdeeds. Ars has no issue with it.
http://arstechnica.com/busines...
There are already so many parameters that a photograph can use to tell their own story (framing, focal length, depth of field, ...) that a few touches such as correcting white balance or exposure would pale in comparison. In fact these touches can be used to make the picture actually more faithful, by removing camera artefacts.
There is a good example somewhere where people complained against advertisers and as evidence submitted a picture taken with telephoto lens, making their city look cluttered with billboards. Advertisers responded by taking a picture of the same area but this time with a wide angle lens, for a totally different effect.
As for the speed of processing, good photographers can do basic editing on a RAW in a couple of minutes.
If Reuters no longer accepts images in a particular format, it's their business.
Exactly. So send them JPEGs and STFU.
I am pretty sure the real issue is file size and standards, not doctoring. As manufacturers keep ridiculously upping sensor MP size, photo sizes continue to balloon to larger and larger sizes. RAW files are notoriously huge and non-standard. The extra processing they are referring to is probably just the need to convert those various RAW files back to JPEG, which takes/wastes time/energy by their staff.
You would have to be a pretty big idiot to think that JPEG files are harder to doctor than RAW files. Any photo format can be used when exporting a doctored image... has nothing to do with how it is saved.
They probably profiled people and found that senders of RAW were more likely to be doctored. Also, having an uncompressed image makes it easier to doctor files later.
Yes, would-be fakers can work around this by sending in JPEG format with no compression (I think), and doctoring then sending; but it's not as stupid as the summary makes it out to be.
First shot fired. Now it's a little less convenient for fakers.
Bet you anything they've a managed workflow system and their solution can't deal with raw files.
It's actually worse than that: they aren't merely saying, "Don't send us raw files" (Note no caps -- "raw" isn't an abbreviation); they're saying "Don't send us anything that was even *processed* from raw files." It's as if the raw processing algorithms in the camera are somehow sacrosanct, but the equivalent algorithms run in Lightroom is suspect.
In fact, I think it would be harder to doctor a RAW format because all image sensors have random imperfections, their own physical "fingerprint" that can be traced back to a specific camera. (These imperfections are fixed in processing. All serious cameras have a built-in imperfection reference map created during manufacture and testing. More serious cameras let you update this manually too) Not to mention doctoring a RAW would require inanimate knowledge of the imaging sensor.
I'm not sure what you're talking about by "imperfection reference map" -- do you mean dust delete data? That isn't built-in; you need to take a reference photo of something white in order to generate that. Some software processors also have hot/dead pixel detection.
Otherwise, there is definitely nothing in serious cameras (I assuming that the Canons and Nikons that the vast majority of photojournalists journalists use are "serious") that has any sort of built-in calibration for random imperfections in the sensor. While I have no doubt that, given enough samples and enough time, you might be able to find a way to "fingerprint" a camera, in most cases, sensor noise (whether photon-shot noise, readout noise, or others) is going to significantly overpower any sort of unique characteristics.
Most cameras will let you shoot both Raw & jpg at the same time. Some pro cameras even have dual memory cards which will allow you to store raw on one ( preferably the bigger / faster one ) and jpg on the other. Grab your shot, submit it quickly via .jpg and use your raw file to impress folks with your post production skills later on :D
The format requirement change really only does two things:
1) It cuts down storage requirements significantly. Full size 14-bit Raw image on my Nikon D4s is almost 20MB. Full size .jpg at the fine setting is 8MB.
( The D4s only has a 16mp sensor. Crank that up a bit and the file sizes get rather ludicrous. )
2) Separates the pros from the amateurs. A pro knows how to get a good shot without resorting to post to fix things they should have got right in the camera.
( like exposure and white balance )
No. JPEG is 8-bit color, while RAW is typically 12 or 14 bits with expanded dynamic range and gamut. This makes RAW useful in case you get the exposure or the white balance wrong, among other things.
Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
"[H]igher quality publications"... That would rule out Reuters. Granted, they're not the AP, but still...
Is a JPEG at 0% compression a RAW image?
It would be close but not exact. The way you would get close is to set the 8x8 quantization matrix to all 1's. In JPEG compression, the image is divided into 8x8 blocks, discrete cosine transformed, elementwise divided by an 8x8 quantization matrix, rounded to the nearest integer, and then (usually) Huffman encoded. The primary problem with being perfectly lossless is that the DCT produces a fractional result. So even if you set the quantization matrix to all 1's, the rounding step would lose information.
Care to enlighten me as to how one sets jpeg compression to 0%?
It's not easy to do in most image editors; even the highest (12) quality setting in Photoshop has quantization. You can do it in ImageMagick, however.
Also, no, RAW formats are not simply uncompressed, but largely unprocessed data as well (certainly less processed than what you get from an out of camera tif or jpf.)
Raw formats are indeed compressed; they're just losslessly compressed.
Finally, there is a true lossless JPEG format, though it is distinct from the usual JPEGs.
Make them think it was doctored by a child.
They have no idea how real photography works. JPG is a 'final' format. You capture an image on an SLR as RAW so you get all of the information the sensor can give you, and then you process it to pull the JPGs you want to give to the user of your shots. In journalism, many photographs are taken under marginal conditions, such as four stops below optimum in a sandstorm. Shooting RAW gives you the most latitude to recover usable images that might give us the ability to identify a terrorist. You can apply high dynamic range processing to a single RAW frame to show detail not recoverable any other way, and given a bracket of five RAW frames one stop apart, even handheld, you can postprocess them into a great picture.
Yes, today's journalism photography is being done with many devices that shoot JPG as their native mode, and as any photographer will tell you, the best camera in the world is the one you have with you. But anyone who prohibits high-detail RAW imagery is a person who does not deserve to be in journalism. Manufacturers have responded to the phone-photography challenge with formats like Micro Four Thirds, which gives you SLR versatility in a compact body and lens format that you can take to wherever the news is being made.
They want a JPG so that they can get it up on the website and social media sites as fast as possible. It pays for them to be the first to get a picture or story out. They don't want a photographer to spend the time converting the image on their computer.
But what is perfectly fine is to shoot in RAW + JPG and send in the JPG right away. Then after the event the photographer would take the best shot or two and do the minimal amount of adjustments allowed to make the image more appropriate for newspapers or magazines.
A camera sensor is arranged like this. The sensing elements aren't paired up with 1 red, 1 green, and 1 blue, and the values stored by the elements have a larger range than the values in a JPEG (0-4096, say, instead of 0-255). A RAW image is essentially a direct dump from the image sensor, minimally processed and stored in a known format (CR2 is an example). A rasterized image is rendered from the raw data (includes white balancing and other processing), and then it's encoded into a standard image format (usually JPEG).
It is pitch black. You are likely to be eaten by a grue.
Reuters doesn't want the hassle of dealing with RAW files. They're huge. The formats are many. They also require extra handling. They also don't need them for their uses.
No, that's wrong. Reuters doesn't want images generated from RAW files. That's something entirely different.
Crumb's Corollary: Never bring a knife to a bun fight.
RAW files are a dump of the camera sensor. They generally require processing to produce something useful (although both Canon and Nikon RAW files now include a JPEG embedded into the file format for preview). The reason why you would want a dump of the camera sensor is so that you can do post-production work like adjusting the white balance, adjust the exposure, etc. JPEGs already have this data "baked" and product much poorer results when trying to do this post-production work.
The whole purpose of shooting raw images is to do advanced processing later. However, any such processing involves creative choice which alters the image to the taste of the person doing the processing. It's easy to alter the white point and have some journalistically important details lost in the shadows.
Also in a high stakes case suspected forgery, it may be possible to detect forged images by looking at minute noise and encoding choices made by a particular camera model. Faking these details well enough to fool the experts would be beyond the expertise of most would-be forgers.
Of course, Reuters could ask for RAW files themselves and have even more fidelity/authentication potential. But those files are huge, many journalists do not have a fast internet connection where they work, and the publisher would need expertise on RAW workflows.
All in all, I think it's a reasonable decision and will be successful against unintentional/unconscious alterations and causual forgery.
Not to mention doctoring a RAW would require inanimate knowledge of the imaging sensor.
Well that settles it. I like my knowledge to be lustful and spry. Inanimate knowledge just isn't the same.
vi ~/.emacs # I'm probably going to Hell for this.
Wrong. JPEG at 0% compression is very much different from a RAW image.
Adobe has a "standardized" RAW format which is bunk and not used by anything much.
RAW formats are specific to each camera and usually patterned after the sensor, which is usually (but not always) a Bayer pattern, R-G-B pixels of different sizes in a grid - often there will be two R's and a single G and B in a square, or some variation. Unless it's a Foveon sensor (and 99.9999% aren't) there is only one R, G, or B per "pixel." A RAW processor knows the geometry of this and uses it to interpolate into R-G-B-per-pixel that JPEGs use.
Furthermore, RAW is usually a higher bit depth. A few years ago 14 bits per sample was the standard for Canon. "Why do you need more than 8 bits per sample?" you ask. Well, try restoring blown highlights from a JPEG vs a RAW, or doing any number of color manipulation. 16.7 million colors is a lot, but if your picture is all green, then it's really only 256 colors and it will be posterized.
OK. A new drug relieves a doctor from choosing between an old, expensive, and ineffective treatment versus no treatment at all.
Contribute to civilization: ari.aynrand.org/donate
And thus, this is exactly my problem with "JPEG ONLY"... The camera does quite a bit more than just apply white balance and compress down to a JPEG from the sensor RAW data. Tonal curves, lens correction, sensor correction, sharpening, and a whole bunch of other processing happens during this conversion process. Most of what can be done in Lightroom can be saved as presets (using other tools) and loaded directly into the camera to be applied to a JPEG automatically on capture. Is this then wrong, because the camera did it rather than the user in the end? The results are still the same.
Care to enlighten me as to how one sets jpeg compression to 0%?
Why did such a stupid question get modded up?
It's simple: you open your digital camera's settings menu, find the option to control the JPEG compression level, and set it to 0%.
Jesus, even my shitty old Android phone has a camera app that allows you to do that!
My camera has low-mid-high compression levels. Which one is 0%? Are any of them zero? I don't even see that compression level in my image editor, the closest I have is "quality level: 100" ,which is not nearly the same as 0% compression.
Compression of RAW to JPEG and the alteration of JPEG images leaves a distinct signature in it's Error Level Analysis results. Using a simple utility like http://www.impulseadventure.co... to automatically prescreen images would relieve a huge burden from their shoulders. For authenticity, requesting the RAW after the JPEG to see if the compression gradients are uniform would work as a nice level of security as well.
Where genius and insanity become confused true wisdom is found
this has nothing to do with tech...
raw *are* sensor data... and you can photoshop a jpeg the same way you photoshop a raw...
that's bs. just cropping change the point of view, and it's doable with jpeg trivially... http://images.sodahead.com/pol...
I can make you a JPEG file that is -90% smaller.
Detailed instructions and all necessary code will be embedded in the IPTC comment tag.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
I suspect that you misremember or that the one who told you this was lying. At least, I can't find any information that confirms this. To the contrary, this book uses the Vietnam war as evidence for the importance of treatment for survival.
https://books.google.nl/books?...
Avantslash: low-bandwidth mobile slashdot.
I would suspect that "often" is really "always". A typical Canon RAW file, for example, has 14 bits per pixel. Because of the extra precision, the effective dynamic range of a RAW file is dramatically wider than the dynamic range of a JPEG image. For example, if you have the following samples in the RAW image:
65500, 65532, 65515, 65533, 65473, 65535
And you convert that to JPEG, you'll get:
255, 255, 255, 255, 255, 255
Iff you later need to pull the highs down to make them less blown out, if you're starting with the RAW image, you'll get a fairly accurate rendition of those values (up to the limits of the sensor), whereas if you start with the JPEG image, you'll get white blotches, because there's no detail there to recover. For recovering highlights and/or shadows, JPEG doesn't even come close to RAW, and can't. There's just too much data lost.
Check out my sci-fi/humor trilogy at PatriotsBooks.
Step 1: Create a large blob of irrelevant code and instructions.
Step 2: Store it in an EXIF tag.
Check out my sci-fi/humor trilogy at PatriotsBooks.
Also, no, RAW formats are not simply uncompressed
In fact they often are. They're losslessly compressed. And they use JPEG-like algorithms to do it!
systemd is Roko's Basilisk.
Biotech isn't tech? That's news to me. Somebody call Reuters.
Okay, here's one:
Q: You're standing near a railroad track. A train is heading towards a group of blind people. You do not have time to run and get them off the tracks. You're standing near a switch, however, and can redirect the train to a different track. Unfortunately, on that second track lies a small child, strapped into a car seat. Do you pull the switch and kill the person who would not have died otherwise, or leave the switch and allow all those other people to die?
A: Neither. You pull out your cell phone or train radio, call dispatch, and tell them to stop the d**n train.
There. That's an example of tech solving an ethical problem.
Check out my sci-fi/humor trilogy at PatriotsBooks.
It's only about two things:
1. RAW formats contain "salt" checksums, algorithm is not known, so you cannot really fake it no matter how did you try, in fact, manufacturer can add one to JPEG's too, but they aren't doing it
2. RAW is MUCH larger than JPEG, probably, JPEG's are enough for them
"It feels like I'm at the Zoo when reading this thread - I'm frightened, but it's interesting" (c)
Except you're in Belgium, and they don't understand what you're saying.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
To answer in a more technical way (than "use ImageMagick").
JPEG encoding inherently can be completely lossless. 8x8 pixel squares of pixel values are converted to 8x8 matrices of frequency components - transforming the representation of data as a superposition of specific sine waves of fixed set of frequencies and parametrized amplitudes. Due to small area and range of values being covered, this mapping is lossless - the data is sufficient to recreate the exact image, errors of the "floating point nature" of sine waves being less than 1 bit of value representation.
Then, depending on the settings of the software - the "compression rate", the parameters of lowest values and of highest frequencies are replaced by zeros. The "quality" parameter decides how many, and how significant ones. Unlike with direct value function which would leave black pixels, with sine waves this leaves the characteristic "artifacts" of JPEG, a kind of wavy imprecision along any sharp edges, some colors being misrepresented etc. This is not very visible to human eye, and you can get away with zeroing half and more of the parameters without significantly altering the perceived image.
And then this is compressed using a standard lossless data compression.
The gain comes from the fact that strings of zeros, repeating zeros and such compress very well - much better than "random" data of standard image.
Of course if you don't strip any zeros and keep all the values, JPEG will be lossless and you won't gain anything size-wise, the compression being equivalent to standard lossless ones. And still you can lose relative to RAW, because the original (input) data uses 8-bit color channels, so 3 8x8 matrices of bytes per one "block". RAW can keep much more bits per pixel.
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
You can think of it, very basically, as Reuters insisting on a 4x6 physical print, rather than wanting the negative.
In this analogy, it's easier to work with the print than the negative, but if you want to scan it back to digital, blow it up, whatever, you're losing quality. With the negative, you have more work to do, but you have a higher-quality starting point, and you can do all sorts of work with it and get far better results than by working from the 4x6 print.
Vintage computer games and RPG books available. Email me if you're interested.
This assumes a) you happen to have a train radio and work for the train company, or b) you have a cell phone, you have cell service, you have the number of some sort of central switching office, they'll take your call, they'll believe who you are, you can describe where the train is and what signal needs to be changed in a way that they'll understand, that they'll believe you, that they'll do what you ask, that they have time to switch the signal, that there aren't further problems down the line that switching the train might just make worse (perhaps the carseat child is on a siding that currently holds a bunch of loaded cars waiting for a locomotive, say, 500 meters away and around a blind corner,) that the train can stop in time (Sir Issac Newton is still the deadliest son-of-a-bitch in space) and so on. Tech has done nothing here.
Vintage computer games and RPG books available. Email me if you're interested.
Need a real-time voice translator then. Another tech thingy.
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
They want to stop people from screwing with photos...but their solution is basically to say "let the camera make its best guess at what you were trying to capture". So instead of spending time in the dark room trying to get the best print possible (starting from a raw file), you are handing it over to to a pimply kid to run through an automated machine (letting the camera guess at brightness/contrast/white balance/etc).
Sure, you could do some unacceptable retouching in the darkroom...but most of what you are trying to do is get the photo to come out right. Same with raw files...you don't need a raw file to Photoshop an extra explosion into the background (although it might make it a little easier to make it hard to detect)...but you do want the raw file when you are trying to correct your exposure. This is particularly important for photojournalists who are not working in studio conditions. Maybe you were shooting in a combat zone and your only non-blurry shots came out way under-exposed...having the raw file gives you the most detail and ability to correct the image back to something usable.
Bottles.
FTFY.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
I cannot fathom any explanation as to why they press so hard on presenting photos and video as is, but feel free to be as creative as possible with the text and words. My guess is that cameramen are considered second class citizens as opposed to the anchors, and they actively want to prevent them from doing anything creative.
Plus raw is an adjective, not an acronym. Anyone who uppercases it should be shot. CR2 files *are* raw. Would NEF or DNG files be okay given their dumb wording??
Umm... what gave you that idea? There are -lots- of ways to edit jpg's and most other image formats.
That said, police photographers have a protocol for ensuring that original images are digitally signed as they leave the camera (I don't know the details) so that they can be trusted in court as evidence.
Was supposed to be a joke, didn't work I guess.
Anyone who uppercases it should be shot.
Whoa, dude, take a pill!
I am currently beneath your threshold
Sounds like your camera may just be total shit, even compared to camera apps on ancient Android phones!
100% quality level, when dealing with JPEGs, implies 0% compression. JPEG is lossy compression. 100% quality means there's no loss. Thus there's no (that is, 0%) JPEG compression going on. If there were JPEG compression happening, then the quality level inherently could not be 100%, because JPEG is a lossy format.
Sounds like you have no idea of what you are writing about. My camera captures a 24Mb raw file. The same capture at the highest quality JPG setting is 5Mb. So what happened to my other 19Mb of image data in your 0% compression scheme?
I am currently beneath your threshold
Just doesn't work on the "Sheldon"s: those that do not understand the concept of sarcasm...