Google Brain Creates Technology That Can Zoom In, Enhance Pixelated Images (softpedia.com)

Google Brain has created new software that can create detailed images from tiny, pixelated images. If you've ever tried zooming in on an image, you know that it generally becomes more blurry. You'd just get larger pixels and not a clear image. Google's new software effectively extracts details from a few source pixels to enhance pixelated images. Softpedia reports: For instance, Google Brain presented some 8x8 pixel images which it then turned into some pretty clear photos where you can actually tell facial features apart. What is this sorcery, you ask? Well, it's Google combining two neural networks. The first one, the conditioning network, works to map the 8x8 pixel source image against other high-resolution images. Basically, it downsizes other high-res images to the same 8x8 size and tries to make a match on the features. Then, the second network comes into play, called the prior network. This one uses an implementation of PixelCNN to add realistic, high-res details to that 8x8 source image. If the networks know that one particular pixel could be an eye, when you zoom in, you'll see the shape of an eye there. Or an eyebrow, or a mouth, for instance. The technology was put to the test and it was quite successful against humans. Human observers were shown a high-resolution celebrity face vs. the upscaled image resulted from Google Brain. Ten percent of the time, they were fooled. When it comes to the bedroom images used by Google for the testing, 28 percent of humans were fooled by the computed image.

Comments

Google can put together images

[hackwrench]

Google can put together images based on smaller images that look like faces.

No CSI

[Thelasko]

I don't care how fancy the algorithm is, the original data was lost. This is still just a guess about the original content. It's just a better guess than was possible before.

I just hope law enforcement doesn't think they can use this to solve any crimes.

Re:No CSI

[rmdingler]

Law enforcement would never rely upon unproven methods to improve conviction rates.

Re:No CSI

[Thelasko]

Sure, it could narrow a search. However, this shouldn't be used as evidence in a trial, or even to obtain a warrant.

Re:No CSI

[Hognoxious]

Absolutely not. If that was allowed to happen then before you know it they'd be acting as an army (except with different coloured uniforms) and we all know that'll never happen because FREEDOM and NUMBER ONE!

Re:No CSI

[PPH]

I just hope law enforcement doesn't think they can sell this to a jury.

FTFY.

CSI has already proved problematic in that jurors have developed unreasonable expectations of what is possible.

Re:No CSI

[wbr1]

This needs so many more points. As someone who has been to prison - guilty as charged and pled to it, there are many who were not. There are plenty who were guilty in prison, but one that is not is too many.

One particular case stands out to me. Navy soldier, on leave, drunk. Seen in an altercation in a bar with someone. that someone was known for instigating fights. Person winds up stabbed to death later.

The suspect is arrested, and drunk, and with a huge lack of sleep, under duress from trained psychological tactics confesses. He later recants. Blood at the scene does not match hos or the victims blood type. None matches his. No physical evidence shows he was there. Convicted of 2nd degree murder.

He has a family member (through an attorney) years later make an inquiry if the evidence is still in storage. Murder cases are supposed to have evidence kept for a very long time (if not forever) in my state. They are told yes. Attorney gets innocence project involved. A few months later the innocence project requests evidence, and is told sorry, it was 'lost' in a move between labs, never to be found again.

This man did more than two decades before mandatory release. For a crime he possibly did not commit. He is not the only one with stories like this.

Find this video from 2006 and ask has it gotten any better? http://www.cnn.com/CNN/Program...

"The degree of civilisation in a society is revealed by entering its prisons." -Fyodor Dostoyevsky
"If you want to see the scum of the earth, go to any prison - at shift change" - Paul Harvey

Re:No CSI

[mmell]

That's part of the problem though, isn't it? We all root for the protagonists on CSI, Criminal Minds, Law & Order, and at least half a dozen other police dramas when they routinely use some mystical all-knowing government database or "hack" into government and private (corporate) databases and nail the bad guy beyond all shadow of doubt. Many of us would groan when the protagonists would "hack" into live bank ATM security cameras and enhance the medallion number of a cab half a block down from a dozen greyscale pixels (my own wife constantly reminds me "it's only television" when I point out how ridiculous it is that they can create data out of the ether).

So - will this algorithm always come up with medallion #1313, because that's what it "learned" a medallion number is? When the system "guesses" at the missing data based on what it "learned" during "training", will it always identify Christy Brinkley and Chuck Norris as the perpetrators? Some of this is valid - I can often read information which is blurry or even pixelated, but not always. Intuitively differentiating an 8 from a 0 based on how dark that few pixels are is different from proving that it was an 8 or a 0.

One other concern - in the past, enhanced video has been useful in court, when there was someone able to explain exactly what transformations were done, someone prepared to prove why the enhanced video is still representative of the portrayed reality. Learning systems may often produce surprisingly accurate results from seemingly inadequate data, but it's considerably harder to prove that the enhanced information accurately portrays reality - perhaps impossible, given the complexity of such systems.

Blade Runner-esque?

[threadsafe]

Is this Blade Runner-esque? Decker summoned some wicked camera technology. Don't bother me with those pesky limits to the physical laws.

Re:Blade Runner-esque?

[threadsafe]

I'm actually a dog on the internet because I didn't proof or spell check my previous reply.

Re:Blade Runner-esque?

[FatdogHaiku]

I'm actually a dog on the internet because I didn't proof or spell check my previous reply.

Just paws before you hit Submit...

Ideal test case

[fibonacci8]

Feed it minecraft screenshots and japanese porn, and see what the result is.

Re:Ideal test case

[Djoulihen]

Well, if you are into that kind of stuff (neural network generated kind-of porn but not really), I suggest you check this out: https://open_nsfw.gitlab.io/ (Warning: somehow NSFW, or just weird, I don't know, but it will give you nightmares).

It's about time!

[npslider]

So in other words... from a small picture of the earth viewed from orbit, Google can now show me my house AND the address on the UPS package sitting at my doorstep?

Amazing!

Re:It's about time!

[mmell]

Yes, they can! And the flip-side of this technology is that you can zip a 500GB database down to 200GB, zip that down to 700MB, zip that down to a few hundred KB . . . when it's 1B, you can embed it on a carbon atom on my DNA and I can carry it back to Kronos, for the glory of the Empire!

What about zoom out ?

[alexhs]

If it doesn't do uncrop, it's lame.

Re:What about zoom out ?

[K. S. Kyosuke]

Already done. ;)

"...it was quite successful against humans..."

[turkeydance]

now we know the perception and purpose,

no need to photoshop

[zlives]

this is perfect for your "real" profile pic on dating sites, just upload a google enhanced image of your self created from your 8x8 pixel image. yes this celebrity is really me.

They've been watching too much TV

[taustin]

TV Detective: "We have this security video showing the murder."

TV Lab Rat: "It's too grainy to tell how is it?"

TV Detective: "Can't you enhance it?"

TV Lab Rat: "Sure. Who do you want it to look like?"

Really impressive progress

[PhrostyMcByte]

A while ago, someone made the nnedi upsampler that uses neural networks to upsample. It's still one of the best image upsamplers available.

Google's approach is quite a bit different. Where nnedi worked to better extract detail out of what was already in the image, Google seems to literally fill in detail that was probably in the source but maybe not. Much, I guess, like how our own memories work. It's an interesting approach and the results look quite fantastic. My only question is how well it will work on a random sampling.

Re:Stupid

[ceoyoyo]

I see you still don't actually know what a Fourier transform is.

Bad summary

[swillden]

The summary's explanation of what this does isn't correct. It says:

Google's new software effectively extracts details from a few source pixels to enhance pixelated images.

It doesn't extract details from a few source pixels. It invents details to add to those source pixels, based on the knowledge that the pixelated image is of a face, and of what faces look like. It produces something that plausibly fits the input data. How close this is to the original image, pre-pixelation, depends on what images were in its training set.

This is an interesting piece of work, but it doesn't mean that you can recover data that has been discarded.

They do that in movies all the time

[OneHundredAndTen]

Does this imply that movies have been lying to us all along? :-)

Come on Google

[CODiNE]

At least make the world a better place...
by making this a MAME scaler.

Re:Stupid

[PopeRatzo]

You will get smacked down if you try to feign knowledge in those domains because those of us who know our shit have zero tolerance for bullshit

No, once information has been lost it's gone. No amount of time or frequency domain analysis will restore it.

https://arxiv.org/pdf/1101.007...

Re:Stupid

[elgatozorbas]

Anti-aliasing is not "providing data that is not actually there", anti-aliasing is removing data that is actually not there, more specifically high-frequency aliases of the low-frequency information. Hence the name anti-aliasing.
In practice, this means that the best any algorithm can do to improve a (general) blocky image, is smooth the edges of the blocks. What this algorithm apparently does, as far as I understand, is just correlate the blocky image to a library of known facial features. This has nothing to do with FFT or anti-aliasing whatsoever.

Apart from that, unless your name is "M", I am afraid

I know this because I am friends with both of the "B"s in BBM.

doesn't really mean anything.

Re:Stupid

[AmiMoJo]

A Fourier transform takes a signal and converts it into a bunch of sine waves that when combined will reproduce the original signal.

A signal is just varying amplitude over time. A sine wave is a signal where the amplitude is sin(t), where t is time. It turns out that all signals can be constructed by combining sine waves. The more sine waves you have the close to the original signal you can get.

Why it is useful to convert a signal to sine waves? Say you decide you are going to use 20,000 sine waves, the first one a 1Hz wave, the second one a 2Hz wave, 3Hz, 4Hz all the way up to 20,000Hz. You use a Fourier transform to convert the signal to those sine waves. Now you increase the amplitude of the low frequency sine waves, say 200Hz and below. Now do another Fourier transform to convert from sine waves back into a signal. Congratulations, you just pumped up the bass on your music.

In technical terms the signal is in the time domain, and the sine waves are in the frequency domain.

Disclaimer: This is a simplification, it's more complex than this but without getting into the heavy maths of it, this is basically what's going on.