New AI Model Fills in Blank Spots in Photos (nikkei.com)

A new technology uses artificial intelligence to generate synthetic images that can pass as real. From a report, shared by a reader (the link may be paywalled): The technology was developed by a team led by Hiroshi Ishikawa, a professor at Japan's Waseda University. It uses convolutional neural networks, a type of deep learning, to predict missing parts of images. The technology could be used in photo-editing apps. It can also be used to generate 3-D images from real 2-D images. The team at first prepared some 8 million images of real landscapes, human faces and other subjects. Using special software, the team generated numerous versions for each image, randomly adding artificial blanks of various shapes, sizes and positions. With all the data, the model took three months to learn how to predict the blanks so that it could fill them in and make the resultant images look identical to the originals. The model's learning algorithm first predicts and fills in blanks. It then evaluates how consistent the added part is with its surroundings.

Comments

It might be ok and awful at the same time.

[Mal-2]

I bet it will be pretty good in some contexts, and most likely an improvement overall compared to content-aware fills. However, when it completely falls on its face I bet it will be even funnier than the way content-aware fill blows up. Lower rate of occurrence, but much more hilarity when it happens.

Alternate Headline

[OzPeter]

The alternate headline is:

Computer program analyzes data and based on that analysis invents new data that seems plausible to most people

Re: Alternate Headline

[OzPeter]

I think it's really deeper tho. If you see just the tip of a football an NFL running back is holding, your brain has a plausible idea that he's holding a football, (not a puppy with a butt that looks like a football) and that will be confirmed when he spikes it in the end zone. Building most probable world models from partial information is a key thing human intelligence does.

Nice straw man bro!

There is a difference between inferring that a complete football is attached to the tip and creating (from scratch) an image of the hidden parts of the football. Because until you actually see it, the hidden parts of the football may well be painted to look like a puppy.

As in the case of TFA where they created from scratch a shelf full of books without knowing if that shelf actually contained books or a bust of Plato. And in fact from all we know of the "before" image presented in the TFA there might have been a bust of Plato on the shelf - or even no shelf at all.

Re: Alternate Headline

[javaman235]

The feeling of surprise is when your brain encounters something different from the world model *it already built* from partial information and held subconsciously. It's a part of intelligence.

I mean yeah, if someone peddles this is purely accurate info it's BS, but you can't deny that statements about what's probably in a concealed area, based on lots of experience/training are useful if taken for what they are.

Can it capture the whole boob?

[Anonymous Coward]

Many celebrities like to show off underboob, side boob, cleavage and the occasional nip slip.

Does this software allow us to piece together a whole CELEBRITY BREAST?

Not bad, but not great

[lurker412]

The example shown in the linked article doesn't hold up under scrutiny. Look at the blue-green books on the center-right--the convergence of the shelves is wrong and the corner is not rendered correctly. Assuming this was a one-step edit, it's probably better than Photoshop's current content aware fill, but it still requires additional work to escape detection.

Re:Anti-shake

[Solandri]

Hopefully this can be applied to anti-shake filters where existing solutions do a really poor job of inventing blurry missing data.

The data is actually still there, not missing. It's just smeared across several pixels instead of each point in space corresponding to a single pixel. You can use a deconvolution filter to un-smear it. Same goes for out-of-focus photos.

In the case of camera shake, if the camera would record its movements while the photo was being taken and included that info in the photo, you could apply a perfect deconvolution filter that would almost completely eliminate the blur (it becomes less accurate near the edges because you've permanently lost info when parts of the photo moved out of the frame).

Photoshop CC already includes this filter. I suspect the only reason it isn't yet built into phones to automatically de-blur photos with camera shake is because it's fairly processor intensive, making it easier just to take another photo. (Samsung's strategy is to take multiple photos held in a temporary memory buffer, then the phone determines which is sharpest, and lets you choose if you want to keep it, then deletes the others.)