vacancies advertise contact news tip The Vault
facebook rss twitter

Nvidia shows off AI tech for realistic reconstruction of photos

by Mark Tyson on 24 April 2018, 11:10


Quick Link:

Add to My Vault: x

If you have an image with missing information, perhaps there was a hole or torn section, or it’s a corrupted digital image with missing pixels, then Nvidia AI researchers have an interesting new AI technique for you. A state-of-the-art deep learning method that can edit images or reconstruct a corrupted image has been devised and demonstrated by Nvidia researchers, lead by Guilin Liu.

The process you see in the video above is called ‘image inpainting’. In the examples provided you can easily understand that this technique can be used for AI-pixel detail re-creation and drawing in missing content – it can be used to get rid of incongruous photo elements too.

In the researchers’ own words, from the recently published paper hosted by Cornell University, it is claimed that “our model can robustly handle holes of any shape, size location, or distance from the image borders. Previous deep learning approaches have focused on rectangular regions located around the centre of the image, and often rely on expensive post-processing,” the Nvidia researchers stated. “Further, our model gracefully handles holes of increasing size.”

As you might guess, the AI neural network has been through various levels of training. The researchers generated over 55,000 photo masks of random streaks and holes (irregular shapes), with 25,000 of them, in six categories, used for training. The AI learned from various marks (from the photo masks) being superimposed upon various images. Later on, the AI was tested using masks not included within the training programme. As a side note, the researchers employed Nvidia Tesla V100 GPUs and the cuDNN-accelerated PyTorch deep learning framework for their work.

Nvidia’s research team are particularly proud of how their AI doesn’t lead to artifacts such as colour discrepancy and blurriness. In other image repair methods fixing up this very issue is “expensive” in terms processing “and may fail,” they note.

Nvidia is attending ICLR 2018 at the Vancouver Convention Centre, 30th April – 3rd May, 2018, to showcase this and similar deep learning research.

HEXUS Forums :: 8 Comments

Login with Forum Account

Don't have an account? Register today!
I'm always impressed with the research nvidia does in the realms of image processing/manipulation using deep learning/AI - some of the whitepapers are mind bending, but really interesting reads.
Yes. Because all humanity needs is another app that creates fakery. Well done Nvidia…
Does it work if pieces of clothing are in the way ;););)
Yes. Because all humanity needs is another app that creates fakery. Well done Nvidia…

Sometimes “fakery” is better than nothing. Many years ago I worked for an agency that did photo restoration - war era photographs that were water/fire/sun/etc damaged, often completely missing large portions of the original image. With the right training data something like this could turn an hours-long job in to a few seconds with quite decent results

So yes, very well done Nvidia.
what's with her eyes though - that is not realistic IMO and looks like a botched photoshop. Compare left eye (as we look at it) in particular.