If you have an image with missing information, perhaps there was a hole or torn section, or it’s a corrupted digital image with missing pixels, then Nvidia AI researchers have an interesting new AI technique for you. A state-of-the-art deep learning method that can edit images or reconstruct a corrupted image has been devised and demonstrated by Nvidia researchers, lead by Guilin Liu.
The process you see in the video above is called ‘image inpainting’. In the examples provided you can easily understand that this technique can be used for AI-pixel detail re-creation and drawing in missing content – it can be used to get rid of incongruous photo elements too.
In the researchers’ own words, from the recently published paper hosted by Cornell University, it is claimed that “our model can robustly handle holes of any shape, size location, or distance from the image borders. Previous deep learning approaches have focused on rectangular regions located around the centre of the image, and often rely on expensive post-processing,” the Nvidia researchers stated. “Further, our model gracefully handles holes of increasing size.”
As you might guess, the AI neural network has been through various levels of training. The researchers generated over 55,000 photo masks of random streaks and holes (irregular shapes), with 25,000 of them, in six categories, used for training. The AI learned from various marks (from the photo masks) being superimposed upon various images. Later on, the AI was tested using masks not included within the training programme. As a side note, the researchers employed Nvidia Tesla V100 GPUs and the cuDNN-accelerated PyTorch deep learning framework for their work.
Nvidia’s research team are particularly proud of how their AI doesn’t lead to artifacts such as colour discrepancy and blurriness. In other image repair methods fixing up this very issue is “expensive” in terms processing “and may fail,” they note.
Nvidia is attending ICLR 2018 at the Vancouver Convention Centre, 30th April – 3rd May, 2018, to showcase this and similar deep learning research.
