Recent advancements in deep learning techniques such as Convolutional Neural Networks(CNN) and Generative Adversarial Networks(GAN) have achieved breakthroughs in the problem of semantic image inpainting, the task of reconstructing missing pixels in given images. While much more effective than conventional approaches, deep learning models require large datasets and great computational resources for training, and inpainting quality varies considerably when training data vary in size and diversity. To address these problems, we present in this paper a inpainting strategy of \textit{Comparative Sample Augmentation}, which enhances the quality of training set by filtering out irrelevant images and constructing additional images using information about the surrounding regions of the images to be inpainted. Experiments on multiple datasets demonstrate that our method extends the applicability of deep inpainting models to training sets with varying sizes, while maintaining inpainting quality as measured by qualitative and quantitative metrics for a large class of deep models, with little need for model-specific consideration. Semantic image inpainting, the task of reconstructing missing pixels in images, has various applications in computer vision problems such as computational photography and image restoration BID7 ). Although there has been substantial progress in relevant research, image inpainting still remains a great challenge due to the difficulty to synthesize missing pixels that are visually and semantically coherent with surrounding existing background pixels. Such an issue becomes especially apparent when the amount of available training image data is limited due to the current limitations of deep models in representing possible latent features.Current solutions to the inpainting problem mainly belong in two groups: traditional patch-based learning methods and deep learning methods. Traditional methods often directly utilize background information by assuming that information of missing patches can be found in background regions BID0 ), leading to poor performances in reconstructing complex non-repeating structures in the inpainting areas and in capturing high-level semantics. Deep learning neural methods, on the other hand, exploit deep models to extract representations of latent space of existing pixels and transform inpainting into a conditional pixel generation problem BID6 , BID7 ). While these approaches did produce images of significantly higher quality, they generally require an enormous amount of highly varied training data for model training, a requirement of which makes it impossible to apply these strategies when the set of available training data is limited. Recent research BID2 ) also suggest that the performances of neural networks vary considerably in a variety of tasks when input images contain adversarial noise that potentially affects latent space, showing that countering adversarial examples is a key in boosting deep learning models.To address these issues, we propose in this paper a simple black-box strategy that easily adapts to existing generative frameworks without model specific considerations and extends deep neural network strategies to the cases with varying amounts of training data. The contribution of this paper can be summarized as follows:• We designed an effective strategy of selecting relevant samples by constructing a similarity measure based on color attributes of the inpainting image and the training dataset and selecting the K-most-relevant pictures.• Our algorithm also bolsters local image qualities by adding new images created through white-noise addition on the original inpainting image.• We have also conducted detailed set of experiments using the state-of-the-art generative inpainting structures, and demonstrate that our method achieves better inpainting results when the available training data is not necessarily abundant.
We introduced a strategy which enables inpainting models on datasets of various sizes Help image inpainting using GANs by using a comparative augmenting filter and adding random noise to each pixel.