Username   Password       Forgot your password?  Forgot your username? 


A Novel Image Inpainting Method for Object Removal based on Structure Sparsity

Volume 14, Number 11, November 2018, pp. 2777-2788
DOI: 10.23940/ijpe.18.11.p24.27772788

Lei Zhang and Minhui Chang

School of Mathematics and Information Technology, Yuncheng University, Yuncheng, 044000, China

(Submitted on August 21, 2018; Revised on September 9, 2018; Accepted on October 13, 2018)


In the traditional image inpainting method for object removal, for each target patch, the entire source region must be traversed to search for the exemplar patch, which may make the restoration process time-consuming and affect the restoration efficiency. Even worse, the target patch may be replaced by an inappropriate exemplar patch during the process, which will introduce some unexpected objects in the restored image and make the result unable to meet the requirements of visual consistency. In view of these problems, we propose a novel image inpainting method for object removal based on structure sparsity. First, we calculate the structure sparsity of the target patch, and then identify the local characteristics of the region where the target patch is located. Then, we set different search regions for the target patches according to different regional characteristics. Finally, we find the exemplar patch in the search region and restore the target patch. Experiments on a number of natural images show that the proposed method can reduce the restoration time and improve the restoration efficiency. Additionally, it can prevent the mismatch to some extent and improve the restoration effect.


References: 29

                  1. C. Guillemot and O. M. Le, “Image Inpainting: Overview and Recent Advances,” IEEE Signal Processing Magazine, Vol. 31, No. 1, pp. 127-144, 2014
                  2. W. Wang and Y. Jia, “Damaged Region Filling and Evaluation by Symmetrical Exemplar-based Image Inpainting for Thangka,” Eurasip Journal on Image & Video Processing, Vol. 38, pp. 1-13, 2017
                  3. N. Bhardwaj and S. Agarwal, “Review of Image Defect Detection and Inpainting Techniques and Scope for Improvements,” International Journal of Emerging Sciences, Vol. 5, No. 1, pp. 49-67, 2015
                  4. I. C. Chang, J. C. Yu, and C. C. Chang, “A Forgery Detection Algorithm for Exemplar-based Inpainting Images Using Multi-region Relation,” Image and Vision Computing, Vol. l, No. 1, pp. 57-71, 2013
                  5. L. Zhang, B. Kang, B. Liu, and Z. Bao, “A New Inpainting Method for Object Removal based on Patch Local Feature and Sparse Representation,” International Journal of Innovative Computing, Information and Control, Vol. 12, No. 1, pp. 113-124, 2016
                  6. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image Inpainting,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417-424, 2000
                  7. T. F. Chan, J. Shen, and H. M. Zhou, “Total Variation Wavelet Inpainting,” Journal of Mathematical Imaging and Vision, Vol. 25, No. 1, pp. 107-125, 2006
                  8. T. F. Chan and J. Shen, “Nontexture Inpainting by Curvature-Driven Diffusions,” Journal of Visual Communication and Image Representation, Vol. 12, No. 4, pp. 436-449, 2001
                  9. L. Zhang, B. Kang, B. Liu, and F. Zhang, “Image Inpainting based on Exemplar and Sparse Representation,” International Journal of Signal Processing, Image Inpainting and Pattern Recognition, Vol. 9, No. 9, pp. 177-188, 2016
                  10. L. Dai, D. H. Jiang, B. Ding, and J. K. Hahn, “Improved Digital Image Restoration Algorithm based on Criminisi,” Journal of Digital Information Management, Vol. 14, No. 5, pp. 302-310, 2016
                  11. M. S. Ishi, L. Singh, and M. Agrawal, “A Review on Image Inpainting to Restore Image,” IOSR Journal of Computer Engineering, Vol. 13, No. 6, pp. 8-13, 2013
                  12. A. Criminisi, P. Pérez, and K. Toyama, “Region Filling and Object Removal by Exemplar-based Image Inpainting,” IEEE Transactions on Image Processing, Vol. 13, No. 9, pp. 1200-1212, 2004
                  13. A. Wong and J. Orchard, “A Nonlocal-means Approach to Exemplar-based Inpainting,” in Proceedings of 15th IEEE International Conference on Image Processing, pp. 2600-2603, 2008
                  14. A. A. Efros and T. K. Leung, “Texture Synthesis by Non-parametric Sampling,” in Proceedings of IEEE International Conference on Computer Vision, pp. 1033-1038, 1999
                  15. L. J. Deng, T. Z. Huang, and X. L. Zhao, “Exemplar-based Image Inpainting Using a Modified Priority Definition,” PLOS One, Vol. 10, No. 10, pp. 1-18, 2015
                  16. J. Lin, D. Deng, J. Yan, and X. Lin, “Self-Adaptive Group based Sparse Representation for Image Inpainting,” Journal of Computer Applications, Vol. 37, No. 4, pp. 1169-1173, 2017
                  17. M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An Algorithm for Designing Over-Complete Dictionaries for Sparse Representation,” IEEE Transactions on Signal Processing, Vol. 54, No. 11, pp. 4311-4322, 2006
                  18. M. Elad, J. L. Starck, P. Querre, and D. L. Donoho, “Simultaneous Cartoon and Texture Image Inpainting Using Morphological Component Analysis (MCA),” Applied and Computational Harmonic Analysis, Vol. 19, No. 3, pp. 340-358, 2005
                  19. Y. Huang, K. Li, and M. Yang, “An Improved Image Inpainting Algorithm based on Image Segmentation,” Procedia Computer Science, No. 107, pp. 796-801, 2017
                  20. V. K. Alilou and F. Yaghmaee, “Exemplar-based Image Inpainting Using Svd-based Approximation Matrix and Multi-Scale Analysis,” Multimedia Tools and Applications, Vol. 76, No. 5, pp. 7213-7234, 2017
                  21. J. Wang, K. Lu, D. Pan, and B. K. Bao, “Letters: Robust Object Removal with an Exemplar-based Image Inpainting Approach,” Neurocomputing, No. 123, pp. 150-155, 2014
                  22. X. Xi, F. Wang, and Y. Liu, “Improved Criminisi Algorithm based on a New Priority Function with the Gray Entropy,” in Proceedings of 2013 9th International Conference on Computational Intelligence and Security, 2013
                  23. Q. Zhang, J. Lin, and Y. Liu, “A Fast Image Inpainting Scheme Using Local Average Gray Entropy,” Computer Applications and Software, Vol. 31, No. 10, pp. 206-208, 2014
                  24. W. Zou, Z. Zhou, and Y. Wang, “Image Inpainting Algorithm based on Non-Subsampled Contourlet Transform,” Journal of Computer Applications, Vol. 37, No. 2, pp. 553-558, 2017
                  25. A. Nan and X. Xi, “An Improved Criminisi Algorithm based on a New Priority Function and Updating Confidence,” in Proceedings of International Conference on Biomedical Engineering and Informatics, pp. 885-889, 2014
                  26. Y. F. Liu, F. L. Wang, and X. Y. Xi, “Enhanced Algorithm for Exemplar-based Image Inpainting,” in Proceedings of 2013 9th International Conference on Computational Intelligence and Security (CIS), pp. 209-213, 2013
                  27. B. Shen, W. Hu, Y. Zhang, and Y. J. Zhang, “Image Inpainting via Sparse Representation,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 697-700, 2009
                  28. Z. Xu and J. Sun, “Image Inpainting by Patch Propagation Using Patch Sparsity,” IEEE Transactions on Image Processing, Vol. 19, No. 5, pp. 1153-1165, 2010
                  29. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour Detection and Hierarchical Image Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 5, pp. 898-916, 2011


                                  Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

                                  This site uses encryption for transmitting your passwords.