Username   Password       Forgot your password?  Forgot your username? 

 

Colorization for Anime Sketches with Cycle-Consistent Adversarial Network

Volume 15, Number 3, March 2019, pp. 910-918
DOI: 10.23940/ijpe.19.03.p20.910918

Guanghua Zhang, Mengnan Qu, Yuhao Jin, and Qingpeng Song

School of Information Science and Engineering, Hebei University of Science and Technology, Shijiazhuang, 050018, China

(Submitted on October 24, 2018; Revised on November 26, 2018; Accepted on December 28, 2018)

Abstract:

Coloring animation sketches has always been a complex and interesting task, but as the sketch is the first part of animation creation that neither presents gray value nor presents semantic information, the lack of real animation sketches is the biggest difficulty in current model training. It is also usually expensive to collect such data. In recent years, some methods based on generative adversarial networks (GANs) have achieved great success. They can generate colorized anime illustration on given sketches. Many existing sketch coloring tools are based on this supervised learning method, but the marking of data is particularly important for supervised learning, and much time is spent on the marking of data. To address these challenges, we propose a novel approach for unsupervised learning based on U-net and periodic consistent confrontation. Specifically, we combine the periodic consistent antagonism framework with the U-net structure and residual network, enabling us to robustly train a deep network to make the resulting images more natural and realistic. We also adopted some special data generation methods, so that our model can not only color anime sketches but also extract line drafts from colored pictures. By comparing the mainstream models of supervised learning, we show that the image processed by the proposed method can achieve a similar effect.

 

References: 26

        1. L. A. Gatys, A. S. Ecker, and M. Bethge, “Image Style Transfer using Convolutional Neural Networks,” in Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414-2423, IEEE, 2016
        2. L. A. Gatys, A. S. Ecker, and M. Bethge, “A Neural Algorithm of Artistic Style,” arXiv Preprint arXiv: 1508. 06576, 2015
        3. L. A. Gatys, M. Bethge, A. Hertzmann, and E. Shechtman, “Preserving Color in Neural Artistic Style Transfer,” arXiv Preprint arXiv: 1606. 05897v1, 2016
        4. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., “Generative Adversarial Nets,” in Proceedings of International Conference on Neural Information Processing Systems, pp. 2672-2680, MIT Press, 2014
        5. A. Radford, L. Metz, and S. Chintala, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks,” arXiv preprint arXiv: 1511. 06434, 2015
        6. J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks,” in Proceedings of IEEE International Conference on Computer Vision, pp. 2242-2251, IEEE Computer Society, 2017
        7. A. Odena, C. Olah, and J. Shlens, “Conditional Image Synthesis with Auxiliary Classifier GANs,” arXiv Preprint arXiv: 1610. 09585v4, 2016
        8. M. Mirza and S. Osindero, “Conditional Generative Adversarial Nets,” in Proceedings of International Conference on Neural Information Processing Systems, pp. 2672-2680, MIT Press, 2014
        9. X. Mao, Q. Li, H. Xie, R. Y.K. Lau, Z. Wang, and S. P. Smolly, “Least Squares Generative Adversarial Networks,” arXiv Preprint arXiv: 1611. 04076v3, 2016
        10. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234-241, Springer, Cham, 2015
        11. P. Isola, J. Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 5967-5976, IEEE Computer Society, 2017
        12. (https://github.com/pfnet/PaintsChainer, accessed March 29 2017)
        13. L. Zhang, Y. Ji, and X. Lin, “Style Transfer for Anime Sketches with Enhanced Residual U-Net and Auxiliary Classifier GAN,” arXiv Preprint arXiv: 1706. 03319v2, 2017
        14. Y. Liu, Z. Qin, T. Wan, and Z. Luo, “Auto-Painter: Cartoon Image Generation from Sketch by using Conditional Generative Adversarial Networks,” arXiv Preprint arXiv: 1705. 01908, 2017
        15. K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, IEEE Computer Society, 2016
        16. S. Iizuka, E. Simo-Serra, and H. Ishikawa, “Let There Be Color: Joint End-to-End Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification,” ACM Transactions on Graphics, ACM, Vol. 35, No. 110, 2016
        17. R. Zhang, P. Isola, and A. A. Efros, “Colorful Image Colorization,” in Proceedings of European Conference on Computer Vision, pp. 649-666, Springer, Cham, 2016
        18. J. Yu, Z, Lin, J. Yang, X. Shen, X. Lu, and T. S. Huang, “Generative Image Inpainting with Contextual Attention,” arXiv Preprint arXiv: 1801. 07892, 2018
        19. K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv Preprint arXiv: 1409. 1556, 2014
        20. D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context Encoders: Feature Learning by Inpainting,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536-2544, IEEE Computer Society, 2016
        21. D. Berthelot, T. Schumm, and L. Metz, “BEGAN: Boundary Equilibrium Generative Adversarial Networks,” arXiv Preprint arXiv: 1703. 10717, 2017
        22. D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” arXiv Preprint arXiv: 1312. 6114, 2013
        23. C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, et al., “Photo-Realistic Single Image Super-Resolution using a Generative Adversarial Network,” arXiv Preprint arXiv: 1609. 04802, 2016
        24. C. Li and M. Wand, “Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks,” in Proceedings of European Conference on Computer Vision, pp. 702-716, Springer, Cham, 2016
        25. H. Winnemöller, “XDoG: Advanced Image Stylization with Extended Difference-of-Gaussiansm,” in Proceedings of ACM Siggraph/Eurographics Symposium on Non-Photorealistic Animation and Rendering, pp. 147-156, ACM, 2011
        26. “Deepcolor: Automatic Coloring and Shading of Manga-Style Lineart,” (http://kvfrans.com/coloring-and-shading-line-art-automatically-through-conditional-gans/, accessed March 1, 2017)

         

        Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

         
        This site uses encryption for transmitting your passwords. ratmilwebsolutions.com