Username   Password       Forgot your password?  Forgot your username? 


3D Convolutional Neural Network for Semantic Scene Segmentation based on Unstructured Point Clouds

Volume 14, Number 7, July 2018, pp. 1503-1512
DOI: 10.23940/ijpe.18.07.p14.15031512

Rui Zhanga,b, Yan Wangc, Guangyun Lib, Zhen Hana, Junpeng Lia, and Chunying Lia

aNorth China University of Water Resources and Electric Power, Zhengzhou, 450045, China
bInformation Engineering University, Zhengzhou, 450052, China
cZhengzhou Institute of Technology, Zhengzhou, 450044, China

(Submitted on March 19, 2018; Revised on April 23, 2018; Accepted on June 13, 2018)


The use of point cloud datasets is an inevitable trend in the analysis of natural scenes. In this paper, we propose a semantic segmentation network architecture that consumes 3D point clouds directly, which can efficiently avoid mapping 3D point clouds to 2D images. Experimental results indicate strong performance that is on par with or even better than state-of-the-art methods for semantic segmentation on the Stanford semantic parsing dataset.


References: 38

          1. J. M. Alvarez, T. Gevers, Y. LeCun, and A. M. Lopez, “Road Scene Segmentation from a Single Image,” European Conference on Computer Vision. Springer, pp. 376–389, 2012.
          2. N. H. Arachchige, H. G. Maas, “Automatic Building Facade Detection in Mobile Laser Scanner Point Clouds,” In. The German Society for Photogrammetry, Remote Sensing and Geoinformation (DGPF), Potsdam, Germany.2012.
          3. N. H. Arachchige, S. N. Perera, H. G. Maas. “Automatic Processing of Mobile Laser Scanner Point Clouds for Building Facade Detection,” ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XXXIX-B5:187-192, 2012.
          4. I. Armeni, A. Sax, A. R. Zamir, and S. Savarese, “Joint 2D-3D Semantic Data for Indoor Scene Understanding,” ArXiv e-prints, Feb. 2017.
          5. X. Y. Ai, L. Y. Wang. “Extraction of Planar Characteristics of Airborne LiDAR Point Cloud Data,” Journal of Liaoning Technical University:Natural Science,34(2):212-216, 2015.
          6. J. M. Biosca, J. L. Lerma. “Unsupervised Robust Planar Segmentation of Terrestrial Laser Scanner Point Clouds based on Fuzzy Clustering Method,” ISPRS Journal of Photogrammetry & Remote Sensing, 63:84-98, 2008.
          7. J. P. Burochin, B. Vallet, M. Bredif, et al. “Detecting Blind Building Facades from Highly Overlapping Wide Angle Aerial Imagery,” ISPRS Journal of Photogrammetry and Remote Sensing. 96:193-209, 2014.
          8. C. Chen, W. Ke, X. U. Wenxue, et al. “Real-Time Visualizing of Massive Vehicle-Borne Laser Scanning Point Clouds,” Geomatics & Information Science of Wuhan University, 40(9):1163-1168, 2015.
          9. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision Meets Robotics: The Kitti Dataset,” The International Journal of Robotics Research, vol. 32, no. 11, pp. 1231–1237, 2013.
          10. T. Hackel, N. Savinov, L. Ladicky and Jan D. Wegner and K. Schindler and M. Pollefeys, “SEMANTIC3D.NET: A New Large-Scale Point Cloud Classification Benchmark,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-1-W1: 91-98, 2017.
          11. J. Huang, S. You, “Point Cloud Labeling Using 3D Convolutional Neural Network,” International Conference on Pattern Recognition. IEEE, 2017.
          12. B. Li. “3D Fully Convolutional Network for Vehicle Detection in Point Cloud,” Computer Vision and Pattern Recognition. 2017. arXiv:1611.08069 23
          13. Y. Y. Li, R. Bu, M. C. Sun, B. Q. Chen. “PointCNN,” /abs/1801.07791, 2018.
          14. M. D. Li, S. P. Jiang, H. P. Wang. “A RANSCA-Based Stable Plane Fitting Method of Point Clouds,” Science of Surveying and Mapping. 40(1), pp:102 – 106, 2015.
          15. M. L. Li, G. Y. Lin, L. Wang, et al. “Automatic Feature Detecting from Point Clouds Using 3D Hough Transform,” Bulletin of Surveying and Mapping, (2):29-33, 2015.
          16. N. Li, Y. W. Ma, Y. Tang and S. L. Gao. “Segmentation of Building Facade Point Cloud Using RANSAC,” Science of Surveying and Mapping. 36(5): 144-146. 2011.
          17. Y. N. Lin, W. Wei. “Research on Algorithm of Object Tracking based on Generalized Hough Transform,” ZheJiang University, 2013.
          18. Y. Liu, F. Wang, A. M. Dobaie, et al., “Comparison of 2D Image Models in Segmentation Performance for 3D Laser Point Clouds,” Neurocomputing, 251, 2017.
          19. D. Meagher, “Geometric Modeling Using Octree Encoding,” Computer Graphics and Image Processing, 19(2): 129-147, 1982.
          20. D. Maturana, S. Scherer. “3D Convolutional Neural Networks for Landing Zone Detection from LiDAR,” IEEE International Conference on Robotics and Automation. IEEE, pp:3471-3478, 2015.
          21. C. R. Qi, H. Su, K. Mo, and L. J. Guibas. “Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation,” arXiv preprint arXiv:1612.00593, 2016.
          22. A. Quadros, J. Underwood, and B. Douillard, “An Occlusion-Aware Feature for Range Images,” Robotics and Automation, ICRA’12. IEEE International Conference on, May pp:14-18 2012.
          23. C. R. Qi, L. Yi., H. Su, & Guibas, L. J. “Pointnet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space,” 2017 arXiv:1706.02413v1
          24. G. Ros, J. M. Alvarez. “Unsupervised Image Transformation for Outdoor Semantic Labeling,” Intelligent Vehicles Symposium. IEEE, pp:537-542, 2015.
          25. R. Richter, M. Behrens, J. Döllner. “Object Class Segmentation of Massive 3D Point Clouds of Urban Areas Using Point Cloud Topology,” International Journal of Remote Sensing. Vol. 34, No. 23, 8408–8424, 2013.
          26. G. Ros, S. Ramos, M. Granados, A. Bakhtiary, D. Vazquez, and A. M. Lopez, “Vision-Based Offline-Online Perception Paradigm for Autonomous Driving,” Applications of Computer Vision (WACV), 2015 IEEE Winter Conference on. IEEE, pp. 231- 238, 2015.
          27. K. Simonyan and A. Zisserman. “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv preprint arXiv:1409.1556, 2014.
          28. B. Song, Z. Wang and L. Sheng, “A New Generic Algorithm Approach to Smooth Path Planning for Mobile Robots,” Assembly Automation, Vol. 36, No. 2, Apr. 2016, pp. 138-145.
          29. B. Song, Z. Wang, L. Zou, “On Global Smooth Path Planning for Mobile Robots using a Novel Multimodal Delayed PSO Algorithm,” Cognitive Computation, 9(1):1-13, 2017.
          30. O. Vinyals, S. Bengio, and M. Kudlur. “Order matters: Sequence to Sequence for Sets,” arXiv preprint arXiv:1511.06391, 2015.
          31. P. Vracar, I. Kononenko, and M. Robnik-Sikonja. “Obtaining Structural Descriptions of Building Facades,” Computer Science and Information Systems 13(1):23–43, 2015.
          32. Y. M. Wang, M. Guo. “A Combined 2D and 3D Spatial Indexing of Very Large Point-cloud Data Sets,” Acta Geodaetica et Cartographica Sinica,41(4):605-612, 2012.
          33. J. S. Yang. “A Method of Combing the Model of the Global Quadtree Index with Local KD-tree for Massive Airborne LiDAR Point Cloud Data Organization,” Geomatics and Information Science of Wuhan University,39(8):918-922, 2014.
          34. J. S. Yang, X. F. Huang. “A Hybrid Spatial Index for Massive Point Cloud Data Management and Visualization,” Transaction in GIS, 18(S1):97-108, 2014.
          35. B. S. Yang, F. X. Liang, R. G. Huang. Progress, “Challenges and Perspectives of 3D LiDAR Point Cloud Processing,” Acta Geodaetic et Cartographica Sinica, 46(10):1509-1516.DOI: 10.11947/j.AGCS.2017.20170351, 2017.
          36. R. Zhang, S. A. Candra, K. Vetter, and A. Zakhor, “Sensor Fusion for Semantic Segmentation of Urban Scenes,” Robotics and Automation (ICRA), IEEE International Conference on. IEEE, pp. 1850-1857, 2015.
          37. R. Zhang, G. Y. Li, L. Wang, M. L. Li et al., “New Method of Hybrid Index for Mobile LiDAR Point Cloud Data,” Geomatics and Information Science of Wuhan University, 2017. DOI: 10.13203/j.whugis20160441
          38. D. Y. Zhang, W. Q. Wu, M. P. Wu, et al. “Plane Landmark Detection from Lidar Data Based on3D Hough Transform,” Journal of National University of Defense Technology, 32(2):130-134, 2010.


                  Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

                  Download this file (14-IJPE-2018-07-14.pdf)IJPE-2018-07-14[3D Convolutional Neural Network for Semantic Scene Segmentation based on Unstructured Point Clouds]981 Kb
                  This site uses encryption for transmitting your passwords.