A Survey of the Inadequacies in Traffic Sign Recognition Systems for Autonomous Vehicles

doi:10.23940/ijpe.20.10.p10.15881597

Int J Performability Eng ›› 2020, Vol. 16 ›› Issue (10): 1588-1597.doi: 10.23940/ijpe.20.10.p10.15881597

Previous Articles Next Articles

A Survey of the Inadequacies in Traffic Sign Recognition Systems for Autonomous Vehicles

Angelica F. Magnussen^a, Nathan Le^b, Linghuan Hu^c, and W. Eric Wong^c,*

^aCollege of Engineering, University of Texas at Arlington, Arlington, TX, 76019, USA;
^bCollege of Science and Engineering, Texas A&M University, Corpus Christi, TX, 78412, USA;
^cComputer Science Department, University of Texas at Dallas, Richardson, TX, 75080, USA

Submitted on ; Revised on ; Accepted on
Contact: * E-mail address: ewong@utdallas.edu
About author:Angelica F. Magnussen is a B.S. student in computer science at the University of Texas at Arlington. Her research interests include software engineering, software testing, artificial intelligence, and distributed systems. Contact her at angelica.magnussen@mavs.uta.edu.
Nathan Le is a B.S. student in Computer Science at Texas A&M University Corpus Christi. His research interests include artificial intelligence and autonomous vehicles. Contact him at nle10@islander.tamucc.edu.
Linghuan Hu is a Ph.D. student in software engineering at the University of Texas at Dallas. His research interests include combinatorial testing and test generation. Hu received a M.S. in software engineering from the University of Texas at Dallas. Contact him at linghuan.hu@utdallas.edu.
W. Eric Wong is a professor and the director of the Advanced Research Center for Software Testing and Quality Assurance in Computer Science at the University of Texas at Dallas. His research focuses on helping practitioners improve the quality of software while reducing the cost of production.
Wong received a Ph.D. in computer science from Purdue University, West Lafayette, Indiana. He is the Editor-in-chief of IEEE Transactions on Reliability and the corresponding author for this article. Contact him at ewong@utdallas.edu.

Abstract

Abstract: Traffic sign recognition systems are crucial for autonomous vehicles. They assist autonomous driving systems by collecting road-related information, such as speed limits, stop signs, etc., that are necessary for safe driving. However, as evidenced by recent autonomous vehicle crashes and recognition system failure-related studies, there are serious concerns about the inadequacies of the traffic sign recognition systems and their used techniques. In response to the industrial needs and to help practitioners improve the reliability and safety of the traffic sign recognition systems, this paper discusses the general architectural outline of traffic sign recognition systems and the challenges that must be overcome, in order for traffic sign recognition systems to be safe and reliable. An in-depth discussion of various solutions is given to provide practitioners valuable insight into the improvement of traffic sign recognition systems.

Key words: autonomous vehicles, traffic sign recognition system, software safety

Angelica F. Magnussen, Nathan Le, Linghuan Hu, and W. Eric Wong. A Survey of the Inadequacies in Traffic Sign Recognition Systems for Autonomous Vehicles [J]. Int J Performability Eng, 2020, 16(10): 1588-1597.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. S. Singh, “Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey,” Traffic Safety Facts Crash Stats, Report No. DOT HS 812 115, National Highway Traffic Safety Administration, Washington, DC, February 2015, (https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812115)
2. P. McCausland, “Self-Driving Uber Car that Hit and Killed Woman Did Not Recognize that Pedestrians Jaywalk,” NBCNews.com, NBCUniversal News Group, 11 Nov. 2019 (www.nbcnews.com/tech/tech-news/self-driving-uber-car-hit-killed-woman-did-not-recognize-n1079281)
3. B. Vlasic and Neal E. Boudette, “Self-Driving Tesla Was Involved in Fatal Crash, U.S. Says,” The New York Times, 30 June 2016, (www.nytimes.com/2016/07/01/business/self-driving-tesla-fatal-crash-investigation.html)
4. D. R.Bruno and F. Santos Osorio, “A Comparison of Traffic Signs Detection Methods in 2D and 3D Images for the Benefit of the Navigation of Autonomous Vehicles,” inProceedings of 2018 Latin American Robotic Symposium, 2018 Brazilian Symposium on Robotics (SBR) and 2018 Workshop on Robotics in Education (WRE), pp. 26-32, João Pessoa, 2018, doi: 10.1109/LARS/SBR/WRE.2018.00015
5. D. R.Bruno and F. S. Osorio, “Image Classification System based on Deep Learning Applied to the Recognition of Traffic Signs for Intelligent Robotic Vehicle Navigation Purposes,” inProceedings of 2017 Latin American Robotics Symposium (LARS) and 2017 Brazilian Symposium on Robotics (SBR), pp. 1-6, Curitiba, 2017, doi: 10.1109/SBR-LARS-R.2017.8215287
6. A. Youssef, D. Albani, D. Nardi,D. D. Bloisi, “Fast Traffic Sign Recognition using Color Segmentation and Deep Convolutional Networks,” inProceedings of International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 205-216, 2016, doi:10.1007/978-3-319-48680-2_19
7. L. K.Wahyono and K. Jo, “Traffic Sign Recognition and Tracking for a Vision-based Autonomous Vehicle using Optimally Selected Features,” inProceedings of 2015 54th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), pp. 1419-1422, Hangzhou, 2015, doi: 10.1109/SICE.2015.7285415
8. R. Kulkarni, S. Dhavalikar,S. Bangar, “Traffic Light Detection and Recognition for Self Driving Cars using Deep Learning,” inProceedings of 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), pp. 1-4, Pune, India, 2018, doi: 10.1109/ICCUBEA.2018.8697819
9. B. Sanyal, R. K. Mohapatra,R. Dash, “Traffic Sign Recognition: A Survey,” inProceedings of 2020 International Conference on Artificial Intelligence and Signal Processing (AISP), pp. 1-6, Amaravati, India, 2020, doi: 10.1109/AISP48273.2020.9072976
10. J. W. Cao, C. X. Song, S. L. Peng, F. Xiao,S. X. Song, “Improved Traffic Sign Detection and Recognition Algorithm for Intelligent Vehicles,” Sensors, Vol. 19, No. 18, pp. 4021, 2019, doi:10.3390/s19184021
11. J. Rezgui, A. Hbaieb, L. Chaari,J. Maryland, “Traffic Sign Recognition using Neural Networks Useful for Autonomous Vehicles,” inProceedings of 2019 International Conference on Smart Applications, Communications and Networking (SmartNets), pp. 1-6, Sharm El Sheik, Egypt, 2019, doi: 10.1109/SmartNets48225.2019.9069770
12. H. Lengyel and Z. Szalay, “Traffic Sign Anomalies and Their Effects to the Highly Automated and Autonomous Vehicles,” inProceedings of 35^th International Colloquium on Advanced Manufacturing and Repair Technologies in Vehicle Industry, pp. 193-204, Zielona Góra, 2018
13. S. Zang, M. Ding, D. Smith, P. Tyler, T. Rakotoarivelo,M. A. Kaafar, “The Impact of Adverse Weather Conditions on Autonomous Vehicles: How Rain, Snow, Fog, and Hail Affect the Performance of a Self-Driving Car,” IEEE Vehicular Technology Magazine, Vol. 14, No. 2, pp. 103-111, June 2019, doi: 10.1109/MVT.2019.2892497
14. X. Zhao, P. Sun, Z. Xu, H. Min,H. Yu, “Fusion of 3D LIDAR and Camera Data for Object Detection in Autonomous Vehicle Applications,” IEEE Sensors Journal, Vol. 20, No. 9, pp. 4901-4913, 1 May, 2020, doi: 10.1109/JSEN.2020.2966034
15. Uniqtech, “Multilayer Perceptron (MLP) vs Convolutional Neural Network in Deep Learning,” Medium, Data Science Bootcamp, 13 June 2019, (medium.com/data-science-bootcamp/multilayer-perceptron-mlp-vs-convolutional-neural-network-in-deep-learning-c890f487a8f1)
16. S. P. Kaarmukilan, S. Poddar,A. Thomas K., “FPGA based Deep Learning Models for Object Detection and Recognition Comparison of Object Detection Comparison of Object Detection Models using FPGA,” inProceedings of 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC), pp. 471-474, Erode, India, 2020, doi: 10.1109/ICCMC48092.2020.ICCMC-00088
17. V. Swaminathan, S. Arora, R. Bansal,R. Rajalakshmi, “Autonomous Driving System with Road Sign Recognition using Convolutional Neural Networks,” inProceedings of 2019 International Conference on Computational Intelligence in Data Science (ICCIDS), pp. 1-4, Chennai, India, 2019, doi: 10.1109/ICCIDS.2019.8862152
18. A. V.Deshpande and M. M. Subashini, “A Survey of Different Image Processing Methods for the Design and Development of an Efficient Traffic Sign Board Recognition System,” inProceedings of 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information and Communication Technology (RTEICT), pp. 340-343, Bangalore, 2017, doi: 10.1109/RTEICT.2017.8256612
19. S. Ray, “A Quick Review of Machine Learning Algorithms,” inProceedings of 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), pp. 35-39, Faridabad, India, 2019, doi: 10.1109/COMITCon.2019.8862451
20. R. Hasegawa, Y. Iwamoto,Y. Chen, “Robust Detection and Recognition of Japanese Traffic Sign in the Complex Scenes based on Deep Learning,” inProceedings of 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), pp. 575-578, Osaka, Japan, 2019, doi: 10.1109/GCCE46687.2019.9015419
21. D. Tabernik and D. Skočaj, “Deep Learning for Large-Scale Traffic-Sign Detection and Recognition,” IEEE Transactions on Intelligent Transportation Systems, Vol. 21, No. 4, pp. 1427-1440, April 2020, doi: 10.1109/TITS.2019.2913588
22. Z. Zhu, J. Lu, R. R. Martin,S. Hu, “An Optimization Approach for Localization Refinement of Candidate Traffic Signs,” IEEE Transactions on Intelligent Transportation Systems, Vol. 18, No. 11, pp. 3006-3016, November 2017, doi: 10.1109/TITS.2017.2665647
23. Eykholt, Kevin, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno,Dawn Song. "Robust physical-world attacks on deep learning visual classification." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1625-1634. 2018.
24. M. Khalilikhah, K. Heaslip,K. Hancock, “Traffic Sign Vandalism and Demographics of Local Population: A Case Study in Utah,” Journal of Traffic and Transportation Engineering (English Edition), Vol. 3, No. 3, pp. 192-202, 2016, doi:10.1016/j.jtte.2015.11.001
25. M. Khalilikhah and K. Heaslip, “The Effects of Damage on Sign Visibility: An Assist in Traffic Sign Replacement,” Journal of Traffic and Transportation Engineering (English Edition), Vol. 3, No. 6, pp. 571-581, 2016, doi:10.1016/j.jtte.2016.03.009
26. H. Lengyel and Z. Szalay, “Test Scenario for Road Sign Recognition Systems with Special Attention on Traffic Sign Anomalies,” inProceedings of 2019 IEEE 19th International Symposium on Computational Intelligence and Informatics and 7th IEEE International Conference on Recent Achievements in Mechatronics, Automation, Computer Sciences and Robotics (CINTI-MACRo), pp. 000193-000198, Szeged, Hungary, 2019, doi: 10.1109/CINTI-MACRo49179.2019.9105238
27. R. Majumdar, A. Mathur, M. Pirron, L. Stegner,D. Zufferey, “Paracosm: A Language and Tool for Testing Autonomous Driving Systems,” arXiv: 1902.01084, February 2019
28. L. Jöckel, M. Kläs,S. Martínez-Fernández, “Safe Traffic Sign Recognition through Data Augmentation for Autonomous Vehicles Software,” inProceedings of 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C), pp. 540-541, Sofia, Bulgaria, 2019, doi: 10.1109/QRS-C.2019.00114
29. J. Su, D. V. Vargas,K. Sakurai, “One Pixel Attack for Fooling Deep Neural Networks,” IEEE Transactions on Evolutionary Computation, Vol. 23, No. 5, pp. 828-841, October 2019, doi: 10.1109/TEVC.2019.2890858
30. A. V. Deshpande, R. Kannan,M. M. Subashini, “Study of Various Image De-Noising Methods Used for the Purpose of Traffic Sign Board Recognition in an Intelligent Advanced Driver Assistance System,” inProceedings of 2018 International Conference on Intelligent and Advanced System (ICIAS), pp. 1-6, Kuala Lumpur, 2018, doi: 10.1109/ICIAS.2018.8540630
31. A. V.Deshpande and M. M. Subashini, “An Investigative Approach Towards Various Image Segmentation Algorithms Used for Traffic Sign Recognition,” inProceedings of 2017 Fourth International Conference on Image Information Processing (ICIIP), pp. 1-6, Shimla, 2017, doi: 10.1109/ICIIP.2017.8313693
32. A. Barodi, A. Bajit, M. Benbrahim,A. Tamtaoui, “An Enhanced Approach in Detecting Object Applied to Automotive Traffic Roads Signs,” inProceedings of 2020 IEEE 6th International Conference on Optimization and Applications (ICOA), pp. 1-6, Beni Mellal, Morocco, 2020, doi: 10.1109/ICOA49421.2020.9094457

A Survey of the Inadequacies in Traffic Sign Recognition Systems for Autonomous Vehicles

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 0

Recommended 0