Testability Metrics for Software Behavioral Models

doi:10.23940/ijpe.17.08.p1.11711182

Abstract

Abstract: Design for testability is one of the important research interests in software engineering. It becomes crucial in model-based testing because software behavioral models can be used to construct test sequences to perform conformance testing. However, testability of models has received less attention in the past. A model with high testability is easy to be used to construct effective test sequences, and conformance testing can also be realized easily. To improve testability of software behavioral models, using the formal method, we present five testability metrics: observability, controllability, test constructibility, performability, and error traceability. Then, a case is studied to evaluate the effectiveness of proposed testability metrics. As a result of the case study, models with higher testability can not only be used to generate executable test sequences, but also the size of the constructed test suite is also smaller. Our research will enrich the modeling theory of model-based testing and can improve the application of this test method in industry.

Submitted on October 29, 2017; Revised on November 12, 2017; Accepted on December 3, 2017
References: 34

Pan Liu. Testability Metrics for Software Behavioral Models [J]. Int J Performability Eng, 2017, 13(8): 1171-1182.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References 0

	D. Amalfitano, A. R. Fasolino, P. Tramontana, B. D. Ta, and A. M. Memon, “MobiGUITAR: Automated Model-Based Testing of Mobile Apps,” IEEE Software, vol. 32, pp. 53-59, 2015
	P. Arcaini, A. Gargantini, and E. Riccobene, “Combining model-based testing and runtime monitoring for program testing in the presence of nondeterminism,” in proceedings of IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, pp. 178-187, Luxembourg, Luxembourg, March 2013
	B. Baudry and Y. L. Traon, “Measuring design testability of a UML class diagram,” Information and Software Technology, vol. 47, pp. 859-879, 2005
	B. Baudry, Y. Le Traon, and G. Sunyé, “Testability analysis of a UML class diagram,” in Proceedings of Eighth IEEE Symposium on Software Metrics, pp. 54-63, Ottawa, Canada, June 2002
	B. Baudry, Y. Le Traon, G. Sunyé, and J. Jezequel, “Measuring and improving design patterns testability,” in Proceedings of Ninth International on Software Metrics Symposium, pp. 50-59, Washington, DC, USA, September 2003
	A. Bertolino and L. Strigini, “On the use of testability measures for dependability assessment,” Software Engineering, IEEE Transactions on, vol. 22, pp. 97-108, 1996
	R. V. Binder, “Design for testability in object-oriented systems,” Communications of the ACM, vol. 37, pp. 87-101, 1994
	R. V. Binder, B. Legeard, and A. Kramer, “Model-based testing: where does it stand?,” Communications of the ACM, vol. 58, 2015
	M. Bruntink and A. van Deursen, “An empirical study into class testability,” Journal of Systems and Software, vol. 79, pp. 1219-1232, 2006
	S. R. Chidamber and C. F. Kemerer, “A metrics suite for object oriented design,” Software Engineering, IEEE Transactions on, vol. 20, pp. 476-493, 1994
	P. Clements, M. J. Escalona, P. Inverardi, et al. “Exploiting software architecture to support requirements satisfaction testing,” in Proceedings of the 19th ACM Sigsoft Symposium and the 13th European Conference on Foundations of Software Engineering, pp:484-487, Szeged, Hungary, September 2011
	H.-E. Eriksson and M. Penker, Business modeling with UML: Wiley Chichester, 2000
	N. E. Fenton and S. L. Pfleeger, Software metrics: a rigorous and practical approach: PWS Publishing Co., 1998
	R. S. Freedman, “Testability of software components,” Software Engineering, IEEE Transactions on, vol. 17, pp. 553-564, 1991
	A. Gonzalez-Sanchez, E. Piel, H.-G. Gross, and A. J. van Gemund, “Minimising the preparation cost of runtime testing based on testability metrics,” in Proceedings of IEEE 34th Annual in Computer Software and Applications Conference, 2010, pp. 419-424, Seoul, Korea, July 2010
	S. Jungmayr, “Testability measurement and software dependencies,” Shaker, 2002
	R. Lai, “A survey of communication protocol testing,” Journal of Systems and Software, vol. 62, pp. 21-46, 2002
	D. Lee and M. Yannakakis, “Principles and methods of testing finite state machines-a survey,” Proceedings of the IEEE, vol. 84, pp. 1090-1123, 1996
	P. Liu, H.-K. Miao, H.-W. Zeng, and Y. Liu, “FSM-based testing: Theory, method and evaluation,” Jisuanji Xuebao(Chinese Journal of Computers), vol. 34, pp. 965-984, 2011
	P. Liu, H. Miao, “Theory of Test Modeling Based on Regular Expressions,” Structured Object-Oriented Formal Language and Method. Springer International Publishing, pp:17-31, 2013
	P. Liu, H. Miao, H. Zeng, and L. Cai, “An Approach to Test Generation for Web Applications,” International Journal of u- and e- Service, Science and Technology, vol. 6, p. 16, 2013
	A. Kout, F. Toure, and M. Badri, “An empirical analysis of a testability model for object-oriented programs,” ACM SIGSOFT Software Engineering Notes, vol. 36, pp. 1-5, 2011
	T. J. McCabe, “A complexity measure,” Software Engineering, IEEE Transactions on, pp. 308-320, 1976
	B. A. Nejmeh, “NPATH: A measure of execution path complexity and its applications,” Communications of the ACM, vol. 31, pp. 188-200, 1988
	C. Schulze, D. Ganesan, M. Lindvall, R. Cleaveland, and D. Goldman, “Assessing model-based testing: an empirical study conducted in industry,” in Companion Proceedings of the 36th International Conference on Software Engineering, pp. 135-144, Hyderabad, India, May 2014
	Y.-N. Shen, F. Lombardi, and A. T. Dahbura, “Protocol conformance testing using multiple UIO sequences,” Communications, IEEE Transactions on, vol. 40, pp. 1282-1287, 1992
	J. W. Sheppard and M. Kaufman, “Formal specification of testability metrics in IEEE P1522,” in IEEE Autotestcon Proceedings of IEEE Systems Readiness Technology Conference , pp. 71-82, Valley Forge, PA, USA, August 2001
	Y. Shin, A. Meneely, L. Williams, and J. A. Osborne, “Evaluating complexity, code churn, and developer activity metrics as indicators of software vulnerabilities,” Software Engineering, IEEE Transactions on, vol. 37, pp. 772-787, 2011
	J. Offutt and A. Abdurazik, Generating tests from UML specifications: Springer, 1999
	Y. Le Traon, F. Ouabdesselam, and C. Robach, “Analyzing testability on data flow designs,” in Proceedings of the 11th International Symposium on Software Reliability Engineering, pp. 162-173, San Jose, CA, USA, Oct. 2000.
	J. M. Voas and K. W. Miller, “Semantic metrics for software testability,” Journal of Systems and Software, vol. 20, pp. 207-216, 1993
	J. M. Voas and K. W. Miller, “Software testability: The new verification,” IEEE software, vol. 12, pp. 17-28, 1995
	P.-L. Yeh and J.-C. Lin, “Software testability measurements derived from data flow analysis,” in Proceedings of the Second Euromicro Conference on Software Maintenance and Reengineering, pp. 96-102, Washington, DC, USA, March 1998
	L. Zhao, “A new approach for software testability analysis,” in Proceedings of the 28th international conference on Software engineering, pp. 985-988, Shanghai, China, May 2006.