Username   Password       Forgot your password?  Forgot your username? 

Testability Metrics for Software Behavioral Models

Volume 13, Number 8, December 2017, pp. 1171-1182
DOI: 10.23940/ijpe.17.08.p1.11711182

Pan Liua,b

aCollege of Information and Computer, Shanghai Business School, Shanghai 201400, China
bShanghai Key Laboratory of Computer Software Testing & Evaluating, Shanghai 201112, China

(Submitted on October 29, 2017; Revised on November 12, 2017; Accepted on December 3, 2017)


Design for testability is one of the important research interests in software engineering. It becomes crucial in model-based testing because software behavioral models can be used to construct test sequences to perform conformance testing. However, testability of models has received less attention in the past. A model with high testability is easy to be used to construct effective test sequences, and conformance testing can also be realized easily. To improve testability of software behavioral models, using the formal method, we present five testability metrics: observability, controllability, test constructibility, performability, and error traceability. Then, a case is studied to evaluate the effectiveness of proposed testability metrics. As a result of the case study, models with higher testability can not only be used to generate executable test sequences, but also the size of the constructed test suite is also smaller. Our research will enrich the modeling theory of model-based testing and can improve the application of this test method in industry.


References: 34

      1. D. Amalfitano, A. R. Fasolino, P. Tramontana, B. D. Ta, and A. M. Memon, “MobiGUITAR: Automated Model-Based Testing of Mobile Apps,” IEEE Software, vol. 32, pp. 53-59, 2015
      2. P. Arcaini, A. Gargantini, and E. Riccobene, “Combining model-based testing and runtime monitoring for program testing in the presence of nondeterminism,” in proceedings of IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, pp. 178-187, Luxembourg, Luxembourg, March 2013
      3. B. Baudry and Y. L. Traon, “Measuring design testability of a UML class diagram,” Information and Software Technology, vol. 47, pp. 859-879, 2005
      4. B. Baudry, Y. Le Traon, and G. Sunyé, “Testability analysis of a UML class diagram,” in Proceedings of Eighth IEEE Symposium on Software Metrics, pp. 54-63, Ottawa, Canada, June 2002
      5. B. Baudry, Y. Le Traon, G. Sunyé, and J. Jezequel, “Measuring and improving design patterns testability,” in Proceedings of Ninth International on Software Metrics Symposium, pp. 50-59, Washington, DC, USA, September 2003
      6. A. Bertolino and L. Strigini, “On the use of testability measures for dependability assessment,” Software Engineering, IEEE Transactions on, vol. 22, pp. 97-108, 1996
      7. R. V. Binder, “Design for testability in object-oriented systems,” Communications of the ACM, vol. 37, pp. 87-101, 1994
      8. R. V. Binder, B. Legeard, and A. Kramer, “Model-based testing: where does it stand?,” Communications of the ACM, vol. 58, 2015
      9. M. Bruntink and A. van Deursen, “An empirical study into class testability,” Journal of Systems and Software, vol. 79, pp. 1219-1232, 2006
      10. S. R. Chidamber and C. F. Kemerer, “A metrics suite for object oriented design,” Software Engineering, IEEE Transactions on, vol. 20, pp. 476-493, 1994
      11. P. Clements, M. J. Escalona, P. Inverardi, et al. “Exploiting software architecture to support requirements satisfaction testing,” in Proceedings of the 19th ACM Sigsoft Symposium and the 13th European Conference on Foundations of Software Engineering, pp:484-487, Szeged, Hungary, September 2011
      12. H.-E. Eriksson and M. Penker, Business modeling with UML: Wiley Chichester, 2000
      13. N. E. Fenton and S. L. Pfleeger, Software metrics: a rigorous and practical approach: PWS Publishing Co., 1998
      14. R. S. Freedman, “Testability of software components,” Software Engineering, IEEE Transactions on, vol. 17, pp. 553-564, 1991
      15. A. Gonzalez-Sanchez, E. Piel, H.-G. Gross, and A. J. van Gemund, “Minimising the preparation cost of runtime testing based on testability metrics,” in Proceedings of IEEE 34th Annual in Computer Software and Applications Conference, 2010, pp. 419-424, Seoul, Korea, July 2010
      16. S. Jungmayr, “Testability measurement and software dependencies,” Shaker, 2002
      17. R. Lai, “A survey of communication protocol testing,” Journal of Systems and Software, vol. 62, pp. 21-46, 2002
      18. D. Lee and M. Yannakakis, “Principles and methods of testing finite state machines-a survey,” Proceedings of the IEEE, vol. 84, pp. 1090-1123, 1996
      19. P. Liu, H.-K. Miao, H.-W. Zeng, and Y. Liu, “FSM-based testing: Theory, method and evaluation,” Jisuanji Xuebao(Chinese Journal of Computers), vol. 34, pp. 965-984, 2011
      20. P. Liu, H. Miao, “Theory of Test Modeling Based on Regular Expressions,” Structured Object-Oriented Formal Language and Method. Springer International Publishing, pp:17-31, 2013
      21. P. Liu, H. Miao, H. Zeng, and L. Cai, “An Approach to Test Generation for Web Applications,” International Journal of u- and e- Service, Science and Technology, vol. 6, p. 16, 2013
      22. A. Kout, F. Toure, and M. Badri, “An empirical analysis of a testability model for object-oriented programs,” ACM SIGSOFT Software Engineering Notes, vol. 36, pp. 1-5, 2011
      23. T. J. McCabe, “A complexity measure,” Software Engineering, IEEE Transactions on, pp. 308-320, 1976
      24. B. A. Nejmeh, “NPATH: A measure of execution path complexity and its applications,” Communications of the ACM, vol. 31, pp. 188-200, 1988
      25. C. Schulze, D. Ganesan, M. Lindvall, R. Cleaveland, and D. Goldman, “Assessing model-based testing: an empirical study conducted in industry,” in Companion Proceedings of the 36th International Conference on Software Engineering, pp. 135-144, Hyderabad, India, May 2014
      26. Y.-N. Shen, F. Lombardi, and A. T. Dahbura, “Protocol conformance testing using multiple UIO sequences,” Communications, IEEE Transactions on, vol. 40, pp. 1282-1287, 1992
      27. J. W. Sheppard and M. Kaufman, “Formal specification of testability metrics in IEEE P1522,” in IEEE Autotestcon Proceedings of IEEE Systems Readiness Technology Conference , pp. 71-82, Valley Forge, PA, USA, August 2001
      28. Y. Shin, A. Meneely, L. Williams, and J. A. Osborne, “Evaluating complexity, code churn, and developer activity metrics as indicators of software vulnerabilities,” Software Engineering, IEEE Transactions on, vol. 37, pp. 772-787, 2011
      29. J. Offutt and A. Abdurazik, Generating tests from UML specifications: Springer, 1999
      30. Y. Le Traon, F. Ouabdesselam, and C. Robach, “Analyzing testability on data flow designs,” in Proceedings of the 11th International Symposium on Software Reliability Engineering, pp. 162-173, San Jose, CA, USA, Oct. 2000.
      31. J. M. Voas and K. W. Miller, “Semantic metrics for software testability,” Journal of Systems and Software, vol. 20, pp. 207-216, 1993
      32. J. M. Voas and K. W. Miller, “Software testability: The new verification,” IEEE software, vol. 12, pp. 17-28, 1995
      33. P.-L. Yeh and J.-C. Lin, “Software testability measurements derived from data flow analysis,” in Proceedings of the Second Euromicro Conference on Software Maintenance and Reengineering, pp. 96-102, Washington, DC, USA, March 1998
      34. L. Zhao, “A new approach for software testability analysis,” in Proceedings of the 28th international conference on Software engineering, pp. 985-988, Shanghai, China, May 2006.


          Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

          This site uses encryption for transmitting your passwords.