A Survey of Software Trustworthiness Measurements

doi:10.23940/ijpe.19.09.p9.23642372

Abstract

Abstract: As people depend more and more on software, there exists an increasing demand for software trustworthiness. The quantification of software trustworthiness can not only help users choose the right software, but also provide evidence for increasing the trustworthiness of the design and implementation of software. It has important practical significance to software development and application. In this paper, software trustworthiness measurements are surveyed from two aspects of product and process. The advances in the product-oriented software trustworthiness measurements are summarized in the view of attributes and behaviors, and the research progress of the process-based software trustworthiness measurements is studied from the perspective of expert experience and process data. Lastly, the challenges of software trustworthiness measurements are analyzed．

Key words: software trustworthiness measurement, trustworthy attribute, trustworthy evidence, software behavior, software process

Hongwei Tao, Yixiang Chen, Hengyang Wu, and Rumei Deng. A Survey of Software Trustworthiness Measurements [J]. Int J Performability Eng, 2019, 15(9): 2364-2372.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. J. F. He, Z. G. Shan, J. Wang, G. G. Pu, Y. F. Fang, K. Liu, et al., “Review of the Achievements of Major Research Plan on Trustworthy Software,” Bulletin of National Natural Science Foundation of China, Vol. 32, No. 3, pp. 291-296, March 2018
2. S. B. Cai, Y. Z. Zou, L. S. Shao, B. Xie,W. Z. Shao, “Framework Supporting Software Assets Evaluation on Trustworthiness,” Journal of Software, Vol. 21, No. 2, pp. 359-372, February 2010
3. S. L. Yang, S. Ding,W. Chu, “Trustworthy Software Evaluation using Utility based Evidence Theory,” Journal of Computer Research and Development, Vol. 46, No. 7, pp. 1152-1159, July 2009
4. A. Avizienis, J. C. Laprie, B. Randell,C. Landwehr, “Basic Concepts and Taxonomy of Dependable and Secure Computing,” IEEE Transactions on Dependable and Secure Computing, Vol. 1, No. 1, pp. 11-33, January 2004
5. National Science and Technology Council, “Research Challenges in High Confidence Systems,” in Proceedings of the Committee on Computing Information and Communications Workshop, pp. 1-51, Virginia, USA, August 1997
6. H. W. Chen, J. Wang,W. Dong, “High Confidence Software Engineering Technologies,” Acta Electronica Sinica, Vol. 31, No. 12A, pp. 2-7, December 2003
7. F. B. Schneider, “Trust in Cyberspace,” National Academy Press, Washington, 1998
8. S. Becker, W. Hasselbring, A. Paul, M. Boskovic, H. Koziolek, J. Ploski, et al., “Trustworthy Software Systems: A Discussion of Basic Concepts and Terminology,” ACM SIGSOFT Software Engineering Notes, Vol. 31, No. 6, pp. 1-18, November 2006
9. B. Lang, X. D. Liu, H. M. Wang, B. Xie,X. G. Mao, “A Classification Model for Software Trustworthiness,” Journal of Frontiers of Computer Science and Technology, Vol. 4, No. 3, pp. 231-239, March 2010
10. H. W.Tao and Y. X. Chen, “A Quantitative Relation Model Between Trustworthy Attributes,” in Proceedings of the 3rd International Conference on Quantitative Logic and Soft Computing (QL&SC2012), pp. 197-204, Xi'an, China, April 2012
11. P. Oman and J. Hagemeister, “Metrics for Assessing a Software System Maintainability,” in Proceedings of the 1992 IEEE Conference on Software Maintenance, pp. 337-344, Orlando, USA, November 1992
12. P. Oman and J. Hagemeister, “Construction and Testing of Polynomials Predicting Software Maintainability,” Journal of System and Software, Vol. 24, No. 3, pp. 251-266, March 1994
13. K. Yiannis, H. Ilja, T. Christos,V. Joost, “Interpretation of Source Code Clusters in Terms of the ISO/IEC-9126 Maintainability Characteristics,” in Proceedings of 12th European Conference on Software Maintenance and Reengineering, pp. 63-72, Athens, Greece, April 2008
14. Y. M.Liu and I. Traore, “Properties for Security Measures of Software Products,” Applied Mathematics and Information Sciences, Vol. 1, No. 2, pp. 129-156, May 2007
15. C. Y. Huang, M. R. Lyu,S. Y. Kuo, “A Unified Scheme of Some Nonhomogeneous Poisson Process Models for Software Reliability Estimation,” IEEE Transactions on Software Engineering, Vol. 29, No. 3, pp. 261-269, April 2003
16. B. H. Wang, Y. X. Chen, S. Zhang,H. Y. Wu, “Updating Model of Software Component Trustworthiness based on Users Feedback”, IEEE Access, Vol. 7, pp. 1-8, January 2019
17. G. H. Shen, Z. Q. Huang, B. Xie, Y. Q. Zhu, L. L. Liao, F. Wang, et al., “Survey on Software Trustworthiness Evaluation,” Journal of Software, Vol. 27, No. 4, pp. 1-15, April 2016
18. L. Gu, Y. Guo, H. Wang, Y. Z. Zou, B. Xie,W. Z. Shao, “Runtime Software Trustworthiness Evidence Collection Mechanism based on TPM,” Journal of Software, Vol. 21, No. 2, pp. 373-387, February 2010
19. L. Tan, J. Chen,M. T. Zhou, “Trustworthiness Evidence Collection Mechanism of Running Dynamic Environment of Trusted Terminal,” Acta Electronica Sinica, Vol. 41, No. 1, pp. 77-85, January 2013
20. L. Yuan, H. M. Wang, G. Yin, D. X. Shi,H. B. Mi, “Trustworthy Evaluation Technology of Software based on Roles,” Journal of Beijing University of Technology, Vol. 36, No. 5, pp. 611-615, May 2010
21. L. Yuan, H. M. Wang, G. Yin, D. X. Shi,Y. X. Zhu, “An Automatic Evaluation Evidence Framework for Open Source Software,” Journal of Chinese Computer Systems, Vol. 31, No. 11, pp. 2145-2151, November 2011
22. G. H. Shen, Z. Q. Huang, J. Qian, Y. J. Xu, J. Hao, W. Y. Zhao, et al., “Research on Software Trustworthiness Evaluation Model and Its Implementation,” Journal of Frontiers of Computer Science and Technology, Vol. 5, No. 6, pp. 553-561, June 2011
23. G. Y.Shi and Y. F. Fu, “Establishing Evidence Model based on Multiple Attributes of Trustworthiness,” Journal of Guizhou Normal University (Natural Sciences), Vol. 29, No. 3, pp. 66-74, August 2011
24. X. L. Ding, H. M. Wang, Y. Y. Wang,G. Lu, “Verification Oriented Trustworthiness Evidence and Trustworthiness Evaluation of Software,” Journal of Frontiers of Computer Science and Technology, Vol. 4, No. 1, pp. 46-53, January 2010
25. G. Lu, H. M. Wang,X. G. Mao, “A Cognitive-based Evidence Model for Software Trustworthiness Evaluation,” Journal of Nanjing University (Natural Sciences), Vol. 46, No. 4, pp. 456-463, July 2010
26. H. L. Shi, J. Ma,F. Y. Zou, “A Fuzzy Comprehensive Evaluation Model for Software Dependability based on Entropy Weight,” in Proceedings of 2008 International Conference on Computer Science and Software Engineering, pp. 683-685, Hubei, China, December 2008
27. B. Li and Y. Cao, “An Improved Comprehensive Evaluation Model of Software Dependability based on Rough Set Theory,” Journal of Software, Vol. 4, No. 10, pp. 1152-1159, December 2009
28. L. Shi, S. L. Yang, K. Li,B. G. Yu, “Developing an Evaluation Approach for Software Trustworthiness using Combination Weights and TOPSIS,” Journal of Software, Vol. 7, No. 3, pp. 532-543, March 2012
29. S. Ding, S. L. Yang,C. Fu, “A Novel Evidential Reasoning based Method for Software Trustworthiness Evaluation under the Uncertain and Unreliable Environment,” Expert Systems with Applications, Vol. 39, No. 2012, pp. 2700-2709, August 2012
30. B. Wang, X. S. Zhou,Y. L. Yang, “An Entropy-Weighted Multi-Attribute Combination Method for Software Trustworthiness Classification Assessment,” Microelectronics and Computer, Vol. 31, No. 6, pp. 21-24, June 2014
31. Z. M. Zheng, S. L. Ma, W. Li, X. Jiang, W. Wei, L. L. Ma, et al., “Complexity of Software Trustworthiness and Its Dynamical Statistical Analysis Methods,” Science in China Series F: Information Science, Vol. 52, No. 9, pp. 1651-1657, September 2009
32. X. Zhang, W. Li, Z. M. Zheng,B. H. Guo, “Optimized Statistical Analysis of Software Trustworthiness Attributes,” SCIENCE CHINA Information Sciences, Vol. 55, No. 11, pp. 2508-2520, November 2012
33. H. W. Tao, “Research on the Measurement Models of Software Trustworthiness based on Attributes,” Ph. D. Dissertation, East China Normal University, Shanghai, China, 2011
34. H. W.Tao and Y. X. Chen, “A New Metric Model for Trustworthiness of Softwares,” Telecommunication System, Vol. 51, No. 2-3, pp. 95-105, November 2012
35. D. Taibi, “Towards a Trustworthiness Model for Open Source Software,” Ph. D. Dissertation, University of Insubria, Lombardy, 2010
36. G. N. Si, Y. H. Ren, J. Xu,J. F. Yang, “A Dependability Evaluation Model for Internetware based on Bayesian Network,” Journal of Computer Research and Development, Vol. 49, No. 5, pp. 1028-1038, May 2012
37. H. Q. Wang, H. W. Lv, Q. Zhao, X. K. Dong,G. S. Feng, “Model and Quantification of Autonomic Dependability of Mission-Critical Systems,” Journal of Software, Vol. 21, No. 2, pp. 344-358, February 2010
38. L. Chen, P. Cheng,W. Liu, “The Model and Method of Trustworthiness Level Evaluation for Software Product,” inProceedings of 2010 Sixth International Conference on Natural Computation (ICNC 2010), pp. 709-715, Yantai, China, August 2010
39. S. A. Hofmeyr, S. Forrest,A. Somayaji, “Intrusion Detection using Sequences of System Calls,” Journal of Computer Security, Vol. 6, No. 3, pp. 151-180, August 1998
40. R. Sekar, M. Bendre, D. Dhurjati,P. Bollineni, “A Fast Automaton-based Method for Detecting Anomalous Program Behaviors,” in Proceedings of the IEEE Symposium on Security and Privacy, pp. 144-155, Oakland, USA, February 2001
41. P. Li, H. Park, D. B. Gao,J. M. Fu, “Bridging the Gap Between Data-Flow and Control-Flow Analysis for Anomaly Detection,” inProceedings of the 2008 Annual Computer Security Applications, pp. 392-401, Anaheim, CA, USA, October 2008
42. D. Wagner and D. Dean, “Intrusion Detection via Static Analysis,” inProceedings of the IEEE Symposium on Security and Privacy, pp. 156-168, Oakland, CA, USA, May 2001
43. H. H. Feng, J. T. Giffin, Y. Huang, S. Jha, W. Lee,B. P. Miller, “Formalizing Sensitivity in Static Analysis for Intrusion Detection,” in Proceedings of the IEEE Symposium on Security and Privacy, pp. 194-208, Berkeley, CA, USA, May 2004
44. J. F. Tian, Z. Li,Y. L. Liu, “A Design Approach of Trustworthy Software and Its Trustworthiness Evaluation,” Journal of Computer Research and Development, Vol. 48, No. 8, pp. 1447-1454, August 2011
45. J. F. Tian, J. E. Han, R. Z. Du,Y. Wang, “Creditability Evaluation Model based on Software Behavior Trace,” Journal of Computer Research and Development, Vol. 49, No. 7, pp. 1514-1527, July 2012
46. M. Nami and W. Suryn, “From Requirements to Software Trustworthiness using Scenarios and Finite State Machine,” inProceedings of 38th Annual Conference on IEEE Industrial Electronics Society, pp. 3126-3131, Montreal, QC, Canada, December 2012
47. L. Zhuang, C. X. Shen,M. Cai, “Research on State Space Reduction of Behavior-based Trusted Dynamic Measurement,” Chinese Journal of Computers, Vol. 37, No. 5, pp. 1071-1081, May 2014
48. N. Fu, X. S. Zhou,T. Zhan, “QPi: A Calculus to Enforce Trustworthiness Requirements,” Journal of Computer Research and Development, Vol. 48, No. 11, pp. 2120-2130, November 2011
49. D. Wang, H. Y. Sun, J. Wang,L. H. Fu, “Trusted Analysis Model for Interactive Behavior of a Software System based on Slicing Technology,” Journal of Beijing University of Technology, Vol. 39, No. 5, pp. 713-721, May 2013
50. Y. L. Liu, R. Z. Du, J. L. Feng,J. F. Tian, “Trust Model Of Software Behaviors based On Check Point Risk Assessment,” Journal of Xidian University (Natural Science), Vol. 39, No. 1, pp. 179-184, February 2012
51. Y. Y.Yuan and Q. Han, “A Software Behavior Trustworthiness Measurement Method based on Data Mining,” International Journal of Computational Intelligence Systems, Vol. 48, No. 11, pp. 817-825, September 2011
52. J. Du, “Research on Software Process-Oriented Trustworthiness Evidence Model and Process Trustworthiness Evaluation Method,” Ph. D. Dissertation, University of Chinese Academy of Sciences, Beijing, China, 2012
53. E. Amoroso, T. Nguyen, J. Weiss, J. Watson, P. Lapiska,T. Starr, “Toward an Approach to Measuring Software Trust,” inProceedings of 1991 IEEE Symposium on Research in Security and Privacy, pp. 198-218, Oakland, USA, May 1991
54. Y. Yang, Q. Wang,M. S. Li, “Process Trustworthiness as a Capability Indicator for Measuring and Improving Software Trustworthiness,” inProceedings of the International Conference on Software Process (ICSP 2009), pp. 389-401, Vancouver, Canada, May 2009
55. J. Z. Zhan, “Influences and Control of Distrustable Factor on Software Trustworthiness,” Ph. D. Dissertation, Nanjing University, Nanjing, China, 2011
56. W. X. Zhang, W. H. Liu,X. Wu, “Quantitative Evaluation Across Software Development Life Cycle based on Data Fusion,” Computer Science, Vol. 40, No. 11A, pp. 192-195 , November 2013
57. H. B. Qian, X. J. Zhu,M. Z. Jin, “Research on Testing-based Software Credibility Measurement and Assessment,” in Proceedings of World Congress on Software Engineering (WCSE 2009), pp. 59-64, Xiamen, China, May 2009
58. H. W.Tao and Y. X. Chen, “A Survey of Software Trustworthiness Measurement Validation”, International Journal of Performability Engineering, Vol. 14, No. 9, pp. 2056-2065, September 2018
59. J. Wang, Y. X. Chen, B. Gu, X. Y. Guo, B. H. Wang, S. Y. Jin, et al., “An Approach to Measuring and Grading Software Trust for Spacecraft Software,” Scientia Sinica Techologica, Vol. 45, No. 2, pp. 221-228, February 2015
60. H. W.Tao and J. Zhao, “Source Codes Oriented Software Trustworthiness Measure based on Validation,” Mathematical Problems in Engineering, Vol. 2018, No. 3, pp. 1-10, November 2018

[1]	Jiamei Niu, Xuan Zhang, Ziqi Tang, Jingzhuan Zhao. System Dynamics Simulation of Global Software Development Process [J]. Int J Performability Eng, 2020, 16(2): 171-184.
[2]	Mengyue Wang, Yanfang Ma, Haiyu Pan, Hongwei Tao, and Liang Chen. Trade-off Costs of Software Trustworthy Attributes [J]. Int J Performability Eng, 2019, 15(9): 2422-2431.
[3]	Rumei Deng, Yixiang Chen, Hengyang Wu, and Hongwei Tao. Software Trustworthiness Evaluation using Structural Equation Modeling [J]. Int J Performability Eng, 2019, 15(10): 2628-2635.
[4]	Hongwei Tao and Yixiang Chen. A Survey of Software Trustworthiness Measurement Validation [J]. Int J Performability Eng, 2018, 14(9): 2056-2065.