Alternative Ranking Distance Metrics for Fault-Focused Clustering in Parallel Fault Localization

doi:10.23940/ijpe.23.10.p1.633643

Int J Performability Eng ›› 2023, Vol. 19 ›› Issue (10): 633-643.doi: 10.23940/ijpe.23.10.p1.633643

Alternative Ranking Distance Metrics for Fault-Focused Clustering in Parallel Fault Localization

Yihao Li^a, Pan Liu^b, W. Eric Wong^c,*, Nicholas Chau^c, and Chih-Wei Hsu^c

^aSchool of Information and Electrical Engineering, Ludong University, Yantai, China;
^bSchool of Business Information, Shanghai Business School, Shanghai, China;
^cDepartment of Computer Science, University of Texas at Dallas, Richardson, Texas, USA

Contact: * E-mail address: ewong@utdallas.edu
About author:Yihao Li is an associate professor of Ludong University. His research interests include software testing, software fault localization, and software quality assurance.
Pan Liu is a full professor of Shanghai Business School. His research interests include big data applications, software testing, formal methods, and algorithm design.
W. Eric Wong is a full professor of The University of Texas at Dallas. His research interests include software testing, debugging, risk analysis/metrics, safety, and reliability.
Nicholas Chau is an REU student in Computer Science at the University of Texas at Dallas supported by the U.S. National Science Foundation under Grant 2050869.
Chih-Wei Hsu is a PhD student in Computer Science at the University of Texas at Dallas. His research interests include software testing, debugging, risk analysis/metrics, safety, and reliability.

Abstract

Abstract: Generating fault-focused clusters is a common practice used in parallel fault localization where fault-focused rankings that are likely leading to the same firstly identified faulty statement are grouped together. With respect to the performance of fault-focused clustering and fault localization cost, one critical impact factor is the metric used to measure the distance between two rankings. Current work prefers using Kendall tau distance for its fitness in computing the ranking disagreement. In this paper, we tend to apply two other well-established ranking distance metrics, Spearman’s Footrule and a set intersection-based measure, to fault-focused clustering and cross compare the parallel fault location performance of using these three distinct distance metrics.

Key words: fault-focused clustering, parallel fault localization, ranking distance

Yihao Li, Pan Liu, W. Eric Wong, Nicholas Chau, and Chih-Wei Hsu. Alternative Ranking Distance Metrics for Fault-Focused Clustering in Parallel Fault Localization [J]. Int J Performability Eng, 2023, 19(10): 633-643.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] Vessey I.Expertise in Debugging Computer Programs: A Process Analysis. International Journal of Man-Machine Studies, vol. 23, no. 5, pp. 459-494, 1985.
[2] Abreu R., Zoeteweij P., Golsteijn R., andVan Gemund, A.J. A Practical Evaluation of Spectrum-Based Fault Localization. Journal of Systems and Software, vol. 82, no. 11, pp. 1780-1792, 2009.
[3] Abreu R., Zoeteweij P., andVan Gemund, A.J. Spectrum-Based Multiple Fault Localization. In2009 IEEE/ACM International Conference on Automated Software Engineering, IEEE, pp. 88-99, 2009.
[4] Jones, J.A., Bowring, J.F., and Harrold, M.J. Debugging in Parallel. In Proceedings of the 2007 international symposium on Software testing and analysis, pp. 16-26, 2007.
[5] Jones, J.A. and Harrold, M.J.Empirical Evaluation of the Tarantula Automatic Fault-Localization Technique. InProceedings of the 20th IEEE/ACM international Conference on Automated software engineering, pp. 273-282, 2005.
[6] Liu C., Fei L., Yan X., Han J., andMidkiff S.P.Statistical Debugging: A Hypothesis Testing-Based Approach. IEEE Transactions on software engineering, vol. 32, no. 10, pp. 831-848, 2006.
[7] Naish L., Lee H.J., andRamamohanarao K.A Model for Spectra-Based Software Diagnosis. ACM Transactions on software engineering and methodology (TOSEM), vol. 20, no. 3, pp. 1-32, 2011.
[8] Wong, W.E. and Mathur, A.P.Fault Detection Effectiveness of Mutation and Data Flow Testing. Software Quality Journal, vol. 4
pp. 69-83, 1995.
[9] Wong W.E., Debroy V., Gao R., andLi Y.The DStar Method for Effective Software Fault Localization. IEEE Transactions on Reliability, vol. 63, no. 1, pp. 290-308, 2013.
[10] Wong W.E., Shi Y., Qi Y., andGolden, R. using an RBF Neural Network to Locate Program Bugs. In2008 19th International Symposium on Software Reliability Engineering (ISSRE), IEEE, pp. 27-36, 2008.
[11] Wong W.E., Debroy V., andXu D.Towards Better Fault Localization: A Crosstab-Based Statistical Approach. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 3, pp. 378-396, 2011.
[12] Xie X., Chen T.Y., Kuo F.C., andXu B.A Theoretical Analysis of the Risk Evaluation Formulas for Spectrum-Based Fault Localization. ACM Transactions on software engineering and methodology (TOSEM), vol. 22, no. 4, pp. 1-40, 2013.
[13] Xie X., Wong W.E., Chen T.Y., andXu B.Spectrum-Based Fault Localization: Testing Oracles Are No Longer Mandatory. In2011 11th International Conference on Quality Software, IEEE, pp. 1-10, 2011.
[14] Xu X., Debroy V., Eric Wong, W., and Guo, D. Ties Within Fault Localization Rankings: Exposing and Addressing the Problem. International Journal of Software Engineering and Knowledge Engineering, vol. 21, no. 06, pp. 803-827, 2011.
[15] Liu, C. and Han, J.Failure Proximity: A Fault Localization-Based Approach. InProceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering , pp. 46-56, 2006.
[16] Kendall, M. G. and Gibbons, J. D.Rank Correlation Methods.Edward Arnold, 1990.
[17] Xing E., Jordan M., Russell S.J. and Ng A.Distance Metric Learning, With Application to Clustering with Side-Information.Advances in Neural Inforamtion Processings System, pp 505-512, 2003.
[18] Gao, R. and Wong, W.E.MSeer—An Advanced Technique for Locating Multiple Bugs in Parallel. IEEE Transactions on Software Engineering, vol. 45, pp. 301-318, 2019.
[19] Spearman C.The Proof and Measurement of Association between Two Things. The American Journal of Psychology, vol. 15, no. 1, pp. 72-101, 1904.
[20] Agrawal R.,Comparing Ranked List, https://ragrawal.wordpress.com/2013/01/18/comparing-ranked-list/, accessed on November 11, 2023.
[21] Li, Y. and Liu, P.A Preliminary Investigation on the Performance of SBFL Techniques and Distance Metrics in Parallel Fault Localization. IEEE Transactions on Reliability, vol. 71, no. 2, pp. 803-817, 2022.
[22] Kaufman, L. and Rousseeuw, P.J.Finding Groups in Data: An Introduction to Cluster Analysis. Wiley Interscience, New York, 1990.
[23] Software-artifact Infrastructure Repository, https://sir.csc.ncsu.edu/portal/index.php, accessed on November 21, 2023.
[24] Debroy, V. and Wong, W.E.Insights on Fault Interference for Programs with Multiple Bugs. In2009 20th International Symposium on Software Reliability Engineering, IEEE, pp. 165-174, 2009.
[25] DiGiuseppe, N. and Jones, J.A. on the Influence of Multiple Faults on Coverage-Based Fault Localization. In Proceedings of the 2011 international symposium on software testing and analysis, pp. 210-220, 2011.
[26] Xiaobo Y., Bin L., andShihai W.How Negative Effects a Multiple-Fault Program Can Do to Spectrum-Based Fault Localization. In2019 Prognostics and System Health Management Conference (PHM-Qingdao), IEEE, pp. 1-6, 2019.
[27] Yu Z., Bai C., andCai K.Y.Does the Failing Test Execute a Single or Multiple Faults? An approach to classifying failing tests. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, IEEE, vol. 1, pp. 924-935, 2015.
[28] Lamraoui, S.M. and Nakajima, S.A Formula-Based Approach for Automatic Fault Localization of Multi-Fault Programs. Journal of Information Processing, vol. 24, no. 1, pp. 88-98, 2016.
[29] Song Y., Xie X., Zhang X., Liu Q., andGao R.Evolving Ranking-Based Failure Proximities for Better Clustering in Fault Isolation. InProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, pp. 1-13, 2022.
[30] Steimann, F. and Frenkel, M.Improving Coverage-Based Localization of Multiple Faults using Algorithms from Integer Linear Programming. In2012 IEEE 23rd International Symposium on Software Reliability Engineering, IEEE, pp. 121-130, 2012.
[31] Zakari A., Lee S.P., andChong C.Y.Simultaneous Localization of Software Faults based on Complex Network Theory. IEEE Access, vol. 6, pp. 23990-24002, 2018.
[32] Zakari A., Lee S.P., andHashem I.A.T. A Community-Based Fault Isolation Approach for Effective Simultaneous Localization of Faults.IEEE Access, vol. 7, pp. 50012-50030, 2019.
[33] Song Y., Xie X., Liu Q., Zhang X., andWu X.A Comprehensive Empirical Investigation on Failure Clustering in Parallel Debugging. Journal of Systems and Software, vol. 193, pp. 111452, 2022.
[34] Podgurski, A., Leon, D., Francis, P., Masri, W., Minch, M., Sun, J., and Wang, B. Automated Support for Classifying Software Failure Reports. In 25th International Conference on Software Engineering, 2003. Proceedings, IEEE, pp. 465-475, 2003.

Alternative Ranking Distance Metrics for Fault-Focused Clustering in Parallel Fault Localization

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 0

Recommended 0