A Comparative Study on Similarity Measure Techniques for Cross-Project Defect Prediction
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Software defect prediction is helpful for allocating valuable project resources effectively for software quality assurance activities thanks to focusing on the identified fault-prone modules. If historical data collected within a company is sufficient, a Within-Project Defect Prediction (WPDP) can be utilized for accurate fault-prone module prediction. In case a company does not maintain historical data, it may be helpful to build a classifier towards predicting comprehensible fault prediction based on Cross-Project Defect Prediction (CPDP). Since CPDP employs different project data collected from other organization to build a classifier, the main obstacle to build an accurate classifier is that distributions between source and target projects are not similar. To address the problem, because it is crucial to identify effective similarity measure techniques to obtain high performance for CPDP, In this paper, we aim to identify them. We compare various similarity measure techniques. The effectiveness of similarity weights calculated by those similarity measure techniques are evaluated. The results are verified using the statistical significance test and the effect size test. The results show k-Nearest Neighbor (k-NN), LOcal Correlation Integral (LOCI), and Range methods are the top three performers. The experimental results show that predictive performances using the three methods are comparable to those of WPDP.
Abstract
Software defect prediction is helpful for allocating valuable project resources effectively for software quality assurance activities thanks to focusing on the identified fault-prone modules. If historical data collected within a company is sufficient, a Within-Project Defect Prediction (WPDP) can be utilized for accurate fault-prone module prediction. In case a company does not maintain historical data, it may be helpful to build a classifier towards predicting comprehensible fault prediction based on Cross-Project Defect Prediction (CPDP). Since CPDP employs different project data collected from other organization to build a classifier, the main obstacle to build an accurate classifier is that distributions between source and target projects are not similar. To address the problem, because it is crucial to identify effective similarity measure techniques to obtain high performance for CPDP, In this paper, we aim to identify them. We compare various similarity measure techniques. The effectiveness of similarity weights calculated by those similarity measure techniques are evaluated. The results are verified using the statistical significance test and the effect size test. The results show k-Nearest Neighbor (k-NN), LOcal Correlation Integral (LOCI), and Range methods are the top three performers. The experimental results show that predictive performances using the three methods are comparable to those of WPDP.
참고문헌 (50)
- S. Kim, "Classifying software changes: Clean or buggy?", Softw. Eng. IEEE Trans., 34(2), pp.181~196 (2008)[DOI]
- K. O. Elish, "Predicting defect-prone software modules using support vector machines", J. Syst. Softw., 81(5), pp.649~660 (2008)[DOI]
- E. Arisholm, "A systematic and comprehensive investigation of methods to build and evaluate fault prediction models", J. Syst. Softw., 83(1), pp.2~17 (2010)[DOI]
- T. Hall, "A Systematic Literature Review on Fault Prediction Performance in Software Engineering", IEEE Trans. Softw. Eng., 38(6), pp.1276~1304 (2012)[DOI]
- T. Menzies, "Defect prediction from static code features: current results, limitations, new approaches", Autom. Softw. Eng., 17(4), pp.375~407 (2010)[DOI]
- M. D'Ambros, "Evaluating defect prediction approaches: A benchmark and an extensive comparison", Empir. Softw. Eng., 17(4-5), pp.531~577 (2012)[DOI]
- T. Zimmermann, "Cross-project defect prediction", Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering, pp.91~100 (2009)
- Z. He, "An investigation on the feasibility of cross-project defect prediction", Autom. Softw. Eng., 19(2), pp.167~199 (2011)
- Y. Ma, "Transfer learning for cross-company software defect prediction", Inf. Softw. Technol., 54(3), pp.248~256 (2012)[DOI]
- J. Nam, "Transfer defect learning", Proceedings of the 35th International Conference on Software Engineering, pp.382~391 (2013)
- D. Ryu, "A Hybrid Instance Selection using Nearest-Neighbor for Cross-Project Defect Prediction", J. Comput. Sci. Technol., 30(5), pp.969~980 (2015)[DOI]
- G. Woodbury, "An Introduction to Statistics" (2001), Cengage Learning
- D. Ryu, "Value-cognitive boosting with a support vector machine for cross-project defect prediction", Empir. Softw. Eng., 21(1), pp.43~71 (2016)[DOI]
- B. Turhan, "On the relative value of cross-company and within-company data for defect prediction", Empir. Softw. Eng., 14(5), pp.540~578 (2009)[DOI]
- T. Pang-Ning, "Introduction to Data Mining" (2006)
- T. Grbac, "Stability of Software Defect Prediction in Relation to Levels of Data Imbalance", Proceedings of the 2nd Workshop of SQAMIA (2013)
- N. V Chawla, "SMOTE : Synthetic Minority Over-sampling Technique", J. Artif. Intell. Res., 16, pp.321~357 (2002)
- C. C. Aggarwal, "Outlier Analysis" (2013), Springer New York
- H.-P. Kriegel, "Angle-based outlier detection in high-dimensional data", Proceeding 14th ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (KDD '08), pp.444~452 (2008)
- N. Altman, "An introduction to kernel and nearest-neighbor nonparametric regression", Am. Stat., 46(3), pp.175~185 (1992)
- R. Hamming, "Error Detecting and Error Correcting Codes", Bell Syst. Tech. J., XXIX(2) (1950)
- B. Raman, "Enhancing Learning using Feature and Example selection" (2003)
- E. Parzen, "On estimation of a probability density function and mode", Ann. Math. Stat., 33(3), pp.1065~1076 (1962)[DOI]
- M. Breunig, "LOF: identifying density-based local outliers", ACM Sigmod Rec., pp.1~12 (2000)
- S. Papadimitriou, "LOCI: Fast outlier detection using the local correlation integral", Proc. - Int. Conf. Data Eng., pp.315~326 (2003)
- S. Lloyd, "Least squares quantization in PCM", IEEE Trans. Inf. Theory, 28(2), pp.129~137 (1982)[DOI]
- I. T. Jolliffe, "Principal Component Analysis" (2002), Springer
- T. Kohonen, "Self-organized formation of topologically correct feature maps", Biol. Cybern., 43(1), pp.59~69 (1982)[DOI]
- C. M. Bishop, "Pattern recognition and machine learning" (2006), Springer
- B. Turhan, "Empirical evaluation of the effects of mixed project data on learning defect predictors", Inf. Softw. Technol., 55(6), pp.1101~1118 (2013)[DOI]
- M. Jureczko, "Using Object-Oriented Design Metrics to Predict Software Defects", Models and Methods of System Dependability. Oficyna Wydawnicza Politechniki Wroclawskiej, pp.69~81 (2010)
- [URL]
- K. Beyer, "When is "nearest neighbor" meaningful?" (1999), Springer-Verlag
- S. Lessmann, "Benchmarking Classification Models for Software Defect Prediction: A Proposed Framework and Novel Findings", IEEE Trans. Softw. Eng., 34(4), pp.485~496 (2008)[DOI]
- M. Hall, "The WEKA data mining software: an update", ACM SIGKDD Explor. Newsl., 11(1), pp.10~18 (2009)[DOI]
- P. C. Mahalanobis, "On the generalised distance in statistics", Proceedings of the National Institute of Sciences of India, 2(1), pp.49~55 (1936)
- F. Menzies T, "Data mining static code attributes to learn defect predictors", IEEE Trans. Softw. Eng., 33(1), pp.2~13 (2007)[DOI]
- B. Turhan, "Empirical Evaluation of Mixed-Project Defect Prediction Models", Proceedings of the 37th EUROMICRO Conference on Software Engineering and Advanced Applications, pp.396~403 (2011)
- Y. Kamei, "Revisiting common bug prediction findings using effort-aware models", IEEE Int. Conf. Softw. Maintenance, ICSM (2010)
- S. Wang, "Using Class Imbalance Learning for Software Defect Prediction", IEEE Trans. Reliab., 62(2), pp.434~443 (2013)[DOI]
- M. Friedman, "The use of ranks to avoid the assumption of normality implicit in the analysis of variance", J. Am. Stat. Assoc., (32), pp.675~701 (1937)
- M. Friedman, "A comparison of alternative tests of significance for the problem of m rankings", Ann. Math. Stat., (11), pp.86~92 (1940)
- J. Demsar, "Statistical comparisons of classifiers over multiple data sets", J. Mach. Learn. Res., 7, pp.1~30 (2006)
- J. Tukey, "Comparing individual means in the analysis of variance", Biometrics, (5), pp.99~114 (1949)
- P. Nemenyi, "Distribution-free multiple comparisons" (1963)
- O. J. Dunn, "Multiple comparisons among means", J. Am. Stat. Assoc., (56), pp.52~64 (1961)
- F. Wilcoxon, "Individual comparisons by ranking methods", Biometrics Bull., pp.80~83 (1945)
- A. Arcuri, "A practical guide for using statistical tests to assess randomized algorithms in software engineering", 2011 33rd International Conference on Software Engineering (ICSE), pp.1~10 (2011)
- A. Vargha, "A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong", J. Educ. Behav. Stat., 25(2), pp.101~132 (2000)
- D. M. J. Tax, "DDtools, the Data Description Toolbox for Matlab" (2014)