Analyzing Machine Learning Techniques for Fault Prediction Using Web Applications
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
Web applications are indispensable in the software industry and continuously evolve either meeting a newercriteria and/or including new functionalities. However, despite assuring quality via testing, what hinders astraightforward development is the presence of defects. Several factors contribute to defects and are oftenminimized at high expense in terms of man-hours. Thus, detection of fault proneness in early phases ofsoftware development is important. Therefore, a fault prediction model for identifying fault-prone classes in aweb application is highly desired. In this work, we compare 14 machine learning techniques to analyse therelationship between object oriented metrics and fault prediction in web applications. The study is carried outusing various releases of Apache Click and Apache Rave datasets. En-route to the predictive analysis, theinput basis set for each release is first optimized using filter based correlation feature selection (CFS) method. It is found that the LCOM3, WMC, NPM and DAM metrics are the most significant predictors. The statisticalanalysis of these metrics also finds good conformity with the CFS evaluation and affirms the role of thesemetrics in the defect prediction of web applications. The overall predictive ability of different fault predictionmodels is first ranked using Friedman technique and then statistically compared using Nemenyi post-hocanalysis. The results not only upholds the predictive capability of machine learning models for faulty classesusing web applications, but also finds that ensemble algorithms are most appropriate for defect prediction inApache datasets. Further, we also derive a consensus between the metrics selected by the CFS technique andthe statistical analysis of the datasets.
Abstract
Web applications are indispensable in the software industry and continuously evolve either meeting a newercriteria and/or including new functionalities. However, despite assuring quality via testing, what hinders astraightforward development is the presence of defects. Several factors contribute to defects and are oftenminimized at high expense in terms of man-hours. Thus, detection of fault proneness in early phases ofsoftware development is important. Therefore, a fault prediction model for identifying fault-prone classes in aweb application is highly desired. In this work, we compare 14 machine learning techniques to analyse therelationship between object oriented metrics and fault prediction in web applications. The study is carried outusing various releases of Apache Click and Apache Rave datasets. En-route to the predictive analysis, theinput basis set for each release is first optimized using filter based correlation feature selection (CFS) method. It is found that the LCOM3, WMC, NPM and DAM metrics are the most significant predictors. The statisticalanalysis of these metrics also finds good conformity with the CFS evaluation and affirms the role of thesemetrics in the defect prediction of web applications. The overall predictive ability of different fault predictionmodels is first ranked using Friedman technique and then statistically compared using Nemenyi post-hocanalysis. The results not only upholds the predictive capability of machine learning models for faulty classesusing web applications, but also finds that ensemble algorithms are most appropriate for defect prediction inApache datasets. Further, we also derive a consensus between the metrics selected by the CFS technique andthe statistical analysis of the datasets.
참고문헌 (56)
- P. He, "An empirical study on software defect prediction with a simplified metric set", Information and Software Technology, 59, pp.170~190 (2015)[DOI]
- F. Rahman, "Recalling the imprecision of cross-project defect prediction", Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (2012)
- S. H. Kan, "Metrics and Models in Software Quality Engineering, 2nd ed" (2003), Addison-Wesley
- S. Lessmann, "Benchmarking classification models for software defect prediction: a proposed framework and novel findings", IEEE Transactions on Software Engineering, 34(4), pp.485~496 (2008)[DOI]
- T. Gyimothy, "Empirical validation of object-oriented metrics on open source software for fault prediction", IEEE Transactions on Software Engineering, 31(10), pp.897~910 (2005)[DOI]
- T. Menzies, "Data mining static code attributes to learn defect predictors", IEEE Transactions on Software Engineering, 33(1), pp.2~13 (2007)[DOI]
- J. Demsar, "Statistical comparisons of classifiers over multiple data sets", Journal of Machine Learning Research, 7, pp.1~30 (2006)
- M. D'Ambros, "Evaluating defect prediction approaches: a benchmark and an extensive comparison", Empirical Software Engineering, 17(4-5), pp.531~577 (2012)[DOI]
- J. Bansiya, "A hierarchical model for object-oriented design quality assessment", IEEE Transactions on Software Engineering, 28(1), pp.4~17 (2002)[DOI]
- S. R. Chidamber, "A metrics suite for object oriented design", IEEE Transactions on Software Engineering, 20(6), pp.476~493 (1994)[DOI]
- T. Fawcett, "ROC graphs: notes and practical considerations for researchers", Machine Learning, 31(1), pp.1~38 (2004)
- M. A. Hall, "Correlation-based feature selection for machine learning", pp.1998
- M. A. Hall, "Practical feature subset selection for machine learning", Proceedings of the 21st Australasian Computer Science Conference, pp.181~191 (1998)
- M. A. Hall, "The WEKA data mining software: an update", ACM SIGKDD Explorations Newsletter, 11(1), pp.10~18 (2009)[DOI]
- Q. Song, "A fast clustering-based feature subset selection algorithm for high-dimensional data", IEEE Transactions on Knowledge and Data Engineering, 25(1), pp.1~14 (2013)[DOI]
- V. Bolon-Canedo, "A review of microarray datasets and applied feature selection methods", Information Sciences, 282, pp.111~135 (2014)[DOI]
- D. Liu, "Recent advances in wavelength selection techniques for hyperspectral image processing in the food industry", Food and Bioprocess Technology, 7(2), pp.307~323 (2014)[DOI]
- A. B. de Carvalho, "Predicting fault proneness of classes through a multiobjective particle swarm optimization algorithm", Proceedings of the 20th IEEE International Conference on Tools with Artificial Intelligence, pp.387~394 (2008)
- C. Catal, "An artificial immune system approach for fault prediction in object-oriented Software", Proceedings of the 2nd International Conference on Dependability of Computer System, pp.238~245 (2007)
- B. Henderson-Sellers, "Object-Oriented Metrics: Measures of Complexity" (1996), Prentice-Hall
- K. Dejaeger, "Toward comprehensible software fault prediction models using Bayesian network classifiers", IEEE Transactions on Software Engineering, 39(2), pp.237~257 (2013)[DOI]
- E. Arisholm, "A systematic and comprehensive investigation of methods to build and evaluate fault prediction models", Journal of System and Software, 83(1), pp.2~17 (2010)[DOI]
- T. J. McCabe, "A complexity measure", IEEE Transactions on Software Engineering, 2(4), pp.308~320 (1976)
- Structured Testing: A Testing Methodology Using the Cyclomatic Complexity Metric, "A. H. Watson", National Institute of Standards and Technology, ~ (1996)
- M. H. Halstead, "Elements of Software Science" (1977), North-Holland
- C. Catal, "A systematic review of software fault prediction studies", Expert Systems Applications, 36(4), pp.7346~7354 (2009)[DOI]
- J. Kennedy, "Particle swarm optimization", Proceedings of 4th IEEE International Conference on Neural Networks, pp.1942~1948 (1995)
- Y. Shi, "A modified particle swarm optimizer", Proceedings of IEEE International Conference on Evolutionary Computation, pp.69~73 (1998)
- F. Wilcoxon, "Individual comparisons by ranking methods", Biometrics Bulletin, 1(6), pp.80~83 (1945)[DOI]
- Y. Singh, "Application of support vector machine to predict fault prone classes", ACM SIGSOFT Software Engineering Notes, 34(1), pp.1~6 (2009)
- H. M. Olague, "Empirical validation of three software metrics suites to predict fault-proneness of object-oriented classes developed using highly iterative or agile software development processes", IEEE Transactions on software Engineering, 33(6), pp.402~419 (2007)[DOI]
- G. J. Pai, "Empirical analysis of software fault content and fault proneness using Bayesian methods", IEEE Transactions on Software Engineering, 33(10), pp.675~686 (2007)[DOI]
- S. Kanmani, "Object-oriented software fault prediction using neural networks", Information and Software Technology, 49(5), pp.483~492 (2007)[DOI]
- D. Azar, "An ant colony optimization algorithm to improve software quality prediction models: case of class stability", Information and Software Technology, 53(4), pp.388~393 (2011)[DOI]
- S. Di Martino, "A genetic algorithm to configure support vector machines for predicting fault-prone components", Product-Focused Software Process Improvement, pp.247~261 (2011)
- A. Okutan, "Software defect prediction using Bayesian networks", Empirical Software Engineering, 19(1), pp.154~181 (2014)[DOI]
- Y. Zhou, "On the ability of complexity metrics to predict fault-prone classes in object oriented systems", Journal of Systems and Software, 83(4), pp.660~674 (2010)[DOI]
- Y. Zhou, "Empirical analysis of object oriented design metrics for predicting high severity faults", IEEE Transactions on Software Engineering, 32(10), pp.771~784 (2006)[DOI]
- D. Bowes, "Mutation-aware fault prediction", Proceedings of the 25th International Symposium on Software Testing and Analysis, pp.330~341 (2016)
- R. Malhotra, "An empirical comparison of machine learning techniques for software defect prediction", Proceedings of the 8th International Conference on Bioinspired Information and Communications Technologies, pp.320~327 (2014)
- R. Malhotra, "Defect collection and reporting system for Git based open source software", Proceedings of the International Conference on Data Mining and Intelligent Computing, pp.1~7 (2014)
- P. A. Devijver, "Pattern Recognition: A Statistical Approach" (1982), Prentice-Hall
- A. Bhattacharyya, "On a measure of divergence between two statistical populations defined by their probability distributions", Bulletin of the Calcutta Mathematical Society, 35, pp.99~109 (1943)
- H. Chernoff, "A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations", The Annals of Mathematical Statistics, 23(4), pp.493~507 (1952)[DOI]
- E. Patrick, "Nonparametric feature selection", IEEE Transactions in Information Theory, 15(4), pp.577~584 (1969)[DOI]
- Z. Xu, "The impact of feature selection on defect prediction performance: an empirical comparison", Proceedings of the 27th International Symposium on Software Reliability Engineering, pp.309~320 (2016)
- M. Stone, "Cross-validatory choice and assessment of statistical predictions", Journal of the Royal Statistical Society Series B (Methodological), 36(2), pp.111~114 (1974)
- W. Fu, "Tuning for software analytics: is it really necessary?", Information and Software Technology, 76, pp.135~146 (2016)[DOI]
- F. Sarro, "A further analysis on the use of genetic algorithm to configure support vector machines for inter-release fault prediction", Proceedings of the 27th Annual ACM Symposium on Applied Computing, pp.1215~1220 (2012)
- A. Arcuri, "On parameter tuning in search based software engineering", Search Based Software Engineering, pp.33~47 (2011)
- M. Friedman, "A comparison of alternative tests of significance for the problem of m rankings", The Annals of Mathematical Statistics, 11(1), pp.86~92 (1940)[DOI]
- P. B. Nemenyi, "Distribution-free multiple comparisons", pp.1963
- S. C. Misra, "Relationships between selected software measures and latent bug-density: guidelines for improving quality", Computational Science and Its Applications, pp.724~732 (2003)
- Validating Object-Oriented Design Metrics on a Commercial Java Application, "D. Glasberg", National Research Council of Canada, ~ (2000)
- S. M. A. Shah, "An overview of software defect density: a scoping study", Proceedings of the 19th Asia-Pacific Software Engineering Conference, pp.406~415 (2012)
- B. Ghotra, "Revisiting the impact of classification techniques on the performance of defect prediction models", Proceedings of the 37th International Conference on Software Engineering, pp.789~800 (2015)