关键词: 软件缺陷预测、跨版本缺陷预测、超参数优化

Abstract: In the field of machine learning, hyperparameters are one of the key factors that affect prediction performance. Previous studies have shown that optimizing hyperparameters can improve the performance of inner-version defect prediction and cross-project defect prediction, but the impact on the performance of cross-version defect prediction is unclear. This paper explores the impact of hyperparameter optimization on cross-version defect prediction. The optimization goal is the model's optimal AUC performance of 10-fold cross-validation in previous version (training set). Five classical defect prediction models (Decision Tree, K-Nearest Neighbors, Random Forests, Support Vector Machine and Multi-Layer Perceptron) and four common hyperparameter optimization algorithms (Bayesian Optimization Based on TPE and Sequential Model-Based Optimization for General Algorithm Configuration, Random Search and Simulated Annealing) are selected and the empirical study is conducted on PROMISE datasets. The results indicate that: (1) There is an obvious improvement in the AUC index of cross-version defect prediction after optimizing the Decision Tree, K-Nearest Neighbors and Multi-Layer Perceptron models. (2) The optimal models still maintains the same stability as the default hyperparametric models. (3) Hyperparameter optimization take 1-2 minutes for all models on average except the complicated multi-layer perceptron model, time cost is within an acceptable range. The above results indicate that the hyperparameter optimization of the model should be considered in the process of cross-version defect prediction to improve its performance.

Key words: Software Defect Prediction, Cross-Version Defect Prediction, Hyperparameter Optimization