Examining the performance of kernel methods for software defect prediction based on support vector machine

Support Vector Machine (SVM) has been widely used to build software defect prediction models. Prior studies compared the accuracy of SVM to other machine learning algorithms but arrives at contradictory conclusions due to the use of different choices of kernel functions and metrics. Such a contradictory conclusion raises an important question about the performance of kernel functions, across different experimental conditions. To this end, the present study examines the impact and stability of four kernel functions with feature selection on the performance of SVM for software defect prediction. Strictly speaking, we examine the performance of nonlinear kernel functions against linear kernel function based on different experimental parameters such as data granularity, imbalance ratio of the dataset, and feature subsets. A large-scale study has been conducted using four kernel functions, ten feature subset selection thresholds based on the Information gain algorithm, 38 public datasets and one evaluation measure. This has resulted in 1520 experiments. The findings demonstrate that: 1) Not all nonlinear kernel functions significantly outperform linear, only RBF surpasses linear and other nonlinear kernel functions. 2) We don't have significant difference between kernel functions w.r.t. data granularity, we only found significant difference between RBF and other kernel function based on 'function' data granularity. 3) we also found that RBF can work significantly better than linear and other nonlinear function over datasets with very high and high imbalance ratios. 4) The performances of all kernel functions fluctuate over different feature subsets; However, using top 40% of the features would work best with all kernel functions. To conclude, we can recommend using SVM with RBF kernel for defects datasets because the performance of other kernel functions is limited.(c) 2022 Elsevier B.V. All rights reserved.

Automated summary

This study examines the impact and stability of four kernel functions (including linear and non-linear) with feature selection on the performance of SVM for software defect prediction. The findings show that only RBF outperforms linear kernel function and is more effective for datasets with high imbalance ratios. Different feature subsets affect the performances of all kernel functions, but using the top 40% of features yields the best results. Therefore, it is recommended to use SVM with RBF kernel for defect datasets.