Optimal Accuracy Selection for Gaussian Multiclass SVM through optimization of kernel scale and box constraint

Abstract

The best accuracy of multiclass max-win-voting SVM with Gaussian radial basis function (RBF) depends on optimal parameter selection; sigma (σ) and box constraint (C). Accuracy is the most important measure to evaluate the performance of SVM. There are training accuracy and test accuracy which are estimated on the training subset and the test subset, respectively. The training accuracy is a reference to check over-fitting or under-fitting problems by comparing it with the test accuracy. If the training accuracy is high while the test accuracy is much lower, it implies that an over-fitting problem occurs. If both the test accuracy and the training accuracy are very low, an under-fitting problem occurs. The Gaussian radial basis function (RBF) is a widely used kernel function in SVM. The kernel parameter σ is most crucial to maintain high performance of the Gaussian SVM. Most previous studies on this topic are based on optimization search algorithms that result in large computation load. In this paper, we propose an analytical algorithm to determine the optimal σ with the principle of maximizing between- class separability and minimizing within-class separability. An attractive advantage of the proposed algorithm is that no optimization search process is required, and thus the selection process is less complex and more computationally efficient. After optimal σ is selected, box constraint parameter is easily searched using simple iterative method. Experimental results on three real world datasets demonstrate that the proposed algorithm give best accuracy when using it for the Gaussian multiclass SVM. Keywords— parameter selection, Gaussian radial basis function, class separability, support vector machine, distance similarity