Online SVR Training by Solving the Primal Optimization Problem

Online estimation of regression functions becomes important in presence of drifts and rapid changes in the training data. In this article we propose a new online training algorithm for SVR, called Priona, which is based on the idea of computing approximate solutions to the primal optimization problem. For the solution of the primal SVR problem we investigated the trade-off between computation time and prediction accuracy for the gradient, diagonally scaled gradient, and Newton descent direction. The choice of a particular buffering strategy did not influence the performance of the algorithm. By using a line search Priona does not require a priori selection of a learning rate which facilitates its practical application. On various benchmark data sets Priona is shown to perform better in terms of prediction accuracy in comparison to the Norma and Silk online SVR algorithms. Further, tests on two artificial data sets show that the online SVR algorithms are able to track temporal changes and drifts of the regression function, if the buffer size and learning rate are selected appropriately.