An Ensemble Deep Learning Method for Vehicle Type Classification on Visual Traffic Surveillance Sensors

Visual traffic surveillance systems play important roles in intelligent transport systems nowadays. The first step of a visual traffic surveillance system usually needs to correctly detect objects from images or videos and classify them into different categories (e.g., car, truck, and bus). This paper aims to introduce a new vehicle type classification scheme on the images acquired from multi-view visual traffic surveillance sensors. Most image classification algorithms focus on maximizing the percentage of the correct predictions, which have a deficiency that the images from minority categories are prone to be misclassified as the dominant categories. To address this challenge of classifying imbalanced data acquired from visual traffic surveillance sensors, we propose a method, which integrates deep neural networks with balanced sampling in this paper. The proposed method consists of two main stages. In the first stage, data augmentation with balanced sampling is applied to alleviate the unbalanced data set problem. In the second stage, an ensemble of convolutional neural network models with different architectures is constructed with parameters learned on the augmented training data set. Experiments on the MlOvision traffic camera dataset classification challenge data set demonstrate that the proposed method is able to enhance the mean precision of all categories, in the condition of high overall accuracy, compared with the baseline algorithms.