STDnet-ST: Spatio-temporal ConvNet for small object detection

Object detection through convolutional neural networks is reaching unprecedented levels of precision. However, a detailed analysis of the results shows that the accuracy in the detection of small objects is still far from being satisfactory. A recent trend that will likely improve the overall object detection suc-cess is to use the spatial information operating alongside temporal video information. This paper intro-duces STDnet-ST, an end-to-end spatio-temporal convolutional neural network for small object detection in video. We define small as those objects under 16 x 16 px, where the features become less distinc-tive. STDnet-ST is an architecture that detects small objects over time and correlates pairs of the top-ranked regions with the highest likelihood of containing those small objects. This permits to link the small objects across the time as tubelets. Furthermore, we propose a procedure to dismiss unprofitable object links in order to provide high quality tubelets, increasing the accuracy. STDnet-ST is evaluated on the publicly accessible USC-GRAD-STDdb, UAVDT and VisDrone2019-VID video datasets, where it achieves state-of-the-art results for small objects. (c) 2021 Elsevier Ltd. All rights reserved.

Automated summary

Object detection using convolutional neural networks has achieved unprecedented levels of accuracy, but there is still room for improvement in detecting small objects. Utilizing spatial information alongside temporal video data is a new trend that can potentially enhance overall object detection performance. STDnet-ST is an end-to-end spatio-temporal convolutional neural network designed for detecting small objects in video, achieving state-of-the-art results on various video datasets.