Efficient Storage of Multi-Sensor Object-Tracking Data

The rapid development of Internet of Things (IoT) enables people to track objects by deploying multiple sensors, e.g., to track people in indoor spaces using RFID sensors. Multi-sensor object-tracking data are usually produced as records, which are thereby organized into small files and written to servers. However, the small-size property and high arriving-rate of multi-sensor object-tracking data will result in poor I/O performance of file systems such as HDFS. In this paper, we propose the first read/write-optimized solution for storing multi-sensor object-tracking data on HDFS. In particular, we exploit a distributed caching mechanism and a parallel file-merging policy to improve the I/O performance of HDFS. With our design, object-tracking data are first cached by a Distributed Memory File System (DMFS) on top of HDFS. These data are further merged into large files, which are then flushed to HDFS in parallel. We demonstrate that this mechanism is able to improve the write throughput of HDFS and outperforms existing centralized-cache-based approaches. In addition, in order to improve the search performance of object queries over multi-sensor object-tracking data, we propose a Sensor-Dependence Graph (SDG) to model sensor dependence and further present an SDG-based algorithm to efficiently cluster sensors. The object-tracking data from the sensors in the same cluster are merged into the same large files, which can reduce file scans during query processing and therefore improve search performance. We conduct extensive experiments to evaluate the performance of our proposal. The results suggest the efficiency of our proposal with respect to disk-write throughput, memory-write throughput, search performance, and sensor clustering.