Effects of Video Encoding on Camera-Based Heart Rate Estimation

Objective: Public databases are important for evaluating and comparing different methods and algorithms for camera-based heart rate estimation. Because uncompressed video requires huge file sizes, a need for compression algorithms exists to store and share video data. Due to the optimization of modern video codecs for human perception, video compression can influence heart rate estimation negatively by reducing or eliminating small color changes of the skin (PPG) that are needed for camera based heart rate estimation. In this paper, we contribute a comprehensive analysis to answer the question of how to compress video without compromising PPG information. Methods: To analyze the influence of video compression, we compare the effect of several encoding parameters: two modern encoders (H264, H265), compression rate, resolution changes using different scaling algorithms, color subsampling, and file size on two publicly available datasets. Results: We show that increasing the compression rate decreases the accuracy of heart rate estimation, but that resolution can be reduced (up to a cutoff point) and color subsampling can be applied for reducing file size without a big impact on heart rate estimation. Conclusions: From the results, we derive and propose guidelines for the recording and encoding of video data for camera-based heart rate estimation. Significance: The paper can sensitize the research community toward the problems of video encoding, and the proposed recommended practices can help with conducting future experiments and creating valuable datasets that can be shared publicly. Such datasets would improve comparability and reproducibility in the research field.