4.7 Article

Estimating Heart Rate and Rhythm via 3D Motion Tracking in Depth Video

Journal

IEEE TRANSACTIONS ON MULTIMEDIA
Volume 19, Issue 7, Pages 1625-1636

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TMM.2017.2672198

Keywords

Biomedical monitoring; image denoising; signal analysis

Funding

  1. JSPS [15K12072]
  2. European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant [734331]
  3. Grants-in-Aid for Scientific Research [15K12072] Funding Source: KAKEN

Ask authors/readers for more resources

Low-cost depth sensors, such as Microsoft Kinect, have potential for noncontact health monitoring that is robust to ambient lighting conditions. However, captured depth images typically suffer from high acquisition noise, and hence, processing them to estimate biometrics is difficult. In this paper, we propose to capture depth video of a human subject using Kinect 2.0 to estimate his/her heart rate and rhythm; as blood is pumped from the heart to circulate through the head, tiny oscillatory head motion due to Newtonian mechanics can be detected for periodicity analysis. Specifically, we first restore a captured depth video via a joint bit-depth enhancement/denoising procedure, using a graph-signal smoothness prior for regularization. Second, we track an automatically detected head region throughout the depth video to deduce 3D motion vectors. The detected vectors are fed back to the depth restoration module in a loop to ensure that the motion information in two modules is consistent, improving performance of both restoration and motion tracking. Third, the computed 3D motion vectors are projected onto its principal component for 1D signal analysis, composed of trend removal, bandpass filtering, and wavelet-based motion denoising. Finally, the heart rate is estimated via Welch power spectrum analysis, and the heart rhythm is computed via peak detection. Experimental results show accurate estimation of the heart rate and rhythm using our proposed algorithm as compared to rate and rhythm estimated by a portable oximeter.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Energy & Fuels

Low-Frequency Non-Intrusive Load Monitoring of Electric Vehicles in Houses with Solar Generation: Generalisability and Transferability

Apostolos Vavouris, Benjamin Garside, Lina Stankovic, Vladimir Stankovic

Summary: The paper proposes a scalable methodology for detecting household EV charging events and load consumption from smart meter data, emphasizing the generalisability of the approaches across similar houses and different geographical regions and EV charging profiles. The effectiveness of different performance and generalisation loss metrics is also discussed.

ENERGIES (2022)

Article Energy & Fuels

Transfer learning for multi-objective non-intrusive load monitoring in smart building

Dandan Li, Jiangfeng Li, Xin Zeng, Vladimir Stankovic, Lina Stankovic, Changjiang Xiao, Qingjiang Shi

Summary: This study explores the importance of reducing carbon emissions in buildings to overall greenhouse gas emissions reductions and proposes a method for estimating appliance power consumption based on transfer learning. By utilizing a one-to-many model and incorporating appliance transfer learning and cross-domain transfer learning, this approach effectively estimates the power consumption of all appliances.

APPLIED ENERGY (2023)

Article Chemistry, Analytical

Graph-Based Feature Weight Optimisation and Classification of Continuous Seismic Sensor Array Recordings

Jiangfeng Li, Lina Stankovic, Vladimir Stankovic, Stella Pytharouli, Cheng Yang, Qingjiang Shi

Summary: This paper proposes a novel multi-channel event-detection scheme based on Neyman-Pearson lemma and Multi-channel Coherency Migration (MCM) on the stacked signal across multi-channels to improve the classification of seismic signals. It also uses graph-based feature weight optimization as feature selection to enhance signal classification. Experimental results show that this method can identify 614 more seismic events compared to traditional detection approaches, and feature selection provides more focused feature sets while improving the classification performance.

SENSORS (2023)

Review Chemistry, Analytical

A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation

Dong Han, Beni Mulyana, Vladimir Stankovic, Samuel Cheng

Summary: This review provides an overview of recent advances in deep reinforcement learning algorithms for robotic manipulation tasks. It covers the fundamental ideas of reinforcement learning and the parts of a reinforcement learning system. Various deep reinforcement learning algorithms, including value-based methods, policy-based methods, and actor-critic approaches, are discussed. The review also examines the challenges and solutions when applying these algorithms to robotics tasks, and highlights unsolved research issues and future directions for the subject.

SENSORS (2023)

Article Chemistry, Analytical

Explainability-Informed Feature Selection and Performance Prediction for Nonintrusive Load Monitoring

Rachel Stephen Mollel, Lina Stankovic, Vladimir Stankovic

Summary: With the global roll-out of smart metering, the potential of higher resolution energy readings is being tapped into for accurate billing, improved demand response, and better-tuned tariffs. This paper focuses on the trustworthiness of the NILM model and proposes a naturally interpretable decision tree approach, as well as explainability tools, to improve appliance classification performance and feature selection. Experimental results show significant improvements in the classification performance of appliances like the toaster, dishwasher, and washing machine.

SENSORS (2023)

Article Environmental Sciences

Synthetic Data Generation for Deep Learning-Based Inversion for Velocity Model Building

Apostolos Parasyris, Lina Stankovic, Vladimir Stankovic

Summary: In recent years, deep learning has been used in seismic inversion to analyze the subsurface, such as improving velocity model building. This study focuses on deep-learning-based inversion for velocity model building, using a conditional generative adversarial network (PIX2PIX) with ResNet-9 as the generator. Mathematical methodology is also used to generate samples of multi-stratified heterogeneous velocity models for training the architecture. The results show that the proposed architecture achieves state-of-the-art performance in reconstructing velocity models using only one seismic shot, reducing cost and computational complexity. The solution is also shown to be applicable to linear multi-layer models, curved or folded structures, structures with salt bodies, and higher-resolution structures built from geological images through quantitative and qualitative evaluation.

REMOTE SENSING (2023)

Article Energy & Fuels

An active learning framework for the low-frequency Non-Intrusive Load Monitoring problem

Tamara Todic, Vladimir Stankovic, Lina Stankovic

Summary: With the widespread deployment of smart meters, Non-Intrusive Load Monitoring (NILM) has emerged as a promising application for informing energy management within buildings. However, existing deep learning NILM models have limitations in terms of flexibility and scalability. In this study, an active learning framework is proposed to improve transferability and reduce the cost of labeling, achieving optimal accuracy-labelling effort trade-off. The results demonstrate the potential of this approach in improving the performance of NILM models and reducing computational resources needed.

APPLIED ENERGY (2023)

Article Chemistry, Analytical

Unsupervised Cluster Analysis of Walking Activity Data for Healthy Individuals and Individuals with Lower Limb Amputation

Alexander Jamieson, Laura Murray, Vladimir Stankovic, Lina Stankovic, Arjan Buis

Summary: This study performs an unsupervised cluster analysis on the activities of individuals with lower limb amputation (ILLAs) and individuals without gait impairment in free-living conditions. The use of a thigh-based accelerometer allows for the clustering of physical activity data, with tSNE showing the best performance. The analysis identifies clusters in walking-based activities, specifically ground walking and stair ambulation.

SENSORS (2023)

Article Geochemistry & Geophysics

Domain Knowledge Informed Multitask Learning for Landslide-Induced Seismic Classification

Jiangfeng Li, Minxiang Ye, Lina Stankovic, Vladimir Stankovic, Stella Pytharouli

Summary: This study proposes a multitask learning scheme that utilizes physical characteristics of seismic wave propagation and a 3-D P-wave velocity model for signal representation learning. The results show that this approach outperforms state-of-the-art methods.

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS (2023)

Article Geochemistry & Geophysics

Microseismic Event Classification With Time-, Frequency-, and Wavelet-Domain Convolutional Neural Networks

Jiaxin Jiang, Vladimir Stankovic, Lina Stankovic, Emmanouil Parastatidis, Stella Pytharouli

Summary: Passive seismics are used to understand underground processes and predict their effects. Manual detection of seismic events is time-consuming and prone to inconsistency, so an automated approach based on convolutional neural networks (CNN) is proposed. Three different CNN architectures are evaluated using continuous seismometer recordings from a landslide area, showing excellent performance. The proposed network can also detect earthquake events in a different seismic area, demonstrating its potential to replace manual labeling.

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING (2023)

Article Automation & Control Systems

Knowledge Distillation for Scalable Nonintrusive Load Monitoring

Giulia Tanoni, Lina Stankovic, Vladimir Stankovic, Stefano Squartini, Emanuele Principi

Summary: This article proposes a knowledge distillation approach for NILM, aiming to reduce model complexity and improve generalization on unseen data domains. Experimental results show that the approach outperforms benchmark methods in multilabel appliance classification.

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS (2023)

Proceedings Paper Construction & Building Technology

Benefits of Three-Phase Metering for Load Disaggregation

Apostolos Vavouris, Lina Stankovic, Vladimir Stankovic, Jiufeng Shi

Summary: This paper explores how three-phase metering can benefit nonintrusive load monitoring (NILM), especially for appliances that are difficult to disaggregate and not widely reported. The emphasis is on the performance and loss introduced when using different levels of granularity of low-frequency data for load disaggregation.

PROCEEDINGS OF THE 2022 THE 9TH ACM INTERNATIONAL CONFERENCE ON SYSTEMS FOR ENERGY-EFFICIENT BUILDINGS, CITIES, AND TRANSPORTATION, BUILDSYS 2022 (2022)

Proceedings Paper Construction & Building Technology

Using Explainability Tools to Inform NILM Algorithm Performance: A Decision Tree Approach

Rachel Stephen Mollel, Lina Stankovic, Vladimir Stankovic

Summary: This paper demonstrates the importance of using explainability tools to explain the outcomes of a decision tree multi-classification approach for NILM, as well as how model explainability informs feature selection and ultimately improves performance.

PROCEEDINGS OF THE 2022 THE 9TH ACM INTERNATIONAL CONFERENCE ON SYSTEMS FOR ENERGY-EFFICIENT BUILDINGS, CITIES, AND TRANSPORTATION, BUILDSYS 2022 (2022)

Article Engineering, Electrical & Electronic

How Likely is a Random Network Graph Shift-Enabled?

Liyan Chen, Samuel Cheng, Vladimir Stankovic, Lina Stankovic, Qingjiang Shi

Summary: This article discusses the shift-enabled property of random network graphs, finding that the considered unweighted connected random network graphs are shift-enabled with high probability when the number of edges is moderately high, while very dense and fully connected graphs are not. Weighted connected graphs are usually shift-enabled, unless the number of edges is very low.

IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS (2022)

Proceedings Paper Computer Science, Artificial Intelligence

Quantification of Dairy Farm Energy Consumption to Support the Transition to Sustainable Farming

Tamara Todic, Lina Stankovic, Vladimir Stankovic, Jiufeng Shi

Summary: In this research, a deep learning-based load disaggregation approach was used to quantify individual energy consumption of milk production-related devices in dairy farms. The experiments conducted in three dairy farms in Germany showed that load disaggregation from a single aggregate meter is a viable and cost-effective alternative to accurately measure electricity consumption and provide decision support.

2022 IEEE INTERNATIONAL CONFERENCE ON SMART COMPUTING (SMARTCOMP 2022) (2022)

No Data Available