4.7 Article

Multi-sensor information fusion based on machine learning for real applications in human activity recognition: State-of-the-art and research challenges

期刊

INFORMATION FUSION
卷 80, 期 -, 页码 241-265

出版社

ELSEVIER
DOI: 10.1016/j.inffus.2021.11.006

关键词

Wearable device; Information fusion; Human activity recognition; Machine learning; Deep learning; Transfer learning

资金

  1. National Natural Science Foundation of China [61803072, 61873044, 61903062]
  2. Natural Science Foundation of Liaoning Province, China [2021-MS-111]
  3. Fundamental Research Funds for the Central Universities, China [DUT20JC03, DUT20JC44]

向作者/读者索取更多资源

This paper introduces common wearable sensors, smart wearable devices, and key application areas, proposing fusion methods for multi-modality and multi-location sensors. It comprehensively surveys important aspects of wearable sensor fusion methods in human activity recognition, including new technologies in unsupervised learning and transfer learning, while also discussing open research issues that need further investigation and improvement.
This paper firstly introduces common wearable sensors, smart wearable devices and the key application areas. Since multi-sensor is defined by the presence of more than one model or channel, e.g. visual, audio, environmental and physiological signals. Hence, the fusion methods of multi-modality and multi-location sensors are proposed. Despite it has been contributed several works reviewing the stateoftheart on information fusion or deep learning, all of them only tackled one aspect of the sensor fusion applications, which leads to a lack of comprehensive understanding about it. Therefore, we propose using a more holistic approach in order to provide a more suitable starting point from which to develop a full understanding of the fusion methods of wearable sensors. Specifically, this review attempts to provide a more comprehensive survey of the most important aspects of multi-sensor applications for human activity recognition, including those recently added to the field for unsupervised learning and transfer learning. Finally, the open research issues that need further research and improvement are identified and discussed.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

Article Computer Science, Theory & Methods

Trust in Edge-based Internet of Things Architectures: State of the Art and Research Challenges

Lidia Fotia, Flavia Delicato, Giancarlo Fortino

Summary: The Internet of Things (IoT) enables smart objects to provide smart services inserted into information networks for human beings. The introduction of edge computing in IoT reduces decision-making latency, saves bandwidth resources, and expands cloud services at the network's edge. However, decentralized trust management poses challenges for edge-based IoT systems. Trust management is crucial for reliable mining and data fusion, improved user privacy and data security, and context-aware service provisioning.

ACM COMPUTING SURVEYS (2023)

Review Computer Science, Information Systems

AI-enabled IoT penetration testing: state-of-the-art and research challenges

Claudia Greco, Giancarlo Fortino, Bruno Crispo, Kim-Kwang Raymond Choo

Summary: This paper provides a comprehensive review of literature on penetration testing of IoT devices and systems. It identifies existing and potential IoT penetration testing applications and proposed approaches, and highlights recent advances in AI-enabled penetration testing methods at the network edge.

ENTERPRISE INFORMATION SYSTEMS (2023)

Article Computer Science, Information Systems

Tolerance Analysis of Cyber-Manufacturing Systems to Cascading Failures

Xiuwen Fu, Pasquale Pace, Gianluca Aloi, Antonio Guerrieri, Wenfeng Li, Giancarlo Fortino

Summary: In this study, a interdependent network model for cyber-manufacturing systems (CMS) is developed based on the perspective of physical-service networking. The proposed realistic cascading failure model takes into account the load distribution characteristics of the physical network and the service network. The experiments confirm that attacks on the physical network are more likely to trigger cascading failures and cause more damage, and interdependency failures are the main cause of performance degradation in the service network during cascading failures, while isolation failures are the main cause of performance degradation in the physical network during cascading failures.

ACM TRANSACTIONS ON INTERNET TECHNOLOGY (2023)

Article Engineering, Electrical & Electronic

Kinematics Analysis of Arms in Synchronized Canoeing With Wearable Inertial Measurement Unit

Long Liu, Jiayi Liu, Sen Qiu, Zhelong Wang, Hongyu Zhao, Masood Habib, Yongzhen Wang

Summary: This article proposes a motion-capture method based on inertial sensors to analyze the synchronized movements of two canoeists. The results show a significant correlation between the shoulder joint angles of the synchronized canoeists and the ability to analyze posture coordination. This research can help evaluate the synchronization effect of synchronized canoeing and improve technical movements, as well as assist coaches in selecting athletes with matching skills and styles.

IEEE SENSORS JOURNAL (2023)

Review Chemistry, Analytical

Swarm Intelligence in Internet of Medical Things: A Review

Roohallah Alizadehsani, Mohamad Roshanzamir, Navid Hoseini Izadi, Raffaele Gravina, H. M. Dipu Kabir, Darius Nahavandi, Hamid Alinejad-Rokny, Abbas Khosravi, U. Rajendra Acharya, Saeid Nahavandi, Giancarlo Fortino

Summary: Continuous advancements in technologies like the internet of things and big data analysis have enabled information sharing and smart decision-making using everyday devices. Swarm intelligence algorithms facilitate constructive interaction among individuals regardless of their intelligence level to address complex nonlinear problems. This paper examines the application of swarm intelligence algorithms in the internet of medical things, with a focus on wearable devices in healthcare. It reviews existing works on utilizing swarm intelligence in tackling IoMT problems such as disease prediction, data encryption, and resource allocation. The paper concludes with research perspectives and future trends.

SENSORS (2023)

Article Chemistry, Analytical

A Hybrid Generic Framework for Heart Problem Diagnosis Based on a Machine Learning Paradigm

Alaa Menshawi, Mohammad Mehedi Hassan, Nasser Allheeib, Giancarlo Fortino

Summary: A generic framework has been developed for heart problem diagnosis using a hybrid of machine learning and deep learning techniques. The framework utilizes a novel voting technique based on the prediction probabilities of multiple models to eliminate bias. Experimental results show that the framework outperforms single machine learning models, classical stacking techniques, and traditional voting techniques, achieving an accuracy of 95.6%.

SENSORS (2023)

Review Chemistry, Analytical

At the Confluence of Artificial Intelligence and Edge Computing in IoT-Based Applications: A Review and New Perspectives

Amira Bourechak, Ouarda Zedadra, Mohamed Nadjib Kouahla, Antonio Guerrieri, Hamid Seridi, Giancarlo Fortino

Summary: Given its advantages, edge computing has emerged as key support for intelligent applications and 5G/6G IoT networks. However, there are concerns about its capabilities to handle the computational complexity of machine learning techniques for big IoT data analytics. This paper aims to explore the confluence of AI and edge computing in various application domains to leverage existing research and identify new perspectives.

SENSORS (2023)

Article Chemistry, Analytical

MEMS Devices-Based Hand Gesture Recognition via Wearable Computing

Huihui Wang, Bo Ru, Xin Miao, Qin Gao, Masood Habib, Long Liu, Sen Qiu

Summary: This paper investigates static and dynamic gesture recognition methods based on miniature inertial sensors. Hand-gesture data are obtained through a data glove and preprocessed using Butterworth low-pass filtering and normalization algorithms. The random forest algorithm achieves the highest recognition accuracy and shortest recognition time for static gestures. The addition of the attention mechanism significantly improves the recognition accuracy of the LSTM model for dynamic gestures, with a prediction accuracy of 98.3%, based on the original six-axis dataset.

MICROMACHINES (2023)

Review Computer Science, Artificial Intelligence

Disclosing Edge Intelligence: A Systematic Meta-Survey

Vincenzo Barbuto, Claudio Savaglio, Min Chen, Giancarlo Fortino

Summary: The Edge Intelligence (EI) paradigm is a promising solution to the limitations of cloud computing in the development and provision of next-generation Internet of Things (IoT) services. This paper provides a systematic analysis of the state-of-the-art manuscripts on EI, exploring the past, present, and future directions of the EI paradigm and its relationships with IoT and cloud computing.

BIG DATA AND COGNITIVE COMPUTING (2023)

Article Engineering, Electrical & Electronic

Multi-Object Tracking Based on a Novel Feature Image With Multi-Modal Information

Yi An, Jialin Wu, Yunhao Cui, Huosheng Hu

Summary: This paper proposes a multi-object tracking framework based on the multi-modal information of 3D point clouds and color images. The method combines point cloud and image data for object detection and constructs a height-intensity-density image for object tracking. It also introduces a new rotation kernel correlation filter for object prediction and develops object retention and re-recognition modules to overcome object matching failure. Experimental results on the KITTI dataset demonstrate that the proposed method outperforms existing traditional multi-object tracking methods.

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (2023)

Article Chemistry, Analytical

Dataglove for Sign Language Recognition of People with Hearing and Speech Impairment via Wearable Inertial Sensors

Ang Ji, Yongzhen Wang, Xin Miao, Tianqi Fan, Bo Ru, Long Liu, Ruicheng Nie, Sen Qiu

Summary: This study proposes a low-cost data glove solution that utilizes multiple inertial sensors to achieve efficient and accurate sign language recognition, enabling seamless communication between deaf and able-bodied individuals. Four machine learning models and an attention-based mechanism of long and short-term memory neural networks were employed to recognize 20 different types of dynamic sign language data. The results show that the proposed Attention-BiLSTM and RF algorithms have the highest performance in recognizing the twenty dynamic sign language gestures, with accuracies of 98.85% and 97.58% respectively, providing evidence for the feasibility of the proposed data glove and recognition methods. This study serves as a valuable reference for the development of wearable sign language recognition devices and promotes easier communication between deaf and able-bodied individuals.

SENSORS (2023)

Article Engineering, Multidisciplinary

A Novel Homomorphic Encryption and Consortium Blockchain-Based Hybrid Deep Learning Model for Industrial Internet of Medical Things

Aitizaz Ali, Muhammad Fermi Pasha, Antonio Guerrieri, Antonella Guzzo, Xiaobing Sun, Aamir Saeed, Amir Hussain, Giancarlo Fortino

Summary: This paper proposes a hybrid deep learning model for Industrial Internet of Medical Things (IIoMT) that addresses security challenges using homomorphic encryption (HE) and blockchain technology, providing higher privacy and security. By deploying a pre-trained model on edge devices and utilizing a consortium blockchain for data sharing and updating, the model can effectively classify and train local models while delivering higher efficiency and low latency.

IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING (2023)

Article Computer Science, Cybernetics

Behavioral Modeling and Prediction in Social Perception and Computing: A Survey

Zhihan Lv, Chen Cheng, Antonio Guerrieri, Giancarlo Fortino

Summary: More data are generated through mobile network technology, giving birth to the cyber-physical social intelligent ecosystem (C & P-SIE). This survey studies the development of physical social intelligence, discussing its applications in various domains such as intelligent transportation, healthcare, public service, economy, and social networking. It also explores the future prospects of behavior modeling in C & P-SIE under information security, data-driven techniques, and cooperative artificial intelligence technologies. This research provides a theoretical foundation and new opportunities for the digital and intelligent development of smart cities and social systems.

IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS (2023)

Article Computer Science, Cybernetics

A Social Edge-Based IoT Framework Using Reputation-Based Clustering for Enhancing Competitiveness

Giancarlo Fortino, Lidia Fotia, Fabrizio Messina, Domenico Rosaci, Giuseppe M. L. Sarne

Summary: This article introduces a multi-agent SIoT architecture that incorporates a reputation system based on clustering of smart objects, providing reliability for transactions in SIoT scenarios. By enabling feedback between smart objects, and communication between edge servers and the cloud, reputation values are updated, enhancing the trustworthiness of objects.

IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS (2023)

Article Computer Science, Information Systems

Deep Learning Based Fusion Model for Multivariate LTE Traffic Forecasting and Optimized Radio Parameter Estimation

Syed Tauhidun Nabi, Md. Rashidul Islam, Md. Golam Rabiul Alam, Mohammad Mehedi Hassan, Salman A. AlQahtani, Gianluca Aloi, Giancarlo Fortino

Summary: This research utilizes 6.2 million real network time series LTE data traffic and other associated parameters to build a traffic forecasting model using multivariate feature inputs and deep learning algorithms, which can forecast traffic at a granular eNodeB-level and provide eNodeB-wise forecasted PRB utilization.

IEEE ACCESS (2023)

Article Computer Science, Artificial Intelligence

Ultrametrics for context-aware comparison of binary images

C. Lopez-Molina, S. Iglesias-Rey, B. De Baets

Summary: Quantitative image comparison is a critical topic in image processing literature, with diverse applications. Existing measures of comparison often overlook the context in which the comparison takes place. This paper presents a context-aware comparison method for binary images, tested on the BSDS500 benchmark.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Preemptively pruning Clever-Hans strategies in deep neural networks

Lorenz Linhardt, Klaus-Robert Mueller, Gregoire Montavon

Summary: This paper investigates the issue of mismatches between the decision strategy of the explainable model and the user's domain knowledge, and proposes a new method EGEM to mitigate hidden flaws in the model. Experimental results demonstrate that the approach can significantly reduce reliance on Clever Hans strategies and improve the accuracy of the model on new data.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Dual-level Deep Evidential Fusion: Integrating multimodal information for enhanced reliable decision-making in deep learning

Zhimin Shao, Weibei Dou, Yu Pan

Summary: This paper proposes a novel algorithm, Dual-level Deep Evidential Fusion (DDEF), to integrate multimodal information at both the BBA level and multimodal level, aiming to enhance accuracy, robustness, and reliability. The DDEF approach utilizes the Dirichlet framework and BBA methods for effective uncertainty estimation and employs the Dempster-Shafer Theory for dual-level fusion. The experimental results show that the proposed DDEF outperforms existing methods in synthetic digit classification and real-world medical prognosis after BCI treatment.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Multi-modal detection of fetal movements using a wearable monitor

Abhishek K. Ghosh, Danilo S. Catelli, Samuel Wilson, Niamh C. Nowlan, Ravi Vaidyanathan

Summary: The inability of current FM monitoring methods to be used outside clinical environments has made it challenging to understand the nature and evolution of FM. This investigation introduces a novel wearable FM monitor with a heterogeneous sensor suite and a data fusion architecture to efficiently capture and separate FM from interfering artifacts. The performance of the device and architecture were validated through at-home use, demonstrating high accuracy in detecting and recognizing FM events. This research is a major milestone in the development of low-cost wearable FM monitors for pervasive monitoring of FM in unsupervised environments.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

ADCT-Net: Adaptive traffic forecasting neural network via dual-graphic cross-fused transformer

Jianlei Kong, Xiaomeng Fan, Min Zuo, Muhammet Deveci, Xuebo Jin, Kaiyang Zhong

Summary: In this study, we propose an intelligent traffic flow prediction framework based on the adaptive dual-graphic transformer with a cross-fusion strategy, aiming to uncover latent graphic feature representations that transcend temporal and spatial limitations. By establishing a traffic spatiotemporal prediction model using a cross-fusion attention mechanism, our proposed model achieves superior prediction performance on practical urban traffic flow datasets, particularly for long-term predictions.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Component similarity based conflict analysis: An information fusion viewpoint

Huilai Zhi, Jinhai Li

Summary: This article addresses the issue that conflict analysis based on single-valued information systems is no longer valid. It proposes a conflict analysis method based on component similarity, which uses three-way n-valued concept lattices to handle set-valued formal contexts and realizes fast conflict analysis from an information fusion viewpoint. Experimental results verify the effectiveness of this method in reducing time consumption.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Mining and fusing unstructured online reviews and structured public index data for hospital selection

Huchang Liao, Jiaxin Qi, Jiawei Zhang, Chonghui Zhang, Fan Liu, Weiping Ding

Summary: In this paper, a hospital selection approach based on a fuzzy multi-criterion decision-making method is proposed. This approach considers sentiment evaluation values of unstructured data from online reviews and structured data of public indexes simultaneously. The methodology involves collecting and processing online reviews, classifying topics and sentiments, quantifying sentiment analysis results using fuzzy numbers, and obtaining final preference scores of hospitals based on patients' preferences. A case study and robustness analysis are conducted to validate the effectiveness of the method.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Fake news detection: Taxonomy and comparative study

Faramarz Farhangian, Rafael M. O. Cruz, George D. C. Cavalcanti

Summary: The proliferation of social networks has posed challenges in combating fake news, but automatic fake news detection using artificial intelligence has become more feasible. This paper revisits the definitions and perspectives of fake news and proposes an updated taxonomy, based on multiple criteria, for the field. The study finds that optimal feature extraction techniques vary depending on the dataset, and context-dependent models based on transformer models consistently exhibit superior performance.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

A dynamic multiple classifier system using graph neural network for high dimensional overlapped data

Mariana A. Souza, Robert Sabourin, George D. C. Cavalcanti, Rafael M. O. Cruz

Summary: In this study, a dynamic selection technique is proposed to handle sparse and overlapped data. The technique leverages the relationships between instances and classifiers to learn a dynamic classifier combination rule. Experimental results show that the proposed method outperforms static selection and other dynamic selection techniques.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

A clustering method based on multi-positive-negative granularity and attenuation-diffusion pattern

Bin Yu, Ruihui Xu, Mingjie Cai, Weiping Ding

Summary: This paper introduces a clustering method based on non-Euclidean metric and multi-granularity staged clustering to address the challenges posed by complex spatial structure data to traditional clustering methods. The method improves the similarity measure and employs an attenuation-diffusion pattern for local to global clustering, achieving good clustering results.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Fast metric multi-view hashing for multimedia retrieval

Jian Zhu, Pengbo Hu, Bingqian Li, Yi Zhou

Summary: The acquisition of multi-view hash representation for heterogeneous data is highly important for multimedia retrieval. Current approaches suffer from limited retrieval precision due to insufficient integration of multi-view features and failure to effectively utilize metric information from diverse samples. In this paper, we propose an innovative method called Fast Metric Multi-View Hashing (FMMVH), which demonstrates the superiority of gate-based fusion over traditional methods. We also introduce a novel deep metric loss function to leverage metric information from dissimilar samples. By optimizing and employing model compression techniques, our FMMVH method significantly outperforms existing state-of-the-art methods on benchmark datasets, with up to 7.47% improvement in mean Average Precision (mAP).

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

SwinWave-SR: Multi-scale lightweight underwater image super-resolution

Fayaz Ali Dharejo, Iyyakutti Iyappan Ganapathi, Muhammad Zawish, Basit Alawode, Moath Alathbah, Naoufel Werghi, Sajid Javed

Summary: The resource-limited nature of underwater vision equipment affects underwater robotics and ocean engineering tasks. Super-resolution methods, particularly using Vision Transformers (ViTs), have emerged to enhance low-resolution underwater images. However, ViTs face challenges in handling severe degradation in underwater imaging. In contrast, Multi-scale ViTs (MViTs) overcome these challenges by preserving long-range dependencies through evolving channel capacity. This study proposes a novel algorithm, SwinWave-SR, for efficient and accurate multi-scale super-resolution for underwater images.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Federated split learning for sequential data in satellite-terrestrial integrated networks

Weiwei Jiang, Haoyu Han, Yang Zhang, Jianbin Mu

Summary: This study incorporates federated learning and split learning paradigms with satellite-terrestrial integrated networks and introduces a split-then-federated learning framework and federated split learning with long short-term memory to handle sequential data in STINs. The proposed solution is demonstrated to be effective through a case study of electricity theft detection based on a real-world dataset.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Software defined radio frequency sensing framework for Internet of Medical Things

Najah Abuali, Mohammad Bilal Khan, Farman Ullah, Mohammad Hayajneh, Hikmat Ullah, Shahid Mumtaz

Summary: The demand for innovative solutions in biomedical systems for precise diagnosis and management of critical diseases is increasing. A promising technology, non-invasive and intelligent Internet of Medical Things (IoMT) system, emerges to assess patients with reduced health risks. This research introduces a comprehensive framework for early diagnosis of respiratory abnormalities through RF sensing and SDR technology. The results highlight the superior performance of deep learning frameworks in classifying respiratory anomalies.

INFORMATION FUSION (2024)

Article Computer Science, Artificial Intelligence

Global-local fusion based on adversarial sample generation for image-text matching

Shichen Huang, Weina Fu, Zhaoyue Zhang, Shuai Liu

Summary: In the era of adversarial machine learning (AML), developing robust and generalized algorithms has become a key research topic. This study proposes a global similarity matching module and a global-local cognition fusion training mechanism based on relationship adversarial sample generation to improve image-text matching algorithm. Experimental results show significant improvements in accuracy and robustness, performing well in facing security challenges and promoting the fusion of visual and linguistic modalities.

INFORMATION FUSION (2024)