Article
Chemistry, Analytical
Ang Ji, Yongzhen Wang, Xin Miao, Tianqi Fan, Bo Ru, Long Liu, Ruicheng Nie, Sen Qiu
Summary: This study proposes a low-cost data glove solution that utilizes multiple inertial sensors to achieve efficient and accurate sign language recognition, enabling seamless communication between deaf and able-bodied individuals. Four machine learning models and an attention-based mechanism of long and short-term memory neural networks were employed to recognize 20 different types of dynamic sign language data. The results show that the proposed Attention-BiLSTM and RF algorithms have the highest performance in recognizing the twenty dynamic sign language gestures, with accuracies of 98.85% and 97.58% respectively, providing evidence for the feasibility of the proposed data glove and recognition methods. This study serves as a valuable reference for the development of wearable sign language recognition devices and promotes easier communication between deaf and able-bodied individuals.
Article
Chemistry, Analytical
Eko Sakti Pramukantoro, Akio Gofuku
Summary: Heartbeat monitoring is crucial for early detection of cardiovascular disease. This paper proposes a wearable device-based heartbeat classifier trained using machine learning and deep learning methods, achieving high accuracy for real-time monitoring.
Article
Computer Science, Information Systems
Ramin Fallahzadeh, Zhila Esna Ashari, Parastoo Alinia, Hassan Ghasemzadeh
Summary: In recent years, there has been an increasing amount of research on autonomous activity recognition models for deployment in new settings. However, current research lacks comprehensive frameworks for transfer learning, particularly when dealing with partially available data in these new settings. To address this, the researchers propose OptiMapper, a novel uninformed cross-subject transfer learning framework that extracts abstract knowledge across subjects and utilizes it to develop personalized and accurate activity recognition models. The experimental results demonstrate that OptiMapper achieves high accuracy in activity recognition.
IEEE TRANSACTIONS ON MOBILE COMPUTING
(2023)
Article
Chemistry, Multidisciplinary
Yutaka Yoshida, Emi Yuda
Summary: It is reported that workouts can relieve daily stress and improve mental and physical health. This study attempted to detect workouts using machine learning based on multiple types of biological information obtained from a wearable wristband sensor. The detection accuracy of random forest (RF) and support vector machine (SVM) was high, with RF achieving recall, precision, and F-score values of 0.962, 0.963, and 0.963, respectively. The importance of feature values used for detection showed that sleep state, skin temperature, and pulse rate accounted for approximately 86.3% of the total.
APPLIED SCIENCES-BASEL
(2023)
Review
Engineering, Electrical & Electronic
E. Ramanujam, Thinagaran Perumal, S. Padmavathi
Summary: Human Activity Recognition (HAR) is the field of inferring human activities from signals acquired through sensors of smartphones and wearable devices, mainly for smart home and elderly care. Current techniques mostly use Deep Learning for feature extraction and classification efficiency, but there are challenges and issues that require future research and improvements.
IEEE SENSORS JOURNAL
(2021)
Article
Engineering, Electrical & Electronic
Mahdi Pedram, Ramesh Kumar Sah, Seyed Ali Rokni, Marjan Nourollahi, Hassan Ghasemzadeh
Summary: Advances in embedded systems have led to the integration of wearable sensors in health monitoring. However, due to the personalized nature of human movement and the limitations of embedded sensors, a resource-efficient framework is needed for real-time activity recognition.
IEEE SENSORS JOURNAL
(2022)
Article
Engineering, Electrical & Electronic
Myung-Kyu Yi, Wai-Kong Lee, Seong Oun Hwang
Summary: Human Activity Recognition (HAR) is an important part of human life care. Through rigorous analysis of various HAR datasets, we propose a lightweight approach using statistical feature extraction to discriminate between static and dynamic activities. By replacing the first-level ML classifier with this technique, we achieve higher accuracy with less computational and memory consumption. The proposed HAR method, combined with Random Forest and Convolutional Neural Networks, achieves state-of-the-art results and is practical for wearable devices using a single accelerometer.
IEEE TRANSACTIONS ON CONSUMER ELECTRONICS
(2023)
Article
Computer Science, Theory & Methods
Fuqiang Gu, Mu-Huan Chung, Mark Chignell, Shahrokh Valaee, Baoding Zhou, Xue Liu
Summary: This study provides a comprehensive survey on recent advances and challenges in human activity recognition (HAR) with deep learning, highlighting the lack of in-depth research on deep learning methods in HAR.
ACM COMPUTING SURVEYS
(2021)
Article
Engineering, Electrical & Electronic
Rebeen Ali Hamad, Longzhi Yang, Wai Lok Woo, Bo Wei
Summary: Existing models for human activity recognition based on sensor data have achieved state-of-the-art performances. However, training separate models for each domain is time-consuming and computationally expensive. To address this issue, we propose a multi-domain learning network that transfers knowledge across related domains and mitigates isolated learning paradigms using a shared representation.
IEEE SENSORS JOURNAL
(2022)
Article
Chemistry, Analytical
Mohamed Elshafei, Diego Elias Costa, Emad Shihab
Summary: This research investigates the impact of muscle fatigue on Human Activity Recognition (HAR) systems, using biceps concentration curls as an example. Findings show that fatigue prolongs completion time of later sets and decreases muscular endurance, leading to changes in data patterns and affecting the performance of subject-specific and cross-subject models. Feedforward Neural Network (FNN) exhibits the best performance in both types of models.
Article
Computer Science, Information Systems
Thien Huynh-The, Cam-Hao Hua, Nguyen Anh Tu, Dong-Seong Kim
Summary: This study proposes a fusion model for activity recognition by combining deep convolutional neural networks with traditional handcrafted features, achieving high accuracy in activity recognition in multisensor systems, outperforming other state-of-the-art approaches.
IEEE INTERNET OF THINGS JOURNAL
(2021)
Article
Engineering, Electrical & Electronic
Yves Luduvico Coelho, Francisco de Assis Souza dos Santos, Anselmo Frizera-Neto, Teodiano Freire Bastos-Filho
Summary: Human Activity Recognition (HAR) has gained increasing attention from researchers and industry, requiring the design of small, lightweight, powerful, and low-cost smart sensors for practical HAR systems on wearable devices. Edge computing presents an energy-efficient solution that offers real-time response and privacy requirements for HAR applications.
IEEE SENSORS JOURNAL
(2021)
Article
Computer Science, Artificial Intelligence
Ankit Thakkar, Kinjal Chaudhari
Summary: Investing in financial markets aims to gain higher benefits, but predicting market dynamics is challenging due to the complex nature of the market. Fusion techniques can integrate data and characteristics to enhance prediction accuracy in stock market applications. Major applications include stock price prediction, risk analysis, index prediction, and portfolio management.
INFORMATION FUSION
(2021)
Article
Computer Science, Artificial Intelligence
Zhiwen Xiao, Xin Xu, Huanlai Xing, Fuhong Song, Xinhan Wang, Bowen Zhao
Summary: This paper presents a federated learning system for human activity recognition, HARFLS, which includes a perceptive extraction network (PEN) that extracts features through a feature network and a relation network. Compared to other systems, it achieves higher recognition accuracy, especially on the WISDM and PAMAP2 datasets.
KNOWLEDGE-BASED SYSTEMS
(2021)
Article
Computer Science, Artificial Intelligence
Ayokunle Olalekan Ige, Mohd Halim Mohd Noor
Summary: With the development of deep learning, numerous models have been proposed for human activity recognition. However, activity recognition remains challenging due to the complexity of specific activity patterns. Existing models that address this challenge are often bulky and complex, making them unsuitable for resource-constrained embedded systems. This research proposes an efficient and lightweight deep learning model that achieves high recognition accuracy while minimizing resource consumption.
COMPUTATIONAL INTELLIGENCE
(2023)
Article
Computer Science, Theory & Methods
Lidia Fotia, Flavia Delicato, Giancarlo Fortino
Summary: The Internet of Things (IoT) enables smart objects to provide smart services inserted into information networks for human beings. The introduction of edge computing in IoT reduces decision-making latency, saves bandwidth resources, and expands cloud services at the network's edge. However, decentralized trust management poses challenges for edge-based IoT systems. Trust management is crucial for reliable mining and data fusion, improved user privacy and data security, and context-aware service provisioning.
ACM COMPUTING SURVEYS
(2023)
Review
Computer Science, Information Systems
Claudia Greco, Giancarlo Fortino, Bruno Crispo, Kim-Kwang Raymond Choo
Summary: This paper provides a comprehensive review of literature on penetration testing of IoT devices and systems. It identifies existing and potential IoT penetration testing applications and proposed approaches, and highlights recent advances in AI-enabled penetration testing methods at the network edge.
ENTERPRISE INFORMATION SYSTEMS
(2023)
Article
Computer Science, Information Systems
Xiuwen Fu, Pasquale Pace, Gianluca Aloi, Antonio Guerrieri, Wenfeng Li, Giancarlo Fortino
Summary: In this study, a interdependent network model for cyber-manufacturing systems (CMS) is developed based on the perspective of physical-service networking. The proposed realistic cascading failure model takes into account the load distribution characteristics of the physical network and the service network. The experiments confirm that attacks on the physical network are more likely to trigger cascading failures and cause more damage, and interdependency failures are the main cause of performance degradation in the service network during cascading failures, while isolation failures are the main cause of performance degradation in the physical network during cascading failures.
ACM TRANSACTIONS ON INTERNET TECHNOLOGY
(2023)
Article
Engineering, Electrical & Electronic
Long Liu, Jiayi Liu, Sen Qiu, Zhelong Wang, Hongyu Zhao, Masood Habib, Yongzhen Wang
Summary: This article proposes a motion-capture method based on inertial sensors to analyze the synchronized movements of two canoeists. The results show a significant correlation between the shoulder joint angles of the synchronized canoeists and the ability to analyze posture coordination. This research can help evaluate the synchronization effect of synchronized canoeing and improve technical movements, as well as assist coaches in selecting athletes with matching skills and styles.
IEEE SENSORS JOURNAL
(2023)
Review
Chemistry, Analytical
Roohallah Alizadehsani, Mohamad Roshanzamir, Navid Hoseini Izadi, Raffaele Gravina, H. M. Dipu Kabir, Darius Nahavandi, Hamid Alinejad-Rokny, Abbas Khosravi, U. Rajendra Acharya, Saeid Nahavandi, Giancarlo Fortino
Summary: Continuous advancements in technologies like the internet of things and big data analysis have enabled information sharing and smart decision-making using everyday devices. Swarm intelligence algorithms facilitate constructive interaction among individuals regardless of their intelligence level to address complex nonlinear problems. This paper examines the application of swarm intelligence algorithms in the internet of medical things, with a focus on wearable devices in healthcare. It reviews existing works on utilizing swarm intelligence in tackling IoMT problems such as disease prediction, data encryption, and resource allocation. The paper concludes with research perspectives and future trends.
Article
Chemistry, Analytical
Alaa Menshawi, Mohammad Mehedi Hassan, Nasser Allheeib, Giancarlo Fortino
Summary: A generic framework has been developed for heart problem diagnosis using a hybrid of machine learning and deep learning techniques. The framework utilizes a novel voting technique based on the prediction probabilities of multiple models to eliminate bias. Experimental results show that the framework outperforms single machine learning models, classical stacking techniques, and traditional voting techniques, achieving an accuracy of 95.6%.
Review
Chemistry, Analytical
Amira Bourechak, Ouarda Zedadra, Mohamed Nadjib Kouahla, Antonio Guerrieri, Hamid Seridi, Giancarlo Fortino
Summary: Given its advantages, edge computing has emerged as key support for intelligent applications and 5G/6G IoT networks. However, there are concerns about its capabilities to handle the computational complexity of machine learning techniques for big IoT data analytics. This paper aims to explore the confluence of AI and edge computing in various application domains to leverage existing research and identify new perspectives.
Article
Chemistry, Analytical
Huihui Wang, Bo Ru, Xin Miao, Qin Gao, Masood Habib, Long Liu, Sen Qiu
Summary: This paper investigates static and dynamic gesture recognition methods based on miniature inertial sensors. Hand-gesture data are obtained through a data glove and preprocessed using Butterworth low-pass filtering and normalization algorithms. The random forest algorithm achieves the highest recognition accuracy and shortest recognition time for static gestures. The addition of the attention mechanism significantly improves the recognition accuracy of the LSTM model for dynamic gestures, with a prediction accuracy of 98.3%, based on the original six-axis dataset.
Review
Computer Science, Artificial Intelligence
Vincenzo Barbuto, Claudio Savaglio, Min Chen, Giancarlo Fortino
Summary: The Edge Intelligence (EI) paradigm is a promising solution to the limitations of cloud computing in the development and provision of next-generation Internet of Things (IoT) services. This paper provides a systematic analysis of the state-of-the-art manuscripts on EI, exploring the past, present, and future directions of the EI paradigm and its relationships with IoT and cloud computing.
BIG DATA AND COGNITIVE COMPUTING
(2023)
Article
Engineering, Electrical & Electronic
Yi An, Jialin Wu, Yunhao Cui, Huosheng Hu
Summary: This paper proposes a multi-object tracking framework based on the multi-modal information of 3D point clouds and color images. The method combines point cloud and image data for object detection and constructs a height-intensity-density image for object tracking. It also introduces a new rotation kernel correlation filter for object prediction and develops object retention and re-recognition modules to overcome object matching failure. Experimental results on the KITTI dataset demonstrate that the proposed method outperforms existing traditional multi-object tracking methods.
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
(2023)
Article
Chemistry, Analytical
Ang Ji, Yongzhen Wang, Xin Miao, Tianqi Fan, Bo Ru, Long Liu, Ruicheng Nie, Sen Qiu
Summary: This study proposes a low-cost data glove solution that utilizes multiple inertial sensors to achieve efficient and accurate sign language recognition, enabling seamless communication between deaf and able-bodied individuals. Four machine learning models and an attention-based mechanism of long and short-term memory neural networks were employed to recognize 20 different types of dynamic sign language data. The results show that the proposed Attention-BiLSTM and RF algorithms have the highest performance in recognizing the twenty dynamic sign language gestures, with accuracies of 98.85% and 97.58% respectively, providing evidence for the feasibility of the proposed data glove and recognition methods. This study serves as a valuable reference for the development of wearable sign language recognition devices and promotes easier communication between deaf and able-bodied individuals.
Article
Engineering, Multidisciplinary
Aitizaz Ali, Muhammad Fermi Pasha, Antonio Guerrieri, Antonella Guzzo, Xiaobing Sun, Aamir Saeed, Amir Hussain, Giancarlo Fortino
Summary: This paper proposes a hybrid deep learning model for Industrial Internet of Medical Things (IIoMT) that addresses security challenges using homomorphic encryption (HE) and blockchain technology, providing higher privacy and security. By deploying a pre-trained model on edge devices and utilizing a consortium blockchain for data sharing and updating, the model can effectively classify and train local models while delivering higher efficiency and low latency.
IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING
(2023)
Article
Computer Science, Cybernetics
Zhihan Lv, Chen Cheng, Antonio Guerrieri, Giancarlo Fortino
Summary: More data are generated through mobile network technology, giving birth to the cyber-physical social intelligent ecosystem (C & P-SIE). This survey studies the development of physical social intelligence, discussing its applications in various domains such as intelligent transportation, healthcare, public service, economy, and social networking. It also explores the future prospects of behavior modeling in C & P-SIE under information security, data-driven techniques, and cooperative artificial intelligence technologies. This research provides a theoretical foundation and new opportunities for the digital and intelligent development of smart cities and social systems.
IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
(2023)
Article
Computer Science, Cybernetics
Giancarlo Fortino, Lidia Fotia, Fabrizio Messina, Domenico Rosaci, Giuseppe M. L. Sarne
Summary: This article introduces a multi-agent SIoT architecture that incorporates a reputation system based on clustering of smart objects, providing reliability for transactions in SIoT scenarios. By enabling feedback between smart objects, and communication between edge servers and the cloud, reputation values are updated, enhancing the trustworthiness of objects.
IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
(2023)
Article
Computer Science, Information Systems
Syed Tauhidun Nabi, Md. Rashidul Islam, Md. Golam Rabiul Alam, Mohammad Mehedi Hassan, Salman A. AlQahtani, Gianluca Aloi, Giancarlo Fortino
Summary: This research utilizes 6.2 million real network time series LTE data traffic and other associated parameters to build a traffic forecasting model using multivariate feature inputs and deep learning algorithms, which can forecast traffic at a granular eNodeB-level and provide eNodeB-wise forecasted PRB utilization.
Article
Computer Science, Artificial Intelligence
C. Lopez-Molina, S. Iglesias-Rey, B. De Baets
Summary: Quantitative image comparison is a critical topic in image processing literature, with diverse applications. Existing measures of comparison often overlook the context in which the comparison takes place. This paper presents a context-aware comparison method for binary images, tested on the BSDS500 benchmark.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Lorenz Linhardt, Klaus-Robert Mueller, Gregoire Montavon
Summary: This paper investigates the issue of mismatches between the decision strategy of the explainable model and the user's domain knowledge, and proposes a new method EGEM to mitigate hidden flaws in the model. Experimental results demonstrate that the approach can significantly reduce reliance on Clever Hans strategies and improve the accuracy of the model on new data.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Zhimin Shao, Weibei Dou, Yu Pan
Summary: This paper proposes a novel algorithm, Dual-level Deep Evidential Fusion (DDEF), to integrate multimodal information at both the BBA level and multimodal level, aiming to enhance accuracy, robustness, and reliability. The DDEF approach utilizes the Dirichlet framework and BBA methods for effective uncertainty estimation and employs the Dempster-Shafer Theory for dual-level fusion. The experimental results show that the proposed DDEF outperforms existing methods in synthetic digit classification and real-world medical prognosis after BCI treatment.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Abhishek K. Ghosh, Danilo S. Catelli, Samuel Wilson, Niamh C. Nowlan, Ravi Vaidyanathan
Summary: The inability of current FM monitoring methods to be used outside clinical environments has made it challenging to understand the nature and evolution of FM. This investigation introduces a novel wearable FM monitor with a heterogeneous sensor suite and a data fusion architecture to efficiently capture and separate FM from interfering artifacts. The performance of the device and architecture were validated through at-home use, demonstrating high accuracy in detecting and recognizing FM events. This research is a major milestone in the development of low-cost wearable FM monitors for pervasive monitoring of FM in unsupervised environments.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Jianlei Kong, Xiaomeng Fan, Min Zuo, Muhammet Deveci, Xuebo Jin, Kaiyang Zhong
Summary: In this study, we propose an intelligent traffic flow prediction framework based on the adaptive dual-graphic transformer with a cross-fusion strategy, aiming to uncover latent graphic feature representations that transcend temporal and spatial limitations. By establishing a traffic spatiotemporal prediction model using a cross-fusion attention mechanism, our proposed model achieves superior prediction performance on practical urban traffic flow datasets, particularly for long-term predictions.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Huilai Zhi, Jinhai Li
Summary: This article addresses the issue that conflict analysis based on single-valued information systems is no longer valid. It proposes a conflict analysis method based on component similarity, which uses three-way n-valued concept lattices to handle set-valued formal contexts and realizes fast conflict analysis from an information fusion viewpoint. Experimental results verify the effectiveness of this method in reducing time consumption.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Huchang Liao, Jiaxin Qi, Jiawei Zhang, Chonghui Zhang, Fan Liu, Weiping Ding
Summary: In this paper, a hospital selection approach based on a fuzzy multi-criterion decision-making method is proposed. This approach considers sentiment evaluation values of unstructured data from online reviews and structured data of public indexes simultaneously. The methodology involves collecting and processing online reviews, classifying topics and sentiments, quantifying sentiment analysis results using fuzzy numbers, and obtaining final preference scores of hospitals based on patients' preferences. A case study and robustness analysis are conducted to validate the effectiveness of the method.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Faramarz Farhangian, Rafael M. O. Cruz, George D. C. Cavalcanti
Summary: The proliferation of social networks has posed challenges in combating fake news, but automatic fake news detection using artificial intelligence has become more feasible. This paper revisits the definitions and perspectives of fake news and proposes an updated taxonomy, based on multiple criteria, for the field. The study finds that optimal feature extraction techniques vary depending on the dataset, and context-dependent models based on transformer models consistently exhibit superior performance.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Mariana A. Souza, Robert Sabourin, George D. C. Cavalcanti, Rafael M. O. Cruz
Summary: In this study, a dynamic selection technique is proposed to handle sparse and overlapped data. The technique leverages the relationships between instances and classifiers to learn a dynamic classifier combination rule. Experimental results show that the proposed method outperforms static selection and other dynamic selection techniques.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Bin Yu, Ruihui Xu, Mingjie Cai, Weiping Ding
Summary: This paper introduces a clustering method based on non-Euclidean metric and multi-granularity staged clustering to address the challenges posed by complex spatial structure data to traditional clustering methods. The method improves the similarity measure and employs an attenuation-diffusion pattern for local to global clustering, achieving good clustering results.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Jian Zhu, Pengbo Hu, Bingqian Li, Yi Zhou
Summary: The acquisition of multi-view hash representation for heterogeneous data is highly important for multimedia retrieval. Current approaches suffer from limited retrieval precision due to insufficient integration of multi-view features and failure to effectively utilize metric information from diverse samples. In this paper, we propose an innovative method called Fast Metric Multi-View Hashing (FMMVH), which demonstrates the superiority of gate-based fusion over traditional methods. We also introduce a novel deep metric loss function to leverage metric information from dissimilar samples. By optimizing and employing model compression techniques, our FMMVH method significantly outperforms existing state-of-the-art methods on benchmark datasets, with up to 7.47% improvement in mean Average Precision (mAP).
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Fayaz Ali Dharejo, Iyyakutti Iyappan Ganapathi, Muhammad Zawish, Basit Alawode, Moath Alathbah, Naoufel Werghi, Sajid Javed
Summary: The resource-limited nature of underwater vision equipment affects underwater robotics and ocean engineering tasks. Super-resolution methods, particularly using Vision Transformers (ViTs), have emerged to enhance low-resolution underwater images. However, ViTs face challenges in handling severe degradation in underwater imaging. In contrast, Multi-scale ViTs (MViTs) overcome these challenges by preserving long-range dependencies through evolving channel capacity. This study proposes a novel algorithm, SwinWave-SR, for efficient and accurate multi-scale super-resolution for underwater images.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Weiwei Jiang, Haoyu Han, Yang Zhang, Jianbin Mu
Summary: This study incorporates federated learning and split learning paradigms with satellite-terrestrial integrated networks and introduces a split-then-federated learning framework and federated split learning with long short-term memory to handle sequential data in STINs. The proposed solution is demonstrated to be effective through a case study of electricity theft detection based on a real-world dataset.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Najah Abuali, Mohammad Bilal Khan, Farman Ullah, Mohammad Hayajneh, Hikmat Ullah, Shahid Mumtaz
Summary: The demand for innovative solutions in biomedical systems for precise diagnosis and management of critical diseases is increasing. A promising technology, non-invasive and intelligent Internet of Medical Things (IoMT) system, emerges to assess patients with reduced health risks. This research introduces a comprehensive framework for early diagnosis of respiratory abnormalities through RF sensing and SDR technology. The results highlight the superior performance of deep learning frameworks in classifying respiratory anomalies.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Shichen Huang, Weina Fu, Zhaoyue Zhang, Shuai Liu
Summary: In the era of adversarial machine learning (AML), developing robust and generalized algorithms has become a key research topic. This study proposes a global similarity matching module and a global-local cognition fusion training mechanism based on relationship adversarial sample generation to improve image-text matching algorithm. Experimental results show significant improvements in accuracy and robustness, performing well in facing security challenges and promoting the fusion of visual and linguistic modalities.
INFORMATION FUSION
(2024)