Article
Engineering, Electrical & Electronic
Jing Li, Jianming Zhu, Chang Li, Xun Chen, Bin Yang
Summary: In this paper, a convolution-guided transformer fusion network for infrared and visible image fusion is proposed. This method considers both local features and long-range dependences by alternately using convolution modules and transformer modules to capture image features. Experimental results demonstrate the effectiveness of this method.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
(2022)
Article
Computer Science, Information Systems
Hongmei Wang, Lin Li, Chenkai Li, Xuanyu Lu
Summary: In this paper, a novel autoencoder-based image fusion network is proposed, which combines CNN and Transformer to simultaneously capture the local and global features of the source images. Additionally, contrast and gradient enhancement feature extraction blocks are designed for infrared and visible images respectively to maintain the information specific to each source. Experimental results demonstrate that the proposed network outperforms state-of-the-art methods in preserving both the clear target and detailed information of infrared and visible images.
Article
Engineering, Electrical & Electronic
Zhisheng Gao, Qiaolu Wang, Chenglin Zuo
Summary: This study proposes a total variation optimization framework based on deep learning for infrared and visible image fusion, which can obtain optimization solutions through neural network learning. The experiments show that this method has competitive advantages in image fusion.
SIGNAL IMAGE AND VIDEO PROCESSING
(2022)
Article
Instruments & Instrumentation
Xiaowen Liu, Jing Li, Xin Yang, Hongtao Huo
Summary: This paper proposes a novel infrared and visible image fusion method based on cross-modal extraction strategy (CMEVIF). By establishing an interactional relationship between dual independent extraction networks, the model is able to extract sufficient deep features. Experimental results demonstrate the effectiveness and superiority of this method in both objective metrics and visual impressions.
INFRARED PHYSICS & TECHNOLOGY
(2022)
Article
Optics
Bowen Wang, Yan Zou, Linfei Zhang, Yuhai Li, Qian Chen, Chao Zuo
Summary: In this paper, a deep-learning-based infrared-visible images fusion method is proposed, which utilizes an encoder-decoder architecture. By reformulating the image fusion task and designing corresponding loss functions, the proposed method maintains the structure and intensity ratio of the infrared-visible image. Experimental results demonstrate that the proposed method outperforms other state-of-the-art approaches in terms of visual effects and objective assessments, and can stably provide high-resolution reconstruction results.
OPTICS AND LASERS IN ENGINEERING
(2022)
Article
Computer Science, Artificial Intelligence
Xingchen Zhang, Yiannis Demiris
Summary: Visible and infrared image fusion (VIF) has gained considerable attention for its applications in various tasks, and there has been an increasing number of deep learning-based VIF methods proposed in recent years. This paper presents a comprehensive review of these methods, discussing motivation, taxonomy, recent developments, datasets, evaluation methods, and future prospects in detail. It serves as a valuable reference for VIF researchers and those interested in this rapidly developing field.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Information Systems
Kanika Bhalla, Deepika Koundal, Surbhi Bhatia, Mohammad Khalid Imam Rahmani, Muhammad Tahir
Summary: This study proposes a hybrid algorithm for integrating multi-features among two heterogeneous images. The algorithm utilizes fuzzification and siamese convolutional neural network to extract prominent features and high-frequency information, and produces focus maps containing detailed integrated information. The experimental results show that the proposed technique achieves the best qualitative and quantitative results compared to existing methods.
CMC-COMPUTERS MATERIALS & CONTINUA
(2022)
Article
Computer Science, Information Systems
Yong Yang, Chengrui Gao, Zhangqiang Ming, Jixiang Guo, Edou Leopold, Junlong Cheng, Jie Zuo, Min Zhu
Summary: In this paper, a fusion method for infrared and visible images, called LatLRR-CNN, is proposed, which combines latent low-rank representation (LatLRR) and convolutional neural network (CNN). This method can prevent information loss, lack of imaging quality, and the need for complex fusion rules or networks. The infrared or visible images are first decomposed into low-rank parts and salient parts using LatLRR, and then these two parts are separately fused using CNN. Finally, the fused low-rank part and the fused salient part are merged to obtain the fused image. Experimental results on publicly accessible datasets show that our method outperforms state-of-the-art methods in terms of objective metrics and visual effects. Specifically, the average of our method on the Nato sequence reaches 7.59, MI reaches 2.89, SD reaches 57.77, and VIf reaches 0.51.
MULTIMEDIA TOOLS AND APPLICATIONS
(2023)
Article
Engineering, Electrical & Electronic
Han Xu, Xinya Wang, Jiayi Ma
Summary: In this article, a novel decomposition method for visible and infrared image fusion (DRF) is proposed, which disentangles images into scene- and sensor modality-related representations and applies different fusion strategies, leading to comparable performance in terms of visual effect and quantitative metrics compared to the state of the art.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
(2021)
Article
Computer Science, Artificial Intelligence
Wei Tang, Fazhi He, Yu Liu
Summary: This paper proposes an infrared and visible image fusion method based on Transformer and cross-correlation, called TCCFusion. The method uses a local feature extraction branch and a global feature extraction branch to preserve local and global useful information. A cross-correlation loss is used to train the fusion model. Experimental results show that TCCFusion outperforms state-of-the-art algorithms in both visual quality and quantitative assessments.
PATTERN RECOGNITION
(2023)
Article
Computer Science, Information Systems
Yinhui Luo, Xingyi Wang, Yuezhou Wu, Chang Shu
Summary: In this work, a detail-aware deep homography estimation network is proposed to improve the performance of homography estimation for infrared and visible images. By designing a shallow feature extraction network and a detail feature loss, more detailed information is preserved and an adaptive feature registration rate is used for computation.
Article
Environmental Sciences
Qin Pu, Abdellah Chehri, Gwanggil Jeon, Lei Zhang, Xiaomin Yang
Summary: In remote sensing, the fusion of infrared and visible images is commonly used to synthesize a fused image with abundant common and differential information from the source images. Existing fusion networks based on deep learning fail to effectively integrate this information. To address this, we propose a dual-head fusion strategy and contextual information awareness fusion network (DCFusion) to preserve meaningful information. Firstly, we extract multi-scale features using multiple convolution and pooling layers. Then, we fuse the different modal features using a dual-headed fusion strategy (DHFS) from the encoder. Finally, we reconstruct the fused image using a contextual information awareness module (CIAM). Extensive experiments on MSRS and TNO datasets demonstrate good performance in target maintenance and texture preservation for fusion images.
Article
Instruments & Instrumentation
Jianming Zhang, Wenxin Lei, Shuyang Li, Zongping Li, Xudong Li
Summary: This paper proposes a novel algorithm for infrared and visible image fusion. The algorithm decomposes the input image into different layers using a guided filter, and then fuses them using entropy-based fusion module, maximum absolute value rule, and a mask-guided deep convolutional neural network. Experimental results demonstrate that the algorithm achieves good performance in both subjective and objective evaluation.
INFRARED PHYSICS & TECHNOLOGY
(2023)
Article
Instruments & Instrumentation
Shi Yi, Gang Jiang, Xi Liu, Junjie Li, Ling Chen
Summary: This study proposes an infrared and visible image fusion network with composite auto encoder and transformer-convolutional parallel mixed fusion strategy, which achieves outstanding feature fusion performance, preserves detailed source image information, reduces fusion noise level, and enhances contrast effect and edge sharpness of fused objects.
INFRARED PHYSICS & TECHNOLOGY
(2022)
Article
Engineering, Electrical & Electronic
Jinfu Li, Lei Liu, Hong Song, Yuqi Huang, Junjun Jiang, Jian Yang
Summary: This paper proposes a heterogeneous dual-branch multi-cascade fusion network, named DCTNet, for infrared and visible image fusion. The network exploits CNN and Transformer to handle features from different modalities and uses an adaptive fusion interaction module for feature fusion. Experimental results demonstrate that the proposed method outperforms existing methods in quantitative and qualitative evaluations and shows promising performance in downstream object detection and semantic segmentation applications.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
(2023)
Article
Computer Science, Artificial Intelligence
C. Lopez-Molina, S. Iglesias-Rey, B. De Baets
Summary: Quantitative image comparison is a critical topic in image processing literature, with diverse applications. Existing measures of comparison often overlook the context in which the comparison takes place. This paper presents a context-aware comparison method for binary images, tested on the BSDS500 benchmark.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Lorenz Linhardt, Klaus-Robert Mueller, Gregoire Montavon
Summary: This paper investigates the issue of mismatches between the decision strategy of the explainable model and the user's domain knowledge, and proposes a new method EGEM to mitigate hidden flaws in the model. Experimental results demonstrate that the approach can significantly reduce reliance on Clever Hans strategies and improve the accuracy of the model on new data.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Zhimin Shao, Weibei Dou, Yu Pan
Summary: This paper proposes a novel algorithm, Dual-level Deep Evidential Fusion (DDEF), to integrate multimodal information at both the BBA level and multimodal level, aiming to enhance accuracy, robustness, and reliability. The DDEF approach utilizes the Dirichlet framework and BBA methods for effective uncertainty estimation and employs the Dempster-Shafer Theory for dual-level fusion. The experimental results show that the proposed DDEF outperforms existing methods in synthetic digit classification and real-world medical prognosis after BCI treatment.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Abhishek K. Ghosh, Danilo S. Catelli, Samuel Wilson, Niamh C. Nowlan, Ravi Vaidyanathan
Summary: The inability of current FM monitoring methods to be used outside clinical environments has made it challenging to understand the nature and evolution of FM. This investigation introduces a novel wearable FM monitor with a heterogeneous sensor suite and a data fusion architecture to efficiently capture and separate FM from interfering artifacts. The performance of the device and architecture were validated through at-home use, demonstrating high accuracy in detecting and recognizing FM events. This research is a major milestone in the development of low-cost wearable FM monitors for pervasive monitoring of FM in unsupervised environments.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Jianlei Kong, Xiaomeng Fan, Min Zuo, Muhammet Deveci, Xuebo Jin, Kaiyang Zhong
Summary: In this study, we propose an intelligent traffic flow prediction framework based on the adaptive dual-graphic transformer with a cross-fusion strategy, aiming to uncover latent graphic feature representations that transcend temporal and spatial limitations. By establishing a traffic spatiotemporal prediction model using a cross-fusion attention mechanism, our proposed model achieves superior prediction performance on practical urban traffic flow datasets, particularly for long-term predictions.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Huilai Zhi, Jinhai Li
Summary: This article addresses the issue that conflict analysis based on single-valued information systems is no longer valid. It proposes a conflict analysis method based on component similarity, which uses three-way n-valued concept lattices to handle set-valued formal contexts and realizes fast conflict analysis from an information fusion viewpoint. Experimental results verify the effectiveness of this method in reducing time consumption.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Huchang Liao, Jiaxin Qi, Jiawei Zhang, Chonghui Zhang, Fan Liu, Weiping Ding
Summary: In this paper, a hospital selection approach based on a fuzzy multi-criterion decision-making method is proposed. This approach considers sentiment evaluation values of unstructured data from online reviews and structured data of public indexes simultaneously. The methodology involves collecting and processing online reviews, classifying topics and sentiments, quantifying sentiment analysis results using fuzzy numbers, and obtaining final preference scores of hospitals based on patients' preferences. A case study and robustness analysis are conducted to validate the effectiveness of the method.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Faramarz Farhangian, Rafael M. O. Cruz, George D. C. Cavalcanti
Summary: The proliferation of social networks has posed challenges in combating fake news, but automatic fake news detection using artificial intelligence has become more feasible. This paper revisits the definitions and perspectives of fake news and proposes an updated taxonomy, based on multiple criteria, for the field. The study finds that optimal feature extraction techniques vary depending on the dataset, and context-dependent models based on transformer models consistently exhibit superior performance.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Mariana A. Souza, Robert Sabourin, George D. C. Cavalcanti, Rafael M. O. Cruz
Summary: In this study, a dynamic selection technique is proposed to handle sparse and overlapped data. The technique leverages the relationships between instances and classifiers to learn a dynamic classifier combination rule. Experimental results show that the proposed method outperforms static selection and other dynamic selection techniques.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Bin Yu, Ruihui Xu, Mingjie Cai, Weiping Ding
Summary: This paper introduces a clustering method based on non-Euclidean metric and multi-granularity staged clustering to address the challenges posed by complex spatial structure data to traditional clustering methods. The method improves the similarity measure and employs an attenuation-diffusion pattern for local to global clustering, achieving good clustering results.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Jian Zhu, Pengbo Hu, Bingqian Li, Yi Zhou
Summary: The acquisition of multi-view hash representation for heterogeneous data is highly important for multimedia retrieval. Current approaches suffer from limited retrieval precision due to insufficient integration of multi-view features and failure to effectively utilize metric information from diverse samples. In this paper, we propose an innovative method called Fast Metric Multi-View Hashing (FMMVH), which demonstrates the superiority of gate-based fusion over traditional methods. We also introduce a novel deep metric loss function to leverage metric information from dissimilar samples. By optimizing and employing model compression techniques, our FMMVH method significantly outperforms existing state-of-the-art methods on benchmark datasets, with up to 7.47% improvement in mean Average Precision (mAP).
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Fayaz Ali Dharejo, Iyyakutti Iyappan Ganapathi, Muhammad Zawish, Basit Alawode, Moath Alathbah, Naoufel Werghi, Sajid Javed
Summary: The resource-limited nature of underwater vision equipment affects underwater robotics and ocean engineering tasks. Super-resolution methods, particularly using Vision Transformers (ViTs), have emerged to enhance low-resolution underwater images. However, ViTs face challenges in handling severe degradation in underwater imaging. In contrast, Multi-scale ViTs (MViTs) overcome these challenges by preserving long-range dependencies through evolving channel capacity. This study proposes a novel algorithm, SwinWave-SR, for efficient and accurate multi-scale super-resolution for underwater images.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Weiwei Jiang, Haoyu Han, Yang Zhang, Jianbin Mu
Summary: This study incorporates federated learning and split learning paradigms with satellite-terrestrial integrated networks and introduces a split-then-federated learning framework and federated split learning with long short-term memory to handle sequential data in STINs. The proposed solution is demonstrated to be effective through a case study of electricity theft detection based on a real-world dataset.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Najah Abuali, Mohammad Bilal Khan, Farman Ullah, Mohammad Hayajneh, Hikmat Ullah, Shahid Mumtaz
Summary: The demand for innovative solutions in biomedical systems for precise diagnosis and management of critical diseases is increasing. A promising technology, non-invasive and intelligent Internet of Medical Things (IoMT) system, emerges to assess patients with reduced health risks. This research introduces a comprehensive framework for early diagnosis of respiratory abnormalities through RF sensing and SDR technology. The results highlight the superior performance of deep learning frameworks in classifying respiratory anomalies.
INFORMATION FUSION
(2024)
Article
Computer Science, Artificial Intelligence
Shichen Huang, Weina Fu, Zhaoyue Zhang, Shuai Liu
Summary: In the era of adversarial machine learning (AML), developing robust and generalized algorithms has become a key research topic. This study proposes a global similarity matching module and a global-local cognition fusion training mechanism based on relationship adversarial sample generation to improve image-text matching algorithm. Experimental results show significant improvements in accuracy and robustness, performing well in facing security challenges and promoting the fusion of visual and linguistic modalities.
INFORMATION FUSION
(2024)