Review
Engineering, Electrical & Electronic
Haithem Hermessi, Olfa Mourali, Ezzeddine Zagrouba
Summary: The article discusses the development and application of medical image fusion methods, exploring the theoretical backgrounds and approaches of different fusion categories, summarizing the pros and cons of each category, and proposing directions for future research.
Review
Computer Science, Information Systems
Mohammed Ali Saleh, AbdElmgeid A. A. Ali, Kareem Ahmed, Abeer M. M. Sarhan
Summary: Image fusion is a promising field in image processing, especially in medical diagnosis and clarification of medical images. It enhances the quality of medical images by combining multiple images from different modalities. Choosing the best fusion technique is a crucial task in image fusion assessment.
Review
Biology
Muhammad Adeel Azam, Khan Bahadar Khan, Sana Salahuddin, Eid Rehman, Sajid Ali Khan, Muhammad Attique Khan, Seifedine Kadry, Amir H. Gandomi
Summary: This article provides a comprehensive overview of multimodal medical image fusion methodologies, databases, and quality measurements. Medical imaging modalities are categorized based on radiation, visible-light imaging, microscopy, and multimodal imaging. Fusion techniques are classified into categories including frequency fusion, spatial fusion, decision-level fusion, deep learning, hybrid fusion, and sparse representation fusion. The associated diseases for each modality and fusion approach are presented, and quality assessment fusion metrics are also discussed.
COMPUTERS IN BIOLOGY AND MEDICINE
(2022)
Review
Computer Science, Information Systems
Sajid Ullah Khan, Mir Ahmad Khan, Muhammad Azhar, Faheem Khan, Youngmoon Lee, Muhammad Javed
Summary: Medical imaging has been widely used in diagnosing various disorders, but the challenge lies in accurate disease identification and improved therapies. Multi modal image fusion (MMIF) aims to combine complementary information from different imaging modalities to improve the quality and clear assessment of medical related problems. This review provides a detailed overview of medical imaging modalities, multimodal medical image databases, MMIF steps/rules, methods, performance evaluation, and future directions. It is expected to be valuable in developing more effective medical image fusion methods for clinical diagnosis.
JOURNAL OF KING SAUD UNIVERSITY-COMPUTER AND INFORMATION SCIENCES
(2023)
Article
Engineering, Electrical & Electronic
Chengchao Wang, Rencan Nie, Jinde Cao, Xue Wang, Ying Zhang
Summary: Multimodal medical image fusion is a technique that aims to merge saliency and complementary information from different source images to assist in biomedical diagnoses. In this paper, the authors propose a new information gate network (IGNFusion) and a Siamese multi-scale cross attention fusion module (SMSCAFM) to optimize the fusion process, achieving significant improvements over existing methods on multiple datasets.
IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING
(2022)
Article
Mathematical & Computational Biology
Shuaiqi Liu, Mingwang Wang, Lu Yin, Xiuming Sun, Yu-Dong Zhang, Jie Zhao
Summary: Medical image fusion algorithm based on structure preservation and deep learning is proposed in this study. The algorithm decomposes the source images into base layer components and detail layer components using a two-scale decomposition method. It then utilizes an iterative joint bilateral filter and a convolutional neural network to fuse the components of the base layer and detail layer, respectively. The experimental results demonstrate that the proposed algorithm outperforms state-of-the-art techniques in medical image fusion.
FRONTIERS IN COMPUTATIONAL NEUROSCIENCE
(2022)
Article
Medicine, General & Internal
Maruturi Haribabu, Velmathi Guruviah
Summary: Multimodal medical image fusion (MMIF) is the process of merging different modalities of medical images into a single output image to improve clinical applicability. A new approach to intuitionistic fuzzy set-based MMIF has been proposed, which includes fuzzification, intuitionistic fuzzy image creation, and fusion rule. The proposed method provides a better-quality fused image and outperforms existing methods in terms of various performance metrics.
Article
Engineering, Electrical & Electronic
Yanyu Liu, Ruichao Hou, Dongming Zhou, Rencan Nie, Zhaisheng Ding, Yanbu Guo, Li Zhao
Summary: This method utilizes spectral total variation transform for multimodal medical image fusion, decomposing source images into different components and merging them using local structural patch measurement and spatial frequency dual-channel spiking cortical model, resulting in more competitive and satisfactory results.
INTERNATIONAL JOURNAL OF IMAGING SYSTEMS AND TECHNOLOGY
(2021)
Article
Computer Science, Artificial Intelligence
Junchao Zhang, Yidong Luo, Junbin Huang, Ying Liu, Jiayi Ma
Summary: A multi-exposure image fusion method based on perception enhanced structural patch decomposition (PESPD-MEF) is proposed, which decomposes and fuses image patches to generate informative and perception-realistic results. The method outperforms state-of-the-art methods in terms of perceptual realism for both multi-exposure images and single low-light image enhancement.
INFORMATION FUSION
(2023)
Review
Computer Science, Information Systems
Shatabdi Basu, Sunita Singhal, Dilbag Singh
Summary: Medical image fusion is a relevant field that has wide applications in disease diagnosis and prediction. It aims to combine multiple images of the same or different modality to enhance the image content and provide more information about diseases using easily available image scans. Multimodal medical image fusion can improve the quality and accuracy of medical images for diagnosis and treatment planning.
MULTIMEDIA TOOLS AND APPLICATIONS
(2023)
Article
Neurosciences
Yi Li, Junli Zhao, Zhihan Lv, Zhenkuan Pan
Summary: This article introduces a multimode medical image fusion method using CNN and supervised learning, which effectively improves fusion effect, image detail clarity, and time efficiency. Experimental results indicate that this method performs well in terms of visual quality and various quantitative evaluation criteria.
FRONTIERS IN NEUROSCIENCE
(2021)
Article
Computer Science, Artificial Intelligence
Wei Tang, Fazhi He, Yu Liu, Yansong Duan
Summary: Multimodal medical image fusion, the merging of information from different modalities, is crucial for comprehensive diagnosis and surgical navigation. Existing deep learning-based methods have improved fusion results but still lack satisfactory performance. In this study, we propose an unsupervised method called MATR that uses a multiscale adaptive Transformer. MATR achieves accurate fusion by adapting the convolutional kernel based on global context and enhancing global semantic extraction. The network architecture is designed to capture useful multimodal information from different scales. The proposed method outperforms other methods in visual quality and quantitative evaluation, and shows good generalization capability.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2022)
Article
Engineering, Biomedical
Yuan Gao, Shiwei Ma, Jingjing Liu, Yanyan Liu, Xianxia Zhang
Summary: An improved image fusion algorithm based on visual salience detection is proposed in this study to fuse the salient features of different modal medical images into a single image, effectively improving the visual quality of the image and preserving the salient features of tissues. By applying this multimodal medical image fusion method, the proposed method shows advantages in retaining image salient features with higher objective indexes, clearer edge contour, higher overall contrast, and no ringing effect and artifacts compared to other models.
BIOMEDICAL SIGNAL PROCESSING AND CONTROL
(2021)
Article
Geochemistry & Geophysics
Yuli Sun, Lin Lei, Li Liu, Gangyao Kuang
Summary: Multimodal change detection (MCD) is a challenging topic in remote sensing due to the unavailability of directly comparing multimodal images. This article proposes a structural regression fusion (SRF)-based method to reduce the influence of structural asymmetry and improve image transformation performance in MCD. SRF incorporates fusion into the regression process and yields three types of constraints to perform the fused image transformation. The proposed SRF is verified on six real datasets and outperforms some state-of-the-art methods.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2023)
Article
Computer Science, Information Systems
Muhammad Touseef Irshad, Hafeez Ur Rehman
Summary: In medical imaging, the use of different imaging modalities has significantly improved diagnostic information available to physicians. However, existing methods for medical image fusion suffer from loss of key features and introduction of unwanted artifacts. In this study, a method based on gradient compass is proposed, which effectively fuses a pair of multimodal medical images by constructing edge maps and performing adaptive pixel fusion based on statistical properties. Evaluation on CT and MRI images show that the proposed algorithm outperforms existing methods by transferring only relevant information from the source image to the fused image.
Article
Geochemistry & Geophysics
Hangyuan Lu, Yong Yang, Shuying Huang, Wei Tu
Summary: This letter proposes an efficient pansharpening approach based on texture correction and detail refinement to obtain high-quality high spatial resolution multispectral images. Experimental results demonstrate the high efficiency and quality of this method.
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS
(2022)
Article
Geochemistry & Geophysics
Yong Yang, Wei Tu, Shuying Huang, Hangyuan Lu, Weiguo Wan, Lixin Gan
Summary: The paper proposes a novel dual-stream convolutional neural network with residual information enhancement (DSCNN-RIE) for pansharpening, which can efficiently extract spatial information at different resolutions and transfer complementary information between different resolutions, thereby improving the quality of the pansharpened image.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2022)
Article
Computer Science, Artificial Intelligence
Hangyuan Lu, Yong Yang, Shuying Huang, Wei Tu, Weiguo Wan
Summary: A unified pansharpening model based on band-adaptive gradient and detail correction is proposed in this study, achieving accurate spatial structure for the estimated HRMS image. By exploring gradient relationship and defining detail correction constraint, the proposed method outperforms state-of-the-art pansharpening methods in terms of fusion quality and computational efficiency.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2022)
Article
Computer Science, Theory & Methods
Yong Yang, Wei Hu, Shuying Huang, Wei Tu, Weiguo Wan
Summary: This paper proposes an image enhancement network based on multi-stream information supplement, which utilizes an information complementary module to obtain feature information from structures of various scales, and employs a joint loss function to guide the network training, achieving better performance in low-light image restoration.
MULTIDIMENSIONAL SYSTEMS AND SIGNAL PROCESSING
(2022)
Article
Engineering, Electrical & Electronic
Yong Yang, Chenxu Wan, Shuying Huang, Hangyuan Lu, Weiguo Wan
Summary: This paper proposes a new pansharpening approach based on adaptive high-frequency fusion and injection coefficients optimization, which can accurately fuse a panchromatic image with a multispectral image to generate a high spatial-resolution multispectral image.
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING
(2023)
Article
Geochemistry & Geophysics
Hangyuan Lu, Yong Yang, Shuying Huang, Xiaolong Chen, Biwei Chi, Aizhu Liu, Wei Tu
Summary: DL-based pansharpening methods have advantages in extracting spectral-spatial features, but often ignore the local inner connection between source images and HRMS. To address this, a lightweight network based on AWFLN is proposed, which includes a detail extraction model and a residual multiple receptive-field structure to fully extract features. Experimental results show that AWFLN outperforms traditional and state-of-the-art methods in terms of subjective and objective evaluations.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2023)
Article
Computer Science, Artificial Intelligence
Yong Yang, Wei Hu, Shuying Huang, Weiguo Wan, Juwei Guan
Summary: In this paper, a progressive dehazing network (PDN) is proposed to gradually remove haze by constructing preliminary and fine dehazing modules. Experimental results show that the proposed method outperforms some state-of-the-art dehazing methods in terms of visual comparison and objective evaluation.
INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS
(2023)
Article
Computer Science, Artificial Intelligence
Zhiwei Liu, Menghan Gan, Li Xiong, Xiaofeng Mao, Yue Que
Summary: This paper introduces a method to improve the detection performance of small objects by using Swin Transformer as the backbone network and proposing a multilevel receptive field expansion network (MRFENet). The MRFENet, combined with receptive field expansion blocks (RFEBs), retains small object context cues and acquires receptive fields for adaptive detection tasks. A union loss function is designed to enhance the localization ability. Experimental results on the MS COCO dataset show that MRFENet has a significant improvement compared to other state-of-the-art methods, validating its effectiveness in utilizing small object information.
IET IMAGE PROCESSING
(2023)
Article
Engineering, Electrical & Electronic
Shuying Huang, Zixiang Song, Yong Yang, Weiguo Wan, Xiangkai Kong
Summary: This article proposes a multiattention generative adversarial network (MAGAN) for infrared and visible image fusion, which achieves image feature extraction and fusion through a multiattention generator and two multiattention discriminators. Experimental results show that MAGAN outperforms some state-of-the-art models in terms of visual effects and quantitative metrics.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
(2023)
Article
Geochemistry & Geophysics
Zhao Su, Yong Yang, Shuying Huang, Weiguo Wan, Jiancheng Sun, Wei Tu, Changjie Chen
Summary: This research proposes a novel synergistic transformer and CNN method for pansharpening. It utilizes a parallel U-shaped feature extraction module to extract features from LRMS and PAN images, and then integrates the features using a feature fusion module to achieve high-quality pansharpening results.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2023)
Article
Geochemistry & Geophysics
Wei Tu, Yong Yang, Shuying Huang, Weiguo Wan, Lixin Gan, Hangyuan Lu
Summary: Pansharpening is a remote sensing image processing technology that generates a high-resolution multispectral image by fusing low-resolution multispectral and panchromatic images. This article proposes an end-to-end multiscale and multidistillation dilated network (MMDN) for pansharpening. The MMDN utilizes a clique structure-based multiscale dilated block and a multidistillation residual information block to extract spatial details and capture the spatial structure at different scales. Experimental results demonstrate the superiority of the MMDN method in objective and subjective evaluations.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2022)
Article
Engineering, Electrical & Electronic
Yong Yang, Danjie Zhang, Weiguo Wan, Shuying Huang
Summary: A novel multiscale exposure image fusion method based on multivisual feature measurement and detail enhancement is proposed, which achieves better fusion performance by measuring the visual features of the source images and optimizing the weight maps.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
(2022)
Article
Computer Science, Information Systems
Yong Yang, Juwei Guan, Shuying Huang, Weiguo Wan, Yating Xu, Jiaxiang Liu
Summary: This paper proposes an end-to-end rain removal network based on the progressive residual detail supplement method. By iterative design, it removes rain information from coarse to fine, retains more edge details, and achieves high-quality results for image denoising tasks.
IEEE TRANSACTIONS ON MULTIMEDIA
(2022)
Article
Engineering, Electrical & Electronic
Yong Yang, Hangyuan Lu, Shuying Huang, Weiguo Wan, Luyi Li
Summary: This article presents a novel pansharpening method based on the VFOG model and optimized injection gains to address the issues of spectral distortion and difficulties in obtaining appropriate injection gains in existing pansharpening techniques. Experimental results demonstrate that the proposed method offers superior spectral and spatial fidelity compared to existing algorithms.
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING
(2022)
Article
Engineering, Electrical & Electronic
Yong Yang, Wang Zhang, Shuying Huang, Weiguo Wan, Jiaxiang Liu, Xiangkai Kong
Summary: This article presents a novel infrared and visible image fusion method based on dual-kernel side window filtering and detail optimization with S-shaped curve transformation. The proposed method achieves better fusion performance through saliency region highlighting and detail optimization.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
(2022)