4.7 Article

Iterative structure transformation and conditional random field based method for unsupervised multimodal change detection

Journal

PATTERN RECOGNITION
Volume 131, Issue -, Pages -

Publisher

ELSEVIER SCI LTD
DOI: 10.1016/j.patcog.2022.108845

Keywords

Unsupervised change detection; KNN graph; Image transformation; Multimodal; Conditional random field

Funding

  1. National Natural Sci-ence Foundation of China [61971426, 12171481, 4210010534]
  2. Natural Science Basic Research Plan in Shaanxi Province of China [2022JQ-694]

Ask authors/readers for more resources

This paper proposes an unsupervised iterative structure transformation and conditional random field (IST-CRF) based multimodal change detection (MCD) method, which combines structure transformation with a random field framework to obtain an optimal change map within a global probabilistic model. Experimental results demonstrate the effectiveness of the proposed method.
Change detection between heterogeneous images has become an increasingly interesting research topic in remote sensing. The different appearances and statistics of heterogeneous images bring great challenges to this task. In this paper, we propose an unsupervised iterative structure transformation and conditional random field (IST-CRF) based multimodal change detection (MCD) method, combining an imaging modality-invariant based structure transformation method with a random filed framework specifically designed for MCD, to acquire an optimal change map within a global probabilistic model. IST-CRF first constructs graphs to represent the structures of the images, and transforms the heterogeneous images to the same differential domain by using graph based forward and backward structure transformations. Then, the change vectors are calculated to distinguish the changed and unchanged areas. Finally, in order to classify the change vectors and compute the binary change map, a CRF model is designed to fully explore the spectral-spatial information, which incorporates the change information, local spatially-adjacent neighbor information, and global spectrally-similar neighbor information with a random field framework. As the changed samples will influence the structure transformation and reduce the quality of change vectors, we use an iterative framework to propagate the CRF segmentation results back to the structure transformation process that removes the changed samples, and thus improve the accuracy of change detection. Experiments conducted on different real data sets show the effectiveness of IST-CRF. Source code of the proposed method will be made available at https://github.com/yulisun/IST-CRF .(c) 2022 Elsevier Ltd. All rights reserved.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Computer Science, Artificial Intelligence

Nonlocal patch similarity based heterogeneous remote sensing change detection

Yuli Sun, Lin Lei, Xiao Li, Hao Sun, Gangyao Kuang

Summary: This paper proposes a new change detection method based on similarity measurement between heterogeneous images, which constructs a graph for each patch to establish a connection between heterogeneous data and measures the change level by comparing graph structures, achieving robust change detection results without leakage of heterogeneous data.

PATTERN RECOGNITION (2021)

Article Geochemistry & Geophysics

A Cross-Layer Nonlocal Network for Remote Sensing Scene Classification

Ming Li, Lin Lei, Yuli Sun, Xiao Li, Gangyao Kuang

Summary: In this letter, a novel scene classification framework called CL-NL-Net is proposed to improve global scene understanding ability by capturing long-range correlations between different layers. Experimental results show that the method is competitive in terms of classification accuracy.

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS (2022)

Article Geochemistry & Geophysics

Adaptive Local Structure Consistency-Based Heterogeneous Remote Sensing Change Detection

Lin Lei, Yuli Sun, Gangyao Kuang

Summary: This study proposes an unsupervised change detection method based on adaptive local structure consistency for heterogeneous remote sensing images. Experimental results demonstrate the effectiveness of the proposed method.

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS (2022)

Article Geochemistry & Geophysics

Sparse-Constrained Adaptive Structure Consistency-Based Unsupervised Image Regression for Heterogeneous Remote- Sensing Change Detection

Yuli Sun, Lin Lei, Dongdong Guan, Ming Li, Gangyao Kuang

Summary: The study proposes an unsupervised image regression-based change detection method using structure consistency, which adaptively constructs a similarity graph and a superpixel-based Markovian segmentation model to detect changes effectively.

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING (2022)

Article Geochemistry & Geophysics

Dynamic-Hierarchical Attention Distillation With Synergetic Instance Selection for Land Cover Classification Using Missing Heterogeneity Images

Xiao Li, Lin Lei, Yuli Sun, Gangyao Kuang

Summary: This article proposes a novel dynamic-hierarchical attention distillation network (DH-ADNet) with multimodal synergetic instance selection (MSIS) for land cover classification using missing data modalities. The method utilizes multimodal data and attention mechanism to achieve better classification performance.

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING (2022)

Article Geochemistry & Geophysics

Structure Consistency-Based Graph for Unsupervised Change Detection With Homogeneous and Heterogeneous Remote Sensing Images

Yuli Sun, Lin Lei, Xiao Li, Xiang Tan, Gangyao Kuang

Summary: This article proposes a structure consistency-based method for change detection in remote sensing images. By comparing the structures of two images instead of pixel values, the method demonstrates strong robustness and applicability to various scenarios. Additionally, the method shows effectiveness in both homogeneous and heterogeneous change detection, as well as in the seldom-studied case of heterogeneous change detection with multichannel synthetic aperture radar (SAR) images. Through analysis and improvements on the nonlocal patch-based graph (NLPG), the method is made more accurate and robust.

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING (2022)

Article Geography, Physical

Structured graph based image regression for unsupervised multimodal change detection

Yuli Sun, Lin Lei, Xiang Tan, Dongdong Guan, Junzheng Wu, Gangyao Kuang

Summary: This research proposes an unsupervised image regression method based on the inherent structure consistency between heterogeneous images for change detection in multimodal remote sensing images. The proposed method effectively addresses the problem of comparing heterogeneous images and achieves improved detection accuracy compared to state-of-the-art algorithms.

ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING (2022)

Article Environmental Sciences

Auto-Weighted Structured Graph-Based Regression Method for Heterogeneous Change Detection

Lingjun Zhao, Yuli Sun, Lin Lei, Siqian Zhang

Summary: This paper proposes an auto-weighted structured graph (AWSG)-based regression method for heterogeneous change detection using remote sensing images. The method learns the image structure and performs structure regression to detect changes. Experimental results and comparisons demonstrate the effectiveness of the proposed approach.

REMOTE SENSING (2022)

Article Computer Science, Information Systems

Cooperative Multi-Agent Reinforcement-Learning-Based Distributed Dynamic Spectrum Access in Cognitive Radio Networks

Xiang Tan, Li Zhou, Haijun Wang, Yuli Sun, Haitao Zhao, Boon-Chong Seet, Jibo Wei, Victor C. M. Leung

Summary: This article investigates the distributed dynamic spectrum access problem for multiusers in a cognitive radio network. By utilizing deep recurrent Q-networks and cooperative multi-agent reinforcement learning, a distributed offline training and online execution framework is proposed to maximize network throughput. Experimental results show that the algorithm outperforms the state-of-the-art in terms of successful access rate and collision rate.

IEEE INTERNET OF THINGS JOURNAL (2022)

Article Environmental Sciences

Change Alignment-Based Image Transformation for Unsupervised Heterogeneous Change Detection

Kuowei Xiao, Yuli Sun, Lin Lei

Summary: This study proposes a change detection framework based on change alignment, utilizing complementary information from forward and backward difference maps to enhance domain transformation effects and improve heterogeneous CD performance.

REMOTE SENSING (2022)

Article Geochemistry & Geophysics

Structural Regression Fusion for Unsupervised Multimodal Change Detection

Yuli Sun, Lin Lei, Li Liu, Gangyao Kuang

Summary: Multimodal change detection (MCD) is a challenging topic in remote sensing due to the unavailability of directly comparing multimodal images. This article proposes a structural regression fusion (SRF)-based method to reduce the influence of structural asymmetry and improve image transformation performance in MCD. SRF incorporates fusion into the regression process and yields three types of constraints to perform the fused image transformation. The proposed SRF is verified on six real datasets and outperforms some state-of-the-art methods.

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING (2023)

Article Geochemistry & Geophysics

Change Alignment-Based Graph Structure Learning for Unsupervised Heterogeneous Change Detection

Kuowei Xiao, Yuli Sun, Gangyao Kuang, Lin Lei

Summary: This paper proposes a new method called CAGSL for heterogeneous change detection, which uses graph structure learning and change alignment constraint to improve the detection accuracy, and its effectiveness is demonstrated through experiments.

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS (2023)

Article Geochemistry & Geophysics

Graph Signal Processing for Heterogeneous Change Detection

Yuli Sun, Lin Lei, Dongdong Guan, Gangyao Kuang, Li Liu

Summary: This article presents a new strategy for solving the heterogeneous change detection (HCD) problem using graph signal processing (GSP). The authors analyze the HCD problem from both the vertex domain and the spectral domain, proposing filtering methods and spectral analysis models for detecting changes between heterogeneous images. The effectiveness of these methods is demonstrated through experiments on real datasets.

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING (2022)

Article Computer Science, Artificial Intelligence

Image Regression With Structure Cycle Consistency for Heterogeneous Change Detection

Yuli Sun, Lin Lei, Dongdong Guan, Junzheng Wu, Gangyao Kuang, Li Liu

Summary: This study proposes an image regression method based on structure cycle consistency for change detection in heterogeneous images. The method captures the structure information of the image by constructing a similarity relationship-based graph and performs structure-based regression using an adaptive-weighted distance metric. Experimental results demonstrate the excellent performance of the proposed method.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2022)

Article Computer Science, Artificial Intelligence

Exploiting sublimated deep features for image retrieval

Guang-Hai Liu, Zuo-Yong Li, Jing-Yu Yang, David Zhang

Summary: This article introduces a novel image retrieval method that improves retrieval performance by using sublimated deep features. The method incorporates orientation-selective features and color perceptual features, effectively mimicking these mechanisms to provide a more discriminating representation.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

Region-adaptive and context-complementary cross modulation for RGB-T semantic segmentation

Fengguang Peng, Zihan Ding, Ziming Chen, Gang Wang, Tianrui Hui, Si Liu, Hang Shi

Summary: RGB-Thermal (RGB-T) semantic segmentation is an emerging task that aims to improve the robustness of segmentation methods under extreme imaging conditions by using thermal infrared modality. The challenges of foreground-background distinguishment and complementary information mining are addressed by proposing a cross modulation process with two collaborative components. Experimental results show that the proposed method achieves state-of-the-art performances on current RGB-T segmentation benchmarks.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

F-SCP: An automatic prompt generation method for specific classes based on visual language pre-training models

Baihong Han, Xiaoyan Jiang, Zhijun Fang, Hamido Fujita, Yongbin Gao

Summary: This paper proposes a novel automatic prompt generation method called F-SCP, which focuses on generating accurate prompts for low-accuracy classes and similar classes. Experimental results show that our approach outperforms state-of-the-art methods on six multi-domain datasets.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

Residual Deformable Convolution for better image de-weathering

Huikai Liu, Ao Zhang, Wenqian Zhu, Bin Fu, Bingjian Ding, Shengwu Xiong

Summary: Adverse weather conditions present challenges for computer vision tasks, and image de-weathering is an important component of image restoration. This paper proposes a multi-patch skip-forward structure and a Residual Deformable Convolutional module to improve feature extraction and pixel-wise reconstruction.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

A linear transportation LP distance for pattern recognition

Oliver M. Crook, Mihai Cucuringu, Tim Hurst, Carola-Bibiane Schonlieb, Matthew Thorpe, Konstantinos C. Zygalakis

Summary: The transportation LP distance (TLP) is a generalization of the Wasserstein WP distance that can be applied directly to color or multi-channelled images, as well as multivariate time-series. TLP interprets signals as functions, while WP interprets signals as measures. Although both distances are powerful tools in modeling data with spatial or temporal perturbations, their computational cost can be prohibitively high for moderate pattern recognition tasks. The linear Wasserstein distance offers a method for projecting signals into a Euclidean space, and in this study, we propose linear versions of the TLP distance (LTLP) that show significant improvement over the linear WP distance in signal processing tasks while being several orders of magnitude faster to compute than the TLP distance.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

Learning a target-dependent classifier for cross-domain semantic segmentation: Fine-tuning versus meta-learning

Haitao Tian, Shiru Qu, Pierre Payeur

Summary: This paper proposes a method of target-dependent classifier, which optimizes the joint hypothesis of domain adaptation into a target-dependent hypothesis that better fits with the target domain clusters through an unsupervised fine-tuning strategy and the concept of meta-learning. Experimental results demonstrate that this method outperforms existing techniques in synthetic-to-real adaptation and cross-city adaptation benchmarks.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

KGSR: A kernel guided network for real-world blind super-resolution

Qingsen Yan, Axi Niu, Chaoqun Wang, Wei Dong, Marcin Wozniak, Yanning Zhang

Summary: Deep learning-based methods have achieved remarkable results in the field of super-resolution. However, the limitation of paired training image sets has led researchers to explore self-supervised learning. However, the assumption of inaccurate downscaling kernel functions often leads to degraded results. To address this issue, this paper introduces KGSR, a kernel-guided network that trains both upscaling and downscaling networks to generate high-quality high-resolution images even without knowing the actual downscaling process.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

Gait feature learning via spatio-temporal two-branch networks

Yifan Chen, Xuelong Li

Summary: Gait recognition is a popular technology for identification due to its ability to capture gait features over long distances without cooperation. However, current methods face challenges as they use a single network to extract both temporal and spatial features. To solve this problem, we propose a two-branch network that focuses on spatial and temporal feature extraction separately. By combining these features, we can effectively learn the spatio-temporal information of gait sequences.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

PAMI: Partition Input and Aggregate Outputs for Model Interpretation

Wei Shi, Wentao Zhang, Wei-shi Zheng, Ruixuan Wang

Summary: This article proposes a simple yet effective visualization framework called PAMI, which does not require detailed model structure and parameters to obtain visualization results. It can be applied to various prediction tasks with different model backbones and input formats.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

Disturbance rejection with compensation on features

Xiaobo Hu, Jianbo Su, Jun Zhang

Summary: This paper reviews the latest technologies in pattern recognition, highlighting their instabilities and failures in practical applications. From a control perspective, the significance of disturbance rejection in pattern recognition is discussed, and the existing problems are summarized. Finally, potential solutions related to the application of compensation on features are discussed to emphasize future research directions.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

ECLAD: Extracting Concepts with Local Aggregated Descriptors

Andres Felipe Posada-Moreno, Nikita Surya, Sebastian Trimpe

Summary: Convolutional neural networks are widely used in critical systems, and explainable artificial intelligence has proposed methods for generating high-level explanations. However, these methods lack the ability to determine the location of concepts. To address this, we propose a novel method for automatic concept extraction and localization based on pixel-wise aggregations, and validate it using synthetic datasets.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

Dynamic Graph Contrastive Learning via Maximize Temporal Consistency

Peng Bao, Jianian Li, Rong Yan, Zhongyi Liu

Summary: In this paper, a novel Dynamic Graph Contrastive Learning framework, DyGCL, is proposed to capture the temporal consistency in dynamic graphs and achieve good performance in node representation learning.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

ConvGeN: A convex space learning approach for deep-generative oversampling and imbalanced classification of small tabular datasets

Kristian Schultz, Saptarshi Bej, Waldemar Hahn, Markus Wolfien, Prashant Srivastava, Olaf Wolkenhauer

Summary: Research indicates that deep generative models perform poorly compared to linear interpolation-based methods for synthetic data generation on small, imbalanced tabular datasets. To address this, a new approach called ConvGeN, combining convex space learning with deep generative models, has been proposed. ConvGeN improves imbalanced classification on small datasets while remaining competitive with existing linear interpolation methods.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

H-CapsNet: A capsule network for hierarchical image classification

Khondaker Tasrif Noor, Antonio Robles-Kelly

Summary: In this paper, the authors propose H-CapsNet, a capsule network designed for hierarchical image classification. The network effectively captures hierarchical relationships using dedicated capsules for each class hierarchy. A modified hinge loss is utilized to enforce consistency among the involved hierarchies. Additionally, a strategy for dynamically adjusting training parameters is presented to achieve better balance between the class hierarchies. Experimental results demonstrate that H-CapsNet outperforms competing hierarchical classification networks.

PATTERN RECOGNITION (2024)

Article Computer Science, Artificial Intelligence

CS-net: Conv-simpleformer network for agricultural image segmentation

Lei Liu, Guorun Li, Yuefeng Du, Xiaoyu Li, Xiuheng Wu, Zhi Qiao, Tianyi Wang

Summary: This study proposes a new agricultural image segmentation model called CS-Net, which uses Simple-Attention Block and Simpleformer to improve accuracy and inference speed, and addresses the issue of performance collapse of Transformers in agricultural image processing.

PATTERN RECOGNITION (2024)