4.7 Article

A 3D Reconstruction Framework of Buildings Using Single Off-Nadir Satellite Image

Journal

REMOTE SENSING
Volume 13, Issue 21, Pages -

Publisher

MDPI
DOI: 10.3390/rs13214434

Keywords

3D model reconstruction; building; satellite image; encoder-decoder network

Funding

  1. National Natural Science Foundation of China [62071136, 61801142, 61971153, 62002083]

Ask authors/readers for more resources

In this paper, a novel framework for 3D reconstruction of buildings based on a single off-nadir satellite image is proposed. The framework consists of two convolutional neural networks, Scale-ONet and Optim-Net, which can generate water-tight mesh models with exact shape and rough scale of buildings, and reduce scale errors for these models, respectively. Experimental results show that the framework has good robustness for different input images and can achieve ideal reconstruction accuracy on both model shape and scale of buildings.
A novel framework for 3D reconstruction of buildings based on a single off-nadir satellite image is proposed in this paper. Compared with the traditional methods of reconstruction using multiple images in remote sensing, recovering 3D information that utilizes the single image can reduce the demands of reconstruction tasks from the perspective of input data. It solves the problem that multiple images suitable for traditional reconstruction methods cannot be acquired in some regions, where remote sensing resources are scarce. However, it is difficult to reconstruct a 3D model containing a complete shape and accurate scale from a single image. The geometric constraints are not sufficient as the view-angle, size of buildings, and spatial resolution of images are different among remote sensing images. To solve this problem, the reconstruction framework proposed consists of two convolutional neural networks: Scale-Occupancy-Network (Scale-ONet) and model scale optimization network (Optim-Net). Through reconstruction using the single off-nadir satellite image, Scale-Onet can generate water-tight mesh models with the exact shape and rough scale of buildings. Meanwhile, the Optim-Net can reduce the error of scale for these mesh models. Finally, the complete reconstructed scene is recovered by Model-Image matching. Profiting from well-designed networks, our framework has good robustness for different input images, with different view-angle, size of buildings, and spatial resolution. Experimental results show that an ideal reconstruction accuracy can be obtained both on the model shape and scale of buildings.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Geochemistry & Geophysics

Detect Larger at Once: Large-Area Remote-Sensing Image Arbitrary-Oriented Ship Detection

Nan Su, Zhibo Huang, Yiming Yan, Chunhui Zhao, Shuyuan Zhou

Summary: Ship detection is a major problem in satellite image analysis, especially for fast detection of ships in large-area remote-sensing images. This letter proposes an arbitrary-oriented detector based on YOLO, which can quickly locate ship positions. By reducing parameters, adding deformable convolution, and integrating rotation detection capability without angle regression, the proposed method achieves excellent accuracy and speed.

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS (2022)

Article Environmental Sciences

SII-Net: Spatial Information Integration Network for Small Target Detection in SAR Images

Nan Su, Jiayue He, Yiming Yan, Chunhui Zhao, Xiangwei Xing

Summary: In this paper, a novel ship detection method called SII-Net is proposed to enhance the detection performance of small ships in SAR images. The method utilizes a channel-location attention mechanism and a high-level features enhancement module to improve the accuracy of detection. Experimental results on public datasets show that the proposed method outperforms state-of-the-art detectors, especially for small-sized targets.

REMOTE SENSING (2022)

Article Environmental Sciences

PIIE-DSA-Net for 3D Semantic Segmentation of Urban Indoor and Outdoor Datasets

Fengjiao Gao, Yiming Yan, Hemin Lin, Ruiyao Shi

Summary: This paper proposes a novel 3D semantic segmentation method called PIIE-DSA-net, which combines low-level features and deep features for more reliable feature extraction. Experimental results demonstrate that this method achieves good segmentation results on both indoor and outdoor datasets.

REMOTE SENSING (2022)

Article Environmental Sciences

RANet: A Reliability-Guided Aggregation Network for Hyperspectral and RGB Fusion Tracking

Chunhui Zhao, Hongjiao Liu, Nan Su, Lu Wang, Yiming Yan

Summary: Object tracking using RGB images may fail when the object's color is similar to the background. Hyperspectral images provide more spectral information for RGB-based trackers, but there is currently no fusion tracking algorithm for hyperspectral and RGB images. The proposed reliability-guided aggregation network (RANet) combines hyperspectral and RGB information to improve tracking performance, with the RANet achieving the best performance accuracy among the tested trackers.

REMOTE SENSING (2022)

Article Environmental Sciences

TMTNet: A Transformer-Based Multimodality Information Transfer Network for Hyperspectral Object Tracking

Chunhui Zhao, Hongjiao Liu, Nan Su, Congan Xu, Yiming Yan, Shou Feng

Summary: This study proposes a Transformer-based multimodality information transfer network (TMTNet) to improve hyperspectral object tracking by efficiently transferring multimodality data information composed of RGB and hyperspectral data. Two subnetworks are constructed to transfer multimodality fusion information and robust RGB visual information, respectively. The proposed TMTNet tracker outperforms advanced trackers, demonstrating its effectiveness.

REMOTE SENSING (2023)

Article Environmental Sciences

A Novel Object-Level Building-Matching Method across 2D Images and 3D Point Clouds Based on the Signed Distance Descriptor (SDD)

Chunhui Zhao, Wenxuan Wang, Yiming Yan, Nan Su, Shou Feng, Wei Hou, Qingyu Xia

Summary: A novel object-level building-matching method using cross-dimensional data is proposed in this work. The method utilizes a plug-and-play Joint Descriptor Extraction Module (JDEM) to extract three-dimensional shape information from object-level remote sensing data for matching. The effectiveness of the method is verified using a cross-dimensional object-level data set.

REMOTE SENSING (2023)

Article Environmental Sciences

UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme

Yiming Yan, Weikun Zhou, Nan Su, Chi Zhang

Summary: In this paper, a novel 3D surface reconstruction method called UniRender is proposed to address the challenges in accurately capturing the appearance and geometry of scenes in remote sensing environments using neural rendering. UniRender combines the strengths of surface and volume rendering, incorporates photometric consistency constraints, and improves the sampling strategy to achieve high-quality 3D surface reconstruction.

REMOTE SENSING (2023)

Article Geochemistry & Geophysics

A Cross-Modality Feature Transfer Method for Target Detection in SAR Images

Jiayue He, Nan Su, Congan Xu, Yanping Liao, Yiming Yan, Chunhui Zhao, Wei Hou, Shou Feng

Summary: This article proposes a cross-modality feature transfer (CMFT) method to enhance feature representations in SAR modality by transferring knowledge from RGB modality. Through a multilevel modality alignment network (MMAN), a hard-sample supervision module (HSM), and a feature complementary module (FCM), the non-intuitive feature representations in SAR imaging are addressed. Experimental results demonstrate the superiority of CMFT method in SAR ship detection performance.

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING (2023)

Article Geochemistry & Geophysics

DSTNet: Dynamic-Static Transformer Style Network for Cross-Resolution Vehicle Reidentification

Chunhui Zhao, Minghua Wang, Nan Su, Yiming Yan, Shou Feng

Summary: Introduces a new dynamic-static transformer style network (DSTNet) that aims to reduce the difference between the SR-Recovered image and the HR image and obtain the feature representation of both images. Static and dynamic context information is extracted using CNN and Transformer, respectively, to enhance the feature representation. Different normalization strategies are designed in different depths of the network to obtain invariant information between the SR-Recovered image and the HR image. Experimental results show improvements in rank-5 and mAP performances by 3% and 3.6%, respectively, on datasets with large resolution differences.

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS (2023)

Article Geochemistry & Geophysics

GEOP-Net: Shape Reconstruction of Buildings From LiDAR Point Clouds

Yiming Yan, Zilu Wang, Congan Xu, Nan Su

Summary: This paper proposes an implicit modeling framework called GEOP-Net embedded with high-dimensional geometric features for shape reconstruction of buildings based on LiDAR point clouds. Experimental results show that the proposed method has better accuracy than existing methods and provides a new research idea for building reconstruction.

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS (2023)

No Data Available