Article
Geography, Physical
Max Mehltretter, Christian Heipke
Summary: This study combines the advantages of deep learning and cost volume-based features to propose a new Convolutional Neural Network (CNN) architecture for learning features from volumetric 3D data for uncertainty estimation. Three different uncertainty models are discussed and applied to train the CNN, showing the generality and state-of-the-art accuracy of the proposed method in extensive evaluations on three datasets using three common dense stereo matching methods.
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING
(2021)
Article
Biology
Ruyu Liu, Zhengzhe Liu, Jiaming Lu, Guodao Zhang, Zhigui Zuo, Bo Sun, Jianhua Zhang, Weiguo Sheng, Ran Guo, Lejun Zhang, Xiaozhen Hua
Summary: We propose a sparse-to-dense coarse-to-fine depth estimation solution for colonoscopic scenes based on the direct SLAM algorithm. Our solution utilizes scattered 3D points obtained from SLAM to generate accurate and dense depth, and combines a deep learning-based depth completion network and a reconstruction system to enhance the depth map and reconstruct a detailed 3D model of colons. Experimental results demonstrate the effectiveness and accuracy of our method.
COMPUTERS IN BIOLOGY AND MEDICINE
(2023)
Article
Geography, Physical
Elisavet Konstantina Stathopoulou, Roberto Battisti, Dan Cernea, Andreas Georgopoulos, Fabio Remondino
Summary: To support depth estimation in challenging surfaces scenarios, we propose an extended PatchMatch pipeline using an adaptive accumulated matching cost calculation. Our approach achieves competitive results compared to state-of-the-art methods by favoring the reconstruction of problematic regions while preserving fine details in rich textured regions.
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING
(2023)
Article
Engineering, Electrical & Electronic
Yimei Liu, Yuan Rao, Eric Rigall, Hao Fan, Junyu Dong
Summary: This article proposes a multiview stereo network called CR-MVSNet, which utilizes co-visibility reasoning to achieve reliable multiview similarity measurement and efficient reconstruction. The network outperforms state-of-the-art MVS algorithms on multiple datasets.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
(2023)
Article
Chemistry, Multidisciplinary
Qingyu Tan, Zhijun Fang, Xiaoyan Jiang
Summary: Traditional learning-based multi-view stereo (MVS) methods often face issues of high memory consumption and slow inference due to searching correct depth values from a large number of candidates. To solve these problems, we propose a probabilistic depth sampling technique that selects a small number of candidates from a probability distribution, thereby saving computational resources. Additionally, we introduce a self-supervised training pipeline based on knowledge distillation to handle the challenge of obtaining ground-truth depth for outdoor large-scale scenes. Extensive experiments demonstrate that our approach surpasses other recent learning-based MVS methods on multiple datasets.
APPLIED SCIENCES-BASEL
(2023)
Article
Environmental Sciences
Elisavet Konstantina Stathopoulou, Roberto Battisti, Dan Cernea, Fabio Remondino, Andreas Georgopoulos
Summary: A novel approach is proposed to increase confidence and support depth and normal map estimation by leveraging semantic priors into a PatchMatch-based MVS. During depth estimation optimization, class-specific geometric constraints are imposed using semantic class labels on image pixels.
Article
Geochemistry & Geophysics
Jiayi Li, Xin Huang, Yujin Feng, Zhen Ji, Shulei Zhang, Dawei Wen
Summary: This article introduces a new benchmark dataset called the LuoJia-MVS dataset and a new deep neural network called the HDC-MVSNet. The LuoJia-MVS dataset contains 7972 five-view images with a spatial resolution of 10 cm, pixel-wise depths, and precise camera parameters. The HDC-MVSNet network is designed with a new full-scale feature pyramid extraction module, a hierarchical set of 3-D convolutional blocks, and true 3-D deformable convolutional layers by considering the deformation problem and scale variation issue of aerial images.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2023)
Article
Engineering, Electrical & Electronic
Hengjie Lu, Shugong Xu, Shan Cao
Summary: This study proposes a method to tackle the problem of single-line depth completion, aiming to generate a dense depth map from single-line LiDAR info and aligned RGB image. A network named Semantic Guided Two-Branch Network (SGTBN) is proposed for this task, utilizing semantic information and virtual normal loss in addition to the traditional MSE loss to achieve superior performance in single-line depth completion task.
IEEE SENSORS JOURNAL
(2021)
Article
Chemistry, Analytical
Amin Alizadeh Naeini, Mohammad Moein Sheikholeslami, Gunho Sohn
Summary: This paper presents an improved scheme for depth prediction, which adapts intermediate activation functions and uses user clicks as sparse labels to enhance the network's prediction performance.
Article
Computer Science, Software Engineering
Kangkan Wang, Guofeng Zhang, Jian Yang
Summary: We propose a novel approach to estimate the 3D pose and shape of human bodies from a single depth image. The method combines correspondence learning and parametric model fitting to reconstruct 3D human body models. Experimental results demonstrate that the proposed method outperforms the state-of-the-art methods in terms of reconstruction accuracy.
Article
Environmental Sciences
De Xing, Jinliang Hou, Chunlin Huang, Weimin Zhang
Summary: A novel deep 'area-to-point' snow depth estimation model based on a deep residual learning network combining CNN and residual blocks is proposed. The model utilizes all channels of AMSR2 TB data, MODIS NDSI data, and auxiliary geographic information. The model shows good estimation accuracy and has potential for application in other study regions.
Article
Computer Science, Information Systems
Jongsub Yu, Hyukdoo Choi
Summary: This study introduces an object detector with depth estimation using monocular camera images, focusing on predicting a single depth per object for improved risk assessment in autonomous driving. By adding an additional depth estimation channel to the YOLO v4 network architecture, training on dataset labels and benchmarking against existing models, it shows higher detection performance and speed with comparable depth accuracy.
Article
Chemistry, Analytical
Yijie Zheng, Jianxin Luo, Weiwei Chen, Yanyan Zhang, Haixun Sun, Zhisong Pan
Summary: In this paper, we propose an end-to-end unsupervised multi-view 3D reconstruction network framework based on PatchMatch, called Unsup_patchmatchnet. It significantly reduces memory requirements and computing time, improves reconstruction results through the introduction of a feature point consistency loss function and various self-supervised signals. Experimental results show that compared to networks using the 3DCNN method, the network in this paper reduces memory usage by 80% and running time by more than 50%, with an overall error of reconstructed 3D point cloud of only 0.501 mm, surpassing most current unsupervised multi-view 3D reconstruction networks. Additionally, tests on different datasets demonstrate the network's good generalization.
Article
Environmental Sciences
Zhengchao Lai, Fei Liu, Shangwei Guo, Xiantong Meng, Shaokun Han, Wenhao Li
Summary: Using UAVs for remote sensing provides advantages such as flexibility and low cost, but dense real-time reconstruction for large terrain scenes remains a challenge. This study proposes a novel SLAM-based MVS approach for incrementally generating dense 3D models of terrain, implementing a highly parallel and memory-efficient CUDA-based depth computing architecture on an embedded GPU. Results show the proposed approach outperforms state-of-the-art methods in accuracy and efficiency.
Article
Automation & Control Systems
Yongming Yang, Shuwei Shao, Tao Yang, Peng Wang, Zhuo Yang, Chengdong Wu, Hao Liu
Summary: Monocular depth estimation is critical for spatial perception and 3D navigation in surgery. Existing methods often neglect geometric structural consistency, resulting in performance degradation and distorted 3D reconstruction. To address this, we propose a gradient loss, a normal loss, and a geometric consistency loss to improve depth estimation and anatomical structure reconstruction.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2023)
Article
Energy & Fuels
Paulina Kujawa, Krzysztof Chudy, Aleksandra Banasiewicz, Kacper Lesny, Radoslaw Zimroz, Fabio Remondino
Summary: The porosity of rocks is a crucial parameter in rock mechanics and underground mining, affecting fluid movement and internal processes. Conventional testing methods are complex, while modern technologies are expensive. In this study, a core sample with karst and porous structures was used, and resin was poured to reinforce it. The core was then cut and 3D optical scanning was conducted for porosity assessment, achieving accurate results at a reasonable cost.
Article
Geography, Physical
Elisavet Konstantina Stathopoulou, Roberto Battisti, Dan Cernea, Andreas Georgopoulos, Fabio Remondino
Summary: To support depth estimation in challenging surfaces scenarios, we propose an extended PatchMatch pipeline using an adaptive accumulated matching cost calculation. Our approach achieves competitive results compared to state-of-the-art methods by favoring the reconstruction of problematic regions while preserving fine details in rich textured regions.
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING
(2023)
Article
Geography, Physical
Davide Marelli, Luca Morelli, Elisa Mariarosaria Farella, Simone Bianco, Gianluigi Ciocca, Fabio Remondino
Summary: High-resolution data and accurate ground truth are crucial for evaluating and comparing methods and algorithms effectively. However, acquiring real data that is representative and diverse in a given application domain is often challenging. To address this issue, this paper introduces a new synthetic dataset called ENRICH for testing photogrammetric and computer vision algorithms. Compared to existing datasets, ENRICH provides higher resolution images with various lighting conditions, camera orientations, scales, and fields of view. ENRICH consists of three sub-datasets: ENRICH-Aerial, ENRICH-Square, and ENRICH-Statue, each showcasing different characteristics. The usefulness of this dataset is demonstrated through various photogrammetry and computer vision tasks, such as evaluating hand-crafted and deep learning-based features, examining the effects of ground control points (GCPs) configuration on 3D accuracy, and monocular depth estimation. ENRICH is publicly available at: https://github.com/davidemarelli/ENRICH.
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING
(2023)
Article
Environmental Sciences
Eleonora Grilli, Alessandro Daniele, Maarten Bassier, Fabio Remondino, Luciano Serafini
Summary: Deep learning approaches have become state-of-the-art in domains such as pattern recognition and computer vision, but they require a large amount of training data, which is often a challenge in geospatial and remote sensing fields. Neuro-Symbolic Integration field provides a possible solution by incorporating background knowledge into the neural network's learning pipeline, with one method being KENN (Knowledge Enhanced Neural Networks). Empirical results demonstrate that using KENN for point cloud semantic segmentation tasks improves the performance of the original network and achieves state-of-the-art levels of accuracy.
Article
Chemistry, Multidisciplinary
Esteban Ruiz de Ona, Ines Barbero-Garcia, Diego Gonzalez-Aguilera, Fabio Remondino, Pablo Rodriguez-Gonzalvez, David Hernandez-Lopez
Summary: This article presents PhotoMatch, an open-source tool for multi-view and multi-modal feature-based image matching, including various state-of-the-art methods for preprocessing, feature extraction, and matching. The tool also provides tools for detailed assessment and comparison of different methods, allowing users to select the best combination of methods for each specific dataset.
APPLIED SCIENCES-BASEL
(2023)
Article
Environmental Sciences
Fabio Remondino, Ali Karami, Ziyang Yan, Gabriele Mazzacca, Simone Rigon, Rongjun Qin
Summary: This paper critically analyzes the use of neural radiance fields (NeRFs) for image-based 3D reconstruction and compares them quantitatively with traditional photogrammetry. The strengths and weaknesses of NeRFs are objectively evaluated, and their applicability to different real-life scenarios is discussed. The study compares various NeRF methods using objects with different sizes and surface characteristics, and evaluates the quality of the resulting 3D reconstructions based on multiple criteria. The results demonstrate the superior performance of NeRFs for non-collaborative objects with texture-less, reflective, and refractive surfaces, while photogrammetry outperforms NeRFs for objects with cooperative texture. The complementarity of these methods should be further explored in future research.
Article
Remote Sensing
Pawel Trybala, Jaroslaw Szrek, Fabio Remondino, Paulina Kujawa, Jacek Wodecki, Jan Blachowski, Radoslaw Zimroz
Summary: The research potential in the field of mobile mapping technologies is often hindered by constraints such as expensive hardware, limited access to target sites, and the collection of ground truth data. To address these challenges, the research community often provides open datasets. However, datasets that encompass demanding conditions with synchronized sensors are currently limited. To alleviate this issue, the MIN3D dataset is proposed, which includes data gathered using a wheeled mobile robot in two distinct locations. By sharing this dataset, the aim is to support the development of robust methods for navigation and mapping in challenging underground conditions.
PFG-JOURNAL OF PHOTOGRAMMETRY REMOTE SENSING AND GEOINFORMATION SCIENCE
(2023)
Proceedings Paper
Engineering, Electrical & Electronic
Andrea Masiero, Charles Toth, Fabio Remondino
Summary: The rapid development of autonomous ground vehicle technologies and the proliferation of unmanned aerial system applications have raised the need for safe and effective navigation solutions. While GNSS has been widely used for civilian applications, its reception is unreliable in certain areas. Collaborative navigation offers a potential solution by sharing navigation information among platforms operating in close vicinity. This research investigates the feasibility and performance of collaborative navigation in areas where ground and airborne vehicles share the same space, and initial results of a field test are reported.
2023 IEEE/ION POSITION, LOCATION AND NAVIGATION SYMPOSIUM, PLANS
(2023)
Proceedings Paper
Geography, Physical
M. Peppa, L. Morelli, J. P. Mills, N. T. Penna, F. Remondino
Summary: Accurate and reliable image correspondences are crucial in photogrammetry. Recent research has shown promising results in using machine learning methods to extract tie points, but challenges remain in achieving rotationally invariant features and handling large format imagery.
XXIV ISPRS CONGRESS IMAGING TODAY, FORESEEING TOMORROW, COMMISSION II
(2022)
Proceedings Paper
Geography, Physical
F. Remondino, L. Morelli, E. Stathopoulou, M. Elhashash, R. Qin
Summary: This paper explores learning-based methods for extracting tie points in aerial image blocks and confirms the potential of these methods in finding reliable image correspondences in the aerial block.
XXIV ISPRS CONGRESS IMAGING TODAY, FORESEEING TOMORROW, COMMISSION II
(2022)
Proceedings Paper
Geography, Physical
M. Welponer, E. K. Stathopoulou, F. Remondino
Summary: Despite the recent success of learning-based monocular depth estimation algorithms, they still struggle to produce reliable results in the 3D space without additional scene cues. This study explores supervised CNN architectures for monocular depth estimation and evaluates their potential in 3D reconstruction, introducing a new benchmark for synthetic outdoor scenes.
XXIV ISPRS CONGRESS IMAGING TODAY, FORESEEING TOMORROW, COMMISSION II
(2022)
Proceedings Paper
Geography, Physical
E. M. Farella, L. Morelli, F. Remondino, J. P. Mills, N. Haala, J. Crompvoets
Summary: The article introduces the TIME benchmark, which aims to explore the potential of historical aerial images. The benchmark provides multiple historical aerial image datasets and ancillary data to support the photogrammetric processing of the photos.
XXIV ISPRS CONGRESS IMAGING TODAY, FORESEEING TOMORROW, COMMISSION II
(2022)
Proceedings Paper
Geography, Physical
Ali Karami, Roberto Battisti, Fabio Menna, Fabio Remondino
Summary: This paper provides a general overview of the rising need for high-resolution 3D information in the field of industrial metrology for micro-measurements and quality control of transparent objects. It explores the challenges of optical-based 3D reconstruction methods and systems for such objects and reviews various approaches that have been developed to overcome these challenges. The paper also presents 3D results to demonstrate the advantages and disadvantages of each method in dealing with transparent objects.
XXIV ISPRS CONGRESS IMAGING TODAY, FORESEEING TOMORROW, COMMISSION II
(2022)
Proceedings Paper
Geography, Physical
A. Azimi, A. Hosseininaveh, F. Remondino
Summary: This paper proposes a novel geometric method for key-frame selection based on ORB-SLAM3, which selects key-frames in a completely flexible way regardless of the environment, data, and scene conditions, according to the physics and geometry of the environment. The proposed method is evaluated qualitatively and quantitatively, showing a significant improvement in positioning accuracy, despite an increase in processing time.
XXIV ISPRS CONGRESS IMAGING TODAY, FORESEEING TOMORROW, COMMISSION II
(2022)
