Article
Computer Science, Artificial Intelligence
Lei Han, Dawei Zhong, Lin Li, Kai Zheng, Lu Fang
Summary: This paper proposes a novel view synthesis system based on SRN, which improves the clarity and visual effects of synthesized results by learning residual color instead of radiance color.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2022)
Article
Computer Science, Artificial Intelligence
Kai-En Lin, Guowei Yang, Lei Xiao, Feng Liu, Ravi Ramamoorthi
Summary: Image view synthesis has been successful in reconstructing realistic visuals, but view synthesis of dynamic scenes presents challenges due to lack of high-quality training datasets and a time dimension for videos. Researchers have introduced a multi-view video dataset and a new algorithm that enables stable view extrapolation from dynamic scene videos captured by static cameras. Their method operates in 3D space and demonstrates better temporal stability and visual effects compared to traditional methods.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Yangyang Xu, Xuemiao Xu, Jianbo Jiao, Keke Li, Cheng Xu, Shengfeng He
Summary: This paper introduces a Face Flow-guided Generative Adversarial Network (FFlowGAN) for face synthesis, breaking down large-angle synthesis into small-angle rotations guided by face flow to maintain facial details and feature propagation. Extensive experiments show the effectiveness of this divide-and-conquer strategy, outperforming the state-of-the-art both qualitatively and quantitatively on benchmark datasets.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2021)
Article
Computer Science, Artificial Intelligence
Chengdong Dong, Ajay Kumar
Summary: This paper introduces a new approach to accurately synthesize multi-view contactless 3D fingerprints and addresses the challenges of match accuracy and privacy concerns. The experimental results demonstrate the effectiveness of the proposed method.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Hajime Taira, Masatoshi Okutomi, Torsten Sattler, Mircea Cimpoi, Marc Pollefeys, Josef Sivic, Tomas Pajdla, Akihiko Torii
Summary: This research aims to predict the pose of indoor photographs in a large 3D map. Contributions include a new visual localization method, a dataset for indoor localization, and significant performance improvements on challenging new data.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2021)
Article
Optics
Huachun Wang, Binbin Yan, Xinzhu Sang, Duo Chen, Peng Wang, Shuai Qi, Xiaoqian Ye, Xiao Guo
Summary: The article proposes a novel method based on a position-guiding convolutional neural network for dense view synthesis from sparse views, utilizing depth maps from left, right, and middle views to achieve high-quality dense view synthesis. Experimental results demonstrate that the approach can synthesize high-quality dense views, with SSIM above 0.94, and present continuous and reasonable occlusion relations on a light-field device, showing promising applications in 3D light-field displays.
OPTICS AND LASERS IN ENGINEERING
(2022)
Article
Geochemistry & Geophysics
Yongchang Wu, Zhengxia Zou, Zhenwei Shi
Summary: Novel view synthesis of remote sensing scenes plays a significant role in scene visualization, human-computer interaction, and various downstream applications. This article proposes a novel RS view synthesis method based on implicit neural representations, which outperforms previous state-of-the-art methods in terms of reconstruction accuracy, visual fidelity, and time efficiency.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2022)
Article
Computer Science, Information Systems
M. Shahzeb Khan Gul, M. Umair Mukati, Michel Batz, Soren Forchhammer, Joachim Keinert
Summary: The article introduces a novel deep learning-based light-field view synthesis method, which utilizes convolutional block attention modules to enhance depth image-based rendering, resulting in significant performance gains.
Article
Computer Science, Artificial Intelligence
Adriano Q. de Oliveira, Thiago L. T. da Silveira, Marcelo Walter, Claudio R. Jung
Summary: The study introduces a novel DIBR pipeline for view synthesis that effectively tackles artifacts arising from 3D warping, while maintaining structural characteristics of the scene through an image superpixel algorithm. Comparative analysis demonstrates superior performance in average results across common assessment metrics, with visual results highlighting the technique's potential for real-world applications.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2021)
Article
Computer Science, Artificial Intelligence
Pakkapon Phongthawee, Suttisak Wizadwongsa, Jiraphon Yenphraphai, Supasorn Suwajanakorn
Summary: We propose NeX, a novel approach for real-time novel view synthesis based on enhanced multiplane images (MPI) that can reproduce view-dependent effects. Our technique uses spherical basis functions learned from a neural network to parameterize each pixel and improve fine detail through a hybrid implicit-explicit modeling strategy. Additionally, we introduce an extension to NeX that utilizes knowledge distillation to train multiple MPIs for unbounded 360-degree scenes. Evaluations on multiple benchmark datasets demonstrate that our method outperforms other real-time rendering approaches and can handle challenging view-dependent effects such as rainbow reflections on CDs.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Information Systems
Chi Ho Cheung, Lu Sheng, King Ngi Ngan
Summary: This paper introduces a new method for virtual view synthesis in free-viewpoint communication, showing superior performance in dynamic camera datasets and improvement in static camera datasets.
IEEE TRANSACTIONS ON MULTIMEDIA
(2021)
Article
Computer Science, Artificial Intelligence
Rui Chen, Songfang Han, Jing Xu, Hao Su
Summary: VA-Point-MVSNet is a novel visibility-aware point-based deep framework for multi-view stereo (MVS), which directly processes the target scene as point clouds and predicts depth in a coarse-to-fine manner. The network leverages 3D geometry priors and 2D texture information effectively and processes the point cloud to estimate the 3D flow for each point.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2021)
Article
Engineering, Electrical & Electronic
Yanli Ji, Yang Yang, Fumin Shen, Heng Tao Shen, Wei-Shi Zheng
Summary: Arbitrary-view action recognition remains a challenging problem due to view changes and visual occlusions. To address this issue, researchers have collected a large-scale RGB-D action dataset with diverse data types, rich action performances, and different viewpoints, providing valuable and challenging data for evaluating arbitrary-view recognition.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
(2021)
Article
Computer Science, Artificial Intelligence
Nan Meng, Kai Li, Jianzhuang Liu, Edmund Y. Lam
Summary: This paper introduces a learning-based approach to synthesize views from an arbitrary camera position using a sparse set of images. By jointly modeling the epipolar property and occlusion, the method overcomes inconsistency in the reconstruction process. The proposed method utilizes aperture disparity and warping confidence maps to address pixel shift and occlusion issues.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2021)
Article
Computer Science, Artificial Intelligence
Shubham Tulsiani, Tinghui Zhou, Alexei A. Efros, Jitendra Malik
Summary: We study the consistency between a 3D shape and a 2D observation, and propose a differentiable formulation to compute the gradients of the 3D shape given an observation from any view. Our method uses a differentiable ray consistency term and can incorporate various types of multi-view observations as supervision for learning single-view 3D prediction. We provide empirical analysis and show improvement over existing techniques for single-view object reconstruction.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2022)