Article
Computer Science, Software Engineering
Ziyi Chen, Feng Yu, Minghua Jiang, Hua Wang, Ailing Hua, Tao Peng, Xinrong Hu, Ping Zhu
Summary: The virtual try-on technology satisfies the demands for online shopping and enhances consumers' online clothes experience through image generation technology. The 3D virtual try-on methods, compared with image-based try-on, provide multi-perspective simulation and attract researchers' attention. However, the current methods lack clothing details and have increased implementation costs. To address these issues, we propose a novel 3D virtual try-on network called AFSF-3DVTON, which incorporates appearance flow and shape field to generate high-quality try-on results. Experimental evaluations demonstrate the superiority of the proposed method over state-of-the-art algorithms.
Article
Engineering, Biomedical
G. A. Day, A. C. Jones, R. K. Wilcox
Summary: This study investigates the effects of bone density and shape variation on load transfer in spinal interventions using Statistical Shape and Appearance Modelling. The results demonstrate complex relationships between bone density and shape that affect vertebral stiffness.
JOURNAL OF THE MECHANICAL BEHAVIOR OF BIOMEDICAL MATERIALS
(2022)
Article
Computer Science, Software Engineering
Xiuping Liu, Hua Huang, Weiming Wang, Jun Zhou
Summary: This work introduces a neural network model based on multi-view representations for learning style transformation while preserving contents of 3D shapes. By learning style transformation between different style domains, the model preserves structural details of 3D shapes and outperforms baselines and state-of-the-art approaches in experiments.
Article
Robotics
Moein Shakeri, Shing Yang Loo, Hong Zhang, Kangkang Hu
Summary: This study focuses on the application of polarization imaging in dense map reconstruction, utilizing relative depth information as a prior to improve the accuracy and density of dense reconstruction, especially in regions with poor texture.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2021)
Article
Computer Science, Artificial Intelligence
Yang Yang, Junwei Han, Dingwen Zhang, Qi Tian
Summary: This study proposes a method for 3D shape reconstruction using rich intermediate representations through a newly designed network architecture. By utilizing a two-stream network and a shape transformation network, detailed features of the entire 3D object shapes are successfully reconstructed.
PATTERN RECOGNITION
(2022)
Article
Computer Science, Software Engineering
Biao Zhang, Jiapeng Tang, Matthias Niessner, Peter Wonka
Summary: We introduce 3DShape2VecSet, a novel shape representation for neural fields designed for generative diffusion models. Our new representation encodes neural fields on top of a set of vectors, using concepts such as radial basis function representation, cross attention, and self-attention function. Results demonstrate improved performance in generative applications.
ACM TRANSACTIONS ON GRAPHICS
(2023)
Article
Computer Science, Artificial Intelligence
Zhe Cui, Jianjiang Feng, Jie Zhou
Summary: Compared with contact-based fingerprint acquisition techniques, contactless acquisition has the advantages of less skin distortion, more complete fingerprint area, and hygienic acquisition. However, perspective distortion is a challenge in contactless fingerprint recognition, which changes the ridge frequency and relative minutiae location, and thus degrades the recognition accuracy. We propose a learning-based shape-from-texture algorithm to reconstruct a 3-D finger shape from a single image and unwarp the raw image to suppress the perspective distortion. Our experimental results show that the proposed method has high 3-D reconstruction accuracy and improves matching accuracy for contactless fingerprint recognition.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Xiaoyun Chang, Wentao Yi, Xiangbo Lin, Yi Sun
Summary: Modeling 3D hands with geometry details and appearance can enhance immersion and realism in various applications. To overcome the limitations of traditional representations, we propose the use of implicit function for detailed shape reconstruction and a Structure-aware Signed Distance Function (S-SDF) for reconstructing hand shape at any resolution. Additionally, we introduce a self-supervised appearance synthesis approach to avoid the need for time-consuming 3D texture annotation. Experimental results demonstrate that our method achieves more realistic hand reconstructions compared to previous methods.
IMAGE AND VISION COMPUTING
(2023)
Article
Computer Science, Artificial Intelligence
Jinyu Zhao, Yusuke Monno, Masatoshi Okutomi
Summary: This paper proposes a novel 3D reconstruction method that effectively utilizes geometric, photometric, and polarimetric cues extracted from input multi-view color-polarization images. The method estimates camera poses and an initial 3D model using standard structure-from-motion and multi-view stereo techniques, and then refines the model by optimizing photometric rendering errors and polarimetric errors. Experimental results show that the proposed method can reconstruct detailed 3D shapes without assuming specific surface materials and lighting conditions.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Jose M. Navarro-Jimenez, Jose Aguado, Gregoire Bazin, Vicente Albero, Domenico Borzacchiello
Summary: This study proposes a method for fast digitization using partial scans and statistical shape analysis. By extracting a low-dimensional description of shape variability and reconstruction algorithm, accurate and complete digitization can be achieved. The tests conducted on aeronautical fuselage panels showed an 80% reduction in digitization time while maintaining high precision.
JOURNAL OF INTELLIGENT MANUFACTURING
(2023)
Article
Optics
Xueyan Huang, Yueyi Zhang, Zhiwei Xiong
Summary: This paper presents an event-based structured light system for high-speed 3D scanning, utilizing an event camera and a high-speed digital light projector. By triggering events with a blinking pseudo-random pattern generated by the projector, event frames are generated and digital image correlation is used to calculate displacements for deriving 3D surfaces of target objects.
Article
Computer Science, Information Systems
Bilal Ahmad, Pal Anders Floor, Ivar Farup, Oistein Hovde
Summary: Capsule endoscopy is emerging as an alternative to traditional colonoscopy, utilizing a wireless camera to examine the gastrointestinal tract. The study focuses on reconstructing 3D shapes from wireless capsule endoscopy (WCE) images to provide enhanced viewing for gastroenterologists. Different methods, such as shape from shading (SFS) and shape from focus (SFF), are evaluated and compared. The results show that both methods successfully reconstruct the 3D shapes of colon images, but SFF performs better in retaining details. Additionally, the subjective experiments indicate that contrast enhanced images and 3D models are preferred and considered useful by gastroenterologists.
Article
Food Science & Technology
Meishuan Zhang, Jun Yang, Yiheng Wang, Zhiguo Li, Fideline Tchuenbou-Magaia
Summary: A new bio-microscope technology was developed to reconstruct the 3D shape of fruit cells using three independent views. The method was proven to be accurate in extracting detailed cell morphology information.
FOOD RESEARCH INTERNATIONAL
(2022)
Article
Chemistry, Analytical
Seong Hyun Kim, Sunwon Jeong, Sungbum Park, Ju Yong Chang
Summary: This paper proposes a camera motion agnostic approach for predicting 3D human pose and mesh defined in the world coordinate system. By estimating the difference between two adjacent global poses instead of the global pose coupled to the camera motion, it addresses the issue of estimating pure pose and motion in a moving camera environment.
Article
Environmental Sciences
Rui Guo, Xiaopeng Zhao, Bo Zang, Yi Liang, Jian Bai, Liang Guo
Summary: In this paper, a refined model is proposed to estimate the quad-pol information for the CP mode. The proposed model shows its superiority in estimating the quad-pol information compared to typical reconstruction models. Furthermore, experiments validate the effectiveness of the reconstruction for classification applications using a complex-value convolutional neural network.
Article
Computer Science, Theory & Methods
Daniel Martin, Sandra Malpica, Diego Gutierrez, Belen Masia, Ana Serrano
Summary: Virtual reality (VR) is a rapidly growing technology with the potential to revolutionize content creation and consumption. This survey examines the role and benefits of multimodality in VR, showcasing its importance in enhancing user experience, improving overall performance, and enabling unprecedented abilities in skill and knowledge transfer.
ACM COMPUTING SURVEYS
(2022)
Article
Computer Science, Software Engineering
J. Delanoy, M. Lagunas, J. Condor, D. Gutierrez, B. Masia
Summary: This article presents an image-based editing method that modifies the material appearance of an object by changing high-level perceptual attributes. The method utilizes a two-step generative network to drive appearance changes and generate images with high-frequency details. To train the network, the researchers augmented an existing material appearance dataset with perceptual judgments of high-level attributes obtained through crowd-sourced experiments, and employed training strategies that avoided the need for original-edited image pairs. The perception of appearance in the edited images was validated through a user study.
COMPUTER GRAPHICS FORUM
(2022)
Article
Computer Science, Software Engineering
Shinyoung Yi, Daniel S. Jeon, Ana Serrano, Se-Yoon Jeong, Hui-Yong Kim, Diego Gutierrez, Min H. Kim
Summary: Despite advances in display technology, many applications still rely on outdated psychophysical datasets. We conducted experiments to explore contrast sensitivity and developed a novel function that predicts human contrast sensitivity more accurately. We also provide a practical version that is backward compatible and consistently produces good results in existing applications.
COMPUTER GRAPHICS FORUM
(2022)
Article
Management
Carlos Maranes, Diego Gutierrez, Ana Serrano
Summary: Virtual Reality (VR) is becoming increasingly popular with the commercialization of personal devices. However, the development of Cinematic VR (CVR) content is still in the exploratory stage. Users have partial or complete control of the camera in this medium, which can hinder the delivery of a pre-established narrative. In particular, during movie cuts, viewers may miss key elements of the story. This work explores studies on viewers' behavior and provides guidelines and methods for filmmakers to make decisions during filming and editing.
INTERNATIONAL TRANSACTIONS IN OPERATIONAL RESEARCH
(2023)
Article
Ophthalmology
E. Esteban-Ibanez, T. Perez-Roche, E. Prieto, O. Castillo, A. Fanlo-Zarazaga, A. Alejandre, D. Gutierrez, M. Ortin, V. Pueyo
Summary: Visual acuity and contrast sensitivity are key measures in assessing visual function in children, with digital devices playing an increasingly important role in evaluation. Eye tracking technology enables objective and quick measurement of children's visual function without the need for experienced examiners.
INTERNATIONAL OPHTHALMOLOGY
(2022)
Article
Multidisciplinary Sciences
Sandra Malpica, Belen Masia, Laura Herman, Gordon Wetzstein, David M. Eagleman, Diego Gutierrez, Zoya Bylinskii, Qi Sun
Summary: Time perception is influenced by manipulations to visual inputs, with changes in visual properties impacting both millisecond-level and longer interval time judgments. While high-level cognitive effects elicited by visual inputs affect time perception, they are difficult to measure and control due to confounding semantic information. This study investigates the impact of asemantic visual properties on interval time perception and reveals a consistent pattern where larger visual changes shorten perceived time in intervals up to 3 minutes, contrary to their effect on millisecond-level perception. These findings have significant real-world implications as they can alter participants' time perception.
Article
Computer Science, Software Engineering
Edurne Bernal Berdun, Daniel Martin Serrano, Diego Gutierrez Perez, Belen Masia Corcoy
Summary: Virtual reality has the potential to change the way people consume content, but much is still unknown about the grammar and visual language of this new medium. This study proposes a novel saliency prediction model that combines spherical convolutions and recurrent neural networks to extract and model spatio-temporal features from 360 degrees videos. The model outperforms previous works and successfully mimics human visual attention when exploring dynamic 360 degrees videos.
COMPUTERS & GRAPHICS-UK
(2022)
Article
Computer Science, Software Engineering
Chun-Yu Sun, Yu-Qi Yang, Hao-Xiang Guo, Peng-Shuai Wang, Xin Tong, Yang Liu, Heung-Yeung Shum
Summary: The lack of fine-grained 3D shape segmentation data is a major challenge for developing learning-based 3D segmentation techniques. In this study, we propose an effective semi-supervised method that learns 3D segmentations using a combination of labeled and unlabeled data. Our approach incorporates a novel multilevel consistency loss to ensure consistent network predictions across different levels of perturbed 3D shapes, and a part substitution scheme to augment labeled data for improved training. Extensive validation on different tasks demonstrates the superior performance of our method compared to existing semi-supervised and unsupervised pre-training approaches.
COMPUTATIONAL VISUAL MEDIA
(2023)
Article
Engineering, Manufacturing
Hyeonjoong Jang, Sanghoon Cho, Daniel S. S. Jeon, Dahyun Kang, Myeongho Song, Changhyun Park, Jaewon Kim, Min H. H. Kim
Summary: In this work, a novel visual inspection method using light-field 3D imaging is proposed to detect defects on a display panel. By acquiring high-resolution depth information of defects located inside transparent layers without powering the panel, the physical locations of defects can be estimated. The types of defects and their layer locations can be automatically classified along the depth axis in multiple transparent layers. Experimental results validate the successful detection and classification of various display defects.
IEEE TRANSACTIONS ON SEMICONDUCTOR MANUFACTURING
(2023)
Article
Optics
Young-Gil Cha, Jiwoong Na, Hyun-Kyung Kim, Jae-Myeong Kwon, Seok-Haeng Huh, Seung-Un Jo, Chang-Hwan Kim, Min H. Kim, Ki-Hun Jeong
Summary: This article presents a microlens array camera with variable apertures for high dynamic range imaging. The camera captures low dynamic range images with different F-stop values and reconstructs HDR images with expanded dynamic range and high resolution.
Proceedings Paper
Computer Science, Artificial Intelligence
Andreas Meuleman, Hakyeong Kim, James Tompkin, Min H. Kim
Summary: High-accuracy per-pixel depth is crucial for computational photography, and smartphones now employ multimodal camera systems to achieve this. However, the low resolution and limited active illumination power of ToF sensors make it challenging to produce accurate high-resolution depth. By fusing RGB stereo and ToF information, these issues can be overcome. A key problem, however, is the unknown pose of the floating lens caused by the optical stabilization of the main color sensor, which breaks the geometric relationships between the multimodal image sensors. To address this, an automatic calibration technique based on dense 2D/3D matching is proposed, which can estimate the parameters of the stabilized main RGB sensor from a single snapshot. Fusion is achieved through deep learning using a real-world training dataset with depth supervision. Evaluation results demonstrate higher accuracy compared to existing baselines.
COMPUTER VISION - ECCV 2022, PT I
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Yu Deng, Jiaolong Yang, Jianfeng Xiang, Xin Tong
Summary: The study aims to generate 3D-consistent images with controllable camera poses through 3D-aware image generative modeling. A novel approach is proposed to regulate point sampling and radiance field learning on 2D manifolds, addressing the limitations in handling fine details and stable training in existing generators.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)
(2022)
Article
Computer Science, Artificial Intelligence
Jiapeng Tang, Xiaoguang Han, Mingkui Tan, Xin Tong, Kui Jia
Summary: This paper focuses on the challenging task of learning 3D object surface reconstructions from RGB images and proposes a method that learns and uses a topology-preserved skeletal shape representation to assist the surface reconstruction. Through experiments, the proposed method is shown to be effective and outperforms existing methods in the surface reconstruction task.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2022)
