Article
Computer Science, Software Engineering
T. Chlubna, T. Milet, P. Zemcik
Summary: This paper introduces a light field rendering method that does not require 3D models and only uses scene images to render new views. Addressing the refocusing artifacts of light field approximation, a real-time focusing solution based on statistical analysis is proposed, eliminating the need for precomputed or acquired depth information. Experimental results show that this method can be implemented on a GPU, reducing memory requirements and enabling real-time rendering of high resolution light field data.
COMPUTATIONAL VISUAL MEDIA
(2021)
Article
Computer Science, Software Engineering
Zeqiang Lai, Kaixuan Wei, Ying Fu, Philipp Hartel, Felix Heide
Summary: This article introduces del-Prox, a domain-specific modeling language and compiler for large-scale optimization problems using differentiable proximal algorithms. del-Prox's core feature is full differentiability, supporting hybrid model- and learning-based solvers that integrate proximal optimization with neural network pipelines. With just a few lines of code, del-Prox can generate performant solvers for various image optimization problems, including end-to-end computational optics, image deraining, and compressive magnetic resonance imaging. It can also be used in completely different domains such as energy system planning, outperforming state-of-the-art CVXPY and commercial Gurobi solvers.
ACM TRANSACTIONS ON GRAPHICS
(2023)
Article
Engineering, Electrical & Electronic
Itziar Zabaleta, Marcelo Bertalmio
Summary: This algorithm presents an efficient method for video style transfer, transferring the visual style of a reference image onto unprocessed videos. It improves and adapts color transfer methods based on statistical properties of images, providing excellent transfer results and outperforming other methods in observer preference experiments.
SIGNAL PROCESSING-IMAGE COMMUNICATION
(2021)
Article
Computer Science, Software Engineering
Libing Zeng, Nima Khademi Kalantari
Summary: In this paper, a learning-based test-time optimization approach is proposed to reconstruct geometrically consistent depth maps from a monocular video. The approach optimizes an existing single image depth estimation network on the test example at hand. By introducing pseudo reference depth maps based on the consistency between optical flow displacement and depth-reprojection displacement, inaccurate pseudo reference depth maps are discarded and a confidence map for the reference depth is computed. A loss function is formulated using the pseudo reference depth and the confidence map to efficiently and effectively perform the test-time optimization. The approach is evaluated against state-of-the-art methods and shows better performance in terms of speed and depth map quality.
COMPUTER GRAPHICS FORUM
(2023)
Article
Computer Science, Software Engineering
Chengfang Song, Chunxia Xiao, Yeting Zhang, Haigang Sui
Summary: The paper proposes a divide-and-conquer scheme to remove thin translucent clouds in aerial images, utilizing a veiling metric based on color attenuation prior. By segmenting the image based on cloud concentration, atmospheric light is estimated using a modified local color-line model and scene transmission is refined using weighted L1-norm based contextual regularization. Ground reflection is recovered using an atmospheric scattering model, showing improvement over classical dehazing methods and learning-based declouding methods.
COMPUTER GRAPHICS FORUM
(2021)
Article
Computer Science, Information Systems
Yang Yang, Dan Wu, Lanling Zeng, Zhuoran Li
Summary: The proposed filter is a novel edge-preserving filter based on deep unsupervised learning under the weighted least square framework. It is capable of efficiently smoothing images and suppressing artifacts.
MULTIMEDIA TOOLS AND APPLICATIONS
(2023)
Article
Computer Science, Software Engineering
Yang Yang, Dan Wu, Ling Tang, Lanling Zeng, Zhigeng Pan
Summary: In this paper, a weighted and truncated L-1-regularized optimization model is proposed for image smoothing. A deep unsupervised learning-based filter is also proposed based on the defined loss function. The experimental results show that the proposed filter outperforms state-of-the-art filters in terms of image quality and is highly efficient.
Article
Computer Science, Artificial Intelligence
Sudhakar Kumawat, Tadashi Okawara, Michitaka Yoshida, Hajime Nagahara, Yasushi Yagi
Summary: This paper introduces a deep sensing solution for recognizing human actions directly from coded exposure images. The solution consists of a binary CNN-based encoder network and a 2D CNN for action recognition. The authors also propose a novel knowledge distillation framework and demonstrate improved accuracy compared to previous approaches on various datasets. Additionally, a prototype coded exposure camera is built to validate the feasibility of the deep sensing solution.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Review
Dermatology
Nadia Kashetsky, Kristie Mar, Chaocheng Liu, Jason K. Rivers, Ilya Mukovozov
Summary: Clinical photography is widely used in dermatology for various purposes, including diagnosis, treatment monitoring, medical education, and research. However, there is a lack of comprehensive literature review on this topic. This scoping review summarizes the literature on photography practices in dermatology, with a focus on photography of skin of color, patient preferences, and medical-legal considerations. The findings show that while there is a need for improved representation of skin of color in dermatologic photography, patients generally support medical photography and prefer clinical photographs taken by their own physicians using clinic/hospital-owned cameras.
JOURNAL DER DEUTSCHEN DERMATOLOGISCHEN GESELLSCHAFT
(2023)
Article
Computer Science, Software Engineering
Steven Diamond, Vincent Sitzmann, Frank Julca-Aguilar, Stephen Boyd, Gordon Wetzstein, Felix Heide
Summary: The traditional imaging process involves a series of steps, while advanced processing includes feature extraction, classification, tracking, and fusion. We propose an end-to-end differentiable architecture that performs demosaicking, denoising, deblurring, tone-mapping, and classification, improving perception in low light and challenging conditions.
ACM TRANSACTIONS ON GRAPHICS
(2021)
Article
Computer Science, Artificial Intelligence
Shiqi Chen, Jingwen Zhou, Menghao Li, Yueting Chen, Tingting Jiang
Summary: In this study, a prior quantization model is proposed to correct image degradation in mobile terminals. By integrating multivariate messages and quantifying various priors, targeted optical degradation correction can be achieved. Comprehensive experiments demonstrate the flexibility and potential of the proposed method for image restoration in mass-produced mobile terminals.
PATTERN RECOGNITION LETTERS
(2023)
Article
Computer Science, Information Systems
Yong Yang, Hangyuan Lu, Shuying Huang, Yuming Fang, Wei Tu
Summary: This paper proposes an efficient pan-sharpening model based on conditional random fields to preserve spectral and spatial information while ensuring the sharpness of the fused image. Experimental results show that the proposed method can achieve the best fusion results with high computational efficiency compared to previous state-of-the-art pansharpening methods.
INFORMATION SCIENCES
(2021)
Article
Computer Science, Artificial Intelligence
Ahmet Serdar Karadeniz, Erkut Erdem, Aykut Erdem
Summary: This paper introduces a new method that leverages burst photography to enhance performance and obtain sharper and more accurate RGB images from extremely dark raw images. By using a coarse-to-fine network architecture, high-quality outputs are generated progressively, leading to better image quality by reducing noise level, improving color accuracy, and producing more visually pleasing results.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2021)
Article
Computer Science, Artificial Intelligence
Chongyi Li, Chunle Guo, Linghao Han, Jun Jiang, Ming-Ming Cheng, Jinwei Gu, Chen Change Loy
Summary: This paper provides a comprehensive survey on low-light image enhancement, covering various aspects from algorithm taxonomy to unsolved issues. The researchers also propose a low-light image and video dataset and establish an online platform for generating results. The survey, dataset, and platform can serve as a reference for future research and contribute to the development of this research field.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2022)
Article
Engineering, Electrical & Electronic
Suren Jayasuriya, Odrika Iqbal, Venkatesh Kodukula, Victor Torres, Robert LiKamWa, Andreas Spanias
Summary: Huge advancements have been made in modern image-sensing hardware and visual computing algorithms. However, there is still a gap between hardware and software design in imaging systems, which limits collaboration between research domains. This survey explores existing works that use software-defined imaging (SDI) to replace conventional hardware components, allowing users to program image sensors according to their needs. The scope of the survey covers various imaging systems and discusses the components and future research directions of SDI.
PROCEEDINGS OF THE IEEE
(2023)
Article
Optics
Suyeon Choi, Jonghyun Kim, Yifan Peng, Gordon Wetzstein
Summary: Michelson holography (MH) is a holographic display technology that optimizes image quality for emerging holographic near-eye displays by using two SLMs to cancel out undiffracted light. The system is calibrated using camera-in-the-loop holography techniques and demonstrates state-of-the-art 2D and multi-plane holographic image quality.
Article
Computer Science, Software Engineering
Steven Diamond, Vincent Sitzmann, Frank Julca-Aguilar, Stephen Boyd, Gordon Wetzstein, Felix Heide
Summary: The traditional imaging process involves a series of steps, while advanced processing includes feature extraction, classification, tracking, and fusion. We propose an end-to-end differentiable architecture that performs demosaicking, denoising, deblurring, tone-mapping, and classification, improving perception in low light and challenging conditions.
ACM TRANSACTIONS ON GRAPHICS
(2021)
Article
Computer Science, Software Engineering
Julien N. P. Martel, David B. Lindell, Connor Z. Lin, Eric R. Chan, Marco Monteiro, Gordon Wetzstein
Summary: Neural representations have emerged as a new paradigm in various applications, offering flexibility and accuracy at moderate resolutions but facing challenges in representing large-scale or complex scenes. A new hybrid implicit-explicit network architecture and training strategy has been introduced, showing promising results in fitting high-resolution images and 3D shapes while significantly reducing training times and memory requirements.
ACM TRANSACTIONS ON GRAPHICS
(2021)
Article
Computer Science, Software Engineering
Brooke Krajancich, Petr Kellnhofer, Gordon Wetzstein
Summary: Virtual and augmented reality displays aim to match the perceptual capabilities of the human visual system within limited compute budgets and transmission bandwidths. Foveated graphics techniques exploit spatial acuity fall-off in the periphery of the visual field, but there's less focus on temporal aspects of human vision which vary across the retina. A new model introduced in this study enables prediction of imperceptible temporal information based on spatial frequency, eccentricity, and luminance levels.
ACM TRANSACTIONS ON GRAPHICS
(2021)
Article
Optics
Bahram Javidi, Artur Carnicer, Arun Anand, George Barbastathis, Wen Chen, Pietro Ferraro, J. W. Goodman, Ryoichi Horisaki, Kedar Khare, Malgorzata Kujawinska, Rainer A. Leitgeb, Pierre Marquet, Takanori Nomura, Aydogan Ozcan, YongKeun Park, Giancarlo Pedrini, Pascal Picart, Joseph Rosen, Genaro Saavedra, Natan T. Shaked, Adrian Stern, Enrique Tajahuerce, Lei Tian, Gordon Wetzstein, Masahiro Yamaguchi
Summary: This Roadmap article provides an overview of research activities in the field of digital holography, with sections covering sensing, 3D imaging, displays, virtual and augmented reality, microscopy, cell identification, tomography, label-free live cell imaging, and other applications. Each section represents the author's vision of the significant progress, potential impact, important developments, and challenging issues in the field of digital holography.
Article
Optics
Henry Arguello, Samuel Pinilla, Yifan Peng, Hayato Ikoma, Jorge Bacca, Gordon Wetzstein
Summary: State-of-the-art snapshot spectral imaging systems use color-coded apertures for flexible spatial-spectral modulation, improving spectral reconstruction quality and reducing device size. The use of diffractive optical elements has replaced traditional lenses in some systems to achieve these benefits.
Article
Optics
Ozan Cakmakci, Yi Qin, Peter Bosel, Gordon Wetzstein
Summary: This study demonstrates the first optically see-through full-color volume holographic pancake optic for mobile augmented reality applications, achieved by combining the full-color volume holographic pancake with a flat lightguide to achieve optical see-through property.
Article
Optics
Manu Gopakumar, Jonghyun Kim, Suyeon Choi, Yifan Peng, Gordon Wetzstein
Summary: The study introduces a new algorithmic framework for optimizing high diffraction orders (HDOs) in computer-generated holography, allowing compact holographic displays without the need for optical filtering. By developing a wave propagation model of HDOs and optimizing phase patterns, the method enables HDOs to contribute to forming the image instead of creating artifacts, outperforming previous algorithms in an unfiltered holographic display prototype.
Article
Optics
Hayato Ikoma, Takamasa Kudo, Yifan Peng, Michael Broxton, Gordon Wetzstein
Summary: A hybrid optical-electronic computing approach is introduced to optimize 3D localization performance, achieving significantly higher accuracy than existing approaches, especially in challenging scenarios with high molecular density over large depth ranges. Through extensive simulations and biological experiments, the deep-learning-based microscope demonstrates superior 3D localization accuracy.
Article
Multidisciplinary Sciences
Yifan Peng, Suyeon Choi, Jonghyun Kim, Gordon Wetzstein
Summary: Research has developed a computer-generated holography approach using partially coherent light sources, showing potential to improve speckle characteristics and image quality of holograms. SLEDs have been demonstrated as promising light sources for generating bright, high-contrast, almost speckle-free 2D and 3D holographic images.
Proceedings Paper
Acoustics
Ruangrawee Kitichotkul, Christopher A. Metzler, Frank Ong, Gordon Wetzstein
Summary: Researchers use Stein's unbiased risk estimate (SURE) to develop per-pixel confidence intervals for compressive sensing reconstruction using CNN-based denoisers in the approximate message passing (AMP) framework. These heatmaps inform end-users how much trust to place in an image formed by a CNN, potentially enhancing the utility of CNNs in various computational imaging applications.
2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021)
(2021)
Proceedings Paper
Acoustics
Christopher A. Metzler, Gordon Wetzstein
Summary: The article introduces a CNN architecture for removing colored Gaussian noise and combines it with the VDAMP algorithm, which follows a predictable colored Gaussian distribution. The resulting denoising-based VDAMP algorithm is applied to compressive MRI, outperforming existing techniques significantly.
2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021)
(2021)
Proceedings Paper
Computer Science, Hardware & Architecture
Suyeon Choi, Yifan Peng, Jonghyun Kim, Gordon Wetzstein
Summary: The study introduces a novel holographic display architecture that utilizes two phase-only SLMs to achieve high-quality, contrast-enhanced display experiences. Experimental results demonstrate that the proposed architecture can deliver holographic images with higher contrast and minimal speckle.
OPTICAL ARCHITECTURES FOR DISPLAYS AND SENSING IN AUGMENTED, VIRTUAL, AND MIXED REALITY (AR, VR, MR) II
(2021)
Proceedings Paper
Engineering, Biomedical
Supriya Sathyanarayana, Christoph Leuze, Brian Hargreaves, Bruce L. Daniel, Gordon Wetzstein, Amit Etkin, Mahendra T. Bhati, Jennifer A. McNab
Summary: Repetitive Transcranial Magnetic Stimulation (rTMS) is an important treatment option for medication-resistant depression, but accurate positioning of the electromagnetic coil is crucial for its effectiveness. Existing image-guided neuronavigation systems are accurate but expensive and time-consuming. This study compares three different head tracking methods for a mixed-reality neuronavigation system and found that while all three methods met the accuracy requirements for TMS treatment in initial head poses, only the marker-based method achieved sufficient accuracy for larger head rotations.
MEDICAL IMAGING 2020: IMAGE-GUIDED PROCEDURES, ROBOTIC INTERVENTIONS, AND MODELING
(2021)
Article
Engineering, Electrical & Electronic
Christopher A. Metzler, David B. Lindell, Gordon Wetzstein
Summary: Keyhole imaging technology captures transient measurements along a single optical path to recover the shape and position of objects. It provides a solution to the limitations of existing NLOS techniques, allowing for more efficient imaging without the need to scan large areas.
IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING
(2021)
