4.7 Article

FlexISP: A Flexible Camera Image Processing Framework

Journal

ACM TRANSACTIONS ON GRAPHICS
Volume 33, Issue 6, Pages -

Publisher

ASSOC COMPUTING MACHINERY
DOI: 10.1145/2661229.2661260

Keywords

image processing; image reconstruction

Ask authors/readers for more resources

Conventional pipelines for capturing, displaying, and storing images are usually defined as a series of cascaded modules, each responsible for addressing a particular problem. While this divide-and-conquer approach offers many benefits, it also introduces a cumulative error, as each step in the pipeline only considers the output of the previous step, not the original sensor data. We propose an end-to-end system that is aware of the camera and image model, enforces natural-image priors, while jointly accounting for common image processing steps like demosaicking, denoising, deconvolution, and so forth, all directly in a given output representation (e.g., YUV, DCT). Our system is flexible and we demonstrate it on regular Bayer images as well as images from custom sensors. In all cases, we achieve large improvements in image quality and signal reconstruction compared to state-of-the-art techniques. Finally, we show that our approach is capable of very efficiently handling high-resolution images, making even mobile implementations feasible.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Optics

Hyperspectral phase imaging based on denoising in complex-valued eigensubspace

Igor Shevkunov, Vladimir Katkovnik, Daniel Claus, Giancarlo Pedrini, Nikolay Petrov, Karen Egiazarian

OPTICS AND LASERS IN ENGINEERING (2020)

Article Engineering, Electrical & Electronic

Features of correlation measurements of the parameters of pulsed hyperspectral optical fields using an asymmetric interferometer

M. S. Kulya, V. Ya Katkovnik, K. Egiazarian, N. Petrov

QUANTUM ELECTRONICS (2020)

Article Environmental Sciences

Full-Reference Quality Metric Based on Neural Network to Assess the Visual Quality of Remote Sensing Images

Oleg Ieremeiev, Vladimir Lukin, Krzysztof Okarma, Karen Egiazarian

REMOTE SENSING (2020)

Article Optics

Extended DoF and achromatic inverse imaging for lens and lensless MPM camera based on Wiener filtering of defocused OTFs

Seyyed R. M. Rostami, Vladimir Katkovnik, Karen Egiazarian

Summary: An optimal optical transfer function (OTF) is proposed for RGB inverse imaging to achieve extended depth of field and reduced color aberrations. This new inverse imaging technique, demonstrated in optical setups with lens and lensless, shows better performance in a lensless system designed for the wavelength range of 400 to 700 nm and depth of field range from 0.5 to 1000 m.

OPTICAL ENGINEERING (2021)

Article Environmental Sciences

Lossy Compression of Multichannel Remote Sensing Images with Quality Control

Vladimir Lukin, Irina Vasilyeva, Sergey Krivenko, Fangfang Li, Sergey Abramov, Oleksii Rubel, Benoit Vozel, Kacem Chehdi, Karen Egiazarian

REMOTE SENSING (2020)

Article Chemistry, Multidisciplinary

A Fast Method of Visually Lossless Compression of Dental Images

Sergey Krivenko, Vladimir Lukin, Olha Krylova, Liudmyla Kryvenko, Karen Egiazarian

Summary: This paper proposes a noniterative approach to the visually lossless compression problem of dental images, focusing on lossy compression, preservation of diagnostic information, and noise characteristics of dental images. By analyzing the dependencies of quality metrics on quantization step and considering distortion visibility thresholds, the compression of images is controlled effectively.

APPLIED SCIENCES-BASEL (2021)

Article Optics

Hyperspectral phase retrieval: spectral-spatial data processing with sparsity-based complex domain cube filter

Vladimir Katkovnik, Igor Shevkunov, Karen Egiazarian

Summary: The study introduces a phase retrieval algorithm for hyperspectral imaging that avoids random phase coding commonly used in traditional methods. Through shearography optical setup, the algorithm demonstrates excellent performance in object phase and thickness imaging in simulation and experimental tests.

OPTICAL ENGINEERING (2021)

Article Optics

Polarization holographic recording of vortex diffractive optical elements on azopolymer thin films and 3D analysis via phase-shifting digital holographic microscopy

Veronica Cazac, Elena Achimova, Vladimir Abashkin, Alexandr Prisacar, Constantin Loshmanschii, Alexei Meshalkin, Karen Egiazarian

Summary: This paper presents the fabrication of optical vortex DOEs on photosensitive thin films using analog and digital approaches, and compares the efficiency evolution of the three types of DOEs. The study provides new evidence on the influence of analog and digital generation of spiral wavefront on the performance of vortex DOEs.

OPTICS EXPRESS (2021)

Article Environmental Sciences

Selection of Lee Filter Window Size Based on Despeckling Efficiency Prediction for Sentinel SAR Images

Oleksii Rubel, Vladimir Lukin, Andrii Rubel, Karen Egiazarian

Summary: Radar imaging has many advantages, but SAR images are affected by noise-like speckle. The local statistic Lee filter is a popular despeckling technique, and this study demonstrates how filter parameters can be selected based on efficiency prediction using a trained neural network. Adaptive selection of filter window size can significantly improve filtering efficiency.

REMOTE SENSING (2021)

Article Optics

Power-balanced hybrid optics boosted design for achromatic extended depth-of-field imaging via optimized mixed OTF

Seyyed Reza Miri Rostami, Samuel Pinilla, Igor Shevkunov, Vladimir Katkovnik, Karen Egiazarian

Summary: This paper introduces a power-balanced hybrid optical imaging system with a diffractive computational camera utilizing a refractive lens and multilevel phase mask (MPM). By optimizing the optical power balance and MPM design, the system shows significant advantages in terms of image quality reconstruction and depth-of-field range.

APPLIED OPTICS (2021)

Article Optics

Single-shot pixel super-resolution phase imaging by wavefront separation approach

Peter Kocsis, Igor Shevkunov, Vladimir Katkovnik, Heikki Rekola, Karen Egiazarian

Summary: The proposed method introduces a lensless single-shot phase retrieval approach that separates carrying and object wavefronts. By calibrating discrepancies between computational models and physical elements and implementing pixel super-resolution processing, it reconstructs the object wavefront with correction from the carrying wavefront, demonstrating robustness in simulations and experiments. In phase bio-imaging, it achieves high-quality imaging results and records dynamic scenes efficiently with the single-shot advantage.

OPTICS EXPRESS (2021)

Proceedings Paper Optics

On design of hybrid diffractive optics for achromatic extended depth-of-field (EDoF) RGB imaging

Seyyed Reza Miri Rostami, Samuel Pinilla, Igor Shevkunov, Vladimir Katkovnik, Karen Eguiazarian

Summary: This paper presents a hybrid imaging system that achieves optimized imaging with extended depth of field and reduced chromatic aberrations. A fully differentiable image formation model and neural network techniques are used for imaging optimization. By comparing the effects of different optical parameters on imaging accuracy and quality, the study proposes an application of hybrid optics in compact cameras.

UNCONVENTIONAL OPTICAL IMAGING III (2022)

Proceedings Paper Acoustics

FLASHLIGHT CNN IMAGE DENOISING

Pham Huu Thanh Binh, Cristovao Cruz, Karen Egiazarian

Summary: This paper introduces a learning-based denoising method, FlashLight CNN, which utilizes a deep neural network for image denoising. By combining deep residual networks and inception networks, the proposed approach outperforms current state of the art image denoising methods in quantitative and visual comparisons.

28TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO 2020) (2021)

Proceedings Paper Imaging Science & Photographic Technology

BROADBAND HYPERSPECTRAL PHASE RETRIEVAL FROM NOISY DATA

Vladimir Katkovnik, Igor Shevkunov, Karen Egiazarian

2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP) (2020)

No Data Available