Article
Computer Science, Software Engineering
T. Chlubna, T. Milet, P. Zemcik
Summary: This paper introduces a light field rendering method that does not require 3D models and only uses scene images to render new views. Addressing the refocusing artifacts of light field approximation, a real-time focusing solution based on statistical analysis is proposed, eliminating the need for precomputed or acquired depth information. Experimental results show that this method can be implemented on a GPU, reducing memory requirements and enabling real-time rendering of high resolution light field data.
COMPUTATIONAL VISUAL MEDIA
(2021)
Article
Computer Science, Artificial Intelligence
Christopher Hahne, Amar Aggoun
Summary: Light-field cameras are crucial for rich 3D information retrieval in narrow depth sensing applications. The framework proposed in this research advances previous outcomes by introducing novel micro image scale-space analysis for generic camera calibration and parallax-invariant, cost-effective viewpoint color equalization. Our algorithms outperform state-of-the-art tool chains and existing transport methods, and are released under an open-source license for convenient use by peer researchers, developers, photographers, and data scientists in the field.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2021)
Article
Multidisciplinary Sciences
L. Clermont, W. Uhring, M. Georges
Summary: Understanding stray light is crucial in optical instrument development, with traditional methods limited in providing sufficient information. The new ultrafast time-of-flight stray light characterization method offers identification of stray light origins and aids in system improvement.
SCIENTIFIC REPORTS
(2021)
Article
Multidisciplinary Sciences
Xiaohua Feng, Liang Gao
Summary: LIFT is a novel transient imaging strategy that provides a temporal sequence of over 1000 and efficient light field acquisition for four-dimensional imaging. It enables three-dimensional imaging of light in flight phenomena with a resolution of less than 10 picoseconds, and non-line-of-sight imaging at a video rate of 30 Hz.
NATURE COMMUNICATIONS
(2021)
Article
Engineering, Electrical & Electronic
Zhaolin Xiao, Jinglei Shi, Xiaoran Jiang, Christine Guillemot
Summary: This paper presents a model for axial refocusing precision in light field imaging and demonstrates through experiments that computationally extending the light field baseline can improve the axial refocusing precision.
SIGNAL PROCESSING-IMAGE COMMUNICATION
(2022)
Article
Engineering, Electrical & Electronic
Martin Alain, Aljosa Smolic
Summary: This paper studies the spectral properties of re-parameterized light field and introduces more flexible sampling guidelines. The simulations and results show that there are exciting avenues for practical applications of light fields.
SIGNAL PROCESSING-IMAGE COMMUNICATION
(2022)
Article
Optics
Yin Yongkai, Yu Kai, Yu Chunzhan, Bai Xuechun, Cai Zewei, Meng Xiangfeng, Yang Xiulun
Summary: Light field imaging is a new technology that expands classical optical imaging, providing more possibilities for advancement in imaging technology, especially in the field of computational imaging. By reconstructing missing depth information and 3D structure from 2D images, light field imaging enables 3D imaging with applications in areas such as biological imaging, industrial inspection, automatic navigation, and virtual reality.
CHINESE JOURNAL OF LASERS-ZHONGGUO JIGUANG
(2021)
Article
Computer Science, Artificial Intelligence
Fengqiang Li, Florian Willomitzer, Muralidhar Madabhushi Balaji, Prasanna Rangarajan, Oliver Cossairt
Summary: The paper introduces a novel sensor concept that provides high-precision depth measurements by combining different sensing modalities, achieving depth precision up to 35 μm and point cloud densities matching standard CMOS/CCD detectors.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2021)
Article
Computer Science, Theory & Methods
Rui M. Lourenco, Luis M. N. Tavora, Pedro A. A. Assuncao, Lucas A. Thomaz, Rui Fonseca-Pinto, Sergio M. M. Faria
Summary: In the past decade, there has been a growing number of applications dealing with multidimensional visual information. Among them, the structure tensor-based disparity estimation method has shown great potential. However, this method has some limitations. This paper proposes an improved method that enhances the quality of disparity maps and achieves significant improvements on plane surfaces.
MULTIDIMENSIONAL SYSTEMS AND SIGNAL PROCESSING
(2022)
Article
Computer Science, Artificial Intelligence
Vishwanath Saragadam, Michael DeZeeuw, Richard G. Baraniuk, Ashok Veeraraghavan, Aswin C. Sankaranarayanan
Summary: The study presents a novel video-rate hyperspectral imager that achieves high-quality reconstructions by scene-adaptive spatial sampling guided by super-pixel segmented images, resulting in significantly higher spatial and spectral resolutions compared to traditional hyperspectral cameras.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2021)
Article
Optics
Weihao Wang, Xing Zhao, Zhixiang Jiang, Ya Wen
Summary: In this paper, a deep learning-based method for scattering removal in light field imaging is proposed, which enables high-quality 3D reconstruction and addresses the scattering issue in light field imaging.
CHINESE OPTICS LETTERS
(2022)
Article
Optics
Xin Jin, Kunyi Li, Chuanpu Li, Xufu Sun
Summary: A generic image formation model is proposed for wide-FOV plenoptic cameras using monocentric lens and MLA, approximating the monocentric lens as a superposition of concentric lenses with variable apertures. The validity of the proposed model is verified by comparing real captured PSFs with those generated by the model, showing better quality in reconstructed images compared to subaperture images.
Editorial Material
Optics
Jian Zhao, Mingsheng Li
Summary: The integration of an acousto-optic programmable dispersive filter with spectrally filtered sequentially time all-optical mapping photography enables independent control of frame rate, frame intensity, and exposure time with a simple system design.
LIGHT-SCIENCE & APPLICATIONS
(2022)
Article
Computer Science, Artificial Intelligence
Takuya Funatomi, Takehiro Ogawa, Kenichiro Tanaka, Hiroyuki Kubo, Guillaume Caron, El Mustapha Mouaddib, Yasuyuki Matsushita, Yasuhiro Mukaigawa
Summary: In this study, we propose a method to eliminate temporal illumination variations in whisk-broom hyperspectral imaging. By introducing an additional perpendicular scan and leveraging the sparse structure in the illumination spectrum, we robustly eliminate the effects of illumination variations. We demonstrate the usefulness of our method through capturing historic stained-glass windows of a French cathedral.
INTERNATIONAL JOURNAL OF COMPUTER VISION
(2022)
Article
Physics, Multidisciplinary
Isabella Petrelli, Francesca Santoro, Gianlorenzo Massaro, Francesco Scattarella, Francesco V. Pepe, Francesca Mazzia, Maria Ieronymaki, George Filios, Dimitris Mylonas, Nikos Pappas, Cristoforo Abbattista, Milena D'Angelo
Summary: Correlation Plenoptic Imaging (CPI) is an innovative approach that addresses the trade-off between image resolution and depth of field by utilizing intensity correlations to extract light direction information. A novel reconstruction algorithm based on compressive sensing techniques can reconstruct CPI images with fewer frames.
FRONTIERS IN PHYSICS
(2023)
Article
Computer Science, Artificial Intelligence
Yujie Wang, Chenggang Yan, Yutong Feng, Shaoyi Du, Qionghai Dai, Yue Gao
Summary: Partial point cloud registration transforms partial scans into a common coordinate system, which is crucial for generating complete 3D shapes. Traditional registration methods struggle with small point cloud overlaps, but the STORM method utilizes structure information to accurately detect overlap and generate precise partial correspondences, achieving superior performance.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Multidisciplinary Sciences
Soheil Soltani, Ashkan Ojaghi, Hui Qiao, Nischita Kaza, Xinyang Li, Qionghai Dai, Adeboye O. Osunkoya, Francisco E. Robles
Summary: This study develops a method using deep-ultraviolet microscopy to quantitatively analyze the aggressiveness and grade of prostate cancer. It also provides multiple optical stains through a network model that converts label-free deep-ultraviolet images into virtual stained images. The research has significant implications for improving the diagnosis and treatment of prostate cancer, and can be applied to other tumor types.
SCIENTIFIC REPORTS
(2022)
Article
Optics
Yun Zhao, Hang Chen, Min Lin, Haiou Zhang, Tao Yan, Ruqi Huang, Xing Lin, Qionghai Dai
Summary: Increasing the layer number of on-chip photonic neural networks (PNNs) is important for improving model performance. To address the issue of larger chip area occupancy due to cascading hidden layers, the optical neural ordinary differential equations (ON-ODEs) architecture parameterizes the continuous dynamics of hidden layers with optical ODE solvers. The ON-ODEs demonstrate high accuracy in image classification tasks and improved model classification accuracy for diffraction-based all-optical linear hidden layer.
Article
Computer Science, Artificial Intelligence
Yue Gao, Siqi Li, Yipeng Li, Yandong Guo, Qionghai Dai
Summary: This paper proposes a Fast-Slow joint synthesis framework, named SuperFast, for event-enhanced high-speed video frame interpolation. It divides the task into two sub-tasks, one for high-speed motion contents and the other for relatively slow-motion contents, and utilizes a fusion module to generate the final video frame interpolation results. Experimental results show that the proposed framework achieves state-of-the-art 200x video frame interpolation performance under high-speed motion scenarios.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Yuxiao Cheng, Runzhao Yang, Zhihong Zhang, Jinli Suo, Qionghai Dai
Summary: In this paper, a dual-sensor camera system is proposed to improve video quality under extremely low light conditions by utilizing the correlation between near-infrared (NIR) and RGB spectrum. A compact camera capturing RGB and NIR videos simultaneously is implemented, and a dual-channel multi-frame attention network (DCMAN) is proposed to enhance the quality of low-light RGB and NIR videos. The performance of the camera design and the reconstruction algorithm is validated through experiments on synthetic and real videos.
INFORMATION FUSION
(2023)
Article
Biochemical Research Methods
Zhi Lu, Yu Liu, Manchang Jin, Xin Luo, Huanjing Yue, Zian Wang, Siqing Zuo, Yunmin Zeng, Jiaqi Fan, Yanwei Pang, Jiamin Wu, Jingyu Yang, Qionghai Dai
Summary: Virtual-scanning light-field microscopy (VsLFM) uses a physics-based deep learning model to improve the quality and speed of LFM, reducing motion artifacts and enabling challenging demonstrations such as fast 3D voltage imaging in Drosophila. By constructing a 40 GB high-resolution scanning LFM dataset across different species, VsLFM exploits physical priors and bypasses hardware scanning to achieve ultrafast 3D imaging in various processes such as the beating heart in embryonic zebrafish and neutrophil migration in the mouse liver.
Article
Biochemical Research Methods
Yuanlong Zhang, Guoxun Zhang, Xiaofei Han, Jiamin Wu, Ziwei Li, Xinyang Li, Guihua Xiao, Hao Xie, Lu Fang, Qionghai Dai
Summary: Widefield microscopy allows optical access to large areas of mammalian brains and thousands of neurons, but signal deterioration due to tissue scattering and background contamination makes neuronal extraction challenging. DeepWonder, a deep-learning-based widefield neuron finder, effectively removes background contaminations and achieves high-fidelity neuronal extraction. It enhances signal-to-background ratio by 50-fold and extracts over 14,000 neurons in 17 hours, providing accurate and efficient neuronal segmentation.
Article
Multidisciplinary Sciences
Yitong Chen, Tiankuang Zhou, Jiamin Wu, Hui Qiao, Xing Lin, Lu Fang, Qionghai Dai
Summary: With the explosive growth of global data, there is a growing need for high-throughput processing in image transmission systems. Existing methods rely on electronic circuits, limiting transmission throughput. In this study, an all-optical variational autoencoder called PED is proposed, which maps the physical system of image transmission into an optical generative neural network. This work establishes a large-scale high-throughput unsupervised optical computing framework that reduces system latency and transmission errors.
Article
Computer Science, Artificial Intelligence
Junhao Liang, Weisheng Zhang, Jianghui Yang, Meilong Wu, Qionghai Dai, Hongfang Yin, Ying Xiao, Lingjie Kong
Summary: Tissue biomarkers are important for cancer diagnosis, prognosis, and treatment planning, but there are few robust biomarkers with true value. A human-centric deep learning framework called PathFinder is presented to help pathologists discover new tissue biomarkers. Using sparse multi-class tissue spatial distribution information and attribution methods, PathFinder achieves localization, characterization, and verification of potential biomarkers with state-of-the-art prognostic performance. The framework has identified the spatial distribution of necrosis in liver cancer as a strong predictor of patient prognosis and proposed clinically independent indicators for practical prognosis.
NATURE MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Bo Zhang, Jinli Suo, Qionghai Dai
Summary: Reliable object detection in dark environments is severely challenged by noise and uneven radiance. To address this, we propose illumination-aware spatio-temporal feature fusion modules for low-light video object detection under a TRansformer network structure. Extensive experiments validate the effectiveness of our approach and demonstrate that DVD-TR outperforms state-of-the-art video detectors on a large-scale multi-illuminance dark video benchmark.
INFORMATION FUSION
(2023)
Article
Optics
Yanwang Zhai, Jingtao Fan, Hui Qiao, Tiankuang Zhou, Jiamin Wu, Qionghai Dai
Summary: The rotational Doppler effect (RDE) of a structured light source carrying orbital angular momentum (OAM) has been improved by utilizing a spiral phase spatial filter (SPSF). The model reveals that a rotating rough surface scatters abundant twisted photons carrying varied OAM values, and the OAM spectrum distribution is modulated by its angular coherence of spatial signature. The method of using SPSF on common surfaces with different autocorrelation structures improves efficiency and robustness for rotator detection.
LASER & PHOTONICS REVIEWS
(2023)
Article
Computer Science, Artificial Intelligence
Kaiming Dong, Yuchen Guo, Runzhao Yang, Yuxiao Cheng, Jinli Suo, Qionghai Dai
Summary: Video object detection has seen significant progress in recent decades, but existing detectors struggle with low lighting conditions and motion blur. To address this, we propose a method that multiplexes frame sequences into one snapshot and extracts motion cues for trajectory retrieval. By incorporating a programmable shutter and using a deep network called DECENT, we can effectively retrieve bounding boxes from blurred images of dynamic scenes. We generate quasi-real data for network learning, which allows for high generalization on real dark videos. This approach offers advantages of low bandwidth, low cost, compact setup, and high accuracy, and has been experimentally validated for night surveillance.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2023)
Article
Computer Science, Artificial Intelligence
Donglin Di, Changqing Zou, Yifan Feng, Haiyan Zhou, Rongrong Ji, Qionghai Dai, Yue Gao
Summary: This article proposes a multi-hypergraph based learning framework called HGSurvNet, which achieves an effective high-order global representation of WSIs for patient survival prediction. Extensive validation experiments demonstrate its superiority over state-of-the-art methods.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Zhihong Zhang, Yuxiao Cheng, Jinli Suo, Liheng Bian, Qionghai Dai
Summary: This article presents a novel non-blind deblurring method called INFWIDE, which can effectively tackle the challenges in low-light photography. By employing a two-branch architecture and fusion network, INFWIDE is able to remove noise, hallucinate saturated regions, and achieve high quality night photograph deblurring. The proposed approach demonstrates superior performance in both synthetic and real data experiments.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2023)
Article
Computer Science, Artificial Intelligence
Zhanping Zhou, Chenyang Zhao, Hui Qiao, Ming Wang, Yuchen Guo, Qian Wang, Rui Zhang, Huaiyu Wu, Fajin Dong, Zhenhong Qi, Jianchu Li, Xinping Tian, Xiaofeng Zeng, Yuxin Jiang, Feng Xu, Qionghai Dai, Meng Yang
Summary: This paper proposes a deep learning-based rheumatoid arthritis assistive system that automatically scores and generates interpretable features, improving the decision-making accuracy of radiologists. By leveraging the advantages of multimodal ultrasound images and addressing the limited training data problem, the system achieves excellent performance in testing.
